Ohh I see, okay next pipeline version (coming very very soon 😉 will have the option of function as Task, would that be better for your use case ?
(Also in case of local execution, and I can totally see why this is important, how would you specify where is the current code base ? are you expecting it to be local ?)
I'm all for trying to help with debugging pipeline, because this is really challenging.
BTW: you can run your code as if it is executed from an agent (including the param ove...

Hmm that is a good question, are you mounting the clearml.conf somehow ?

👍

ReassuredTiger98 are you saying you want to be able to run the pipeline as a standalone and as "remote pipeline",
Or is this for a specific step in the pipeline that you want to be able to run standalone/pipelined ?

SmarmySeaurchin8
Something like this one:
vector_series = np.random.randint(10, size=10).reshape(2,5) logger.report_vector(title='vector example', series='vector series', values=vector_series, iteration=0, labels=['A','B'], xaxis='X axis label', yaxis='Y axis label')

For example, for some of our models we create pdf reports, that we save in a folder in the NFS disk

Oh, why not as artifacts ? at least you will be able to access from the web UI, and avoid VFS credential hell 🙂

Regrading clearml datasets:
https://www.youtube.com/watch?v=S2pz9jn26uI

hmm interesting use case, why do you need to add the "--no-binary"

NICE! MagnificentSeaurchin79 could you PR this fix?

BTW: if you could implement _AzureBlobServiceStorageDriver with the new Azure package, it will be great:
Basically update this class:
https://github.com/allegroai/clearml/blob/6c96e6017403d4b3f991f7401e68c9aa71d55aa5/clearml/storage/helper.py#L1620

So inside the pipeline logic you can do Task.current_task().id
Or inside a component Task.current_task().parent

Just making sure, pip package installed on your Conda env, correct?

is the model overridden or its version is automatically increased?

You will have another model, with the same name (assuming the second Task has the same name), but a new ID. So if I understand you correctly, we have auto-versioning :)

you need to set

CLEARML_DEFAULT_BASE_SERVE_URL: 

So it knows how to access itself

Are you doing from keras import ... or from tensorflow.keras import ?

Hi ReassuredTiger98
I think it used to be the default and then it was removed, it has no real affect on performance but it remove all asserts ... what is your use case ? do you see any performance gains ?

Do we support GPUs in a) docker mode b) k8s glue?

yes on both

Is there a good reference to get started with k8s glue?

A few folks here already set it up, do you have a k8s cluster with GPU support ?

using only a subset of the features

ShallowGoldfish8 if you have some parameter that controls it (i.e. select different features) then you can launch it with two sets f parameters.
Am I missing something?
for example:
` my_features_select = {"type": "set_a"}
Task.current_task().connect(my_features_select)

if my_features_select["type"] == "set_a":

do something

else

do something else `wdyt?

I should mention this is run within a TF v1 session context

This should not be connected.

everything gets stored as intended (to clearML dashboard)

So in jupyter it works? But from command line it does not ? what's the difference ?

DefeatedOstrich93 what do you mean by "I am wondering why do I need to create files before applying diff ?"
git diff will not list files unless their are added (they are marked as "untracked") think temp files logs etc. until you add a file to git it will basically ignore that file. Make sense ?

is your server

you mean the app.clear.ml ?

Yes exactly !

Wait, that makes no sense to me. The API from python and the API from the UI are getting the same data from the backend ...
What are you getting with?
from clearml import Task task = Task.get_task(task_id=<put task id here>) print(task.models)

This is why it thinks it's CPU 🙂

correct (i.e. not frontend)

you are correct, I was referring to the template experiment

Hi @<1529271085315395584:profile|AmusedCat74>

ClearML Scheduler where it doesn't reuse the task

What do you mean by doesn't reuse the Task, do you mean you want each time the scheduler is launched to basically overwrite the previous run ?

If i have an alternative location for the vscode, where should i indicate in the configuration?

We might need to add support for that, but it should not be a problem to override (e.g. downloadable link like http/s3/ etc.)
Is this something that is doable ?

This seems to only work for a single file (weights_path implies a single file, not multiple ones). Is that the case?See update_weights_package actually packages an entire folder as zip and will do the extraction when you get it back (check the function docstring, I think you can also specify wildcard etc if needed)

Why do you see this as preferred to the dataset method we have now?

So it answers a few requirements that you raised
It is fully visible as part of the project and se...

I'm assuming your are looking for the AWS autoscaler, spinning EC2 instances up/down and running daemons on them.
https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py

https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler

JuicyDog96 Yes please!
Let me check what's the status with the docs repository, and I'll get back to you soon 🙂