Also, how do pipelines compare here?

Pipelines are a type of Task, so like Tasks you can clone and enqueue them, or set them as the target of the trigger.

the most flexible solution would be to have some way of triggering the execution of a script in the parent task environment,

This is the exact idea of the TriggerScheduler None
What am I missing here?

RattySeagull0 I think you are correct, python 3.6 is the installed inside the docker. Is it important to have 3.7 ? You might need another docker (or change the installation script and install python 3.7 inside)

this is why....

After testing the code again, I see the task parameter dictionary has been removed properly

Great!

However, I still have the same problem with duplicate tasks, as you can see in the image.

Any chance the pipeline script Itself is running from the agent (as opposed to running the pipeline code locally, then the pipelines are executed on the agent)?

Hi SpicyOtter88
plt.plot([0, 1], [0, 1], 'r--', label='')ti cannot have a legend without a label, so it gives it "anonymous" label, I think it should just get "unlabeled 0" wdyt?

Hi @<1524922424720625664:profile|TartLeopard58>

• Opened container ports for VS Code, JupyterLab, and SSH.I think that by default it uses the host network so it can take care of that, are you saying you added k8s integration ?
• Added NodePort to the service to directly access via public IP:NodePort (previously only SSH was available, but now NodePort is added for VS Code and JupyterLab as well), allowing direct access without SSH tunneling.Interesting!
• Considering security vulnerabilitie...

Hi AttractiveCockroach17
In your "Installed Packages" (when the task is in draft mode, you can edit it like any requirements.txt), you need to add:
package @ git+You can also make sure you have in in the first place bu adding
Task.add_requirements("package", "@ git+ ") task = Task.init(...)

where the ui merges the plots just as we want and I was wondering if there is some simple way to do it in the case of all plots.

we can do it for scalars (this is trivial)
We can merge specific plots when they are simple, I think basic histograms.
But for any generic plots we fear the merge will just fail, and this is why it defaults to side by side.

how can I combine two plots in the ui as you mentioned?

The easiest solution is to use, "report_scatter2d", these are specific pl...

Nice!!!

DeterminedToad86
Yes I think this is the issue, on SageMaker a specific compiled version of torchvision was installed (probably part of the image)
Edit the Task (before enqueuing) and change the torchvision URL to:
torchvision==0.7.0Let me know if it worked

Hi GentleSwallow91
I think this would be a good start:
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py
wdyt?

I have a question regarding running the code on the remote machine, each time I run the code I see the console in the ClearML server start downloading all the libraries I used in the code and when I run another code the same thing happens so why it has to download all the libraries again and many times?

I'm assuming you are referring to the installation, the downloaded python packages are cached.
You can turn on full caching by uncommenting the following line:
https://github.com/alleg...

@<1545216077846286336:profile|DistraughtSquirrel81> shoot an email to "support@clear.ml" and provide all the information you can on the "lost account" (i.e. the one you had the data on), this means email account that created it (or your colleagues emails), and any other information that might help to locate it.

Hmm, could it be that the working dir is outside of the git repo?

potential sources of slow down in the training code

Is there one?

clearml doesn’t do any “magic” in regard to this for tensorflow, pytorch etc right?

No 😞 and if you have an idea on how, that will be great.
Basically the problem is that there is no "standard" way to know which layer is in/out

and i found our lab seems only have shared user file because i installed trains on one node, but it doesn’t appear on the others

Do you mean there is no shared filesystem among the different machines ?

Hmm that is a good idea, and I think you are correct, it cannot support it. But it will be easy to do, maybe adding an argument trigger_on_archive ? wdyt?

Sure thing 🙂

Thanks!

Thank you!

Ohh SubstantialElk6 please use agent RC3, (latest RC is somewhat broken sorry, we will pull it out)

RC3 is out with a few fixes, please feel free to test 🙂

ShallowGoldfish8 I believe it was solved in 1.9.0, can you verify?
pip install clearml==1.9.0

Hi GrittyKangaroo27

Is it possible to import user-defined modules when wrapping tasks/steps with functions and decorators?

Sure, any package (local included) can be imported, and will be automatically listed in the "installed packages" section of the pipeline component Task
(This of course assumes that on a remote machine you could do the "pip install <package")
Make sense ?

WickedGoat98 sure that will not be complicated:
try something along the lines of :
agent: networks: - backend container_name: clearml-agent image: allegroai/clearml-agent:latest restart: unless-stopped privileged: true environment: CLEARML_HOST_IP: ${CLEARML_HOST_IP} CLEARML_WEB_HOST: ${CLEARML_WEB_HOST:-} CLEARML_API_HOST: `
CLEARML_FILES_HOST: ${CLEARML_FILES_HOST:-}
CLEARML_API_ACCESS_KEY: ${CLEARML_API_ACCESS_KEY:-}
...

Is this reproducible? I tried to run the same example code on my machine, and it started training ...
Do you have issues with other pytorch examples? Could you try simple reporting example:
https://github.com/allegroai/clearml/blob/master/examples/reporting/scalar_reporting.py

Hmm can you try:
--args overrides="['log.clearml=True','train.epochs=200','clearml.save=True']"

Hi all! Does anyone know a solution to my issue with deploying models saved on azure on the clearml-serving docker container?

Hi NuttyCamel41
The easiest is to map the clearml.conf to both the serving and triton containers in your docker-compose.yaml (or k8s secrets) and make sure the conf file has the credentials to access the azure blob. wdyt ?

What's the pip verison?