Hey @<1577106212921544704:profile|WickedSquirrel54> , I would definitely be interested in this. A gist would be cool too

by special run args?

uses clearml 1.13.2

I want the script to be agnostic to whether it is run using clearml or not, with a particular queue or not

AMAZING!

is it in the OSS version too?

Thanks, I can have docker + poetry execution modes then?

How?

where is it persisted? if I have multiple sessions I want to persist, is that possible?

Amazing. Going to play with it this weekend !

it worked. The env variables definitely do not work! Had to use clearml.conf along with use_credential_chain=True

@<1523701070390366208:profile|CostlyOstrich36> , as written above, I've done that. It still tries to send to 8081

any luck, is this expected?

I've also overriden CLEARML_FILES_HOST= None , and configured it in clearml.conf file. Don't know where its picking 8081 😕

I tried that earlier - that checks out , it matches the s3 path I provide in the conf

None

I'm thinking of using s3fs on the entire /opt/clearml/data folder. What do you think?

None

because, otherwise it becomes a bit of a chicken and egg problem

• update code
• git push
• docker build and push on CI
• use new docker sha for task execution
• update code
• git push
• repeat?

can I combine docker and poetry mode?

thanks!

Also @<1523701070390366208:profile|CostlyOstrich36> - are these actions available for on prem OSS clearml-server deployments too?

nice! I was wondering whether we can trigger it by the UI, like "on publishing" an experiment

@<1537605940121964544:profile|EnthusiasticShrimp49> , now that I have run the task on remote, can I copy the artefacts/files it creates back to my local fs?

Lets say the artefacts are something like
artefacts = [checkpoint.pth, dvc.lock, some_other_dynamically_generated_file]

Hmmm, my only issue there is that not all of my "artefacts" are clearml artefacts.

The files I need are models and other locally modified files that get generated by the clearml task on remote

Thanks! so it seems like the key is the Task.connect and bubble up params to original task, correct?

I do change the task and the project name, the task name change works fine but the project name change silently fails

Would I also be able to change the task name from within the subprocess?

want to work on it together?

Its a simple training loop that trains models for 2-3 epochs for a total of 200-300 iterations, saves a few checkpoints and saves a final model at the end of it