Ok, so after updating to trains==0.16.2rc0, my problem is different: when I clone a task, update its script and enqueue it, it does not have any Hyper-parameters/argv section in the UI

The cloning is done in another task, which has the argv parameters I want the cloned task to inherit from

Yes 🙂 Thanks!

AgitatedDove14 How can I filter out tasks archived? I don't see this option

` # Set the python version to use when creating the virtual environment and launching the experiment
# Example values: "/usr/bin/python3" or "/usr/local/bin/python3.6"
# The default is the python executing the clearml_agent
python_binary: ""
# ignore any requested python version (Default: False, if a Task was using a
# specific python version and the system supports multiple python the agent will use the requested python version)
# ignore_requested_python_version: ...

And since I ran the task locally with python3.9, it used that version in the docker container

What I put in the clearml.conf is the following:

agent.package_manager.pip_version = "==20.2.3" agent.package_manager.extra_index_url: [" "] agent.python_binary = python3.8

So either I specify in the clearml-agent agent.python_binary: python3.8 as you suggested, or I enforce the task locally to run with python3.8 using task.data.script.binary

yes but they are in plain text and I would like to avoid that

how would it interact with the clearml-server api service? would it be completely transparent?

Alright I have a followup question then: I used the param --user-folder “~/projects/my-project”, but any change I do is not reflected in this folder. I guess I am in the docker space, but this folder is not linked to my the folder on the machine. Is it possible to do so?

So I installed docker, added user to group allowed to run docker (not to have to run with sudo, otherwise it fails), then ran these two commands and it worked

Now it starts, I’ll see if this solves the issue

AMI ami-08e9a0e4210f38cb6 , region: eu-west-1a

doesn’t really work unfortunately

I see what I described in https://allegroai-trains.slack.com/archives/CTK20V944/p1598522409118300?thread_ts=1598521225.117200&cid=CTK20V944 :
randomly, one of the two experiments is shown for that agent

thanks for your help anyway AgitatedDove14 !

Thanks a lot AgitatedDove14 !

should I try to roll back to clearml-server 1.0.2? I am very anxious now…

Hi AgitatedDove14 , thanks for the answer! I will try adding 'multiprocessing_context='forkserver' to the DataLoader. In the issue you linked, nirraviv mentionned that forkserver was slower and shared a link to another issue https://github.com/pytorch/pytorch/issues/15849#issuecomment-573921048 where someone implemented a fast variant of the DataLoader to overcome the speed problem.
Did you experiment any drop of performances using forkserver? If yes, did you test the variant suggested i...

AppetizingMouse58 After some thoughts, we decided to install from scratch 0.16, with no data migration, because we believe this was an edge case not worth spending efforts on. Thank you very much for your help there, very appreciated. You guys rock! 🙂

Oops, I spoke to fast, the json is actually not saved in s3

So two possible cases for trains-agent-1: either:
It picks a new experiment -> show randomly one of the two experiments in the "workers" tab no new experiment in default queue to start -> show randomly no experiment or the one that it is running

Nice, seems to work! 🎉

Hi SoggyFrog26 , https://github.com/allegroai/clearml/blob/master/docs/datasets.md

not really, because it is in the middle of the controller task, there are other things to be done afterwards (retrieving results, logging new artifacts, creating new tasks, etc)

AgitatedDove14 This seems to be consistent even if I specify the absolute path to /home/user/trains.conf

Thanks!

AgitatedDove14 awesome! by "include it all" do you mean wizard for azure and gcp?

Why?