Sure! I enqueue the experiment from my local machine:python -m src.train model=my_model loss=my_loss dataset=my_dataset
Then I go to the server and run the experiment and create a copy to run with a new model. On the copy, I go to the script path
and modify it to be:-m src.train model=my_other_model loss=my_loss dataset=my_dataset
The new experiment, even though the script path
has my_new_model
default, starts training using my_model
.
I can also see on the configuration tab -> OmegaConf the properties of my_model
instead of my_new_model
.
GrievingTurkey78 I see,
Basically the arguments after the -m src.train
in the remote execution should be ignored (they are not needed).
Change the m in the Args section under the configuration. Let me know if it solved it.
Side note: When running src.train
as a module the server gets the command as src
and has to be modified to be src.train
The -m src.train
is just the entry script for the execution all the rest is be taken care by the Configuration section (whatever you pass after it will be ignored if you are using Argparse as it is auto-connects with ClearML)
Make sense ?
AgitatedDove14 Thanks! I’ll give it a try! Makes sense 👌
Hi GrievingTurkey78
Could you provide some more details on your use case, and what's expected?