GrievingTurkey78 I see,
Basically the arguments after the -m src.train
in the remote execution should be ignored (they are not needed).
Change the m in the Args section under the configuration. Let me know if it solved it.
Hi GrievingTurkey78
Could you provide some more details on your use case, and what's expected?
Side note: When running src.train
as a module the server gets the command as src
and has to be modified to be src.train
AgitatedDove14 Thanks! I’ll give it a try! Makes sense 👌
The -m src.train
is just the entry script for the execution all the rest is be taken care by the Configuration section (whatever you pass after it will be ignored if you are using Argparse as it is auto-connects with ClearML)
Make sense ?
Sure! I enqueue the experiment from my local machine:python -m src.train model=my_model loss=my_loss dataset=my_dataset
Then I go to the server and run the experiment and create a copy to run with a new model. On the copy, I go to the script path
and modify it to be:-m src.train model=my_other_model loss=my_loss dataset=my_dataset
The new experiment, even though the script path
has my_new_model
default, starts training using my_model
.
I can also see on the configuration tab -> OmegaConf the properties of my_model
instead of my_new_model
.