Interesting...
We could followup the .env configuration, and allow the clearml-task to add configuration files from cmd line. This will be relatively easy to add. We could expand the Environment support (that somewhat exists), and add the ability to read variables from .emv and Add them to an "hyperparemeter" section, named Environment. wdyt?
But instead I'd like to connect the configuration ahead of time while i'm submitting the task from the command line
Interesting use case, do you already have the connect_configuration in the code? or do we need to somehow create it ?
Is there a preferred way of passing environmental variables to a task? I'm willing to do it however, just the best way I've found to do it has been
1. Have a local .env variable on a client machine 2. Start script running on local laptop. connect_configuration pulls from the .env file 3. now in the clearml UI, the configuration_objects dropdown includes the environmental variables from my local machine necessary to run the script 4. I abort the run running from my laptop, and copy it onto the work queue
Hi ShallowArcticwolf27
from the command line to a remote machine while loading a local
.env
file as a configuration object?
Where would the ".env" go to ? Are we trying to pass it to the remote machine somehow ?
I would love to help add the functionality to do either, but adding the functionality to clearml-task sounds very attractive!
Yes, I would like the contents of the .env file to end up in the configuration_objects
Currently, I can do this if I first run the script locally, and then enqueue it to a remote machine through the UI
env_file = task.connect_configuration('.env', name='env_file') load_dotenv(dotenv_path=env_file, override=True)
adding the functionality to clearml-task sounds very attractive!
Hmm, what do you think?parser.add_argument('--configuration', type=str, default=None, help='Specify local configuration file' ) parser.add_argument('--configuration-name', type=str, default=None, help='configuration section name' ) ... with open(args.configuration, 'rt') as f: create_populate.task.set_configuration_object(args.name, config_text=f.read())Add here:
https://github.com/allegroai/clearml/blob/master/clearml/cli/task/main.py
Should work, feel free to PR once you verify 🙂
Feel free to change the argument names
I'll take a look at this this afternoon, thanks for your help