Hi TrickyFox41

Hey since Hydra does not work with

clearml-task

I should shouldn't it? what does not work ?

--args param.ovveride=value does not work with clearml-task

By the way, since if i create the task locally, reset it and enqueue it, it works. This is the workaround that i'm using right now

I see TrickyFox41 try the following:
--args overrides="param=value"Notice this will change the Args/overrides argument that will be parsed by hydra to override it's params

I tried like that clearml-task --script train.py --args overrides="log.clearml=True train.epochs=200 clearml.save=True" --project mornag-plain-dense --name mornag-plain-dense-training --queue tesla-t4 --skip-task-init

It only works with argparse

TrickyFox41 are you saying that if you add Task.init inthe code it works, but when you are calling "clearml-task" it does not work? (in both cases editing the Args/overrides ?

Hmm that should have worked ...
I'm assuming the Task itself is running on a remote agent, correct ?
Can you see the changes in the OmegaConf section ?
what happens when you pass
--args overrides="['dataset.path=abcd']"

So I think there are two bugs here?
--args overrides="key=value" does not work request: add --hydra to override hydra arguments (and if this is added the first one is not needed)Is that correct?

Yes the task is running on a remote agent with the --docker flag
this is the config on the machine(s) running the agent
`
agent {
venvs_cache: {
max_entries: 50
free_space_threshold_gb: -1
path: ~/.clearml/venvs-cache
}
extra_docker_arguments: [
"--network", "host",
"-v", "/home/ubuntu/.ssh:/root/.ssh:ro",
"-v", "/home/ubuntu/.cache:/root/.cache",
]

docker_internal_mounts {
    sdk_cache: "/clearml_agent_cache"
    apt_cache: "/var/cache/apt/archives"
    ssh_folder: "~/.ssh"
    ssh_ro_folder: "/.ssh"
    pip_cache: "/root/.pip-cache"
    poetry_cache: "/root/.pypoetry-cache"
    vcs_cache: "/root/.clearml/vcs-cache"
    venv_build: "~/.clearml/venvs-builds"
    pip_download: "/root/.clearml/pip-download-cache"
}

}

sdk {
aws {
s3 {
host: "...:443"
key: "..."
secret: "..."
secure: true
}
}
}

api {
web_server: https://...
api_server: https://...
file_server: https://...

credentials {

}

} `

No luck with --args overrides="['dataset.path=abcd']"

Hmm can you try:
--args overrides="['log.clearml=True','train.epochs=200','clearml.save=True']"

same result. in print(cfg) there are no overrides

AgitatedDove14 Yes exactly!

What do you have under Hydra section? and OmegaConf section

I used an env variable to avoid creating and endless loop of init/enqueue (using an argument like clearml.queue that would be captured and forwarded to the agent)

i printed cfg in the script and the config has not been overwritten 😢

Related GitHub issue https://github.com/allegroai/clearml/issues/847

This is what it looks like from the web app