Could it be hydra was installed on your laptop via conda not pip?

Yes, while we do use a conda env, our packages are installed using pip . That being said, I have hydra-core==1.1.1 in my local dependencies as well.

Will try this. Thanks for promptly looking into this. Much appreciated!

Thanks for getting back Martin. The hydra example fails when i try to queue it to my local with
Starting Task Execution: Traceback (most recent call last): File "hydra_example.py", line 10, in <module> @hydra.main(config_path="config_files", config_name="config") AttributeError: module 'hydra' has no attribute 'main'

agent default python is set to 3.9.7

I'm looking at the docs on docker mode and running the script. Is this script run after the venv and code dir are setup, or immediately after the container starts but before the environment for running the experiment is setup?

Aah I see it only says Image . Somehow I hit tunnel vision on Base Docker Image as stated in the docs and couldn't identify both to mean the same thing 😅 thanks

hydra dep does show up

I thought the agent created a new conda env and installed all packages, recorded during initial task run, from scratch (except for caching with venv). Is that not the case?

I think the fire + hydra combination is not an issue anymore. We're going to separate the 2 out, and I tried it last night and argument modification and passing worked fine with hydra only.
In any case, thanks for you help Martin!

Thanks! I'll give the RC a shot.

I tried using 1.2.0rc1 but it doesn't work as expected. We have a bunch of options for fire in the entrypoint, but irrespective of whichever I enter on the command line, fire still just executes the first command that was defined in my dictionary under fire.Fire({...}) . It however routes to the correct command if I use 1.1.6 which tells me that this is being caused by some issue with 1.2.0rc1

Will do!

For hydra-core:
` ...

• humanfriendly==10.0
• hydra==2.5
• idna==3.3
  ... `

Sorry for the delay CostlyOstrich36 here's the relevant lines from the console:
` ...
File "/home/binoyloaner/miniconda3/envs/DS974/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/binoyloaner/miniconda3/envs/DS974/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 103, in forward
return F.linear(input, self.weight, self.bias)
File "/home/binoyloaner/miniconda3/envs/DS974/lib/python3....

The Agent pulls the Task, and then reproduces it, and now it will execute the extra_docker_shell_script that was put in the configuration file.Does this imply the former? Env is fully setup, then script is run, then experiment is started by calling the executable?

Ok. I think I misunderstood what you said. I thought you meant you've already opened a bug ticket. If that's not the case, do you want to me create one on github?

We have run experiments in the past (before I put ClearML into my code) which has logged scalars, plots etc. to local tensorboard. Is there any way to import this data to ClearML cloud for tracking, visualization and comparison?

the CML free SaaS offering. It'll probably hit https://app.clear.ml/api if I'm not wrong

Just general company policy - the team should own the workspace, not a single IC on the team so anyone can make changes in case of lets say personnel changes for example

https://github.com/allegroai/clearml-server/issues/120

the state of the Task changes immediately when it crashes ?

I think so. It goes from running to completed immediately on crash

clearml's callback is never called

yeah I suspect that's what might be happening which is why I was inquiring as to how and where exactly in the CML code that happens. Once I know, I can then place breakpoints in the critical regions and debug to see what's going in.

The toy task is marked "Failed".

Do you want me to try running it manually?

Thanks! Do you have a public bug tracker? If yes, are you able to share the issue number so I can follow it?
I need to put it into my code, so will be eagerly waiting for the fix

Yes I believe it's hydra too, so just learning how CML determines process status will be really helpful

It did indeed. Thanks!

Yes

Sorry if I sounded curt. Didn't mean to. To clarify, I've created my account using Google SSO on http://app.clear.ml , and am currently on the Free tier. I am pushing all my data onto CML's servers. This error happens when I try to query those servers for the metrics and variants for a particular task of mine.

I think there's some confusion here. I'm not running the server. My metrics are getting logged to the CML cloud.