Can you test with the hydra example? if the example works, any chance you can send a toy to reproduce it ?
https://github.com/allegroai/clearml/tree/master/examples/frameworks/hydra

DepressedChimpanzee34
I might have an idea , based on the log you are getting LazyCompletionHelp in stead of str
Could it be you installed hyrda bash completion ?
https://github.com/facebookresearch/hydra/blob/3f74e8fced2ae62f2098b701e7fdabc1eed3cbb6/hydra/_internal/utils.py#L483

DepressedChimpanzee34 I cannot find cfg.py here
https://github.com/allegroai/clearml/tree/master/examples/frameworks/hydra/config_files
(or anywhere else)

Yes, I think we just found out it breaks clearml 🙂
could you test with the latest stable, just in case ?
(I'll make sure we have an RC that supports the hydra dev version)

could you test with the latest stable, just in case - unfortunately, no, it’s not that easy to make my code working with that hydra(

thank you for your help!

let me check when a fix can be deployed for Hydra...

DepressedChimpanzee34
What's the hydra version ?
I tested with 1.1.0dev3 and it worked for me

Hmm let me rerun (offline mode right ?)

the offline error is different

no, online

well at this point I'm not sure it is still essential, we have 3 run-modes offline, local-server, cloud-sever and this option made it work for all of them.. can be that it is not required anymore and its just legacy..

LOL, sure if you have so many setups, that makes sense 🙂

this is strange.. you ran it with the dataclass config I added?

Yes but I had to remove the:
from config_files import cfgand instead used:
@hydra.main(config_path="config_files", config_name="config") def my_app():I need help in reproducing it.

Hi AgitatedDove14 , I am using the pre release of hydra together with the latest clearml, and previous clearml server version (I didn't have time yet to update)..
I'm getting similar error to the mentioned in the post:
2021-05-06 15:48:07,359 - root - WARNING - 2021-05-06 15:48:07,359 - root - WARNING - File "/home/user/.local/lib/python3.8/site-packages/clearml/backend_api/services/v2_9/tasks.py", line 1067, in description self.assert_isinstance(value, "description", six.string_types) 2021-05-06 15:48:07,359 - root - WARNING - 2021-05-06 15:48:07,359 - root - WARNING - File "/home/user/.local/lib/python3.8/site-packages/clearml/backend_api/session/datamodel.py", line 116, in assert_isinstance raise TypeError("Expected %s of type %s, got %s" % (field_name, expected, type(value).__name__)) 2021-05-06 15:48:07,360 - root - WARNING - 2021-05-06 15:48:07,360 - root - WARNING - TypeError: Expected description of type (<class 'str'>,), got LazyCompletionHelp 2021-05-06 15:48:07,360 - root - WARNING - 2021-05-06 15:48:07,360 - root - WARNING - Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. Process finished with exit code 1any ideas how to solve this issue?

it happens on task init

dev6

I get it with the simplest config if I define it as a dataclass, using the example you share as a basis:
config_files/cfg.py
` from hydra.core.config_store import ConfigStore
from dataclasses import dataclass

@dataclass
class MasterConfig:
test: str = 'test'

cs = ConfigStore.instance()
cs.store(name="config", node=MasterConfig) and for the main I had to make some small changes to connect my local server (I'm sharing them just in case): hydra_example.py: # ClearML - Hydra Example

import hydra

from omegaconf import OmegaConf

from clearml import Task
from dataclasses import dataclass
from config_files import cfg

@dataclass
class CLEARML_SERVER_CONFIG:
API_HOST: str
WEB_HOST: str
FILE_SERVER: str
OUTPUT_URI: str
KEY: str
SECRET: str
# HOST: str

@hydra.main(config_name="config")
def my_app(cfg):
# type (DictConfig) -> None

CLEARML_LOCAL = CLEARML_SERVER_CONFIG(API_HOST=' ` ` ',
                                      WEB_HOST=' ` ` ',
                                      FILE_SERVER=' ` ` ',
                                      OUTPUT_URI=' ` ` ',
                                      KEY='2CBR16L9T9R6ANYTAA5K',
                                      SECRET='pK)i-(ImwRPyL(Eu1nKB5cdy(Y+9_XkGxLrpZmWw0^Abyc7&+b')
# if credentials are not set should offline doesn't work
Task.set_credentials(api_host=CLEARML_LOCAL.API_HOST, web_host=CLEARML_LOCAL.WEB_HOST,
                     files_host=CLEARML_LOCAL.FILE_SERVER, key=CLEARML_LOCAL.KEY,
                     secret=CLEARML_LOCAL.SECRET)
task = Task.init(project_name="examples", task_name="hydra configuration")

if name == "main":
my_app() `and the output error is the same as I shared above

Why do you need to have the configuration added manually ? isn't the cleaml.conf easier ? If not I think OS environments are easier no?- well at this point I'm not sure it is still essential, we have 3 run-modes offline, local-server, cloud-sever and this option made it work for all of them.. can be that it is not required anymore and its just legacy..
I run run above code, everything worked with no exception/warning...- this is strange.. you ran it with the dataclass config I added?
What is the try/except solves exactly ? - it solved the issue I reported
TypeError: Expected description of type (<class 'str'>,), got LazyCompletionHelp

I found a super weird fix for this error.. no idea why it works but maybe it can help with debugging it..
try: task = Task.init(project_name="examples", task_name="hydra configuration", reuse_last_task_id=False) except: task = Task.init(project_name="examples", task_name="hydra configuration", reuse_last_task_id=True)

Hi DepressedChimpanzee34
Why do you need to have the configuration added manually ? isn't the cleaml.conf easier ? If not I think OS environments are easier no? I run run above code, everything worked with no exception/warning... What is the try/except solves exactly ?

if I remove the import and use the path I get this:
raise MissingConfigException(hydra.errors.MissingConfigException: Cannot find primary config 'config'. Check that it's in your config search path.

its the module where the cfg is defined, like in the example you shared config_files/cfg.py

and the path it shows is correct..

from the example I shared above
config_files/cfg.py
` from hydra.core.config_store import ConfigStore
from dataclasses import dataclass

@dataclass
class MasterConfig:
test: str = 'test'

cs = ConfigStore.instance()
cs.store(name="config", node=MasterConfig) `

Any chance you can zip the entire folder? I can't figure out what's missing, specifically "from config_files" , i.e. I have no packages nor file named config_files

DepressedChimpanzee34
I have sad news, it is working just fine for me 😞
Python3.6 Ubuntu hydra-core==1.1.0.dev6

its only these two files.. nothing else

😕 well I have py38 and debian.. strange I'll give it another look

Python3.8 I can quickly check, give me a minute

Noooooooooo, it is still working 🙂

thanks AgitatedDove14 , Not that I know I didn't do any special setup other than the straightforward pip install hydra-core --pre

AgitatedDove14 , the hydra people say its not on "their side".. so it is still a mystery to me