I would like to schedule a Task for remote execution. What is the proper way to pass parameters through clearml-task to a task running with Hydra? For example, I would like to schedule a task where some model parameters are overriden. Using local dev, I would use CLI arguments to override:
python src/train_vae.py model.beta_scheduler.target_rate=8.0But what is the appropriate equivalent using scheduled tasks for remote execution?

I have tried two approaches:

Approach A: argparse_args
Issue here is that the args are not passed along to hydra, for some reason?

` task = Task.create(
project_name="Sandbox",
task_name="Testing",
task_type="custom",
script="src/train_vae.py",
requirements_file="environment/requirements.txt",
add_task_init_call=False, # called explicitly by script
argparse_args=[("model.beta_scheduler.target_rate", "8.0")],
)

Task.enqueue(task, queue_name="default") .. but this is not being registered by my hydra script, which looks like this: def init_clearml(config: DictConfig):
task: Task = Task.init(
project_name="Sandbox",
task_name="Sandbox",
)
out_config = OmegaConf.to_container(config)
out_config = task.connect(out_config, "HPO")
out_config = out_config._to_dict()
out_config = OmegaConf.create(out_config)
assert isinstance(out_config, DictConfig)
return out_config

@hydra.main(version_base=None, config_path="configs", config_name="default")
def main(cfg: DictConfig):
cfg = init_clearml(cfg)
... `
Approach B: Using task parameters
Issue here is that numeric parameters are stored as numerics, but read out as strings.

An alternative approach I did, was to use the task set_parameter:
task: Task = Task.create( project_name="Sandbox", task_name="Scheduled", task_type="custom", script="src/train_vae.py", requirements_file="environment/requirements.txt", add_task_init_call=False, ) task.set_parameter("HPO/model/beta_scheduler/target_rate", 8.0, value_type=float) Task.enqueue(task, queue_name="default")Since in our script we connect to the HPO object. But here the issue is that numeric types are being read out as strings when using the connect handle , even though they are successfully stored as float:
task.get_parameters(cast=True) # output: {'HPO/model/beta_scheduler/target_rate': 8.0}
Update: Workaround

I found a workaround in updating our init_clearml to use the SDK that enables casting:
` def init_clearml(config: DictConfig):
task: Task = Task.init(...)
# Overwrite the hydra config with parameters from ClearML
if hpo_dict := task.get_parameters_as_dict(cast=True).get("HPO"):
config = OmegaConf.merge(config, hpo_dict)

out_config = OmegaConf.to_container(config)
out_config = task.connect(out_config, "HPO")
return config `