Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
I Would Like To Schedule A Task For Remote Execution.

I would like to schedule a Task for remote execution. What is the proper way to pass parameters through clearml-task to a task running with Hydra? For example, I would like to schedule a task where some model parameters are overriden. Using local dev, I would use CLI arguments to override:
python src/train_vae.py model.beta_scheduler.target_rate=8.0But what is the appropriate equivalent using scheduled tasks for remote execution?

I have tried two approaches:

Approach A: argparse_args
Issue here is that the args are not passed along to hydra, for some reason?

` task = Task.create(
add_task_init_call=False, # called explicitly by script
argparse_args=[("model.beta_scheduler.target_rate", "8.0")],

Task.enqueue(task, queue_name="default") .. but this is not being registered by my hydra script, which looks like this: def init_clearml(config: DictConfig):
task: Task = Task.init(
out_config = OmegaConf.to_container(config)
out_config = task.connect(out_config, "HPO")
out_config = out_config._to_dict()
out_config = OmegaConf.create(out_config)
assert isinstance(out_config, DictConfig)
return out_config

@hydra.main(version_base=None, config_path="configs", config_name="default")
def main(cfg: DictConfig):
cfg = init_clearml(cfg)
... `
Approach B: Using task parameters
Issue here is that numeric parameters are stored as numerics, but read out as strings.

An alternative approach I did, was to use the task set_parameter:
task: Task = Task.create( project_name="Sandbox", task_name="Scheduled", task_type="custom", script="src/train_vae.py", requirements_file="environment/requirements.txt", add_task_init_call=False, ) task.set_parameter("HPO/model/beta_scheduler/target_rate", 8.0, value_type=float) Task.enqueue(task, queue_name="default")Since in our script we connect to the HPO object. But here the issue is that numeric types are being read out as strings when using the connect handle , even though they are successfully stored as float:
task.get_parameters(cast=True) # output: {'HPO/model/beta_scheduler/target_rate': 8.0}
Update: Workaround

I found a workaround in updating our init_clearml to use the SDK that enables casting:
` def init_clearml(config: DictConfig):
task: Task = Task.init(...)
# Overwrite the hydra config with parameters from ClearML
if hpo_dict := task.get_parameters_as_dict(cast=True).get("HPO"):
config = OmegaConf.merge(config, hpo_dict)

out_config = OmegaConf.to_container(config)
out_config = task.connect(out_config, "HPO")
return config `
Posted one year ago
Votes Newest


Nice SoreHorse95 !
BTW: you can edit the entire omegaconf yaml externally with set/get configuration object (name = OmegaConf) , do notice you will need to change Hydra/allow_omegaconf_edit to true

Posted one year ago