Hi WittyOwl57 ,
Can you give a screenshot of how it's saved in the UI currently? Also can you look at the developer tools and see what tasks.get_configuration_names
& tasks.get_configurations
return when looking at the experiment's configurations in the UI
Hi WittyOwl57 ,
The function is :
task.get_configuration_object_as_dict ( name="name" )
with task being your Task object.
You could find a bunch of pretty similar functions in the docs. Have a look at here : https://clear.ml/docs/latest/docs/references/sdk/task#get_configuration_object_as_dict
AgitatedDove14 Yes! That would be exactly what I want (i.e. get_configuration_as_dict
.) Alas, no such thing exists in 1.4.1. Is that supposed to come in a next version?
CostlyOstrich36{"meta":{"id":"3cceedbbc65d480096ebb02b5aba5902","trx":"3cceedbbc65d480096ebb02b5aba5902","endpoint":{"name":"tasks.get_configurations","requested_version":"2.17","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"configurations":[{"task":"9910bc0401804bb787312d39c3a251ff","configuration":[{"name":"filter","value":"inference = [{\n type = \"StreamFilter\"\n params {\n context = \"full\"\n op = \"or\"\n lower_bounds {\n key = 4\n mouse = 4\n }\n }\n }]\ntrain {\n users {\n op = \"and\"\n lower_bounds {\n min_sessions = 24\n }\n }\n}","type":"dictionary"}]}]}}
ah, I thought you meant something clearml specific.
WittyOwl57
To get task Id's use (e.g. all the tasks of a specific project):task_ids = Task.query_tasks(project_name="examples", task_filter={'status': ["completed"])
Then per task:for t_id in tasks_id: t = Task.get_task(t_id) conf_dict = t.get_configuration_as_dict(name="filter") task_param = t.get_parameters() task_param['filter'] = conf_dict # this is to enable to forcefully update parameters post execution t.mark_started(force=True) # update hyper-parameters back t.set_parameters(task_param) # mark as completed back t.mark_completed(force=True)
wdyt?
EDIT: I would first test on a single Task 😉
This is how a configuration item looks like:<tasks.ConfigurationItem: { "name": "filter", "value": "inference = [{\n type = \"StreamFilter\"\n params {\n context = \"full\"\n op = \"or\"\n lower_bounds {\n key = 16\n mouse = 32\n }\n }\n }]\ntrain {\n users {\n op = \"and\"\n lower_bounds {\n min_sessions = 32\n }\n }\n}", "type": "dictionary" }>
The value
is a string that prints pretty but I'm not sure how to parse back. If it was a yaml string i would print as pretty or prettier, but I could also convert it back into a dict.
Hi SweetBadger76 , thanks, I think I've made it work. The main point of confusion was between dealing with different type of Task
objects (i.e. clearml.backend_api.services.v2_13.tasks.Task
returned by get_all
, which don't have any of those methods).
Interestingly, set_parameters
didn't just work as expected, I had to flatten the dicts myself (which clearml apparently does on its own when I call set_parameters
on a new task.
Thank you all. 🙏
I'm not sure how to access the developer tools 😳