Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Everyone, I Have Another Question: Is It Possible To Change Agent Config For Each Task? E.G.

Hey everyone,
I have another question: Is it possible to change agent config for each task? E.g. agent.package_manager.system_site_packages = true should be task-wise imho, not agent-wise.

  
  
Posted one year ago
Votes Newest

Answers 17


ReassuredTiger98 anything in the configuration file can be overruled 🙂
https://clear.ml/docs/latest/docs/configs/configuring_clearml

  
  
Posted one year ago

So if understand correctly, something like this should work?

task = Task.init(...) task.connect_configuration( {"agent.package_manager.system_site_packages": False} ) task.execute_remotely(queue_name, clone=False, exit_process=True)

  
  
Posted one year ago

no this should work with this one. I'll double check if I'm remembering it correctly but I thought you should be able to start a task after loading your own configuration object, where can set the agent.package_manager.system_site_packages = true .

  
  
Posted one year ago

I am currently on the Open Source version, so no Vault. The environment variables are not meant to used on a per task basis right?

  
  
Posted one year ago

Is this not something completely different?
This will just change the way to local repository is analyzed, but nothing about the agent.

  
  
Posted one year ago

I mean if I do CLEARML_DOCKER_IMAGE=my_image clearml-task something something it will not work, right?

  
  
Posted one year ago

Hi ReassuredTiger98 !

I'm not sure the above will work. Maybe I can help in another way though: when you want to set agent.package_manager.system_site_packages = true does that mean you have a docker container with some of the correct packages installed? In case you use a docker container, there is little no real need to create a virtualenv anyway and you might use the env var CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1 to just install all packages in the root environment.

Because every task gets its own new clean docker container, there is no problem with using the root env. The nice thing is that in that way you get the system packages + any other ones that are installed by the Task.

Was that the outcome you meant? If so, please let me know when you tested it if it works for you 🙂 In the meantime we can think about your idea of making agent.package_manager.system_site_packages task-specific.

  
  
Posted one year ago

ReassuredTiger98 you can set different parameters per task:
https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#configuration

  
  
Posted one year ago

Hi KindChimpanzee37 I was more asking about the general idea to make these settings task-specific, but thank you for the suggestion anyways, I will definitely apply it.

  
  
Posted one year ago

Or you can just load a config file or object: https://clear.ml/docs/latest/docs/references/sdk/task/#connect_configuration

  
  
Posted one year ago

Hi TimelyMouse69
Thank you for answering, but I do not think these methods do allow me to modify anything the is set in clearml.conf. Rather they just do logging.

  
  
Posted one year ago

It should, or you might need to nest the objects.
Edit: I asked, it won't there's a difference in configs I mixed up.

  
  
Posted one year ago

Or you can give it a configuration object: https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#configuration-objects

  
  
Posted one year ago

Maybe this opens up another question, which is more about how clearml-agent is supposed to be used. The "pure" way would be to make the docker image provide everything and clearml-agent should do not setup at all.
What I currently do instead is letting the docker image provide all system dependencies and let clearml-agent setup all the python dependencies. This allows me to reuse a docker image for more different experiments. However, then it would make sense to have as many configs as possible task-specific or task-overridable.

  
  
Posted one year ago

Maybe this is something that is only possible with the vault of the enterprise version?

  
  
Posted one year ago

Okay, no worries. I will check first. Thanks for helping!

  
  
Posted one year ago
1K Views
17 Answers
one year ago
one year ago
Tags