Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
There Is Some Specificity With The Way We Setup Our Environment At My Company That Prevents Me From Using The Full Features Of

There is some specificity with the way we setup our environment at my company that prevents me from using the full features of clearml-agent and forces me to override env setup in a not so flexible way and use standalone mode but I’m hoping it might make sense to “support” this.

Basically, when the agent clones the repo and switches into the dev, I want it to run a script that is in the repo that:
Installs some pip dependencies (I kind of want these regardless of any extra deps specific to the task). Generate some py files/symlinks. This is needed when you use custom C extensions / ANTLR / Protobuf so I’m guessing if other people have this in their repo they’ll want something similar.
I was hoping to do this through the docker setup script but it is quite hacky and inflexible because by then the repo has not been cloned yet. I am hoping this can be handled by the clearml-agent such that a simple clearml-agent execute --id would work.

What do you think?

  
  
Posted 2 years ago
Votes Newest

Answers 9


Hi LazyTurkey38 ,

Basically, when the agent clones the repo and switches into the dev, I want it to run a script that is in the repo that:
Installs some pip dependencies (I kind of want these regardless of any extra deps specific to the task).

what about adding all your packages to the task? can this help? you can do it with Task.add_requirements(package_name, version)

Generate some 

py

 files/symlinks. This is needed when you use custom C extensions / ANTLR / Protobuf so I’m guessing if other people have this in their repo they’ll want something similar.

How do you generate those?

  
  
Posted 2 years ago

So when the repo is cloned and venv is created and activated I want to executed this from the repo: tools/setup_dependencies.sh

  
  
Posted 2 years ago

So just after the clone, before creating the env?

  
  
Posted 2 years ago

That won’t work 😕
The docker shell script runs too early in the process.
I want to inject a bash command after the repo has been clone (and maybe even after the venv has been installed).

  
  
Posted 2 years ago

If you want to force install some packages with the agent you can add those here -
https://github.com/allegroai/clearml-agent/blob/21c4857795e6392a848b296ceb5480aca5f98e4b/docs/clearml.conf#L74

  
  
Posted 2 years ago

I want to inject a bash command after the repo has been clone (and maybe even after the venv has been installed).

LazyTurkey38 the created venv inherits from the system environment, so in theory you can do all the installation on the system python and the created venv will just inherit the packages, no?
(btw: just to clarify, there is only one entry point for the custom bash script and that is before everything, so users can configure the container before the agent starts)

  
  
Posted 2 years ago

TimelyPenguin76 After creating the venv (so I don’t have to do it myself). Once an env is there, I need to run a script while the env is activated from the root of the repo.

  
  
Posted 2 years ago

For your second question, those are generated using custom tooling, it relies on the build system to be setup which is guaranteed by the docker image used. So I don’t think this is a case of supporting a specific env setup or build tool but just allowing custom script for env setup step / building code.
WDYT?

  
  
Posted 2 years ago