Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Clearml Community, Is There A Way To Install Additional Packages On Top Of Base Docker Env When Using

Hi clearml community, is there a way to install additional packages on top of base docker env when using Task.force_store_standalone_script() and task.execute_remotely() ?

I noticed that task.set_packages() function call specifically mentioned that this is ignored when running remotely

  
  
Posted 4 months ago
Votes Newest

Answers 4


ignore the indentation, didn't align when I copy over

  
  
Posted 4 months ago

Any followup on this question? Recap:
Task,add_requirements() doesn't seem to do install the package from my experiment

Additionally, as alternative of add_requirements() if I can't get it working, is there an example of using docker bash init script you can point me to

  
  
Posted 4 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36> , I tried out Task.add_requirements way to add packages, but it doesn't seem to be working as I expected. here is the snippet i used to setup this up:

Task.force_store_standalone_script()
    add_packages = ["fastparquet"]
    for pkg in add_packages:
        Task.add_requirements(pkg)
    task = Task.init(project_name=project_name, task_name=task_name)
    task.set_base_docker(docker_arguments="--env CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=0")
    task.execute_remotely(queue_name=queue)

...
master_df.to_parquet(dataset_path) -> this is what I use to test whether fastparquet is installed properly or not

And I would get this complain from pandas:

File "/usr/local/lib/python3.8/dist-packages/pandas/io/parquet.py", line 60, in get_engine
    raise ImportError(
ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'.
  
  
Posted 4 months ago

Hi @<1797800418953138176:profile|ScrawnyCrocodile51> , you can use Task.add_requirements to add any packages. Additionally, you can also install packages with the docker bash init script

  
  
Posted 4 months ago
423 Views
4 Answers
4 months ago
4 months ago
Tags