Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Does Clearml Creates Separate Virtual Environments For Each Pipeline Steps When Running Remotely?

does clearml creates separate virtual environments for each pipeline steps when running remotely?

  
  
Posted 10 months ago
Votes Newest

Answers 9


so inside /Users/adityachaudhry/.clearml/venvs-builds.1/3.10/task_repository/ I have my git repo, I have one component that make a dataset directory inside this git repo, but when the other component starts executing this dataset directory is not there

  
  
Posted 10 months ago

how's that?

  
  
Posted 10 months ago

Hi @<1610083503607648256:profile|DiminutiveToad80>
Yes, it does. They are also cached by default (on the machine with the agent)
None

  
  
Posted 10 months ago

what do you mean? the same env for all components ? if they are using/importing exactly the same packages, and using the same container, then yes it could

  
  
Posted 10 months ago

is there a way that there is only one environment for the whole pipeline?

  
  
Posted 10 months ago

Yes, it recreates the venv (or fetches it from cache) if you need your dataset, use Dataset class (it will cache it persistently, so no need to re-download)

  
  
Posted 9 months ago

This means all the components of the pipeline use the exact same packages, and then it will just reuse the venv. Make sense ?

  
  
Posted 10 months ago

yes same env for all the components

  
  
Posted 10 months ago

so my model is not able to find the dataset

  
  
Posted 10 months ago
562 Views
9 Answers
10 months ago
9 months ago
Tags