Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Currently Experimenting With Clearml Pipelines In Offline Environments And I Am Trying To Establish A Best Practice For Changing And Maintaining Dependencies In The Pipeline Tasks. I Noticed That Clearml Automatically Pulls The Requirements.Txt

Hi,

I am currently experimenting with ClearML pipelines in offline environments and I am trying to establish a best practice for changing and maintaining dependencies in the pipeline tasks.

I noticed that clearML automatically pulls the requirements.txt file from my gitlab repository and installs the dependencies listed in the file. However, I noticed that when I update the requirements.txt file and push it to gitlab, clearML does not pull this updated requirements.txt file. I noticed that it seems to be using a cache called "vcs_cache" and thus disabled it, however it is still not using the updated requirements.txt

Could I get some assistance with how to configure the settings to always pull the update requirements.txt file? Or is there any other recommended best practice for updating and maintaining dependencies in clearML pipelines?

  
  
Posted 2 years ago
Votes Newest

Answers 6


Hi ShakyOstrich31 ,

Can you verify that you did push the updated code into your repository?

  
  
Posted 2 years ago

ShakyOstrich31

I am reusing an old task ...

Which means that the old Task stores the requirements on the Task itself (see "Installed Packages" section), Notice it also stores the exact git commit to use.
When you are cloning the Task (i.e. in the pipeline), you should probably:
set the commit / branch to the latest in the branch clear the "installed packages" section, which would cause the agent to use the "requirements.txt" stored in the git repo itself.As far as I understand this should solve the issue,
wdyt?

  
  
Posted 2 years ago

Hi, SuccessfulKoala55 I am reusing an old task (I did not run my python file to upload a new task draft). What I have been doing is re-running only the pipeline controller file to start a new pipeline.

Could you direct me on how to clear the saved commit ID and use the master branch on the web app please? I cant seem to find the option.

  
  
Posted 2 years ago

ShakyOstrich31 maybe the issue is the commit num? Are you running a new task after pushing the requirements file or is it an old task you're reusing? Once executed, ClearML will record the commit ID that was executed and use the same one - You can clear this and use the master branch from the WebApp

  
  
Posted 2 years ago

Hi CostlyOstrich36 ,

Yes I just double-checked and I did push the updated requirements.txt

  
  
Posted 2 years ago

Hi CostlyOstrich36 ,

  
  
Posted 2 years ago
959 Views
6 Answers
2 years ago
one year ago
Tags