Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, We Run A Pipeline Using The Pipelinecontroller, When We Do It From

Hey, we run a pipeline using the PipelineController, when we do it from team member 1 computer, we see clearml-agent installing lots of unrelevant packages, and eventually fail. when team member 2 runs it, other packages are being ran, and the task run smoothly
what can cause this problem?

  
  
Posted one year ago
Votes Newest

Answers 32


thank you! i'll try it

  
  
Posted one year ago

Basically, it looks like the agent installs an outdated pip version and this should fix it, and hopefully install your packages correctly

  
  
Posted one year ago

Hi @<1544853695869489152:profile|NonchalantOx99> ! In your clearml.conf , try to set, at the end of the whole file, outside any curly brackets, agent.package_manager.pip_version: "23.1.2"

  
  
Posted one year ago

OK, we'll take a look

  
  
Posted one year ago

basically, only test.py need the packages, but for somereason pipeline_test installs them as well

  
  
Posted one year ago

i reproduced it here:

  
  
Posted one year ago

Well, can you share some of the code (or at least the imports it has)? The SDK created the installed packages list to make sure the task will not fail when being run by the agent

  
  
Posted one year ago

only clearml probably

  
  
Posted one year ago

why those library need to run on a pipelinecontroller task, this task requires no libraries at all

  
  
Posted one year ago

but why does it matter if i ran it on a remote agent?

  
  
Posted one year ago

Well, what are the local libraries? obviously you're using some of them, so they need to be there

  
  
Posted one year ago

is it possible to tell him not the install my local libraries all at once? instead of manually saying ignore_requirements?

  
  
Posted one year ago

You can use Task.ignore_requirements("pywin32") in your code - that should solve it

  
  
Posted one year ago

and then it works, doesn't try to install any other packages

  
  
Posted one year ago

then it works
i opened a new, clean venv just now

  
  
Posted one year ago

what happens if you uninstall it and than try to run the code?

  
  
Posted one year ago

it is installed as a pip package
but i am not using it in the code

  
  
Posted one year ago

Are you sure you're not using pywin32? The ClearML SDK does static code analysis and should only store the packages you reference directly

  
  
Posted one year ago

yeah

  
  
Posted one year ago

just a sec

  
  
Posted one year ago

Can you please re-post the conf file (without the credentials 🙂 )

  
  
Posted one year ago

not using it in the code

  
  
Posted one year ago

So the question is why did pywin32 == 305 got into the installed packages - are you using that in your code?

  
  
Posted one year ago

Also, It would help to see the clearml.conf file from the local environment that started running this task

  
  
Posted one year ago

And what are the installed packages showing in the task UI panel?

  
  
Posted one year ago

from which we run the task

  
  
Posted one year ago

for some reason the agent tries to install the locally installed pip packages

  
  
Posted one year ago

I'd appreciate that as there's more info there that might help understand this.

  
  
Posted one year ago

do you want the entire log files? (it is a pipeline, and i can't seem to find the "Task" itself, to download the logs)

  
  
Posted one year ago

the successful, which is aborted for some reason (but at least the enviorment is setup correctly)
the end of it is :

  • urllib3==1.26.15
  • virtualenv==20.23.0
  • wcwidth==0.2.6
  • Werkzeug==2.3.2
  • widgetsnbextension==4.0.7
  • xgboost==1.7.5
  • yarl==1.9.2
    Environment setup completed successfully
    Starting Task Execution:
    2023-04-29 21:41:02
    Process terminated by user
  
  
Posted one year ago