Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Team,When Clearml-Agent Is Used To Run The Code,I T Will Setup The Environment ,How It Take The Python Package Version?

Hi Team,when clearml-agent is used to run the code,i t will setup the environment ,how it take the python package version?

  
  
Posted one year ago
Votes Newest

Answers 15


Can you provide an example?

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> Waiting for your response.

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> i want agent should take packages which is there in my system.how can i do that?

  
  
Posted one year ago

Hi @<1533257278776414208:profile|SuperiorCockroach75> , the clearml experiment manager will try to detect your package requirements from its original environment. Meaning that if you run the code and it imports e.g. SQLAlchemy, then it will log the exact version of SQLAlchemy you have installed locally.

When you run only get_data,py locally and have the experiment manager track it, can you then look at the task that is made in the clearml webUI and check the installed packages section? That will be the packages that will be used when executing the task remotely later.

It should be exactly the same version as your local package, which you can check using pip freeze | grep SQL . In this case it could mean your local installed version is 2 and you should downgrade locally, so clearml will pick up the correct version.

If you're using a requirements.txt file like above, be sure to set the version as fixed! e.g.SQLAlchemy==x.x.xinstead of justSQLAlchemy` . In the latter case, python will install the latest available version of the package, which may lead to version 2.

Also, please be aware that by default the agent will use a specific python version (depending on which dockerfile it's using to run tasks in). This could be a different version than your local one and lead to mismatches in package versions.

  
  
Posted one year ago

@<1533257278776414208:profile|SuperiorCockroach75> did you add specific packages with constraints? Can you share the code?

  
  
Posted one year ago

@<1523701435869433856:profile|SmugDolphin23> ,i have added pacakges in pipeline componenet,but it is not taking that version.
Note: i am running this code using agent.

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> I have shared above.

  
  
Posted one year ago

Hi @<1533257278776414208:profile|SuperiorCockroach75> Try setting packages in your pipline component to your requirements.txt or simply add the list of packages (with the specific versions). None

  
  
Posted one year ago

image
image

  
  
Posted one year ago

@<1523701087100473344:profile|SuccessfulKoala55> I have copied code from None , there is one file pipeline.py in which we have two function pipe.start_locally(run_pipeline_steps_locally=True) and pipe.start(). when i am running code with pipe.start_locally its working fine with package SQLAlchemy version of 1.4.39.when i am trying to run with pipe.strat(), along with running agent it creating virtula environment with SQLAIchemy of version 2. i want 1.4.39 how can i setup?see the attach snap i am getting error for this version

  
  
Posted one year ago

Hi SuperiorCockroach75 , I assume you mean having a clearml-agent running your task remotely. In that case, the task (after being run once locally) already has the python packages requirements embedded into it, so the agent can use these in order to find out the correct packages to install

  
  
Posted one year ago

Hi @<1523701087100473344:profile|SuccessfulKoala55> ,No its taking different version of pacakges if u compare with local execution.

  
  
Posted one year ago

Hi @<1533257278776414208:profile|SuperiorCockroach75>

I must say I don't really know where this comes from. As far as I understand the agent should install the packages exactly as they are saved on the task itself. Can you go to the original experiment of the pipeline step in question (You can do this by selecting the step and clicking on Full Details" in the info panel), there under the execution tab you should see which version the task detected.

The task itself will try to autodetect the packages, but you can override these if needed using the packages parameter like @<1523701435869433856:profile|SmugDolphin23> has said. Can you please send me a screenshot with the correct SQLAlchemy version set using this method? Thanks!

  
  
Posted one year ago

@<1523701118159294464:profile|ExasperatedCrab78> ,in my system i have SQLAlchemy version 1.4.39,i have run locally and it is is running fine,when i am trying to run through agent it taking sqlalchemy version 2.0.0 which is not there in my system.and i have provided in requirement.txt SQLAlchemy==1.4.39.

  
  
Posted one year ago

@<1523701118159294464:profile|ExasperatedCrab78> can you also give brief descriptionhow the agent reuse environment?

  
  
Posted one year ago