Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone, I'M New To Clearml And I Want To Run A Task On A Remote Host Inside A Docker Container. The Python Interpreter I Need To Use Is Located At

Hi everyone,

I'm new to ClearML and I want to run a task on a remote host inside a Docker container.
The Python interpreter I need to use is located at /opt/conda/bin/python3 . Where do I specify this information in ClearML?

Also, if the Docker container doesn't exist on the remote host, can ClearML pull it from an artifactory?

Thanks in advance

  
  
Posted 11 months ago
Votes Newest

Answers 24


Hi @<1564060263047499776:profile|ThoughtfulCentipede62> , you can specify the Python interpreter in clearml.conf on the remote machine (Search for 'binary' or 'python')
Also, yes ClearML can pull it from an artifactory as long as the machine has access 🙂

  
  
Posted 11 months ago

thanks @<1523701070390366208:profile|CostlyOstrich36> for the fast response!
do I need to update the field python_binary?

  
  
Posted 11 months ago

I don't think you have to, but if your environment is ready you can try 🙂

  
  
Posted 11 months ago

I get the following error:
clearml_agent: ERROR: Python executable with version '/opt/conda/bin/python3' requested by the Task, key 'agent.default_python', not found in path, tried: ('/opt/conda/bin/python3', '/opt/conda/bin/python3')

  
  
Posted 11 months ago

How did you set it up? Are you sure that's the correct path?

  
  
Posted 11 months ago

the path is correct.

I updated clearml.conf on the remote host

Set the python version to use when creating the virtual environment and launching the experiment

# Example values: "/usr/bin/python3" or "/usr/local/bin/python3.6"
# The default is the python executing the clearml_agent
python_binary: "/opt/conda/bin/python3"
  
  
Posted 11 months ago

Just making sure - This path, is it inside the docker or outside the container?

  
  
Posted 11 months ago

@<1564060263047499776:profile|ThoughtfulCentipede62> how did you run the agent?

  
  
Posted 11 months ago

inside the docker only

  
  
Posted 11 months ago

I'm running a systemd service
ExecStart=/bin/bash -c 'source /home/clearml/.bashrc && clearml-agent daemon --queue gpu_3080 --foreground'

  
  
Posted 11 months ago

so the agent isn't running in docker mode and thus won't run the task inside a docker container

  
  
Posted 11 months ago

is the docker image is the docker that will be used for the task?

  
  
Posted 11 months ago

you need to add --docker <image-name>

  
  
Posted 11 months ago

actually, sometimes I will not use a docker for my tasks. is there a way to support both ways?

  
  
Posted 11 months ago

@<1564060263047499776:profile|ThoughtfulCentipede62> , you can run the agent twice on different queues - one with docker one without

  
  
Posted 11 months ago

on 2 different services?

  
  
Posted 11 months ago

2 different queues

  
  
Posted 11 months ago

You can run agent as many times as you want on your machine

  
  
Posted 11 months ago

the same service will run 2 agents?

  
  
Posted 11 months ago

What do you mean by service?

  
  
Posted 11 months ago

you set up 2 agent runs - one with docker and the other without. Each agent should be listening to a different queue. Makes sense?

  
  
Posted 11 months ago

i have a systemd service that runs the following
ExecStart=/bin/bash -c 'source /home/clearml/.bashrc && clearml-agent daemon --queue gpu_3080 --foreground'

do i need to create another systemd service for the agent using docker?

  
  
Posted 11 months ago

If you want to sometimes run with docker and sometimes without, yes.

  
  
Posted 11 months ago

Thanks a lot!! @<1523701070390366208:profile|CostlyOstrich36> @<1523701087100473344:profile|SuccessfulKoala55>

  
  
Posted 11 months ago
618 Views
24 Answers
11 months ago
11 months ago
Tags