Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Community, I Want To Add 2 Args For Clearml Agent Dockers " ["--Network=Host", "--Ipc=Host"]". I'Ve Tried With Changing Agent.Default_Docker.Arguments => But It Didn'T Work. Later When I Tried With Agent.Extra_Docker_Arguments => It Worked. So I Was Wo

Hi community, I want to add 2 args for clearml agent dockers " ["--network=host", "--ipc=host"]". I've tried with changing agent.default_docker.arguments => but it didn't work. Later when I tried with agent.extra_docker_arguments => it worked. So I was wondering what the different between those 2 options. Is agent.default_docker.arguments only applied for default docker image only, not other images? Thanks

  
  
Posted one month ago
Votes Newest

Answers 3


Here is the clearml.conf

    default_docker: {
        # default docker image to use when running in docker mode
        image: "python3.10-cuda12.2:latest"

        # optional arguments to pass to docker image
        arguments: ["--network=host", "--ipc=host" ]
    }

And here is the code that user used to run task (with python3.8 image)


from clearml import Task
task = Task.init(project_name='my_project', task_name='my_remote_task')

task.set_base_docker('python3.10-cuda12.2:latest')  
task.execute_remotely(queue_name=Default)

When task run, this is the docker run command

Executing: ['docker', 'run', '-t', '--gpus', 'all', '-l', 'clearml-worker-id=ubuntu-s-3090ti:0', '-l', 'clearml-parent-worker-id=ubuntu-s-3090ti:0', '-e', 'CLEARML_WORKER_ID=ubuntu-s-3090ti:0', '-e', 'CLEARML_DOCKER_IMAGE=registry.torus.ai/taureau/python3.8-cuda12.2:latest', '-e', 'CLEARML_TASK_ID=49d14d9ff5f944adad588ab5f05ecebe', '-v', '/root/.gitconfig:/root/.gitconfig', '-v', '/tmp/.clearml_agent.fk2xon5d.cfg:/tmp/clearml.conf', '-e', 'CLEARML_CONFIG_FILE=/tmp/clearml.conf', '-v', '/tmp/clearml_agent.ssh.j2xj7jb_:/.ssh', '-v', '/root/.clearml/apt-cache:/var/cache/apt/archives', '-v', '/root/.clearml/pip-cache:/root/.cache/pip', '-v', '/root/.clearml/pip-download-cache:/root/.clearml/pip-download-cache', '-v', '/root/.clearml/cache:/clearml_agent_cache', '-v', '/root/.clearml/vcs-cache:/root/.clearml/vcs-cache', '-v', '/root/.clearml/venvs-cache:/root/.clearml/venvs-cache', '--rm', 'python3.8-cuda12.2:latest', 'bash', '-c', 'echo \'Binary::apt::APT::Keep-Downloaded-Packages "true";\' > /etc/apt/apt.conf.d/docker-clean ; chown -R root /root/.cache/pip ; export DEBIAN_FRONTEND=noninteractive ; export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL libsm6 libxext6 libxrender-dev libglib2.0-0" ; cp -Rf /.ssh -T ~/.ssh ; [ ! -z $(which git) ] || export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL git" ; declare LOCAL_PYTHON ; [ ! -z $LOCAL_PYTHON ] || for i in {15..5}; do which python3.$i && python3.$i -m pip --version && export LOCAL_PYTHON=$(which python3.$i) && break ; done ; [ ! -z $LOCAL_PYTHON ] || export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL python3-pip" ; [ -z "$CLEARML_APT_INSTALL" ] || (apt-get update -y ; apt-get install -y $CLEARML_APT_INSTALL) ; [ ! -z $LOCAL_PYTHON ] || export LOCAL_PYTHON=python3 ; $LOCAL_PYTHON -m pip install -U "pip<20.2 ; python_version < \'3.10\'" "pip<22.3 ; python_version >= \'3.10\'" ; $LOCAL_PYTHON -m pip install -U clearml-agent ; cp /tmp/clearml.conf ~/default_clearml.conf ; NVIDIA_VISIBLE_DEVICES=all $LOCAL_PYTHON -u -m clearml_agent execute --disable-monitoring  --id 49d14d9ff5f944adad588ab5f05ecebe']
  
  
Posted one month ago

Hi @<1749965229388730368:profile|UnevenDeer21> , can you add the log of the job that failed?

Also, note that you can set these arguments from the webUI on the task level itself as well, Execution tab and then container section

  
  
Posted one month ago

@<1523701070390366208:profile|CostlyOstrich36> Thanks, I know about editing from webUI but for some arg like network and ipc I want to set to default for clearml-agent so our enduser don't need to worry about it. When changing agent.default_docker.arguments in clearml-agent to

["--network=host", "--ipc=host"] 

Then when our user init a tasks with a custom image (different than agent.default_docker.image) => I check the console log and see that in the docker run command, there is no network and ipc option.

  
  
Posted one month ago
101 Views
3 Answers
one month ago
one month ago
Tags