Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey, Is There A Way To Limit The Number Of Tasks Run At The Same Time By An Agent In Service Mode?

Hey, is there a way to limit the number of tasks run at the same time by an agent in service mode?

  
  
Posted 2 years ago
Votes Newest

Answers 11


Ok, if I would like to have a different behaviour I would need one agent per task, right?

  
  
Posted 2 years ago

I'm running the following agent:
clearml-agent --config-file /clearml-cache/config/clearml-cpu.conf daemon --queue cpu default services --docker ubuntu:20.04 --cpu-only --services-mode 4 --detached
The goal is to have an agent that can run multiple cpu only tasks at the same time. I notices that when enqueueing multiple tasks, all except for one stay pending until the first one finished downloading all packages and started with code execution. And then task by task switch to "running" when the previous task is done with "execution preparation". Is there any way to not have this waiting time?

  
  
Posted 2 years ago

Well, when running in services mode, the agent starts running a task, and when the task actually starts running, goes to run the next one, so the behaviour your seeing is correct

  
  
Posted 2 years ago

Hi ClumsyElephant70 ,
Indeed there is - just pass a number immediately after the --services-mode command-line option (e.g. --services-mode 5 )

  
  
Posted 2 years ago

Thanks

  
  
Posted 2 years ago

you can basically take one of the standard python+bullseye images, add some stuff to it, and use it as the default image

  
  
Posted 2 years ago

there's also a tiny python+bullseye image, I think

  
  
Posted 2 years ago

I see... Well, another way to greatly improve the execution time is to use smaller docker images with preinstalled requirements (mostly system stuff such as Python etc. and major python packages) - most of the time is spent on these installations

  
  
Posted 2 years ago

We run a lot of pipelines that are cpu only with some parallel steps. Its just about improving the execution time

  
  
Posted 2 years ago

What's the use-case?

  
  
Posted 2 years ago

Yeah, depending on what you need...

  
  
Posted 2 years ago