Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All! I Have A Question Regarding Clearml-Agents That I Have Not Been Able To Find In The Documentation. I Have Seen An Agent Is Also Called A 'Worker'. When An Agent Is Spinned Up, A New Process Is Spawned? Or Can It Also Be A Thread? What Limits The M

Hi all!
I have a question regarding clearml-agents that I have not been able to find in the documentation. I have seen an agent is also called a 'worker'. When an agent is spinned up, a new process is spawned? Or can it also be a thread? What limits the maximum number of agents that can be started on a single machine? It is the number of CPU cores the machine has?

Posted one year ago
Votes Newest

Answers 7

GiganticTurtle0 Hi 🙂

You can use worker/agent interchangeably. When running an agent a new process is started. I think the only limitation on number of works running is the amount of the resources your system has to spare.

Regarding if it can be a thread I'll give you information soon.

Posted one year ago

Ok! I'll try to spin up an agent with the --service-mode command and I will give you feedback

Posted one year ago

Hi CostlyOstrich36 AgitatedDove14
Oh no, I am not trying to say that I am using each agent to run a single task. I have several agents listening to a number of queues so that they are busy most of the time. As we talked about, it is not possible to run multiple pipelines ( PipelineDecorator.pipeline ) simultaneously in a single process. That's why I had been testing locally launching pipelines in different subprocesses and this way I have managed to run several pipelines concurrently. I was just wondering if instead of using local subprocesses, several agents could serve the same purpose (running several pipelines concurrently)

Posted one year ago

GiganticTurtle0 I'm not sure why a direct correlation between the number of agents and cores is an issue, but the issue is resources - each agent will run a task in a separate process (or docker container) - do you really want each one of your tasks to run on a single core shared with the agent process? Wouldn't you get better performance if each task has more cores to run with? What about GPUs?

Posted one year ago

GiganticTurtle0 notice that when you spin an agent with --services-mode, you basically let it run many Tasks at once (this is in contrast to the default behavior, when you have one Task per agent).

Posted one year ago

. I was just wondering if instead of using local subprocesses, several agents could serve the same purpose (running several pipelines concurrently)

wouldn't --service-mode (read as multiple simultaneous Tasks on the same agent) solve the issue?
(BTW: if you set the pipeline component target queue to "services" , this is exactly what will happen)

Posted one year ago

So, for example, if I have a machine with 64 CPU cores, I will be able to run up to 64 agents (in case my system consists of only this machine), right?

Posted one year ago
7 Answers
one year ago
8 months ago
Similar posts