Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Have A Worker Running In Services Mode With

Hi, I have a worker running in services mode with --services-mode and using a Docker image. I did this as I need to run a pipeline with this agent running in Docker mode - a pipeline apparently needs services mode in order to launch multiple tasks at once (ie. the pipeline job and then the components). As described here: None

However, the pipeline just hangs when it gets to launching the first step and doesn't go any further.

I assigned both the pipeline controller and the component to this worker. Do I rather need to create two agents, one in services mode for the controller and then another one (not in services mode) for the component (which does training and predictions)? But, this seems to defeat the point of being able to run multiple tasks in services mode...

Also, in the link above there is the warning Do not enqueue training or inference tasks into the services queue. They will put an unnecessary load on the server. I am not using the dedicated services queue on the server but I am doing training and inference in the pipeline component.

  
  
Posted 9 months ago
Votes Newest

Answers 5


Ah ok, so if I have two steps, this means 3 queues with 3 agents, one for the controller and one each dedicated to the steps, just to confirm?

  
  
Posted 9 months ago

I assigned both the pipeline controller and the component to this worker. Do I rather need to create two agents, one in services mode for the controller and then another one (not in services mode) for the component (which does training and predictions)? But, this seems to defeat the point of being able to run multiple tasks in services mode...

Yes. Again, the services mode is for special 'system' services if you will. The controller can run on the services agent (although not necessary since you can run the pipeline controller on any queue you'd like) while the steps themselves run on different queues

  
  
Posted 9 months ago

Thanks, I'll have a look at the youtube videos. I've been going over the documentation a lot and haven't found much about actually running and deploying pipelines, and a lot of details seem to be missing.

  
  
Posted 9 months ago

Also, in the link above there is the warning

Do not enqueue training or inference tasks into the services queue. They will put an unnecessary load on the server.

I am not using the dedicated

services

queue on the server but I am doing training and inference in the pipeline component.

Steps of a pipeline should have dedicated queues with relevant resources to them

  
  
Posted 9 months ago

Really depends on how you want to set up your pipeline. I suggest going over the documentation and watching the youtube videos for a better understanding.

  
  
Posted 9 months ago
673 Views
5 Answers
9 months ago
9 months ago
Tags
Similar posts