Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Is It Possible To Run An Agent, Listen To The Services Queue Without Using Docker?

Is it possible to run an agent, listen to the services queue without using docker?

  
  
Posted 3 years ago
Votes Newest

Answers 20


Sure, venv mode

  
  
Posted 3 years ago

Does what you suggested here > https://github.com/allegroai/trains-agent/issues/18#issuecomment-634551232 also applies for containers used by the services queue?

  
  
Posted 3 years ago

It's the safest way to run multiple processes and make sure they are cleaned afterwards ...

  
  
Posted 3 years ago

Isn't it overkill to run a whole ubuntu 18.04 just to run a dead simple controller task?

  
  
Posted 3 years ago

Alright, thanks for the answer! Seems legit then 🙂

  
  
Posted 3 years ago

JitteryCoyote63 What did you have in mind?

  
  
Posted 3 years ago

That gave me

Running in Docker mode (v19.03 and above) - using default docker image: nvidia/cuda running python3

Building Task 94jfk2479851047c18f1fa60c1364b871 inside docker: ubuntu:18.04

Starting docker build
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
ERRO[0000] error waiting for container: context canceled

  
  
Posted 3 years ago

Yes that would work 🙂
You can also put it in the docker compose see TRAINS_AGENT_DEFAULT_BASE_DOCKER

  
  
Posted 3 years ago

ubuntu18.04 is actually 64Mo, I can live with that 😛

  
  
Posted 3 years ago

My bad, alpine is so light it doesnt have bash

  
  
Posted 3 years ago

🤞

  
  
Posted 3 years ago

sh?

  
  
Posted 3 years ago

Thanks! Corrected both, now its building

  
  
Posted 3 years ago

trains-agent build --docker nvidia/cuda --id myTaskId --target base_env_services
It's building a gpu enabled docker...
you might want a diff container or to specific --cpu-only

  
  
Posted 3 years ago

It will automatically switch to docker mode

  
  
Posted 3 years ago

In theory yes, in practice you will be using the same docker image for all the services, and they will never interfere with one another. and you have the option to do more sophisticated stuff, like map the file-server data for a clean up service (should be out in a few days :)) so a balance. Also remember that relatively speaking docker are quite light weight, this is not like saying a VM per service...

  
  
Posted 3 years ago

Does what you suggested here >

Yes, it is basically the same underlying mechanism, only instead of 1-to-1 it's 1-to-many

  
  
Posted 3 years ago

ohh sorry, no

  
  
Posted 3 years ago

btw, I tried with alpine instead of ubuntu:18.04, got :

Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
df20fa9351a1: Pulling fs layer
df20fa9351a1: Verifying Checksum
df20fa9351a1: Download complete
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: "bash": executable file not found in $PATH": unknown.
time="2020-06-09T13:24:55Z" level=error msg="error waiting for container: context canceled"
DONE: Running task 'ae788658786043ef8d3c758815a43eb8', exit status 127

Any idea?

  
  
Posted 3 years ago

Alright, so the steps would be:

trains-agent build --docker nvidia/cuda --id myTaskId --target base_env_services

That would create me a base docker image base_env_services . Then how should I ensure that trains-agent uses that base image for the services queue? My guess is:

trains-agent daemon --services-mode --detached --queue services --create-queue --docker base_env_services --cpu-only

Would that work?

  
  
Posted 3 years ago