Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Again. Is There Any Way To Have Trains-Agent Do A 'Docker Build' On The Dockerfile In The Repository It Pulls And Then Run That Image? I Know I Can Specify The Base Image Trains-Agent Runs The Task In And That Will Get Pulled/Run At Execution Time, But

Hi again. Is there any way to have trains-agent do a 'docker build' on the Dockerfile in the repository it pulls and then run that image? I know I can specify the base image trains-agent runs the task in and that will get pulled/run at execution time, but my project is already set up so that it runs in an environment defined by a Dockerfile, and I'd much rather just have trains-agent just automatically build the image defined there than have to build the image separately and make it available for all the agents to pull.

  
  
Posted 3 years ago
Votes Newest

Answers 13


I'm going to follow your suggestion and just put the extra effort into distributing a pre-built image.

That sounds good 🙂
If you feel the need is important, I do have a hack in mind, it will be doable once we have support for entrypoint "-c python_code_here". But since this is still not available I believe you are right and build an image would be the easiest.

A note on the docker image, remember that when running inside the docker we inherit the system packages installed on the docker, so if you change python packages there will not be any need to build a new image :)

  
  
Posted 3 years ago

trains-agent runs a container from that image, then clones ...

That is correct

I'd like the base_docker_image to not only be defined at runtime

I see, may I ask why not just build it once, push it into artifactory and then have trains-agent use it? (it will be much faster)

  
  
Posted 3 years ago

RobustGoldfish9 I see.
So in theory spinning an experiment on an gent would be clone code -> build docker -> mount code -> execute code inside docker?
(no need for requirements etc.?)

  
  
Posted 3 years ago

I see, would having this feature solve it (i.e. base docker + bash init script)?
https://github.com/allegroai/trains/issues/236

  
  
Posted 3 years ago

Hi RobustGoldfish9 ,

I'd much rather just have trains-agent just automatically build the image defined there than have to build the image separately and make it available for all the agents to pull.

Do you mean there is no docker image in the artifactory built based on your Dockerfile ?

  
  
Posted 3 years ago

I'll keep an eye out for the new entrypoint. Thanks again for the support.

  
  
Posted 3 years ago

It's very close. The only difference is that my team does have access to decide MLOps, and we've already structured our project around docker and our environment is already defined in a Dockerfile and a requirements.txt which is acted upon inside that Dockefile. We don't really need trains to completely manage our dependencies.

  
  
Posted 3 years ago

I had thought of that as a solution for when our code stabilized, but during development I'd rather not have to build/maintain an image and keep updating it as code/environment diverged from it. It would be nicer if everything just got built from the Dockerfile checked out from git.

  
  
Posted 3 years ago

Yeah. We still have a requirements.txt file, but we do pip -r inside the Dockerfile along with all our calls to apt.

  
  
Posted 3 years ago

Prior to trains our workflow was to build an image and spin up a container on one of our GPU machines. I can do the same thing and still take advantage of trains' excellent reporting, but then I lose out on the queues and ability to clone experiments from the webui.

  
  
Posted 3 years ago

I might be a little confused. I'm assuming that when I set a base docker image for a task and run it, trains-agent runs a container from that image, then clones the git repository into that image, then applies all the changes/packages trains detected and runs the script. But what I'd really like to do is for trains-agent to pull a repository that includes a Dockerfile, build that dockerfile, run the resulting container, and then execute the script within it.

  
  
Posted 3 years ago

I think for the time being, I'm going to follow your suggestion and just put the extra effort into distributing a pre-built image.

  
  
Posted 3 years ago

I'd like the base_docker_image to not only be defined at runtime, but also be built at runtime.

  
  
Posted 3 years ago
724 Views
13 Answers
3 years ago
one year ago
Tags
Similar posts