Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi There! I'M Trying To Understand How The

Hi there! I'm trying to understand how the trains-server works. So I have a few questions.

  1. For a linux deployment, we need to use docker-compose which runs allegroai/trains:latest docker image. To understand what the image is doing, I ran
    docker run -it allegroai/trains:latest /bin/bash The server type was not stated. It should be either apiserver, webserver or fileserver.How are these services setup? Can I run the image to understand how these services are launched?

  2. If I have to make any changes to server code, how would I create these images?

  3. I'm not familiar with elasticsearch . What is that used for? Is it just to query mongodb faster? How can I learn more about the connection between the two in trains-server ?

  
  
Posted 4 years ago
Votes Newest

Answers 7


Great to hear SourSwallow36 , contributions are always appreciated 🙂
Regrading (3), MongoDB was not build for large scale logging, elastic-search on the other hand was build and designed to log millions of reports and give you the possibility to search over them. For this reason we use each DB for what it was designed for, MongoDB to store the experiment documents (a.k.a env, meta-data etc.) and elastic-search to log the execution outputs.

Also, I would like to add some other plots to the dashboard. I

Do you mean like a dashboard to web UI home screen?
Or is is part of the training process that you want to visualize?

  
  
Posted 4 years ago

I agree it would be better to have it fully configurable. But if every marginal feature adds complexity, we might have to think how applicable that is to the general use cases. I'm thinking of examples in my domain which might not be useful in other domains. Maybe if that becomes an issue, there could be a domain specific feature base?!

I haven't fully compared all the things that I am currently doing with the in-house tool and what we can do with trains. I think I will have more concrete ideas/ discussion points once I start implementing them in trains. At the moment, my front-end is in python, so I essentially just the add the features. So I don't have other ideas right now. But let's see what we can come up with once I dig in.

I really appreciate the responsiveness of your team. I'm really looking forward to starting working on this and contributing :)

  
  
Posted 4 years ago

I'm thinking of a few plots in my current in-house tooling which are slightly different than the standard charts we look at. For example a custom parallel coordinate chart that can use aggregations, categorical variables, etc.

This can be done by comparing experiments, then check the Hyper-Parameters tab, and select graph from the drop down at the top

So my question in general is pertaining to if I would need to get better at Javascript if I were to make those changes. My guess is yes as those changes seem to be at the web UI level. Does that make sense?

Long story short, yes 🙂 the web UI is obviously javascript, and to add features to it you need to code JS.
Do you have other ideas? Because I think it will be very cool to allow user to fully configure the dashbaord, I'm just not sure how that can be achieved without actually add features into the web app itself.

  
  
Posted 4 years ago

Well put 🙂

  
  
Posted 4 years ago

Thanks, I appreciate the answer!

So not the latter. I can always log metrics during training and visualize them.

I'm thinking of a few plots in my current in-house tooling which are slightly different than the standard charts we look at. For example a custom parallel coordinate chart that can use aggregations, categorical variables, etc.
To move over to trains, I'd like to have all these custom plots in my dashboard. I haven't tried to do them in trains yet (I'm just starting).

So my question in general is pertaining to if I would need to get better at Javascript if I were to make those changes. My guess is yes as those changes seem to be at the web UI level. Does that make sense?

  
  
Posted 4 years ago

Hi  AgitatedDove14
Thanks for the quick response.
I will try that out. Great! This is a great tool and I will start contributing. Why use both and not just one of them? What does one offer that the other doesn't?
Also, I would like to add some other plots to the dashboard. I see the plotting is done using Plotly Javascript. I'm a Python developer and don't know much Javascript. Do you have any suggestions on how to go about that or I should just get going with Javascript?

  
  
Posted 4 years ago

Hi SourSwallow36

  1. The same docker image is used for all three jobs, just because it is easier to manage and faster to download. The full code is available on the trains-server GitHub. If you want to spin the containers manually, check the docker-compose.yml on the main repo, it has all the commands there
  2. Fork the trains-server, commit the changes and don't forget to PR them ;)
  3. Elastic search is a database, we use it to log all the experiments outputs, console logs metrics etc. This is in addition to mongodb that is used to log the experiments environment, metadata etc.
  
  
Posted 4 years ago
982 Views
7 Answers
4 years ago
one year ago
Tags