Thanks, I appreciate the answer!
So not the latter. I can always log metrics during training and visualize them.
I'm thinking of a few plots in my current in-house tooling which are slightly different than the standard charts we look at. For example a custom parallel coordinate chart that can use aggregations, categorical variables, etc.
To move over to trains, I'd like to have all these custom plots in my dashboard. I haven't tried to do them in trains yet (I'm just starting).
So my question in general is pertaining to if I would need to get better at Javascript if I were to make those changes. My guess is yes as those changes seem to be at the web UI level. Does that make sense?
I agree it would be better to have it fully configurable. But if every marginal feature adds complexity, we might have to think how applicable that is to the general use cases. I'm thinking of examples in my domain which might not be useful in other domains. Maybe if that becomes an issue, there could be a domain specific feature base?!
I haven't fully compared all the things that I am currently doing with the in-house tool and what we can do with trains. I think I will have more concrete ideas/ discussion points once I start implementing them in trains. At the moment, my front-end is in python, so I essentially just the add the features. So I don't have other ideas right now. But let's see what we can come up with once I dig in.
I really appreciate the responsiveness of your team. I'm really looking forward to starting working on this and contributing :)
Hi SourSwallow36
- The same docker image is used for all three jobs, just because it is easier to manage and faster to download. The full code is available on the trains-server GitHub. If you want to spin the containers manually, check the docker-compose.yml on the main repo, it has all the commands there
- Fork the trains-server, commit the changes and don't forget to PR them ;)
- Elastic search is a database, we use it to log all the experiments outputs, console logs metrics etc. This is in addition to mongodb that is used to log the experiments environment, metadata etc.
I'm thinking of a few plots in my current in-house tooling which are slightly different than the standard charts we look at. For example a custom parallel coordinate chart that can use aggregations, categorical variables, etc.
This can be done by comparing experiments, then check the Hyper-Parameters tab, and select graph from the drop down at the top
So my question in general is pertaining to if I would need to get better at Javascript if I were to make those changes. My guess is yes as those changes seem to be at the web UI level. Does that make sense?
Long story short, yes 🙂 the web UI is obviously javascript, and to add features to it you need to code JS.
Do you have other ideas? Because I think it will be very cool to allow user to fully configure the dashbaord, I'm just not sure how that can be achieved without actually add features into the web app itself.
Great to hear SourSwallow36 , contributions are always appreciated 🙂
Regrading (3), MongoDB was not build for large scale logging, elastic-search on the other hand was build and designed to log millions of reports and give you the possibility to search over them. For this reason we use each DB for what it was designed for, MongoDB to store the experiment documents (a.k.a env, meta-data etc.) and elastic-search to log the execution outputs.
Also, I would like to add some other plots to the dashboard. I
Do you mean like a dashboard to web UI home screen?
Or is is part of the training process that you want to visualize?
Hi AgitatedDove14
Thanks for the quick response.
I will try that out. Great! This is a great tool and I will start contributing. Why use both and not just one of them? What does one offer that the other doesn't?
Also, I would like to add some other plots to the dashboard. I see the plotting is done using Plotly Javascript. I'm a Python developer and don't know much Javascript. Do you have any suggestions on how to go about that or I should just get going with Javascript?