Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Unanswered
Is The App/Ui/Backend Customizable? Any Tutorials For That?


Hi CleanWhale17 let me see if I can address them all

Email Alert for finished Job(I'm not sure if it's already there).

Slack integration will be public by the end of the weekend 🙂
It is fully customization / extendable, I'll be happy to help .

DVC

Full dataset tracking is supported using the artifacts and the ability to integrate to any central storage (shared folders/ S3 / GS / Azure etc.)
From my experience, it is easier to work with artifacts from Data-Processing Tasks, as Trains offers full caching and flexible Storage options, I always have the feeling "git-alike" commit/pull for dataset is the wrong approach, that said there is nothing that will limit you in integrating DVC into your pipeline.
If you are doing Computer-Vision based DL, which means annotation on json files, and pointers to actual files. Then it makes a lot of sense to have the annotation in a single json file as a Data-Processing Task (fully versioned of course), then from training Task pull the json (caching is supported), then from the json access the actual image files with direct file sharing or Using the Trains StorageManager, that does all the heavy lifting for you and can pull data from S3/Gs/Azure etc, with caching built in.

Apache AirFlow

If you have a K8s cluster and you want production grade orchestration, by all means consider AirFlow or KubeFlow. That said for R&D and constantly changing repositories/requirements, Trains offer the ability to reuse containers (so that you do not end up with a conainer per experiment, then 1000's of unused containers) and also the ability to build a fully standalone container from any experiment (i.e. package an experiment/Task in a container for later use with any orchestration solution)
Last thing, K8s is great for managing resources, not so much for scheduling.
You can use trains-agent as bare metal agent, to run containers on any machine (setup with pip install, it is that easy). Or you can integrate with K8s, there are a few example and documentation on the Nvidia NGC could (we are the leading supported platform for managing experiments on Nvidia K8s clusters)

  
  
Posted 4 years ago
118 Views
0 Answers
4 years ago
one year ago