JitteryCoyote63 Not currently there, but certainly sounds like something to add to the list - Care to https://github.com/allegroai/clearml/issues/new/choose ?
Thanks for clarifying @<1523705301990117376:profile|WickedCat12> .
As I mentioned originally, plotting an arbitrary metric against another is further down the ClearML roadmap.
It'd be great if you use a github issue to help push it through :)
UnevenDolphin73 Am I missing anything in rephrasing your use case to "Have a single autoscaler service multiple queues" (where the autoscaler resource configuration is, in essence, the pool you mention)?
The easy way to do that is to add the desired metrics/params as custom columns, then use the column filters: https://clear.ml/docs/latest/docs/webapp/webapp_exp_table#customizing-the-experiments-table
DepressedChimpanzee34 Always appreciated
HappyDove3 you can get some more insight on the different configuration methods and how to use theme https://clear.ml/docs/latest/docs/fundamentals/hyperparameters
Take a look at https://clear.ml/docs/latest/docs/pipelines/pipelines_sdk_tasks#running-the-pipeline ;
By default pipelines are enqueued for execution by a ClearML Agent. You can explicitly change this behaviour in your code.
@<1523705301990117376:profile|WickedCat12> ClearML Scalars explicitly show metrics time progression (you can display iteration/wall-time).
Plotting one metric against another is a feature that lies further down ClearML's roadmap.
If your metric is reported only once per epoch you can make use of the existing scalars functionality by making use of the iteration parameter when reporting your metric to reflect the epoch instead.
Does this make sense?
DepressedChimpanzee34 Have you noticed the "Show n experiments selected" button on the bottom bar? This effectively toggles your view between whatever is currently sorted/filtered and the current item selection.
To address the scenario you describe: Switch to "Show selected experiments", remove the redundant items, and switch back to the original view: "Show all experiments"
Thoughts?
@<1559349204206227456:profile|BeefyStarfish55> try checking out the general overview on pipelines here , and info on the pipelines UI here .
Each step's arguments (and results) should appear in the steps details panel (which you could then follow to the underlying task for complete, in-depth, details).
@<1523709410411548672:profile|NuttyFox2> Since the default server user configuration does not require authentication, I'm assuming your use case calls for some users being authenticated where others are not?
Such mixed access mode is currently not on the near term roadmap for the OSS server - You should create a feature request to help push it into the development plan.
@<1523701157564780544:profile|TenseOstrich47> Seems like the ClearML website is temporarily down 😞 . Should be resolved soon though.
@<1628927672681762816:profile|GreasyKitten62> When you have specific display considerations, you can implement them through report_table's 'extra_layout' and 'extra_data' parameters
Thanks letting us know @<1784392065820397568:profile|SplendidFox3> - The signup for app.clear.ml had indeed broken down, but we should be back on track - Can you now complete the registration?
@<1523701157564780544:profile|TenseOstrich47> This is typically indicative of insufficient server disk space causing ES to go into read-only mode or turn active shards into inactive or unassigned (see FAQ ).
The disk watermarks controlling the ES free-disk constraints are defined by default as % of the disk space (so it might look to you like you still have plenty of space, but ES thinks otherwise). You can configure di...
Hi JuicyOtter4
The GUI search returns all experiments in the project that have your search string in their task id, name, description or any of their models' names.
You can use regex with the '.*' button in the search bar.
@<1523706095791509504:profile|FiercePenguin76> The "Log" tab has been renamed "Console" in ClearML 0.17.0 - Thanks for pointing out the outdated description.
DepressedChimpanzee34 Apologies for missing your previous comment.
Totally agree that the global selection indicator should maintain its 'clear selection' behaviour even if some/all of the selection is off-screen.
CooperativeSealion8 For future reference, notice there's a configuration reference available at https://allegro.ai/docs/references/trains_ref/
DefeatedCrab47 For the most part, mlflow can serve basic ML models using scikit-learn. In contrast, Trains was designed with more general purpose ML/DL workflows in mind, for which there's no "generic" way to serve models as different scenarios can use different input encoding, models results would be represented in a variety of forms, etc.
Consider also, that creating an HTTP endpoint for model inference is quite a breeze: there are multiple examples of Flask on top of any DL/ML framework w...
DefeatedCrab47 Happy you're finding Trains useful 🙂
but it definitely has it's advantages if TRAINS would support it (early stage Data Science infrastructure).
No doubt, and I definitely see such usable example in the cards for Trains' upcoming versions...
UnevenDolphin73 I think it'd be easier to track as a separate one.
HappyDove3 Notice that in https://github.com/allegroai/clearml/issues/400 the goal is to see a table plot in the UI scalars tab for a specific experiment (with additional discussions on how these will be addressed when comparing experiments).
Note that once you take the approach you suggested of logging your metrics single values, you can configure your experiment comparison scalars view to show single values instead of the time-series graph which I think will provide you with the matrix c...
KindGiraffe71 Have you checked out the https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch-lightning/pytorch_lightning_example.py ? https://clearml.slack.com/archives/CTK20V944/p1616070536033700 previous discussion provides some insight into how it works under the hood.
UnevenDolphin73 Well... not right now... Currently the ClearML UI only partitions internal artifact types.
That said, having user-defined artifact groups sure sounds worth looking into - Care to https://github.com/allegroai/clearml/issues/new/choose ?
If the credentials don't provide access, the calls should fail (there's no fallback - just default values in place of empty configuration).
Notice you explicitly configure all hosts values, so you don't end up using a specific server for API access, and the default demo server for File server access...
WittyOwl57 Is that information available for you on each of the compared experiments when you view them individually?