@<1559349204206227456:profile|BeefyStarfish55> try checking out the general overview on pipelines here , and info on the pipelines UI here .
Each step's arguments (and results) should appear in the steps details panel (which you could then follow to the underlying task for complete, in-depth, details).
DepressedChimpanzee34 ClearML tries to conserve storage by limiting the history length for debug images (see sdk.metrics.file_history_size https://clear.ml/docs/latest/docs/configs/clearml_conf#sdk-section ), though the history can indeed grow large by setting a large value or using a metric/variant naming scheme to circumvent this limit.
Does your use case call for accessing a specific iteration for all images or when looking at a specific image? Note that the debug image viewer (wh...
UnsightlySeagull42 The upgrade process is slightly different depending on the environment in which you've deployed your ClearML server (e.g. for a https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_linux_mac.html#upgrading ).
Note the document you are referring to only applies once when you're moving from the older pre-0.16 versions in which case DB migration is required.
If your server is more up to date (0.16 and newer) you should be OK with the link above.
DepressedChimpanzee34a filter similar to one in the scalars page where you can display a subset of the reported debug images can be usefulThe scalars page provides a metric hide/show control - Is this the one you mean? The debug images page also provides a filter by metric - Depending on your naming policy this can easily be used to focus on more sparsely appearing images.
Else, an example of the filter you were thinking of would be appreciated.
Regardless, direct iteration access cou...
BattyLion34 Adding to AgitatedDove14 hint. See the following docs page: https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_config_for_clearml_server.html
@<1523709410411548672:profile|NuttyFox2> Since the default server user configuration does not require authentication, I'm assuming your use case calls for some users being authenticated where others are not?
Such mixed access mode is currently not on the near term roadmap for the OSS server - You should create a feature request to help push it into the development plan.
WittyOwl57 I just used a couple of the experiments in the https://app.community.clear.ml/projects/764d8edf41474d77ad671db74583528d/ of the free tier server.
AverageRabbit65 Adding to SweetBadger76 's reference, e2e examples are available for the different pipeline implementation methods:
https://clear.ml/docs/latest/docs/guides/pipeline/pipeline_controller
https://clear.ml/docs/latest/docs/guides/pipeline/pipeline_decorator
https://clear.ml/docs/latest/docs/guides/pipeline/pipeline_functions
Hi DefeatedCrab47 ,
The examples folder has just been restructured: Find the example here:
https://github.com/allegroai/trains/blob/master/examples/services/hyper-parameter-optimization/hyper_parameter_optimizer.py
TightElk12 This makes a lot of sense - should make it into one of the coming releases
WittyOwl57 The UI shows repo and package detailed comparison under the "Details"/"Execution" (See sample screenshot), whereas auto-logged environment variables are shown under the "HyperParameters" comparison tab.
What do you find missing beyond those?
WittyOwl57 No worries 🙂 happens to the best!
@<1523706095791509504:profile|FiercePenguin76> The "Log" tab has been renamed "Console" in ClearML 0.17.0 - Thanks for pointing out the outdated description.
IrateDolphin19 ClearML provides for saving files generated as part of your code execution through the https://clear.ml/docs/latest/docs/references/sdk/task#upload_artifact . For your use case, you can have your code thus create the artifact as it runs, you can set the specific storage location when you edit your configuration, through the task's output_uri field.
Does this help?
ScrawnyLion96 Looks like a case of broken links - Check out https://clear.ml/docs/latest/docs/references/api/definitions#tasksexecution and https://clear.ml/docs/latest/docs/references/api/definitions#tasksconfiguration_item
SmarmySeaurchin8 Following up on ColossalDeer61 's hint, notice https://allegroai-trains.slack.com/archives/CTK20V944/p1597248476076700?thread_ts=1597248298.075500&cid=CTK20V944 not-too-old thread on reusing globally installed packages.
GentleSwallow91 For more information, look at what ClearML logs for your experiments: https://docs-testing.allegro.ai/docs/latest/docs/fundamentals/task#logging-task-information
JitteryCoyote63 Not currently there, but certainly sounds like something to add to the list - Care to https://github.com/allegroai/clearml/issues/new/choose ?
DepressedChimpanzee34 Thanks for clarifying where the current debug images display falls short for your use case - Extending the filtering to liken the behaviour of the scalars sound like a great idea 🙂
GreasyPenguin14 When the project description is empty you get a "Add project overview" instead if the "Edit" button:
@<1628927672681762816:profile|GreasyKitten62> When you have specific display considerations, you can implement them through report_table's 'extra_layout' and 'extra_data' parameters
OutrageousSheep60 You can see https://github.com/allegroai/clearml/issues/724 a discussion on the topic.
TL;DR:
Currently the containing project is available in the UI as a tooltip to the dataset name An alternate "Project view" to the datasets page is in the works
WittyOwl57 Is that information available for you on each of the compared experiments when you view them individually?
DefeatedCrab47 Thanks for pointing it out.
We'll get in touch with the PyTorch Lightning team to better understand the code restructure they're effecting (see https://github.com/PyTorchLightning/pytorch-lightning/pull/2384 ).
In the mean time, you can look at the prior version: https://github.com/PyTorchLightning/pytorch-lightning/blob/0.8.1/pytorch_lightning/loggers/trains.py
UnevenDolphin73 Am I missing anything in rephrasing your use case to "Have a single autoscaler service multiple queues" (where the autoscaler resource configuration is, in essence, the pool you mention)?
@<1580367723722969088:profile|SmoothDuck83> CSV export is only available for table plots
@<1580367723722969088:profile|SmoothDuck83> Not every plot is trivially be formed as a table (i.e. CSV), that's why the JSON export is available for all plots.
What were you considering?
RotundHedgehog76 Have you tried clearml-data add --files .
? (Probably best to try on a smaller subset first)