https://clear.ml/docs/latest/docs/guides/set_offline

Sure we do! Btw MiniatureCrocodile39 iirc I answered one of your threads with a recording to a webinar of mine

Shh AgitatedDove14 you're dating yourself 😆

I'm specifically interested in the model-first queries you would like to do (as experiment-first queries are fully featured, we want to understand whats the best way to take that into models)

So basically export a webapp view as csv?

Well, we had a nice video from twimlcon but it is not up yet on our site. I recently gave a very long demo on both basic and semi-advanced clearml usage - you can watch it here
https://youtu.be/VJJsVJiWnYY?t=1774
the slides are here:
https://docs.google.com/presentation/d/1PFPTQkHVGxugruTRFDnuVmS85ziSbNOTixCVQwPMFDI/edit?usp=sharing
code is here:
https://github.com/abiller/events/tree/webinars/webinars/flower_detection_rnd

WickedGoat98 I gave you a slight twitter push 😉 if I were I would make sure that the app credentials you put on your screen shot are revoked 🙏 😇

Thanks for your interest in the enterprise offering 🕶️ I would much rather if we kept this slack workspace for the open-source solution we all know and love. You can email me at mailto:ariel@clear.ml for more info. For a short answer: the Data lineage is about an order of magnitude cooler, and hyperdatasets can be thought of "beyond feature stores for unstructured data". Does this help?

Sorry for being late to the party WearyLeopard29 , if you want to see get_mutable_copy() in the wild you can check the last cell of this notebook:

https://github.com/abiller/events/blob/webinars/videos/the_clear_show/S02/E05/dataset_edit_00.ipynb

Or skip to 3:30 in this video:

https://youtube.com/watch?v=NhSRW12Bn5w&feature=share

Well in general there is no one answer. I can talk about it for days. In ClearML the question is really a non issue since of you build a pipeline from notebooks on your dev in r&d it is automatically converted to python scripts inside containers. Where shall we begin? Maybe you describe your typical workload and intended deployment with latency constraints?

I guess the product offering is not so clear yet (pun intended) the self-deployed option is completely free and open source. The enterprise offering is something entirely different
https://clear.ml/pricing/
The "feature store" you see in the free tier is what I am alluding to

From what i remember the bins in tb are wider. And the tapering off around zero cannot be real since this happens in super sparse modela. Overall if you are sure, than this is a nice issue to open on GitHub.

Hi TenseOstrich47 sorry for the long wait, here is a video + code of how to put any sort of metadata inside your clearml model artifact 😉 We will also be improving this, so if you have feature requests we would love to hear about them
https://www.youtube.com/watch?v=WIZ88SmT58M&list=PLMdIlCuMqSTkXBApOMqg2S5IeVfnkq2Mj&index=12

TenseOstrich47 as i might have stated earlier, I'm doing a low-key build of something like this. Thanks to your question I know what to focus on when showcasing 😇 ♥

Here's what's already been done https://youtu.be/xliX3IhNdmw

Honestly, it looks like the tensorboard representation is the wrong one. Only one way to find out - you need to plot the histogram on your own 😅

Oh hey, did someone mention my name on this thread? MiniatureCrocodile39 did you manage to create a pycharm virtual env with clearml installed?

HugePelican43 as AgitatedDove14 says, that's a slippery slope to out-of-memory land. If you have Nvidia A100 you can use multiple agents in MIG mode, sort of like containerized hardware if you never heard of it.
Other than that I do not recommend. Max out utilisation for each task instead.

Hi! Looks like all the processes are calling torch.save so it's probably reflecting what Lightning did behind the curtain. Definitely not a feature though. Do you mind reporting this to our github repo? Also, are you also getting duplicate experiments?

Hi SubstantialBaldeagle49 ,

certainly if you upload all the training images or even all the test images it will have a huge bandwidth/storage cost (I believe bandwidth does not matter e.g. if you are using s3 from ec2) If you need to store all the detection results (for example, QA, regression testing), you can always save the detections json as artifact and view them later in your dev environment when you need. The best option would be to only upload "control" images and "interesting" im...

I'm going to answer soon 🙏

There short answer is "definitely yes" but to get maximum usage you will probably want to setup priority queues

What about cloning and setting "last commit in branch" ?

Rather unfortunate that such a vendor would risk such an inaccurate comparison...

JealousParrot68 Some usability comments - Since ClearML is opinionated, there are several pipeline workflow behaviors that make sense if you use Datasets and Artefacts interchangeably, e.g. the step caching AgitatedDove14 mentioned. Also for Datasets, if you combine them with a dedicated subproject like I did on my show, then you have the pattern where asking for the dataset of that subproject will always give you the most up-to-date dataset. Thus you can reuse your pipelines without havin...

Hi SubstantialElk6 , have a look at Task.execute_remotely, and it's especially for that. For instance in the recent webinar, I used pytorch-cpu on my laptop and task.execute_remotely. the agent automatically installs the GPU version. Example https://github.com/abiller/events/blob/webinars/webinars/flower_detection_rnd/A1_dataset_input.py

I'm not sure I can help with the technicality, but here is a basic question you'll be aksed - are you able to download anything from your minio using ClearML?

BattyLion34 this is up to the discretion of the meetup organizers. At any case, I am going to use the same demos to create several of my stuffed animal videos (we can also upload the same videos without the stuffed animals if there is demand for that)

https://youtu.be/VJJsVJiWnYY?t=1774
BattyLion34

Sure thing. All you need is the credentials. Did you see my extreme example here? https://youtu.be/qz9x7fTQZZ8

Removed the yay's DeliciousBluewhale87
https://www.youtube.com/watch?v=XpXLMKhnV5k