Oh hey, did someone mention my name on this thread? MiniatureCrocodile39 did you manage to create a pycharm virtual env with clearml installed?

but hey, UnevenDolphin73 nice idea, maybe we should have clearml-around that can report who is using which GPU 😉

Totally within ClearML :the_horns: :the_horns:

if she does not push, trains has a commit id for the task that does not exist on the git server. if she does not commit - trains will hold all the diff from the last commit on the server.

https://youtu.be/VJJsVJiWnYY?t=1774
BattyLion34

Thanks for the report!

I would say that this is opposite of the ClearML vision... Repos are for codes, ClearML server is for logs, stats and metadata. It can also be used for artifacts if you dont have dedicated artifact storage (depending on deployment etc)

Do you mind explaining your viewpoint?

Hi TenseOstrich47 sorry for the long wait, here is a video + code of how to put any sort of metadata inside your clearml model artifact 😉 We will also be improving this, so if you have feature requests we would love to hear about them
https://www.youtube.com/watch?v=WIZ88SmT58M&list=PLMdIlCuMqSTkXBApOMqg2S5IeVfnkq2Mj&index=12

SuccessfulKoala55

that's how it is supposed to work 😉 let us know if it does not.

These are excellent questions. While we are working towards including more of our users stack within the ClearML solution, there is still time until we unveil "the clearml approach" to these. From what I've seen within our community, deployment can anything from a simple launch of a docker built with 'clearml-agent build' to auto training pipelines.

Re triggering - this is why we have clearml-task 😉

https://clear.ml/docs/latest/docs/guides/reporting/plotly_reporting ?

We need SEO for our docs 😉

Hi, I was just answering your previous question. can you explain a bit what you mean by "under utilized"? e.g. do you have 2 gpus and are using only one of them for a task?
or are maxing out resources but do not get to 100% utilization (which might be a data pipeline issue)

Looks like incomplete build of pytorch. What are we looking at. And who's christine?

Well, we had a nice video from twimlcon but it is not up yet on our site. I recently gave a very long demo on both basic and semi-advanced clearml usage - you can watch it here
https://youtu.be/VJJsVJiWnYY?t=1774
the slides are here:
https://docs.google.com/presentation/d/1PFPTQkHVGxugruTRFDnuVmS85ziSbNOTixCVQwPMFDI/edit?usp=sharing
code is here:
https://github.com/abiller/events/tree/webinars/webinars/flower_detection_rnd

Thanks for your interest in the enterprise offering 🕶️ I would much rather if we kept this slack workspace for the open-source solution we all know and love. You can email me at mailto:ariel@clear.ml for more info. For a short answer: the Data lineage is about an order of magnitude cooler, and hyperdatasets can be thought of "beyond feature stores for unstructured data". Does this help?

I've been waiting so eagrly for this, I made a playlist! https://open.spotify.com/playlist/4XBqPUgxHD5dbhcYqANzNo?si=G0E_s-OaQzefKIJ0wDkzHA

HugePelican43 as AgitatedDove14 says, that's a slippery slope to out-of-memory land. If you have Nvidia A100 you can use multiple agents in MIG mode, sort of like containerized hardware if you never heard of it.
Other than that I do not recommend. Max out utilisation for each task instead.

What about cloning and setting "last commit in branch" ?

SubstantialBaldeagle49
hopefully you can reuse the same code you used to render the images until now, just not inside a training loop. I would recommend against integrating with trains, but you can query the trains-server from any app, just make sure you serve it with the appropriate trains.conf and manage the security 😉 you can even manage the visualization server from within trains using trains-agent. Open source is so much fun!

First of all I'd like to thank you for pointing out that our messaging is confusing. We'll fix that.

To the point: Nice interface and optimized db access for feature stores is part of our paid, enterprise offering.

Managing data/features as part of your pipelines and getting version-controlled features == offline feature store

The latter is doable with the current ClearML open source tools, and I intend to show it very soon. But right now you won't have a different pane for DataOps, it'll...

This is definitely getting into an example soon! Props for building something cool

I'm specifically interested in the model-first queries you would like to do (as experiment-first queries are fully featured, we want to understand whats the best way to take that into models)

Are you doing imshow or savefig? Is this the matplotlib oop or original subplot? Any warning message relevant to this?

There are examples but nothing comes to mind when. Thinking about well fleshed out for Bert etc. Maybe someone here can correct me

Awesome! You get to be as specific as you want it to be.

#goodfirstissue AgitatedDove14 🤣

WackyRabbit7 It is conceptually different than actually training, etc.

The service agent is mostly one without a gpu, runs several tasks each on their own container, for example: autoscaler, the orchestrators for our hyperparameter opt and/or pipelines. I think it even uses the same hardware (by default?) of the trains-server.

Also, if I'm not mistaken some people are using it (planning to?) to push models to production.

I wonder if anyone else can share their view since this is a relati...

It depends on what you mean by deployment, and what kind of inference you plan to do (ie rt vs batched etc)
But overall currently serving itself is not handled by the open source offering, mainly because there are so many variables and frameworks to consider.
Can you share some more details about the capabilities you are looking for? Some essentials like staging and model versioning are handled very well...