Don't worry! It's all good 🙂

It's decorators. It's functions as steps, It's the ability to add metric tracking for pipelines. It's a better way to move artifacts between steps (way less boilerplate code). It's pipeline instance versioning (for the upcoming UI changes), it's better step parallelizations (If steps are not dependent on each other, they will automatically be parallelized) and I've probably missed some features....

It's a known fact that documentation always trail features by 3-6 months 😄 We're working on new docs, should be released this week 🙂

Hi SmugTurtle78 , Could you spin up an instance with the same user from AWS CLI?

EnviousStarfish54 BTW, as for absolute reproducibility, you are obviously right. If you use S3 to store the data, and you changed the data in S3 then we can't catch it.
Our design compresses (zips) the files and store them in a version somewhere. If this is modified than you are trying hard to break stuff 🙂 (Although you can). This is not the most efficient space-wise when it comes to images \ videos, for these, you can save links, but I think it's only in the enterprise version but then,...

Let me try

Yes, any need for anything else?

I...Think it's a UI bug? I'll confirm 🙂

SubstantialElk6 Yes! We should have it up, together with the previous one uploaded soon!

Try this, I tested it and it works:
docker=pipe._parse_step_ref("${pipeline.url}")It's hack-ish but it should work. I'll try and get a fix in one of the upcoming SDK releases that supports parsing references for parameters other than kwargs

Hi EnviousStarfish54 If you want to not send info to the server, I suggest you to set an environment variable, this way as long as the machine has this envvar set it won't send to the server

JitteryParrot8 in the new SDK we'll have dataset.add_description() which will do the same as KindChimpanzee37 provided but with a nicer interface 😄

Hmm. seem like there is a problem. Let me check 🙂

Hmm, that's not fun
I'm checking 🙂

Cool!

BTW, just talked to the devs, what happens is that your metrics \ logs are saved locally, then once a task is closed, it's zipped. If you are affraid the instance might be taken from you, first we are planning to release a solution for these situations 🙂 and second your code needs to be aware of the risk and to be able to "resume" training from a specific model snapshot \ iteration.

Task.get_offline_mode_folder()

So I think it's necessary to code defensively and once training is done, upload to a remote location (S3 in your case). If disc is persistent this should be a problem as the logs will be saved. Makes sense?

Cool and impressive are 2 adjective we like to hear 😄

https://clear.ml/docs/latest/docs/guides/set_offline/

Hi SpicyCrab51 , Thanks for the warm words 😄 Happy you enjoy our product!
As for your needs, I suggest you explore our https://clear.ml/docs/latest/docs/hyperdatasets/overview , they indeed were made to solve issues similar to what you're facing!
You can see a talk we gave that cover the Hyperdatasets https://www.youtube.com/watch?v=CcL8NNZfHlY !
Note that it is an enterprise feature, and is not part of the open source.
Contact me if you need more info 🙂

MelancholyElk85 I'll talk to the guy in charge and try! You can find main points here 🙂 https://docs.google.com/document/d/1QlPiDO2EzDq_HRvuVhYwPh5F_IsjgmmRTlPNdHik10k/edit

Hi ScaryBluewhale66 , I believe the new server that's about the be released soon (this \ next week), we'll allow you to report a "single value metric". so if you want to report just a number per experiment you can, then you can also compare between runs.

That's true 🙂 Our SDK is a python based and your code needs to be python code for us to integrate with

Can you check again? It works for me. If you're still not able to reach it, can you send an image of the error you're getting?

JitteryCoyote63 Fair point 😅 , I'll be lying to say we haven't been slow on documenting new features 🙂 That being said, since what you're looking for seems REALLY straightforward (at least to people who know how it works internally 😛 ) we can probably do something about it rather quickly 🙂

CLEARML_NO_DEFAULT_SERVER=1

Hi SillySealion58 , yeah in that case we only look at the filename and not the full path. Let me see what we can do internally! Thanks and happy you found a workaround 😄

VivaciousPenguin66 This is very true! We are trying to explain the benefits of this method. Some people like it and some people like the flexibility. We do have our philosophy in mind when we create "best practices" and obviously features to ClearML but ultimately people should do what makes them the most productive!
If we are getting philosophical, I think it's the state of the industry and as it progresses, these standard methods would become more prominent.
also, to add to what you wrote,...

Hi Sarah, Yes! These are good options 🙂 What you can do with hyperdatasets is visualize a slice (or video) in any format your browser knows how to display natively (png \ jpg \ tiff). What you can also do is that every hyperdataset frame can have a link to the original 3D data, as well as single (or multiple) visualized sources. That way you can browse all your slices but still connect them to the 3D data. Once you use hyperdataset management connected to your experiment you'll be able to ...