Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
JealousMole49
Moderator
4 Questions, 10 Answers
  Active since 29 September 2023
  Last activity 27 days ago

Reputation

0

Badges 1

8 × Eureka!
0 Votes
2 Answers
109 Views
0 Votes 2 Answers 109 Views
28 days ago
0 Votes
1 Answers
400 Views
0 Votes 1 Answers 400 Views
Hi everyone! I keep running into Could not load dataset state Errors when using both the CLI and python version of ClearML-Data . I now can't even delete pre...
7 months ago
0 Votes
13 Answers
132 Views
0 Votes 13 Answers 132 Views
one month ago
0 Votes
2 Answers
513 Views
0 Votes 2 Answers 513 Views
7 months ago
0 Hi Community, We'Re Trying To Move From App.Clear.Ml To A Self-Hosted Clearml Instance, And I Was Wondering If Anyone Figured Out A Way To Transfer Tasks From One To The Other. Specifically, I'D Like To Copy All Datasets (Which Internally Are Represente

You mean re-creating the datasets from scratch? Since I'm using local storage, that would also mean unnecessarily copying data back and fourth only to end up with the same thing on the local disks... Is there really no other way?
I'm very surprised ClearML offers no export / import functionality of any kind...

27 days ago
0 Hi Folks, Tldr: Dataset.Remove_Files() Is Very Slow. How Can I Speed It Up? I'M Working With A Large Raw Dataset That We Are Trying To Use A Small Subset Of. The Data Is Thousands Of Images And A Metadata Json File For Each Image. To Create This Subset

Hi Allen,
I've ran into this exact problem myself, and simply added a function to dataset.py in the clearml package ( clearml/datasets/dataset.py ) that takes a list of files instead of a single file.

It looks like this (I use clearml 1.13.1 ):

28 days ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

Hi Eugen, thanks for the pointers! Is there any documentation about those config values I could read so I better understand what I'm doing?

As for your suggestion for reducing the current usage, is that something I would do in the web dashboard (app.clear.ml) or through the API? I'm not sure what this Dataset Content configuration object is exactly and where I'd have to remove it.

one month ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

@<1523701070390366208:profile|CostlyOstrich36> @<1523701087100473344:profile|SuccessfulKoala55> I'm still urgently looking for a solution for this as we depend on clearML to mange our datasets. Any suggestions how I could find out where my storage space went and free it up?

one month ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

@<1523701087100473344:profile|SuccessfulKoala55> I get a warning in the ClearML Dashboard (app.clear.ml) that my metrics storage is almost full. I have no idea what happens when I reach the limit, but since we are dependent on being able to use the datasets stored in ClearML, I don't really want to find out... (Using the SaaS version)

one month ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

@<1523701070390366208:profile|CostlyOstrich36> Thanks for your reply, unfortunately this is exactly the problem: I simply cannot explain that the storage filled up all of a sudden, so I have no idea what to delete. At least 2/3 of the available space was filled during a time where we didn't run any experiments whatsoever, we only added a bunch of datasets. AFAIK there are no logs/metrics/plots involved, and the data itself is always stored locally.

I tried to contact support but they where ...

one month ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

Thank you so much @<1523701435869433856:profile|SmugDolphin23> , that should help a lot!
Just to be sure I understand correctly: Removing this Content configuration only removes the previews and associated data, but leaves the dataset itself fully accessible, correct?
And since you seem quite knowledgable on the subject: Do you know if there is a way to transfer these tasks from one ClearML server instance to another (specifically from SaaS to a self-hosted instance)?

one month ago
0 Hi Everyone! I'M Currently Using The Free Hosted Version (Open Source) Of Clearml. I'M Mainly Using Clearml-Data At To Manage Our Datasets At The Moment, And I'Ve Already Hit The Limit For The Free Metrics Storage. Since We Didn'T Store A Lot Of Metrics (

H @<1523701087100473344:profile|SuccessfulKoala55> , the data is stored locally for sure - we have more data than even the 100GB artifacts storage would allow. Plus I doubt that the actual dataset would be counted against the metrics storage quota, right?

Your second remark is pretty much what the email support told me before sending me here. The problem is that I still don't know how I can
a) prevent my datasets taking up so much metrics space (can I disable previews?)
b) find and remove a...

one month ago
0 Hi Folks, Tldr: Dataset.Remove_Files() Is Very Slow. How Can I Speed It Up? I'M Working With A Large Raw Dataset That We Are Trying To Use A Small Subset Of. The Data Is Thousands Of Images And A Metadata Json File For Each Image. To Create This Subset

The way I wrote it is a bit of a quick fix with a lot of code duplication, I'm sure it could be implemented in a cleaner way (e.g. having only one remove_files method that can either take a single path or a list of paths).
It's one of those things that I intended to do at some point, but never had the time to clean it up (I did a similar modification for adding lists of files, since this has exactly the same issue if you don't want to add something you can define with a wildcard but only ...

27 days ago
0 Hi Everone! We'Re Trying To Start Using Clearml For Both Experiment Tracking And Data Management, And I Have A Question About Clearml-Data: Is It Possible To Specify Two Different Storage Locations For A Dataset? Background: We'Re Using Both Local Machine

Yes, ideally I'd like to ensure that they are always in sync. They will be updated from time to time, adding new versions and having two separate datasets sounds like I'd always have to do this twice...

7 months ago