Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
CornyHedgehog13
Moderator
3 Questions, 9 Answers
  Active since 28 June 2023
  Last activity one year ago

Reputation

0

Badges 1

5 × Eureka!
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi Guys! Is There Any Way To Get Full State Of Dataset From Somewhere, Except My S3 Bucket? I Need A Mapping Of Files And Batches That Were Uploaded As A Dataset. Maybe This Information Is Also Available In One Of The Clearml Databases?
Hi guys! Is there any way to get full state of dataset from somewhere, except my S3 bucket? I need a mapping of files and batches that were uploaded as a dat...
one year ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Hi Guys, Can I Somehow Link 2 Versions Of Dataset As Parent-Child After I Uploaded Both Of Them? They Have The Exact Same Name But Uploaded As Independent
Hi guys, can I somehow link 2 versions of dataset as parent-child after I uploaded both of them? They have the exact same name but uploaded as independent
one year ago
0 Votes
0 Answers
975 Views
0 Votes 0 Answers 975 Views
Hi Guys! Does Anybody Have Any Experience Of Building Data Catalogue If Your Data Is Stored As Clearml Datatset?
Hi guys! Does anybody have any experience of building data catalogue if your data is stored as clearml datatset?
one year ago
0 Hi Guys! Is There Any Way To Get Full State Of Dataset From Somewhere, Except My S3 Bucket? I Need A Mapping Of Files And Batches That Were Uploaded As A Dataset. Maybe This Information Is Also Available In One Of The Clearml Databases?

I also though clearML writes this mapping ( state.json ) into one of its databases: Mongo, Redis, Elasticsearch.

one year ago
0 Is There An Elegant Way Of Accessing A Specific File Entry From A Dataset Without Using Io Operations To Locate The File From The Cache Folder? The File Is Intended To Be Used To Create A Dataframe. At The Moment I'M Using The Code Below. The Problem Is T

It is deterministic. When you do Dataset.get(), clearML downloads file state.json, where you can see all relative file paths and chunks number

one year ago
0 Hi Everyone! How Can I Create A Pipeline In Draft Mode? In Draft Mode, I Need To Install A Shell Script And Pass Additional Parameters. In Addition, I Would Like To Be Able To Remotely Copy Pipelines By Launching Pipelines With Different Parameters

Also, for some reason I don’t have the ability to copy pipelines. Tell me, is this normal?

Every time you click run a pipeline ran is cloned, so you can pass new parameters right from the UI

one year ago
0 Hi Everyone! How Can I Create A Pipeline In Draft Mode? In Draft Mode, I Need To Install A Shell Script And Pass Additional Parameters. In Addition, I Would Like To Be Able To Remotely Copy Pipelines By Launching Pipelines With Different Parameters

what do you mean by draft? do you want to actually run your pipeline even for testing purposes?

one year ago
0 Is There An Elegant Way Of Accessing A Specific File Entry From A Dataset Without Using Io Operations To Locate The File From The Cache Folder? The File Is Intended To Be Used To Create A Dataframe. At The Moment I'M Using The Code Below. The Problem Is T

You can get a chunk number that contains your file and download that chunk

one year ago
0 Hi Guys, Can I Somehow Link 2 Versions Of Dataset As Parent-Child After I Uploaded Both Of Them? They Have The Exact Same Name But Uploaded As Independent

thank you 🤓

one year ago
0 Hi Guys! Is There Any Way To Get Full State Of Dataset From Somewhere, Except My S3 Bucket? I Need A Mapping Of Files And Batches That Were Uploaded As A Dataset. Maybe This Information Is Also Available In One Of The Clearml Databases?

Thank you @<1523701070390366208:profile|CostlyOstrich36> 🤓

one year ago
0 Hi Guys! Is There Any Way To Get Full State Of Dataset From Somewhere, Except My S3 Bucket? I Need A Mapping Of Files And Batches That Were Uploaded As A Dataset. Maybe This Information Is Also Available In One Of The Clearml Databases?

Hi @<1523701070390366208:profile|CostlyOstrich36> . Thank you for your advise, it definitely makes sense. Regarding to the first point, each dataset has a file state.json . In this file there os a key artifact_name e.g., data , data_001 , etc, and relative path of a file. I thought I can map this key with the chunk number. So, if I pull this file from s3 bucket, I can conclude which chunk I should download to get a specific file. Am I wrong?

one year ago
0 Hi Guys! Is There Any Way To Get Full State Of Dataset From Somewhere, Except My S3 Bucket? I Need A Mapping Of Files And Batches That Were Uploaded As A Dataset. Maybe This Information Is Also Available In One Of The Clearml Databases?

I can add a little piece of context.

  • I want to give my users a way to pic a specific batch to get a file they need. Right now there is no way to download just one specific file from an entire dataset.
  • I need a way to check whether a file has already been uploaded to some other dataset or not.
one year ago