Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hello Community, I'M Reading About

Hello Community,
I'm reading about dataset management and was wondering if there is another way to upload data besides the CLI and the Python SDK.
Unfortunately I couldn't find anything about it until now, which makes me wonder. I am thinking of something like REST api?
The general API reference does not contain anything about data management.
My use case is uploading data from a Java application and I want to avoid writing a wrapper service around the Python SDK, if possible.
Thanks a lot in advance!

Posted 4 months ago
Votes Newest

Answers 5

oh, bummer 😕 but thanks a lot for clarifying

Posted 4 months ago

Hi @<1595225628804648960:profile|TroubledLion34> , I'm afraid you can't upload via an API since what is doing the uploading is the SDK/CLI, however, you can upload files via your java application and then register the dataset via the API

Makes sense?

Posted 4 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36> ,

thanks for your help. Sorry, but I'm not really sure what you're suggesting. To clarify: I want to upload the files from the java application. The upload to "the application servers" is already working, the data should then be forwarded to ClearML. For this forwarding I cannot use the python SDK.

This is planned to be an automated process so I cannot use the CLI either.

If it's really not possible, I will probably write a flask service with an "upload endpoint" which will then use the python sdk.

However, I'd really like to avoid this 😉

Posted 4 months ago

Other frameworks I've used either offer their SDKs in multiple programming languages or hide their logic behind an REST API. From what I've seen in the ClearML python SDK, the logic really is in the SDK.

Just trying to understand how it works in ClearML 🙃

Posted 4 months ago

Long story short - You'll have to write a service to upload.

They way that datasets work - the SDK/CLI actually do the uploading itself. The REST API simply registers them on the backend

Posted 4 months ago
5 Answers
4 months ago
4 months ago