Hi @<1595225628804648960:profile|TroubledLion34> , I'm afraid you can't upload via an API since what is doing the uploading is the SDK/CLI, however, you can upload files via your java application and then register the dataset via the API

Makes sense?

Hi @<1523701070390366208:profile|CostlyOstrich36> ,

thanks for your help. Sorry, but I'm not really sure what you're suggesting. To clarify: I want to upload the files from the java application. The upload to "the application servers" is already working, the data should then be forwarded to ClearML. For this forwarding I cannot use the python SDK.

This is planned to be an automated process so I cannot use the CLI either.

If it's really not possible, I will probably write a flask service with an "upload endpoint" which will then use the python sdk.

However, I'd really like to avoid this 😉

Other frameworks I've used either offer their SDKs in multiple programming languages or hide their logic behind an REST API. From what I've seen in the ClearML python SDK, the logic really is in the SDK.

Just trying to understand how it works in ClearML 🙃

Long story short - You'll have to write a service to upload.

They way that datasets work - the SDK/CLI actually do the uploading itself. The REST API simply registers them on the backend

oh, bummer 😕 but thanks a lot for clarifying