Hi, I think this is what you're looking for - None
Answered
Hi! We'Re Using Clearml In Our Pipeline And Have A Question Regarding Datasets.
Currently, We Export Training Data Directly From Clickhouse To S3 Using A Command Like:
Hi! We're using ClearML in our pipeline and have a question regarding datasets.
Currently, we export training data directly from ClickHouse to S3 using a command like:
INSERT INTO FUNCTION s3(...) SELECT ...
So the data goes straight to S3 without being saved locally.
Is it possible to register this already-uploaded file in S3 as a ClearML Dataset
without re-uploading it (i.e., just referencing the existing S3 file)?
If so, what's the recommended way or API to do that?
Thanks!
451 Views
1
Answer
3 months ago
3 months ago
Tags
Similar posts