Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All! I Just Got Started To Learn Things Here, And I Have Some Questions. How Am I Supposed To Do To Perform An Experiment For Training A Model With The Existing Dataset On Clearml Server? So Basically What I'M Trying To Do Here Is To Get An Existing Da

Hi all!
I just got started to learn things here, and I have some questions.
How am I supposed to do to perform an experiment for training a model with the existing dataset on clearml server?
So basically what I'm trying to do here is to get an existing dataset from the server,
and feed it to my model.
In the process, do I have to download the dataset in local, and then feed it to my model?
Or are there any ways to train my model directly with the existing dataset from the server?
Could anyone please provide a guidance on this?

  
  
Posted 10 months ago
Votes Newest

Answers 5


Hi @<1664079296102141952:profile|DangerousStarfish38> , regarding your first question - You need to finalize the dataset.
Regarding the second, my suggestion would be to expose the selected dataset via a hyperparameter. One example would be through argparser

  
  
Posted 10 months ago

@<1523701070390366208:profile|CostlyOstrich36>
I have a follow-up question for the first question.

I initiated a task, did get_local_copy of a dataset,
and then I executed and finished the task (training).
From web UI, I don't see any information saying that the task and dataset are related or linked.

What should I do to connect or link those two or find the information about it?

  
  
Posted 10 months ago

@<1523701070390366208:profile|CostlyOstrich36>
Actually, I've got another questions about dataset!

I tried add_external_files from AWS S3 as a simple test.
And in web UI, it says it's been uploading for 16hours now.
The zip file I tried to upload is under 50MB.
Is something wrong here?

Also, I'm wondering if I could add files that are not "zipped" files, for example a directory containing various files.

  
  
Posted 10 months ago

Thanks!

  
  
Posted 10 months ago

Hi @<1664079296102141952:profile|DangerousStarfish38> , you would need to download the dataset to local using get_local_copy
I suggest going through the docs:
None

  
  
Posted 10 months ago