Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Is There A Way To Enqueue The Dataset

Hi, is there a way to enqueue the dataset add command on a worker? what are the best practices syncing and adding files?

  
  
Posted 2 years ago
Votes Newest

Answers 4


Alright thanks!

  
  
Posted 2 years ago

EnormousCormorant39 , there are SDK methods for using the datasets. I think this will simplify your process immensely.
https://clear.ml/docs/latest/docs/references/sdk/dataset

Also here is a small example for the usage 🙂
` task = Task.init(project_name="<PROJECT_NAME>", task_name="<TASK_NAME>")

#Create dataset
ds = Dataset.create(dataset_name="<DATASET_NAME>", dataset_project="PROJECT_NAME")
ds.add_files("<PATH_TO_FILE/S>")
ds.upload()
ds.finalize() `

  
  
Posted 2 years ago

Yes. CostlyOstrich36
I’m running on an on-prem machine, and trying to automate the whole process of training. meaning bringing the data -> creating the dataset -> running the training task.

When creating a dataset and adding files it seems there is a Task being created in the background (which is awesome).

My question is - can I run this task on a remote machine instead of the machine which i’m running the CLI on?

  
  
Posted 2 years ago

Hi EnormousCormorant39 ,

is there a way to enqueue the dataset

add

command on a worker

Can you please elaborate a bit on this? Do you want to create some sort of trigger action to add files to a dataset?

  
  
Posted 2 years ago