Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
If I Create A Dataset With

If I create a dataset with Dataset.create(..., use_current_task=True) , that task holds the dataset.
Can I then refer/copy/attach the same dataset to other tasks?
(For now I've been using the set_parameter and get_parameter to store the dataset ID between tasks that refer to the same dataset)

  
  
Posted 2 years ago
Votes Newest

Answers 7


Hey UnevenDolphin73
When you use the parameter "use_current_task" the dataset and the resulting task will be the same (same id). So to retrieve this dataset for using it into another task, use Task.get(...) to retrieve its id.
Then when you will need it into another task, simply retrieve it from within that task by using Dataset.get(dataset_id=...)

  
  
Posted 2 years ago

Hi UnevenDolphin73 ,
Can you elaborate on this:
Can I then refer/copy/attach the same dataset to other tasks?Also, what is your use case? Why do you attach a task to the Dataset. They can work as separate objects.

  
  
Posted 2 years ago

Unfortunately not, each task defines and constructs its own dataset. I want cloned task to save that link 🤔

  
  
Posted 2 years ago

On an unrelated note, when cloning an experiment via the WebUI, shouldn't the cloned experiment have the original experiment as a parent? It seems to be empty

  
  
Posted 2 years ago

Say I have Task A that works with some dataset (which is not hard-coded, but perhaps e.g. self-defined by the task itself).
I'd now like to clone Task A and modify some stuff, but still use the same dataset (no need to recreate it, but since it's not hard-coded, I have to maintain a reference somewhere to the dataset ID).

Since the Dataset SDK offers use_current_task , I would have also expected there to be something like dataset.link(task) or task.register_dataset(ds) 🤔

  
  
Posted 2 years ago

Hey UnevenDolphin73
Is there any particular reason why not to create the dataset ? I mean, you need to use it in different tasks, so it could make sense to create it , for it to exist on its own, and then to use it at will in any task, by simply retrieving its id (using Dataset.get)
Makes sense ?

  
  
Posted 2 years ago

hey
"when cloning an experiment via the WebUI, shouldn't the cloned experiment have the original experiment as a parent? It seems to be empty"

you are right, i think there is a bug here. We will release a fix asap 🙂

  
  
Posted 2 years ago