Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Since Clearml 1.6.3, A Dataset Attached To A Task Now Renames That Task By Adding A

Since ClearML 1.6.3, a dataset attached to a Task now renames that task by adding a .datasets in the path, making it hidden.
Can we control/cancel this? We can't find our running tasks now 😢

  
  
Posted one year ago
Votes Newest

Answers 18


No task, no dataset, just an empty container with no reference to the task it's attached.

It seems to me that it should not move the task if use_current_task=True ?

  
  
Posted one year ago

Can you see your task if you run this minimal example UnevenDolphin73 ?
` from clearml import Task, Dataset

task = Task.init(task_name="name_unique", project_name="project")
d = Dataset.create(dataset_name=task.name, dataset_project=task.get_project_name(), use_current_task=True)
d.upload()
d.finalize() `

  
  
Posted one year ago

It also happens when use_current_task=False though. So the current best approach would be to not combine the task and the dataset?

  
  
Posted one year ago

UnevenDolphin73 Yes it makes sense. At the moment, this is not possible. When using use_current_task=True the task gets attached to the dataset and moved under dataset_project/.datasets/dataset_name . Maybe we could make the task not disappear from its original project in the near future.

  
  
Posted one year ago

Does that make sense SmugDolphin23 ?

  
  
Posted one year ago

Let me test it out real quick.

  
  
Posted one year ago

UnevenDolphin73 can't you find your task/dataset under the Datasets tab?

  
  
Posted one year ago

I don't think the version makes the task disappear. You should still see the task in the Datasets section. Maybe there is something you do with that task/dataset that makes it disappear (even tho it shouldn't)?

  
  
Posted one year ago

Can you please provide a minimal example that may make this happen?

  
  
Posted one year ago

That is, we have something like:
` task = Task.init(...)
ds = Dataset.create(dataset_name=task.name, dataset_project=task.get_project_name(), use_current_task=True)

upload files

dataset.upload(show_progress=True)
dataset.finalize()

do stuff with task and dataset

task.close() `But because the dataset is linked to the task, the task is then moved and effectively becomes invisible 😕

Any thoughts AgitatedDove14 SuccessfulKoala55 ?

  
  
Posted one year ago

I'm not too worried about the dataset appearing (or not) in the Datasets tab. I would like it (the original task ) to to not disappear from the original project I assigned it to

  
  
Posted one year ago

And this is of course strictly with the update to 1.6.3 (or newer) that should support API 2.20

  
  
Posted one year ago

Yes and no SmugDolphin23
The project is listed, but there is no content and it hides my main task that it is attached to.

  
  
Posted one year ago

Some examples of the mess it creates (also posted in the main channel):
A single project now has multiple subprojects The subprojects have the .datasets hidden subproject (with really frustrating project names) The subprojects are empty To access the original project, I have to go twice into the same project because of these hidden projects Because of these hidden subprojects, I cannot delete a project that has 0 experiments

  
  
Posted one year ago

UnevenDolphin73 The task shouldn't disappear when using use_current_task=False . There might be something else that makes it disappear.

  
  
Posted one year ago

Yes, that one shows up. I forgot to mention we also set the version explicitly, but that just creates a duplicate dataset under Datasets and anyway our main Task is now hidden from the original project.
So project project exists, but it is empty.

  
  
Posted one year ago

Ah right, I missed that in the codebase. It just adds the .dataset convention to the dataset task.

  
  
Posted one year ago

SmugDolphin23 we've been working with this for 2 weeks now, and it creates a lot of junk in our UI. Is there anyway to have better control over this?

  
  
Posted one year ago
548 Views
18 Answers
one year ago
one year ago
Tags