Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everyone, I Am A New User For Clearml, I Have One Question: I Created The Dataset, And Upload Files Successfully By Class

Hello everyone,
I am a new user for ClearML, I have one question:

I created the dataset, and upload files successfully by class https://clear.ml/docs/latest/docs/references/sdk/dataset :
When I wanted to remove a specific file in existing data, it failed in ClearML Dashboard(Web APP). Any solutions?

Python code:
from clearml import Dataset
data_set = dataset.Dataset.create( dataset_name="test_dataset", dataset_project="test_project" )
data_set.add_files(path="file.csv")
data_set.upload()
dataset.remove_files(file_name, verbose=True)

thanks and best regards,

  
  
Posted 2 years ago
Votes Newest

Answers 8


Hmm, I can't really follow your explanation. The removed file SHOULD not exist right? 😅 And what do you mean exactly with the last sentence? An artifact is an output generated as part of a task. Can you show me what you mean with screenshots for example?

  
  
Posted 2 years ago

Hi ExasperatedCrab78 ,

Thanks for your quick response. 🙂
I did not set dataset.finalize() actually. the dataset status stays running status.

So I execute dataset.remove_files(file_name, verbose=True)
it does not work 😅

  
  
Posted 2 years ago

Sorry, my English is not well.
I use the company's laptop so I can't upload the screenshots because of permission.
Let me think about how to explain this problem.

  
  
Posted 2 years ago

Hi, ZippyWalrus56 , can you add a full print of your console log?

Also if possible provide a code snippet, that can help understand the problem 🙂

  
  
Posted 2 years ago

No error message actually.

I print data_set.list_added_files() , the removed file does not exist.
I want to remove the specific file in an artifact, but I can see the file in the WebAPP instead.

  
  
Posted 2 years ago

Ok, no problem! Take your time, I think I can help you, but I don't understand yet 🙂

  
  
Posted 2 years ago

With what error message did it fail? I would expect it to fail, because you finalized this version of your dataset by uploading it 🙂 You'll need a mutable copy of the dataset before you can remove files from it I think, or you could always remove the file on disk and create a new dataset with the uploaded one as a parent. In that way, clearml will keep track of what changed in between versions.

  
  
Posted 2 years ago

Can you share the exact error message? That will help a ton!

  
  
Posted 2 years ago