Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Clearml. I'M Trying To Look At Datasets Functionality (With The Help Of

Hi clearml. I'm trying to look at Datasets functionality (with the help of https://github.com/allegroai/clearml/blob/master/docs/datasets.md and https://clear.ml/blog/construction-feat-tf2-object-detection-api/ ) and have several questions. 1) Is dataset a separate object that can be used within different project or it is a project part? 2) Can I find a dataset created with cli in Web UI? I don't see it in experiment tabs asI expected? 3) Can I configure clearml to store datasets by default on S3?

  
  
Posted 3 years ago
Votes Newest

Answers 5


Hi HelpfulHare30

1). Great

🙂

  1. As you can see from my screenshot,  Data Processing task is created but I don’t see Datasets tab as I see in 

This dataset tab is part of the enterprise solution, clearml-data is part of the open source ClearML

  1. I see. So need to specify with every cli command/SDK method call

How do you use it with the SDK?

  
  
Posted 3 years ago

TimelyPenguin76 , thank you for explanation. 1). Great. 2) As you can see from my screenshot, Data Processing task is created but I don't see Datasets tab as I see in https://clear.ml/blog/construction-feat-tf2-object-detection-api/ 3) I see. So need to specify with every cli command/SDK method call

  
  
Posted 3 years ago

Hi HelpfulHare30 ,

  1. Is dataset a separate object that can be used within different project or it is a project part?

You can use the dataset in every project you like. Dataset task is part of a specific project, but it can be shared between projects.
dataset_folder = Dataset.get(dataset_id="dataset_id").get_local_copy()

  1. Can I find a dataset created with cli in Web UI?

Yes. it should create a Data processing task.

  1. Can I configure clearml to store datasets by default on S3?

Yes, you can change it with --storage arg.

  
  
Posted 3 years ago

You can change the dataset _task object to have your storage location as output_uri

  
  
Posted 3 years ago

I didn't try yet but thought about dataset.upload(output_url=)

  
  
Posted 3 years ago
602 Views
5 Answers
3 years ago
one year ago
Tags