Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
For The Clearml-Server Component, Can The Clearml File Server Be Configured To Any Kind Of Storage ? Example Hdfs Or Even A Database Etc..

For the Clearml-Server component, can the Clearml File server be configured to any kind of storage ? Example hdfs or even a database etc..

  
  
Posted 2 years ago
Votes Newest

Answers 5


Currently, in the diagram here.. Clearml File server is shown as a local storage drive. Our 2 primary concerns.
Is there any ways , we can scale this file server when our data volume explodes. Maybe it wouldnt be an issue in the K8s environment anyways. Or can it also be configured such that all data is stored in the hdfs (which helps with scalablity). Is there any security to protect this data in this storage ?

  
  
Posted 2 years ago

ok noted... Thanks

  
  
Posted 2 years ago

Hi DeliciousBluewhale87 ,

Can you please elaborate a bit? The fileserver is a very simple component for storing and retrieving data. You can save any form of data on it. What exactly is your use case?

  
  
Posted 2 years ago

can the ClearML File server be configured to any kind of storage ? Example hdfs or even a database etc..

DeliciousBluewhale87 long story short, no 🙂 the file server, will just store/retrieve/delete files from a local/mounted folder

Is there any ways , we can scale this file server when our data volume explodes. Maybe it wouldnt be an issue in the K8s environment anyways. Or can it also be configured such that all data is stored in the hdfs (which helps with scalablity).I would suggest to use object storage for that, basically from the user perspective this is transparent, you can configure your client with (s3/gs/azure are supported):
files_server: " "https://github.com/allegroai/clearml/blob/470aa8c52de34b3e6b9ff543e51d29cc285f0b18/docs/clearml.conf#L10

Is there any security to protect this data in this storage ?

In the open-source version, there is not authentication check to serve specific files, on the paid tiers there is full security/authentication on top of the files server

  
  
Posted 2 years ago
786 Views
5 Answers
2 years ago
one year ago
Tags