Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello All, I’M New To Clearml And Playing Around With The Self-Hosted Version On Aws. How Does Clearml Handles “High-Availability” Requirements, Meaning If I’Ve Run A Few Experiments With Data And Results Inside And For Some Reason The Ec2 Instance Which

hello all, i’m new to ClearML and playing around with the self-hosted version on AWS.
How does ClearML handles “high-availability” requirements, meaning if i’ve run a few experiments with data and results inside and for some reason the EC2 instance which host the ClearML decides to die…
how do i go about recovering those metadata/results/experiments when i spin up another clearML EC2 ?

  
  
Posted 3 years ago
Votes Newest

Answers 3


Hi ComfortableHorse5 ,
This is, as always a matter of effort and deployment. As a rule, it is very unlikely that you AWS storage (EBS) would "die" - usually, when your machine "dies" (for some reason), it is the EC2 instance - in that case, you simply spin up a new instance attached to the same EBS device. To protect against EBS device failure, you should have a periodic backup of your data (daily backup is a very good solution, but you can even do an hourly backup if you'd like) - there are many ways of accomplishing that on AWS.
As always, it is always a question of the effort you put into the deployment. For a fully scalable and state-of-the-art deployment, you can use our https://github.com/allegroai/clearml-server-k8s/tree/master/clearml-server-cloud-ready , but as always, that takes more time and effort to set up and maintain,

  
  
Posted 3 years ago

thanks SuccessfulKoala55 for the reply. Upon spinning up an EC2 via the AMI, there’s only root storage. If i where to add another EBS volume, what kind of data should i be routing to this volume ? is it along the lines of this https://allegro.ai/clearml/docs/docs/concepts_fundamentals/concepts_fundamentals_storage.html#configuring-clearml-for-artifact-storage ?

  
  
Posted 3 years ago

Well, that's one, but for system backup, you basically want to backup and protect the contents of /opt/clearml . BTW, I think the same principles I described apply to the root storage - it's an EBS volume as well.

  
  
Posted 3 years ago
602 Views
3 Answers
3 years ago
one year ago
Tags