Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I’M Trying To Work Out How To Reduce The Amount Of Space Taken Up By Old Tasks On Our Self Hosted Server. I’Ve Deleted About 400 Tasks (200 Or So Were Already Archived) Out Of About 2000 And Was Expecting To See A Reduction In Folder Size In

Hi all, I’m trying to work out how to reduce the amount of space taken up by old tasks on our self hosted server. I’ve deleted about 400 tasks (200 or so were already archived) out of about 2000 and was expecting to see a reduction in folder size in /opt/clearml/data , but it’s exactly the same! Am I right to expect the folder size to reduce? Is there a way of working out which tasks are taking up the most storage?

For context, all artifacts are stored separately and the majority of the disk use is in /opt/clearml/data/elastic_7

  
  
Posted 9 months ago
pic
More
SuperiorPanda77
0 × 1
Report
Votes Newest

Answers 4

Shards that I can see are using a lot of disk space are

  • events-training_stats_scalar
  • events-log
  • And then various worker_stats_*
  
  
Posted 9 months ago
pic
More
SuperiorPanda77
0 × 1
Report

I don't think we configure it in any special way, so the default is probably used

  
  
Posted 8 months ago
pic
More
SuccessfulKoala55
0 × 1
Report

Thanks @<1523701087100473344:profile|SuccessfulKoala55> , I’ve taken a look and is this force merging you’re referring to? Do you know how often ES is configured to merge in clearml server?

  
  
Posted 9 months ago
pic
More
SuperiorPanda77
0 × 1
Report

Hi @<1523701092473376768:profile|SuperiorPanda77> , while data in ES is deleted, compacting the indices is not something ES does immediately - you can look in the EE documentation in how to trigger that yourself (can't look for it now)

  
  
Posted 9 months ago
pic
More
SuccessfulKoala55
0 × 1
Report
Write your answer
563 Views
4 Answers
9 months ago
8 months ago
Tags
clearml