Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
When Using Docker Mode (And Specifically K8S Glue), What Are The Options For Caching? One Option Is Definitely Having A Base Image That Has The Things Needed. Anything Else? Thanks!

When using docker mode (and specifically k8s glue), what are the options for caching? One option is definitely having a base image that has the things needed. Anything else? Thanks!

  
  
Posted 2 years ago
Votes Newest

Answers 9


IF there’s a post-task script, I can add a way to zip and upload pip cache etc to s3 - as in do any caching that I want without having first class support in clearml

  
  
Posted 2 years ago

Gitlab has support for S3 based cache btw.

This might still be considered "slow" compared to local-dist/cluster mount

Would adding support for some sort of post task script help? Is something already there?

Interesting, can you expand on the use case? (currently there is only pre-task script, for setup)

  
  
Posted 2 years ago

Would adding support for some sort of post task script help? Is something already there?

  
  
Posted 2 years ago

Interesting. How do you do PVC? by using the yaml template optionn?

  
  
Posted 2 years ago

One option is definitely having a base image that has the things needed. Anything else? Thanks!

This is a bit complicated, to get the cache to kick in you have to mount an NFS file into the pod as the cache (to create a persistent cache)
Basically, spin NFS pod to store the cache, change the glue job template yaml to mount it into the pod (see default cache folders:
/root/.cache/pip and /root/.clearml/pip-download-cache)
Make sense ?

  
  
Posted 2 years ago

Gitlab has support for S3 based cache btw.

  
  
Posted 2 years ago

pip cache & git cache & venvs cache
Are all supported, you just need to map the folders.
If you do not want to spin a PVC with NFS mount, you can just mount an S3 bucket with s3fs as part of the container extra bash script,
https://github.com/allegroai/clearml-agent/blob/b39b54bbafab39e6731cb742fdf317bc6dcae54a/docs/clearml.conf#L140

s3 FUSE fuse filesystems:
https://github.com/kahing/goofys
https://github.com/s3fs-fuse/s3fs-fuse

WDYT?

  
  
Posted 2 years ago

Yep 🙂

  
  
Posted 2 years ago
529 Views
9 Answers
2 years ago
one year ago
Tags