Hi @<1523701553372860416:profile|DrabOwl94> , this depends on your AWS services choice. Whatever can be mounted into a file system folder can be used as the ClearML cache - a simple solution would be to use a single (relatively cheap) machine with some EBS storage and an NFS server, and have every spot instance mount the NFS into some fixed folder, than make sure the clearml configuration uses that folder as the cache folder
Answered
Hi. Can You Point Me In A Direction Of The Solution Please? We Are Trying To Use Spot Machines On Aws Through Autoscaler In Clearml. The Problem Is We Need To Download Each Time Huge Dataset From S3. Is There Some Cash Solution, Like Attaching Disk For Ex
Hi. Can you point me in a direction of the solution please? We are trying to use spot machines on AWS through autoscaler in ClearML. The problem is we need to download each time huge dataset from S3. Is there some cash solution, like attaching disk for example, so we will not need to download data every time the spot is going down? Thank you.
381 Views
1
Answer
7 months ago
7 months ago
Tags