I added this to each of the containers

    logging:
      options:
	    max-file: 5
        max-size: 10m

strange, I used one of the publicly available AMIs for ClearML (we did not upgrade from the Trains AMI as started fresh)

that should be the case, we have default_output_uri: set to an s3 bucket

Morning, we got to 100% used which is what triggered this investigation. When we initially looked at overlay2 it was using 8GB, so now seems to be acceptable.

not yet, going to try and fix it today.

if I do a df I see this, which is concerning:

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.9G     0  3.9G   0% /dev
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           3.9G  928K  3.9G   1% /run
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/nvme0n1p1   20G  7.9G   13G  40% /
tmpfs           790M     0  790M   0% /run/user/1000

so it looks like the mount points are not created. When do these g...

Hi @<1523701205467926528:profile|AgitatedDove14>
Yes the clearml-server AMI - we want to be able to back it up and encrypt it on our account

Is it possible to use an IAM role rather than user credentials in the clearml.conf file?

thank you

Yep i've done all that, it didn't seem to work until I set the deploy key to write

all sorted, I somehow missed the documentation about the mongodb migration

Thanks. Although it's AWS related, the context was with an error we see within clearml "ValueError: Insufficient permissions for None "

Thanks. Where is the configuration stored on the server? Currently we have deployed an EC2 instance using the marketplace AMI - if this demo is successful we would be looking at splitting the environment into the different AWS services - logging to S3, use of secrets manager, elasticsearch, redis, mongoDB etc

I've looked through the documentation, but didn't initially spot anything that would help with doing this (granted I may have overlooked something)

Is there a way you can allow our account to make a copy of the AMI and store it privately?