Yes, I resolved it. So basically we need to give docker containers extra privileges. add --privileged for single container and in case of docker-compose.yml and privileged: true.
I have mounted my s3 bucket at the location /opt/clearml/data/fileserver/ but I can see my data is not being stored in s3 but its storing in ebs. How so?
I'm assuming the mount was not successful
What you should see is a link to the files server inside clearml, and actual files in your S3 bucket
Hi @<1624579015031394304:profile|JitterySeal56>
... and credentials in clearml.conf file on client side, but I have restrictions of aws keys expiring each hour
This means that you need to configure IAM role on your client machine, the data never goes through the server it is uploaded directly from the dev machine to the S3 bucket.
You can however just store the data on your clearml-files server ...
Check the links that are generated in the ui when you upload an artifact or model
@<1523701205467926528:profile|AgitatedDove14> I was able to mount but if I am running the experiment and uploading data it is going to s3 as well as ebs. Why so it should only go to s3
I can't configure my local machine for IAM Role.
Ok so if we can't send the data to s3 through server, can I mount the s3 bucket as file system on place where dataset is uploaded on server will it solve this problem?
data it is going to s3 as well as ebs. Why so it should only go to s3
This sounds odd, if this is mounted then it goes to the S3 (the link will point to the files server, but it will be stored on the mounted drive i.e. S3)
wdyt?
I have mounted my s3 bucket at the location /opt/clearml/data/fileserver/ but I can see my data is not being stored in s3 but its storing in ebs. How so? It should only be stored in s3. Is there mounting issue in my solution?
can I mount the s3 bucket as file system on place where
you need to mount it where the file server is storing it's files, correct (notice, not the DBs, just the files server)