Like I said above, we want to mount an external EBS volume to the /opt/clearml/data
directory. However, upon the creation of a net new EC2 instance using ClearML's community AMI, this directory already contains sub-directories and files due to the ClearML containers already being up and running. Thus, we can't mount an EBS volume to this directory until we (1) run a docker-compose down
command to stop the running containers and (2) delete the pre-existing files found in the /opt/clearml/data
directory. After having performed those two steps, we are able to mount an external EBS volume to the directory without issue and start the docker containers again with a docker-compose up
command, which then repopulates the necessary sub-directories/files into the /opt/clearml/data
.
The only problem we are facing with this approach is that the clearml-elastic
container repeatedly fails to start up when following these steps, while the other containers have no issue. Why is that? I have attached container logs from the clearml-elastic
container in the file below.