Unanswered
Hi All, I Am Trying To Deploy Clearml Solution On Top Of Amazon Ecs With Fargate. I Am Using Efs For My Data Persistence Layer Where I Have Created Multiple Access Points For Different Components That Need To Be Deployed As Part Of The Clearml Solution. H
Hi @<1523701087100473344:profile|SuccessfulKoala55> , I believe you are right in terms of the paths being internal to the container. It could be that the the integration between the EFS filesystem and the Fargate task is causing issues as well.
here are some of the example of the errors thrown from CloudWatch:
update_from_env.py: error: argument output_file: can't open '/usr/share/nginx/html/configuration.json': [Errno 13] Permission denied: '/usr/share/nginx/html/configuration.json'
/opt/clearml/wrapper.sh: line 82: /etc/nginx/sites-enabled/default: Permission denied
runc create failed: unable to start container process: exec: "/opt/clearml/wrapper.sh": stat /opt/clearml/wrapper.sh: no such file or directory
2024/04/01 13:49:06 [crit] 10#10: pread() "/etc/nginx/sites-enabled/clearml" failed (21: Is a directory)
update_from_env.py: error: argument output_file: can't open '/usr/share/nginx/html/configuration.json': [Errno 13] Permission denied: '/usr/share/nginx/html/configuration.json'
These logs are all from different errors that were shown on different occasions. The changes that I have been making between these runs are mounting different mount points from the EFS Access Point to the Fargate task.
Below is one example of the Fargate task definition that I have tried to run:
{
"taskDefinitionArn": "arn:aws:ecs:eu-west-1:<account-id>:task-definition/LocalDeploymentStageClearMLWebServiceStackstorageServicetasktaskdefinition3F87B49C:11",
"containerDefinitions": [
{
"name": "clearml-webserver",
"image": "allegroai/clearml:latest",
"cpu": 0,
"portMappings": [
{
"name": "web",
"containerPort": 8080,
"hostPort": 8080,
"protocol": "tcp"
}
],
"essential": true,
"command": [
"webserver"
],
"environment": [],
"environmentFiles": [],
"mountPoints": [
{
"sourceVolume": "webData",
"containerPath": "/var/log/clearml",
"readOnly": false
},
{
"sourceVolume": "webData",
"containerPath": "/usr/share/nginx/html",
"readOnly": false
},
{
"sourceVolume": "webData",
"containerPath": "/etc/nginx/sites-enabled/",
"readOnly": false
},
{
"sourceVolume": "webData",
"containerPath": "/opt/clearml",
"readOnly": false
}
],
"volumesFrom": [],
"user": "1000",
"dockerSecurityOptions": [],
"ulimits": [],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "-Local-DeploymentStage-ClearMLWebServiceStack-storageServicetasktaskdefinitionclearmlwebserverLogGroup-bQPMOGsHG7rj",
"awslogs-region": "eu-west-1",
"awslogs-stream-prefix": "/clearml/clearml-webserver"
},
"secretOptions": []
},
"systemControls": [],
"credentialSpecs": []
}
],
"family": "LocalDeploymentStageClearMLWebServiceStackstorageServicetasktaskdefinition3F87B49C",
"taskRoleArn": "arn:aws:iam::<account-id>:role/-Local-DeploymentStag-storageServiceTaskRole63F-DFomn9heIMJt",
"executionRoleArn": "arn:aws:iam::<account-id>:role/-Local-DeploymentStag-storageServicetasktaskdef-oYg0bEXy3X2Z",
"networkMode": "awsvpc",
"revision": 11,
"volumes": [
{
"name": "webData",
"efsVolumeConfiguration": {
"fileSystemId": "fs-123",
"rootDirectory": "/",
"transitEncryption": "ENABLED",
"authorizationConfig": {
"accessPointId": "fsap-123",
"iam": "ENABLED"
}
}
}
],
"status": "ACTIVE",
"requiresAttributes": [
{
"name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
},
{
"name": "ecs.capability.execution-role-awslogs"
},
{
"name": "ecs.capability.efsAuth"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
},
{
"name": "ecs.capability.efs"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
},
{
"name": "com.amazonaws.ecs.capability.task-iam-role"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.25"
},
{
"name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
},
{
"name": "ecs.capability.task-eni"
}
],
"placementConstraints": [],
"compatibilities": [
"EC2",
"FARGATE"
],
"requiresCompatibilities": [
"FARGATE"
],
"cpu": "512",
"memory": "1024",
"registeredAt": "2024-03-31T19:29:15.512Z",
"registeredBy": "arn:aws:sts::<account-id>:assumed-role/Admin/",
"tags": []
}
96 Views
0
Answers
7 months ago
7 months ago