CornyDeer86 you are the best! AnxiousSeal95 thanks for your help also, we have been trying to solve this problem for a long time, and now it works like a magic.
conclusions:
minimal policy needed for the auto-scaling service if we want to specify security group and subnet for RunInstances action permissions and also to use spot instances (as far as we found until now):
still one problem is existing - AWS doesn't support specifications of resources for DescribeInstances/RequestSpotInstances Actions{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "ec2:CancelSpotInstanceRequests", "ec2:RequestSpotInstances", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeInstances" ], "Resource": "*", "Condition": { "StringEquals": { "aws:RequestedRegion": "{region}" } } }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "ec2:TerminateInstances", "ec2:DeleteTags", "ec2:StartInstances", "ec2:CreateTags", "ec2:RunInstances", "ec2:StopInstances", "ec2:GetConsoleOutput" ], "Resource": [ "arn:aws:ec2:{region}:{user_id}:network-interface/*", "arn:aws:ec2:{region}:{user_id}:subnet/subnet-{subnet_id}", "arn:aws:ec2:{region}:{user_id}:key-pair/*", "arn:aws:ec2:{region}:{user_id}:instance/*", "arn:aws:ec2:{region}:{user_id}:volume/*", "arn:aws:ec2:{region}:{user_id}:security-group/sg-{security_group_id}" ], "Condition": { "StringEquals": { "aws:RequestedRegion": "{region}" } } }, { "Sid": "VisualEditor2", "Effect": "Allow", "Action": "ec2:RunInstances", "Resource": "arn:aws:ec2:{region}::image/ami-{ami_id}", "Condition": { "StringEquals": { "ec2:Owner": "amazon" } } } ] }
CostlyOstrich36 AnxiousSeal95
So ,when I was added specific configuration it failed.
Is there a way to lower the needed credentials for specific actions such as: run, stop, start instances etc...? for example: fixing it to work only with conditions of specific subnet, security group and instance types? ( I was trying doing it but as I said it failed with this message:An error occurred (UnauthorizedOperation) when calling the RunInstances operation: You are not authorized to perform this operation. )
Also, I think that maybe there is a bug with the CPU mode: I tried to run tests with instance without GPU , marked the option "Run in CPU mode (no gpus)" and I saw on the experiment logs that its trying to run the docker with "--gpus all" option and failed right after the execution.
Is there a way to lower the needed credentials for specific actions such as: run, stop, start instances etc...? for example: fixing it to work only with conditions of specific subnet, security group and instance types? ( I was trying doing it but as I said it failed with this message:
Can you elaborate on the specific configuration?
Also, I think that maybe there is a bug with the CPU mode: I tried to run tests with instance without GPU , marked the option "Run in CPU mode (no gpus)" and I saw on the experiment logs that its trying to run the docker with "--gpus all" option and failed right after the execution.
Which instance type did you use?
SmugTurtle78 , regarding the CPU only mode - How are you running. Are you using the application in PRO version or are you running through one of the examples?
It is working with "Resource": "*" , I will try to use it and maybe use deny for specific configuration, thanks :)
About the CPU mode, used t3.medium...
About the specific configuration, of course- was trying this policy for example ( when I remove the ec2-vpc condition it works){ "Sid": "GeneralEC2", "Effect": "Allow", "Action": [ "ec2:AttachClassicLinkVpc", "ec2:CancelSpotInstanceRequests", "ec2:CreateFleet", "ec2:Describe*", "ec2:GetConsoleOutput", "ec2:DetachClassicLinkVpc", "ec2:ModifyInstanceAttribute", "ec2:RequestSpotInstances" ], "Resource": "*", "Condition": { "StringEquals": { "aws:RequestedRegion": "{region}" } } }, { "Sid": "RunEC2", "Effect": "Allow", "Action": [ "ec2:RunInstances", "ec2:CreateTags", "ec2:DeleteTags", "ec2:StartInstances", "ec2:StopInstances", "ec2:TerminateInstances" ], "Resource": "*", "Condition": { "StringEquals": { "aws:RequestedRegion": "{region}", "ec2:vpc": "arn:aws:ec2:{region}:{user_id}:vpc/vpc-{subnet_id}" } } }
No, I want to use AWS user with much lower credentials ( Only to relevant for the autoscaler missions)- for example Describe/ RequestSpotInstance/ StopInstances permissions only for the relevant subnet, security group and instance types..
CostlyOstrich36 I would like to add more conditions such as security groups and instance types- this is only an example :)
Hi CostlyOstrich36 AnxiousSeal95 , Do you have any idea ?
SmugTurtle78 , I'll take a look at it shortly 🙂
Hi, SmugTurtle78 ,
Can you please try with "Resource": "*" ?
Also these are the settings that I use, Some might be redundant so consults with your devops guys 🙂{ "Sid": "EC2InstanceManagement", "Effect": "Allow", "Action": [ "ec2:AttachClassicLinkVpc", "ec2:CancelSpotInstanceRequests", "ec2:CreateFleet", "ec2:CreateTags", "ec2:DeleteTags", "ec2:Describe*", "ec2:DetachClassicLinkVpc", "ec2:ModifyInstanceAttribute", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:StartInstances", "ec2:StopInstances", "ec2:TerminateInstances" ], "Resource": "*" }
Hi SmugTurtle78 , sorry for answer in slow-mo 😉 I'm not 100% sure I got the question... you want a a global security group and network for the entire autoscaler instead of per-instance type?
Hi SmugTurtle78 , Could you spin up an instance with the same user from AWS CLI?
Can you try launching a new instance with CPU only and add the log here? I just tried on PRO myself with CPU only and it worked. Can you look at the version of the application you're running with? To see the version you have inside the screen of the application on the top left a small highlighted text "more" if you click on it some of the text will scroll down and show you the version
I tried again and now it is working ,The version is: v1.4.0.
about the options of subnet & security group, I saw it already, but I use it but still I want to give the app lower policy that enable it to run only this network and this security group. CostlyOstrich36
AnxiousSeal95 Maybe you could help me? 🙂
Also in applications I see an option for subnet ID & security group