The task pod (experiment) started reaching out to an IP associated with malicious activity. The IP was associated with 1000+ domain names. The activity was identified in AWS guard duty with a high severity level.

BoredHedgehog47 What is the pod container itself ?
EDIT:
Are you suggesting the default "ubuntu:18.04" is somehow contaminated ?
https://hub.docker.com/layers/library/ubuntu/18.04/images/sha256-d5c260797a173fe5852953656a15a9e58ba14c5306c175305b3a05e0303416db?context=explore

The domain names were bogus sounding businesses

"ipAddressV4": "165.160.15.20", "organization": { "asn": "19574", "asnOrg": "CSC", "isp": "Corporation Service Company", "org": "Corporation Service Company" }, "country": { "countryName": "United States" }, "city": { "cityName": "" }, "geoLocation": { "lat": 37.751, "lon": -97.822 } }, "remotePortDetails": { "port": 80, "portName": "HTTP" }, "localPortDetails": { "port": 9134, "portName": "Unknown" }, "protocol": "TCP", "blocked": false, "localIpDetails": { "ipAddressV4": "10.32.2.13" }

yes

For task execution? It would work, yes, however it would take longer to install, I assume

"additionalInfo": { "inBytes": "438", "localPort": "9134", "outBytes": "401", "unusual": "80", "value": "{\"inBytes\":\"438\",\"localPort\":\"9134\",\"outBytes\":\"401\",\"unusual\":\"80\"}", "type": "default" },

Important part here: Malicious IP: 165.160.15.20

Would using 22.04 Ubuntu still work in the task execution?

This is where we saw the malicious activity

The task pod (experiment) started reaching out to an IP associated with malicious activity. The IP was associated with 1000+ domain names. The activity was identified in AWS guard duty with a high severity level.

Can you share some details? I would really like to get to the bottom of that...

During the task pod runtime

This image is used for the task pods, not the agent pod.

Yes

"title": "Unusual outbound communication seen from EC2 instance i-<> on server port 80.",

Are you suggesting the default "ubuntu:18.04" is somehow contaminated ?

This is an official Ubuntu container (nothing to do with ClearML), this is Very Very odd...

The task pod (experiment) started reaching out to an IP associated with malicious activity. The IP was associated with 1000+ domain names. The activity was identified in AWS guard duty with a high severity level.

What malicious activity?