Maybe you've already seen this and it doesn't help but:
Based on ClearML's S3 setup guide , it looks like you can handle 1 and 2 with Boto3 and have config that sets the keys per session / profile so they would be picked up from a shared config file or aws config file. Not 100% sure if that will avoid the credential expiry mid-task issue but I think it should, maybe with some additional code for you to run a check for cred updates if you get kicked out of your session.
Apologies if I'm repeating things you've read but hope this helps a bit!