Hi UnevenDolphin73 , queues don't have any notion of configuration. Agent running tasks remotely use their own configuration file to execute tasks.

Than yes 😄

I guess it does not do so for all settings, but only those that come from Session()

Well, let me check 🙂

Okay so the only missing thing of the puzzle I think is that it would be nice if this propagates to the autoscaler as well; that then also allows hiding some of the credentials etc 😮

Let me know if you do; would be nice to have control over that 😁

Yeah, obviously it assumes everyone will be talking to the same server 🙂

propagates meaning the autoscaler will use it to spin cloud instances?

Right, but that's as defined in the services agent, which is not immediately transparent

Actually SuccessfulKoala55 , there is something like that happening behind the scenes.
I have an AWS Autoscaler running on a services queue, so the autoscaler inherits the configuration used by the services agent, right?
Now, when my autoscaler launched new EC2 instances, they used the same fileserver as the one that was defined in the services agent too 🤔

Well, that's actually not the current behaviour, and I'm pretty sure we had a good reason for that - I'll try to remember why 🙂

Hi UnevenDolphin73 , as far as I can remember, the autoscaler does not use its own configuration as the default for the instances it spins up

Exactly; the cloud instances (that are run with clearml-agent ) should have that clearml.conf + any changes specified in extra_clearml_configuration for the scaler

Holy crap this was a light-bulb moment, is this listed somewhere in the docs?
It solves so much of my issues xD

and I don't think it's in the docs - we'll add that

Sorry, I misspoke, yes of course, the agents config file, not the queues

As in here SuccessfulKoala55 - https://github.com/allegroai/clearml/blob/master/clearml/automation/cloud_driver.py#L169

and I don't think it's in the docs - we'll add that

Very welcome update, please use some highlighting for it too, it's so important for a complete understanding of how the remote execution works