Yes that is an issue for me, even if we could centralize an environment today, it leaves a concern whenever we add a model that possible package changes are going to cause issues with older models.

yeah changing the environment on the fly is tricky, it basically means spinning an internal http service per model...

Notice you can have many clearml-serving-sessions, they are not limited, so this means you can always spin new serving with new environments. The limitation is changing an environment on the fly

Would the recommendation be to spin up multiple inference containers?

kind of, yes spin multiple clearml-serving-session, essentially each session has it's own environment, and in that environment you can add / remove models on the fly