I see, but to actually serve both models/sessions at the same time, it would require two inference containers, as each inference container can only serve one session at a time?

Hi,
Yes that is an issue for me, even if we could centralize an environment today, it leaves a concern whenever we add a model that possible package changes are going to cause issues with older models. Also it would be nice to have a more direct link between the saved model objects and their serving environment.

Would the recommendation be to spin up multiple inference containers? Also is there a built in way to separate the preprocessing and model inference into separate containers? Part of...