AgitatedDove14 we will try to use Triton, but it’s a bit hard with transformer model.
All extra packages we add in serving)
We made now like this. We in preprocessor.py load our model from S3 bucket and then use it. But it’s maybe not the best solution