I basically would like to know if we can serve the model without tensorrt format which is highly efficient but more complicated to get.

On the helm charts clearml repos, can we use the clearml-serving chart alone ?

Yes I take the export statements from my bash script of the task

One possible solution I could see as well, is putting the data storage to S3 bucket to improve download performance as it is the same cloud provider. No transfer latency.

Related to that,
Is it possible to do Dataset.add_external_files() with source_url and destination_url being two separate azure storage containers?

Okey thanks @<1523701205467926528:profile|AgitatedDove14> and what would be the advantage of using clearm-server on k8s compared to the clearml hosted one?

@<1523701070390366208:profile|CostlyOstrich36> @<1523701205467926528:profile|AgitatedDove14> Any ideas on this one?

I do not remember, but I was afraid.... Thanks for the output ! Maybe in a bad dream ? 😜

I still do not get the K8s clearml server usefulness of it then?

How to set that up inside clearml.conf or something else to know which credentials to load?

How to make sure that the python version is correct?

Yes same error

Because I was ssh-ing to it before the fail. When poetry fails, it installs everything using PIP

Yes should be correct. Inside the bash script of the task.

I will check that. Do you think we could bypass it using Task.create ? And passing all the needed params?

If I may ask as well for another issue in that thread that is taking me a big amount of time:

Poetry Enabled: Ignoring requested python packages, using repository poetry lock file!
Creating virtualenv alfred-Rp77Shgw-py3.9 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file
2023-04-17 10:17:57
Package operations: 351 installs, 1 update, 1 removal
failed installing poetry requirements: Command '['poetry', 'install', '-n']' returned non-zero exit status 1.
Ignorin...

Thanks ! So regarding question2, it means that I can spin up a K8s cluster with triton enabled, and by specifiying the type of model while creating the endpoint, it will use or not the triton engine.
Linked to that, Is the triton engine expecting the tensorrt format or is it just an improvement step compared to other model weights ?

Finally, last question ( I swear 😛 ) : How is the serving on Kubernetes flow supposed to look like? Is it something like that:

• Create en...

In production, we should use the clearml-helm-charts right? Docker-compose in the clearml-serving is more for local testing

Using a pyenv virtual env then exporting LOCALPYTHON env var

I also did that in the following way:

• I put a sleep inside the bash script
• I ssh-ed to the fresh container and did all commands myself (cloning, installation) and again it worked...

but I still had time to go inside the container, export the PATH variables for my poetry and python versions, and run the poetry install command there

How do you explain that it works when I ssh-ed into the same AWS container instance from the autoscaler?

When the task finally failed, I was kicked of from the container

It just allows me to have access to poetry and python installed on hte container

@<1523701070390366208:profile|CostlyOstrich36> poetry is installed as part of the bash script of the task.

The init script of the AWS autoscaler only contains three export variables I set.

@<1523701087100473344:profile|SuccessfulKoala55> Do you think it is possible to ask to run docker mode in the aws autoscaler, and add the cloning and installation inside the init bash script of the task?

Task.set_base_docker 🙂

@<1523701118159294464:profile|ExasperatedCrab78> do you have any inputs for this one? 🙂

@<1523701070390366208:profile|CostlyOstrich36> The base docker image of the AWS autoscaler is nvidia/cuda:10.2-runtime-ubuntu18.04 . According to me, the python version is not set inside the image, but I am might be wrong and it could be the problem indeed... ?

The flask command is ran inside the git project, which is the strange behavior. It is executed in ~/code/repo/ as flask train ...