This is the log i extracted.

unfortunately, our security posture is so strict that we cannot have an agent git user that have unfettered read access to all repos.

clearml-serving does not support Spacy models out of the box among many others and that Clearml-Serving only supports following;
Support Machine Learning Models (Scikit Learn, XGBoost, LightGBM)
Support Deep Learning Models (Tensorflow, PyTorch, ONNX).
An easy way to extend support to different models would be a boon.

I believe in such scenarios, a custom engine would be required. I would like to know, how difficult is it to create a custom engine with clearml-serving? For example, in this...

Hi SuccessfulKoala55 ,i managed to install clearml-agent==1.0.1rc5. However, the same issues occur.

ah thanks. Hopefully the old ones get flushed out by Google soon.

yeah, someone should call them out.

I use the k8s glue.

Hi SuccessfulKoala55 , can i confirm the following comments in the docker-compose.yml ?
And after that to run docker-compose commands without loss of data?

docker-compose down docker-compose up
docker-compose.yml
`
version: "3.6"
services:

apiserver:
command:
- apiserver
container_name: clearml-apiserver
image: allegroai/clearml:latest
restart: unless-stopped
volumes:
- /opt/clearml/logs:/var/log/clearml
- /opt/clearml/config:/opt/clearml/config
#...

From ClearML perspective, how would we enable this, considering we don't have direct control or even IP of the agents

Hi, i was reading this thread and wondered which version of clearml-server and clearml-agent has this taken effect with?

Hi this is the log. I didn't see any attempt from the agent to install virtualenv on the base image.
` 1618369068169 clearml-gpu-id-b926b4b809f544c49e99625380a1534b:gpuGPU-4ad68290-0daf-4634-6768-16fad73d47a3 DEBUG Current configuration (clearml_agent v0.17.2, location: /tmp/.clearml_agent.wgsmv2t9.cfg):

agent.worker_id = clearml-gpu-id-b926b4b809f544c49e99625380a1534b:gpuGPU-4ad68290-0daf-4634-6768-16fad73d47a3
agent.worker_name = clearml-gpu-id-b926b4b809f544c49e99625...

Previously we had similar issues when we switched images used in agent. Might want to check on that.

Ok, that seems clearer, thanks.

The doc also mentioned preconfigured services with selectors in the form of
"ai.allegro.agent.serial=pod-<number>" and a targetPort of 10022. Would you have any examples of how to do this?

I think a related question is, ClearML replies heavily on Triton (Good thing) but Triton only support a few frameworks out of the box. So this 'engine' need to make sure its can work with Triton and use all its wonderful features such as request batching, GPU reuse...etc.

I also think it make sense that when you do certain definitive CI actions like publish, it would support some custom scripts to run.

Hi FriendlySquid61 , AgitatedDove14 , the issue and possible fix is in this issue raise. https://github.com/allegroai/clearml-agent/issues/51

Any comments on using the global python libraries without the need to 'pip install' anything?

Hi SuccessfulKoala55 I was refering to the Task.init() or any other SDK API that we use in our training codes.

No, i can't see the files. But i can see if i don't use ':port' in the URL when uploading. I can't access the machine today, i'll try to check the S3 logs when i'm back.

thanks.

Hi, currently the ClearML SDK only supports python. If i want to run my ML in other languages, can i use a SDK in that language? Or is there other means such as a Web API calls that does the same as the SDK?

Thanks TimelyPenguin76 , is there an env var for the S3 connection as well?

Thanks this would be a good alternative before the enterprise version comes in. How is this different from argparser btw?

alright. 🙂

Is there anyway to see an error log from that?

Sorry AgitatedDove14 can you bump me to that thread?

Although I think you can also pull specific chunks of dataset

How do you do that with clearml-data?

Transform feature engineering and data processing code into recurring data ingestion workflows. Start building data stores, develop, automate, and schedule complex data processing jobs.

Create immutable and differentiable versions on-prem or in the cloud with our data agnostic solution.