will do

use_current_task is False by default, let me see if this helps

I think I figured this out but now I have a problem:

auto_connect_frameworks={ 'xgboost': False, 'scikitlearn': False }

it worked!!!!

Why do I need an output_uri for the model saving? The dataset API can figure this out on its own

clearml-serving needs a "Preprocess" class in a file that is added to the endpoint.

if I swap it to Framework then it autocrecords both

Ok, but I need to think with the head of the DS, this way they only need to remember (and me only need to teach them where to find) one id.

I expect the task to be the main entry point for all their work, and the above interface is easy to remember, check etc etc. Also it is the same as getting artifacts so less friction.

` def get_task(task_id):
return Task.get_task(task_id)

def get_artifact(task_id, artifact_name):
task = Task.get_task(task_id)
return task.artifacts[artifact...

I solve the artifact,dataset,table,scalar anything by simply making foo.run()
return a dictionary like
{ 'artifact_X_train': X_train, 'table_confusion_matrix: cf, 'dataset_x': _ ... }And then call the appropriate logger or artifact uploader or dataset uploader. (In case the dataset uploader I use the foo.output_file which every foo has.

yeah

the current diffs

https://github.com/allegroai/clearml-serving/blob/main/examples/xgboost/preprocess.py

No, pickling is the only thing that will Not trigger clearml (it is just too generic to automagically log)

So what is the mechanism that you "automagically" pick things up (for information, I don't think this is relevant to our usecase)

but here I can tell them: return a dictionary of what you want to save

task = Task.init(...) for foo in foos: data = foo.run() for key,value in data.items(): if key in 'auc', 'f1', etc: logger.log(key, value) elif key.startswith('model'): savemodel etc

TBH ClearML doesn't seem to be picking the model up so I need to do it manually

OSS version

but I am one level lower than top. so:

~/work/repo is the main repo dir

~/work/repo/code/run.py and I am running python run.py in ~/work/repo/code

And then get_model is what I wrote above, just uses the CML API to pick up the right model from the task_id and model_name and the model config contains the class name so get_model has an if/else structure in it to create the right class.

This was not something I was expecting to break.

If you enable nbdime globally and switch virtual environments, then git diff will fail.

docker-compose I guess

Apparently our devops guy figured it out that you needed to have a different port number and a different docker container given 8080 was already occupied

I know there is a aux cfg with key value pairs but how can use it in the python code?
"auxiliary_cfg": { "TASK_ID": "b5f339077b994a8ab97b8e0b4c5724e1", "V": 132 }

these are the service instances (basically increased visibility into what's going on inside the serving containers

But these have: different task ids, same endpoints (from looking through the tabs)

So I am not sure why they are here and why not somewhere else

everything is weird about this. I put two models in the same endpoint, then only one was running, then I started another docker container having a different port number and then the curls with the new model endpoint (with the new port) started working

I don't understand the link between service id-s, service tasks and docker containers

and immediately complained about a package missing, which apparently I can't specify when I establish the model endpoint but I need to re compose the docker container by passing an env variable to it????

but is it true that I can have multiple models on the same docker instance with different endpoints?

When you spin the model you can tell it any additional packages you might need

What does spin mean in this context?

clearml-serving ... ?

wait