I want the model to be stored in a way that clearml-serving can recognise it as a model

Then OutputModel or task.update_output_model(...)
You have to serialize it, in a way that later your code will be able to load it.
With XGBoost, when you do model.save clearml automatically picks and uploads it for you
assuming you created the Task.init(..., output_uri=True)
You can also manually upload the model with task.update_output_model or equivalent with OutputModel class.
if you want to dis...

Hi @<1541954607595393024:profile|BattyCrocodile47>

It seems to me that instead of implementing webhooks to react to things like adding a tag to a model

Did you look at this example ?
None

Can we straightforwardly stream ALL ClearML events to another system?

what would you consider an event?
The "basic" object type is Task, a state in task is changed via an api call, would that be an e...

Yes please 🤩

Assuming git repo looks something like:
.git readme.txt module | +---- script.pyThe working directory should be "."
The script path should be: "-m module.scipt"
And under the Configuration/Args, you should have:
args1 = value args2 = another_value
Make sense?

GreasyPenguin14 yes there is 🙂
https://github.com/allegroai/clearml/issues/209
Set environment variable CLEARML_NO_DEFAULT_SERVER=1

in 

 issues a delete command to the ClearML API server,...

almost, it issues the boto S3 delete commands (directly to the S3 server, not through the cleaml-server)

And that I need to enter an AWS key/secret in the profile page of the web app here? (edited)

correct

Is there a way to force clearml not to upload these models?

DistressedGoat23 is it uploading models or registering them? to disable both set auto_connect_frameworks https://clear.ml/docs/latest/docs/clearml_sdk/task_sdk#automatic-logging

Their name only contain the task name and some unique id so how can i know to which exact training

You mean the models or the experiments being created ?

This is definitely a but, in the super class it should have the same condition (the issue is checking if you are trying to change the "main" task)
Thanks ApprehensiveFox95
I'll make sure we push a fix 🙂

it is just local copy so you can rerun and reconfigure

TrickyFox41 are you saying that if you add Task.init inthe code it works, but when you are calling "clearml-task" it does not work? (in both cases editing the Args/overrides ?

Hi GiganticTurtle0
Let me check

what just happened next time and what is happening underneath.

Not sure I follow, is there still an issue ?

BoredHedgehog47 you need to configure the clearml k8s glue to spin pods (instead of allocating agents per pods statically) does that make sense ?

Thank you WackyRabbit7 please feel free to remind me if it slips away during my night time (yes I do sleep , contrary to common belief :))

what is the best approach to update the package if we have frequent update on this common code?

since this package has an indirect affect on the model endpoint, I would package with the preprocess code of the endpoint.
Each server is updating it's own local copy, and it will make sure it can take it and deploy it hand over hand without breaking its ability to serve these endpoints.
the "wastefulness" of holding multiple copies is negligible when comparing to a situation where everyone ...

Yep that makes sense to me too. Done

But I believe it would be harder for our team to detect and respond to failures in the event handler functions if they were placed there because it seems unclear how we could use our existing systems and practices to do that.

Okay I think this is the issue, handler functions are not "supposed" to fail, they are supposed to trigger Tasks, these can fail.
e.g.:
Model Tag Trigger -> handler function creates a Task -> Task does something, like build container, trigger CI/CD etc -> Task...

CrookedWalrus33 this is odd I tested the exact same code.
I suspect something with the environment maybe?
Whats the python version / OS ? also can you send full pipe freeze?
2022-07-17 07:59:40,339 - clearml.storage - ERROR - Failed uploading: Parameter validation failed: Invalid type for parameter ContentType, value: None, type: <class 'NoneType'>, valid types: <class 'str'>Yes this is odd, it should add the content-type of the file (for example "application/x-tar" but you are getting N...

Hi ReassuredTiger98

However, the clearml-agent also stops working then.

you mean the clearml-agen daemon (the one that spinned the container) is crashing as well ?

So basically the APIClient is a pythonic interface to the RestAPI, so you can do the following
See if this one works
# stats from he last 60 seconds for worker in workers: print(client.workers.get_stats(worker_ids=[worker.id], from_date=int(time()-60),to_date=int(time()), interval=60, ))

Hi AstonishingWorm64
I think you are correct, there is external interface to change the docker.
Could you open a GitHub issue so we do not forget to add an interface for that ?
As a temp hack, you can manually clone "triton serving engine" and edit the container image (under the execution Tab).
wdyt?

at least you did not change permission of your K8s etcd folder 😄

Hi TenderCoyote78

I'm trying to clearml-agent in my dockerfile,

I'm not sure I'm following, Are you traying to create a docker container containing the agent inside? for what purpose ?
(notice that the agent can spin any off the shelf container, there is no need to add the agent into the container it will take of itself when it is running it)

Specifically to your docker file:

RUN curl -sSL

| sh

No need for this line

COPY clearml.conf ~/clearml.conf

Try the ab...

sure 🙂

but it still not is able to run any task after I abort and rerun another task

When you "run" a task you are pushing it to a queue, so how come a queue is empty? what happens after you push your newly cloned task to the queue ?

Hi AdventurousRabbit79
In the wizard
https://github.com/allegroai/clearml/blob/1ab3710074cbfc6a19dd8a57078b10b31b2df31a/examples/services/aws-autoscaler/aws_autoscaler.py#L214

Add the S3 section like you would in the clearml.conf:
https://github.com/allegroai/clearml/blob/1ab3710074cbfc6a19dd8a57078b10b31b2df31a/docs/clearml.conf#L73

So could you re-explain assuming my piepline object is created by 

pipeline = PipelineController(...)

?

pipe.add_step(name='stage_train', parents=['stage_process', ], monitor_artifact=['my_created_artifact'], base_task_project='examples', base_task_name='pipeline step 3 train model', parameter_override={'General/dataset_task_id': '${stage_process.id}'})This will put the artifact names "my_created_artifact" from the step Tas...

main clearml repo?

Yep that sounds right 🙂 thank you!

LOl my pleasure - I guess we should have a link in the doc string of add_requirements to set_packages , I will tell the guys

Python3.8 I can quickly check, give me a minute