I want the model to be stored in a way that clearml-serving can recognise it as a model

Then OutputModel or task.update_output_model(...)
You have to serialize it, in a way that later your code will be able to load it.
With XGBoost, when you do model.save clearml automatically picks and uploads it for you
assuming you created the Task.init(..., output_uri=True)
You can also manually upload the model with task.update_output_model or equivalent with OutputModel class.
if you want to dis...

I would do something like:

` from clearml import Logger

def forward(...):
    self.iteration += 1 
    weights = self.compute_weights(...)
    m = (weights * (target-preds)).mean()
    Logger.current_logger().report_scalar(title="debug", series="mean_weight", value=m, iteration=self.iteration)
    return m `

export CLEARML_DEFAULT_OUTPUT_URI="https://...."Make sense ?

Not at all, we love ideas on improving ClearML.
I do not think there is a need to replace feast, it seems to do a lot, I'm just thinking on integrating it into the ClearML workflow. Do you have a specific use case we can start to work on? Or maybe a workflow that would make sense to implment?

trains-agent RC (which they tell me will be out tomorrow) will have a switch to do that, just so it is easier 🙂

DilapidatedDucks58
did you check:
https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py

Where are you seeing this message?

CooperativeFox72 you can you start by checking the latest RC :)
pip install trains==0.15.2rc0

with conda ?!

and run it locally...

You mean you run the clearml-agent locally ?

Check the log to see exactly where it downloaded the torch from. Just making sure it used the right repository and did not default to the pip, where it might have gotten a CPU version...

Hi TightElk12
Are you looking for a way to set the output_uri from environment variable ? Is this it?

Hi EnthusiasticCoyote38

But one one process finished it changed task status to complete. May be you know some save way to deal with such situation? Or maybe the best way to check task status before upload object?

Well, you can actually forcefully set the state of the Task to running, then add artifacts, then close it?
would that work?

` my_other_task.reload()
my_other_task.mark_started(force=True)
my_other_task.upload_artifact(...)
my_other_task.flush(wait_for_uploads=True)
my_othe...

In both case if I get the element from the list, I am not able to get when the task started. Where is info stored?

If you are using client.tasks.get_all( ...) should be under started field
Specifically you can probably also do:
queried_tasks = Task.query_tasks(additional_return_fields=['started']) print(queried_tasks[0]['id'], queried_tasks[0]['started'],)

it would be nice to group experiments within projects

DilapidatedDucks58 you mean is collapse/expand ? or in something like "sub-project" ?

what if for some old tasks I get WARNING:root:Could not delete Task ID=a0908784a2a942c3812f947ec1f32c9f, 'Task' object has no attribute 'delete'? What's the best way of cleaning them?

This seems like an old SDK no?

Hmm reading this: None
How are you checking the health of the serving pod ?

Hi EnchantingOstrich20
You how doe s clearml get it there?
In runtime it analyzes the code you are running looking for imports then checks the version you have actively used (i.e. active venv / python) and lists it there.
You can also override those in code, or edit them after you clone the ask and before you enqueue it for remote execution

and you have clearml v0.17.2 installed on the "system" packages level, and 0.17.5rc6 installed inside the pyenv venv ?

parser.add_argument( "--dataset_mean", type

=

float, nargs

=

"+", default

=

0.5)

I think providing nargs='+ ' assumes the type is a list. nonetheless we should be able to support it. Could you please add a GitHub issue so we do not forget ?

on the side note, is there any way to automatically give more meaningful names to the running docker containers?

What do you mean by that? running where? and where will you see them ?

https://allegro.ai/docs/references/trains_ref/#agent-section

Hmm seems like everything is working, can you check in the UI if you see the serving session ID in the DevOps project? maybe there are two, and you configured one an dthe docker-compose is running another ?

Hi @<1695969549783928832:profile|ObedientTurkey46>
Use --services-mode in the agent , it will run many Tasks on the same machine, this is usually associated with the services queue, but can be run on any queue. This way you could have the same machine easily running those multiple "control" tasks.
wdyt?

Hi DilapidatedDucks58

eg, we want max validation accuracy and all other metric values for the corresponding epoch

Is this the equivalent of nested sort ?
Wouldn't you get the requested behavior if you add all metric columns but sort based on the "accuracy" column ?

Can I make the Tasks that I'm adding to the pipeline also run locally, such that the entire pipeline runs locally?

Ohh I think only if you have an agent running on your machine.
What is the use case ? (maybe we can add local execution as well?!)

K8s + clearml-agent integration.

Hmm is this an on-prem k8s cluster?

Task.add_requirements does not handle it (traceback in the thread). Any suggestions?

Hmm that is a good point maybe we should fix that 🙂
I'm assuming someone already created this module? Or is it part of the repository?
(if it than the assume this executed from the git root)

Please hit Ctrl-F5 refresh the entire page, see if it is till empty....

No worries, and I will make sure we output a warning if section names are not used 🙂

I put two models in the same endpoint, then only one was running,

without providing version number, you are overriding the models (because this is the same endpoint)

I started another docker container having a different port number and then the curls with the new model endpoint (with the new port) started working

Seems like misconfiguration on the first one?

, which apparently I can't specify when I establish the model endpoint but I need to re compose the docker container by...