What do you mean by a custom queue ?
In the queues page you have a plus button, this will just create a new queue

Hi @<1523701111020589056:profile|DefiantSpider5>
So there are two answers here, I'll start with the open-source version of both

Is there a way in clear ml to interactively view subsets of images based on a lasso of embedding plots

The ClearML Datasets have no "query" capabilities of the data inside the dataset. That means you can see preview images, statistics and download the datasets, but no query capabilities. On the other hand, there is no limitation on the type and format of me...

Hi RoundMosquito25
How did you spin the agent (whats the cmd line? is it in docker mode or venv mode?)
From the console it seems the pip installation inside the container (based on the log this is what I assume) seems like it is stuck ?!

BeefyCow3 not yet 😞

(just using local server not connected to Internet), am I right?

You can if you host your own git server, Or if your code is a single file / jupyter notebook, then the entire code is stored on the Task.
btw: what is the exact setup, how come there is no git repo?

Hi Team,Can i clone experiment shared by some one, via link?

You mean someone that is not in your workspace ? (I'm assuming app.clear.ml ?)

Hi @<1573119955400921088:profile|CloudyPelican46>

On what machine is it best practice to run the clean up service, local machine or should it be on the clearml server ?

The easiest is to run it on the server machine itself, even though in practice you can put it anywhere, but most of the time this service is sleeping and not using so much RAM so it kind of makes sense

Its stored on the Task, you can see it under the execution tab in the UI

Hi @<1524922424720625664:profile|TartLeopard58>

can’t i embed scalars to notion using clearml sdk?

I think that you need the hosted version for it (it needs some special CORS stuff on the server side to make it work)
Did you try in the clearml report? does that work?

Exporter would be nice I agree, not sure it is on the roadmap at the moment 😞
Should not be very complicated to implement if you want to take a stab at it.

AdorableFrog70 taking another look at the MLFlow exporter, it would not be complicated to convert it to MLFlow to ClearML exporter, that would also be cool

BTW: what would be a reason to go back to self-hosted? (not sure about the SaaS cost, but I remember it was relatively cheap)

do I need to have the repo that I am running on my account

If it is a public repo, then no need, credentials are only needed for private repos 🙂
Am I missing something ?

I see, is this what you are looking for?
https://allegro.ai/docs/task.html#trains.task.Task.init

continue_last_task='task_id'

RoughTiger69

Apparently,

, doesn’t populate that dict with

any keys that don’t already exist in it

.

Are you saying new entries are not added to the Dict even if they are on the Task (i.e. only entries that already exist on the dict are populated ?
But you already have all the entries defined here:
https://github.com/allegroai/clearml/blob/721569bb77d89d89e5b4f32a0ed98311c4574650/examples/services/aws-autoscaler/aws_autoscaler.py#L22

Since all this is ha...

MoodyCentipede68 is diagram 2 a batch processing workflow?

If this is the case why not have the stream process call the rest api, then move forward with the result? This way it scales out of the box, the main "conceptual" difference is that the restapi is used internally, and the upside is the event streaming processing becomes part of the application layer, not tied with the compute cost of the model , wdyt?

Oh I see, this seems like Triton configuration issue, usually dim -1 means flexible. I can also mention that serving 1.1 should be released later this week with better multiple input support for triton. Does that make sense?

Here you go:
` @PipelineDecorator.pipeline(name='training', project='kgraph', version='1.2')
def pipeline(...):
return

if name == 'main':
Task.force_requirements_env_freeze(requirements_file="./requirements.txt")
pipeline(...) If you need anything for the pipeline component you can do: @PipelineDecorator.component(packages="./requirements.txt")
def step(data):

some stuff `

Hi CheekyFox58
If you are running the HPO+training on your own machine, it should work just fine in the Free tier

The HPO with the UI and everything, is designed to run the actual training on remote machines, and I think this makes it a Pro feature.

This looks like 'feast' error, could it be a configuration missing?

I think you are correct 😞 Let me make sure we add that (docstring and documentation)

A single query will return if the agent is running anything, and for how long, but I do not think you can get the idle time ...

When we enqueue the task using the web-ui we have the above error

ShallowGoldfish8 I think I understand the issue,
basically I think the issue is:
task.connect(model_params, 'model_params')Since this is a nested dict:
model_params = { "loss_function": "Logloss", "eval_metric": "AUC", "class_weights": {0: 1, 1: 60}, "learning_rate": 0.1 }The class_weights is stored as a String key, but catboost expects "int" key, hence it fails.
One op...

HurtWoodpecker30

The agent uses the

requirements.txt

)

what do you mean by that? aren't the package listed in the "Installed packages" section of the Task?
(or is it empty when starting, i.e. it uses the requirements.txt from the github, and then the agent lists them back into the Task)

Hmm, how does your preprocessing code looks like?

Hi CloudySwallow27

This error occurs randomly during training (in other words training does successfully start).

What's the cleamrl-agent version you are using, and the clearml version ?

JitteryCoyote63 no you should not (unless you already have the Task.init call in your code)
clearml-data add the Task.init call at the beginning of the code in the entry point.
This means you should be able to get Task.current_task() and get back the object.
What do you have under the "uncommitted changes" on the Task that was created?

UnevenDolphin73 clearml.config.get_remote_task_id() will return the Task ID not the Task object. in order to get automagic to work, one h...

With pleasure 🙂

Hi BattyLizard6

does clearml orchestration have the ability to break gpu devices into virtual ones?

So this is fully supported on A100 with MIG slices. That said dynamic multi-tenant GPU on Kubernetes is a Kubernetes issue... We do support multi agents on the same GPU on bare metal, or over shared GPU instances over k8s with:
https://github.com/nano-gpu/nano-gpu-agent
https://github.com/intel/intel-device-plugins-for-kubernetes/tree/main/cmd/gpu_plugin#fractional-resources
http...