JitteryCoyote63 please do not get used to it :D there's an open ticket/feature request to either revert this or let the user/server choose the most comfortable way

Does it make sense to you to run several such glue instances, to manage multiple resource requirements?

Yes, I’ve found that too (as mentioned, I’m familiar with the repository). My issue is still that there is documentation as to what this actually offers.
Is this simply a helm chart to run an agent on a single pod? Does it scale in any way? Basically - is it a simple agent (similiar to on-premise agents, running in the background, but here on K8s), or is it a more advanced one that offers scaling features? What is it intended for, and how does it work?

The official documentation are very spa...

Maybe @<1523701827080556544:profile|JuicyFox94> can answer some questions then…

For example, what’s the difference between agentk8sglue.nodeSelector and agentk8sglue.basePodTemplate.nodeSelector ?
Am I correct in understanding that the former decides the node type that runs the “scaler” (listening to the given agentk8sglue.queue ), and the latter for any new booted instance/pod, that will actually run the agent and the task?
Read: The former can be kept lightweight, as it does no...

Perfect, thanks for the answers Valeriano. These small stuff are missing from the documentation, but I now feel much more confident in setting this up.

AgitatedDove14 yeah I see this now; this was an issue because I later had to "disconnect" the remote task, so it can, itself, create new tasks (using clearml.config.remote.override_current_task_id(None) ). I guess you might remember that discussion? 😁

EDIT: It's the discussion we had here, for reference. https://clearml.slack.com/archives/CTK20V944/p1640955599257500?thread_ts=1640867211.238900&cid=CTK20V944
So probably not needed in JitteryCoyote63 's case, we still have some...

An internal project I've accidentally made with a hidden tag while playing around with the ClearML internal code.

I created a new task with the project name internal tests , and no task name (so it's derived by ClearML).
The task was a simple print out.
The project does not appear in the project space and does not turn up on searches (the task does)

JitteryCoyote63 yes exactly, sorry, I forgot to add the Task.get_task in my response. That's exactly what we do 😅

We just do task.close() and then start a new task.Init() manually, so our "pipelines" are self-controlled

Yes exactly 👍 Good news.

Or is just integrated in the ClearML slack space and for some reason it's showing the clearml address then?

Also full disclosure - I'm not part of the ClearML team and have only recently started using pipelines myself, so all of the above is just learnings from my own trials 😅

Thanks!

Not sure I understand your comment - why not let the user start with an empty comparison page and add them from "Add Experiment" button as well?

My current approach with pipelines basically looks like a GH CICD yaml config btw, so I give the user a lot of control on which steps to run, why, and how, and the default simply caches all results so as to minimize the number of reruns.

The user can then override and choose exactly what to do (or not do).

https://github.com/allegroai/clearml-agent/pull/98 AgitatedDove14 😁

I think -

• Creating a pipeline from tasks is useful when you already ran some of these tasks in a given format, and you want to replicate the exact behaviour (ignoring any new code changes for example), while potentially changing some parameters.
• From decorators - when the pipeline logic is very straightforward and you'd like to mostly leverage pipelines for parallel execution of computation graphs
• From functions - as I described earlier :)

Sounds like incorrect parsing on ClearML side then, doesn't it? At least, it does not fully support MinIO then

I don't imagine AWS users get a new folder named aws-key-region-xyz-bucket-hostname when they download_folder(...) from an AWS S3 bucket, or do they? 🤔

Literally just as you said it - it happened

I'm not sure, I'm not getting anything (this is the only thing I could fin that's weird about this project).
It has a space in the name, has no subprojects, and it just doesn't show up anywhere 🤔

I will! (once our infra guy comes back from holiday and updates the install, for some reason they setup server 1.1.1???)

Meanwhile wondering where I got a random worker from

I'm aware, but it would be much cleaner to define them in the worker's clearml.conf and let ClearML expose them locally to running tasks.

EDIT: Also the above is specifically about serving, which is not the target here 🤔 At least not yet 😄

AgitatedDove14 I will try! I remember there were some issues with it, where I had to resort to this method first, but maybe things have changed since :)

Any simple ways around this for now? @<1523701070390366208:profile|CostlyOstrich36>

But... Which queue does it listen to, and which type of instances will it use etc

So a missing bit of information that I see I forgot to mention, is that we named our packages as foo-mod in pyproject.toml . That hyphen then get’s rewritten as foo_mod.x.y.z-distinfo .

foo-mod @ git+

I think you're interested in the Monitor class:)

Maybe AgitatedDove14 ? 🙃

Thanks SuccessfulKoala55 , I made https://github.com/allegroai/clearml-agent/issues/126 as a suggestion.

Do you have any thoughts on how to expose these... manually?
It does so already for environment variables that prefixed with CLEARML_ , so it would be nice to have some control over that.