Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
UnevenDolphin73
Moderator
106 Questions, 746 Answers
  Active since 10 January 2023
  Last activity 28 days ago

Reputation

0

Badges 1

662 × Eureka!
0 Votes
0 Answers
570 Views
0 Votes 0 Answers 570 Views
Any easy way to ensure the pipline (tasks + controller) are not hidden in the projects UI? EDIT: Apparently not; for the time being I created this https://gi...
one year ago
0 Votes
30 Answers
529 Views
0 Votes 30 Answers 529 Views
Is there a way to set precedence on package managers? If we set an agent to use poetry it will ignore local packages, even if I use the force_requirements_en...
2 years ago
0 Votes
5 Answers
565 Views
0 Votes 5 Answers 565 Views
We're running into errors such as this: Action failed <500/0: tasks.add_or_update_artifacts/v2.10 (Update failed (BSONObj size: 18564134 (0x11B4426) is inval...
one year ago
0 Votes
30 Answers
562 Views
0 Votes 30 Answers 562 Views
Is it possible to avoid the clearml-agent for local installations, and have the file server automatically use an S3 bucket? I've found https://clearml.slack....
2 years ago
0 Votes
1 Answers
524 Views
0 Votes 1 Answers 524 Views
Side note for documentation, Task.project reads > Returns the current Task’s project name. But it actually returns the project ID
one year ago
0 Votes
0 Answers
581 Views
0 Votes 0 Answers 581 Views
The remote workers seem to insist on using old git credentials, even after I stopped them, updated the conf file, and restarted them using it. Are the creden...
2 years ago
0 Votes
2 Answers
521 Views
0 Votes 2 Answers 521 Views
Can we use S3 buckets to cache environments?
one year ago
0 Votes
7 Answers
552 Views
0 Votes 7 Answers 552 Views
Is it expected that K8s helm chart also starts a ClearML worker?
one year ago
0 Votes
3 Answers
544 Views
0 Votes 3 Answers 544 Views
Does anyone know how we can restore ClearML on Helms chart from existing Snapshots (AWS)?
one year ago
0 Votes
30 Answers
608 Views
0 Votes 30 Answers 608 Views
Is there any testing suite that ships with ClearML? If we'd like to make some unit tests for our code?
one year ago
0 Votes
3 Answers
515 Views
0 Votes 3 Answers 515 Views
side question - any plans to include native support for lgbm ?
2 years ago
0 Votes
26 Answers
550 Views
0 Votes 26 Answers 550 Views
I have some code that launches ML tasks and it accepts a YAML file, .env file and various CSVs. What would be the best way to upload these to a clearml task ...
2 years ago
0 Votes
9 Answers
576 Views
0 Votes 9 Answers 576 Views
Hey! Probably missed something, but I recently upgraded to 1.1.1, and I've just noticed that png artifacts are not displayed in the preview. Is this intentio...
2 years ago
0 Votes
11 Answers
572 Views
0 Votes 11 Answers 572 Views
Pipelines suddenly no longer appear in the Pipelines tab, what could/should I look into? EDIT: Using latest ClearML (agent, server, SDK), and creating the pi...
one year ago
0 Votes
2 Answers
562 Views
0 Votes 2 Answers 562 Views
Does ClearML exposes (even temporarily) the contents of e.g. sdk.aws.s3 to the relevant environment variables (i.e. AWS_SECRET_ACCESS_KEY )? Specifically, if...
one year ago
0 Votes
11 Answers
521 Views
0 Votes 11 Answers 521 Views
one year ago
0 Votes
23 Answers
572 Views
0 Votes 23 Answers 572 Views
We're trying to upgrade our ClearML on K8s but we're getting this error - Error: UPGRADE FAILED: cannot patch "clearml-fileserver-data" with kind PersistentV...
one year ago
0 Votes
18 Answers
522 Views
0 Votes 18 Answers 522 Views
Does ClearML somehow remove any loggers from logging module? We suddenly noticed that we have some handlers missing when running in ClearML
one year ago
0 Votes
29 Answers
606 Views
0 Votes 29 Answers 606 Views
We're trying to use the AWS autoscaler and have managed to get it up and running with spinning up instances. However, it does not seem to pull any of the tas...
2 years ago
0 Votes
4 Answers
522 Views
0 Votes 4 Answers 522 Views
Weird error; my local execution hung with ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start 2022-03-07 ...
2 years ago
0 Votes
30 Answers
554 Views
0 Votes 30 Answers 554 Views
Since v1.4.0, our StorageManager.download_folder(..., local_folder='./') is failing - we've had to revert back to 1.3.2. I saw the changelist includes a fix ...
one year ago
0 Votes
3 Answers
664 Views
0 Votes 3 Answers 664 Views
What privileges/IAM role would the AWS autoscaler need?
2 years ago
0 Votes
7 Answers
639 Views
0 Votes 7 Answers 639 Views
Bug report? We noticed that the AWS autoscaler will lose track of instances crashing due to no space left on device, and the ec2 instance will remain running...
11 months ago
0 Votes
6 Answers
594 Views
0 Votes 6 Answers 594 Views
Does the ClearML team still post updates in this channel when new ClearML versions come out? If not, what's the easiest way to keep track of various release ...
one year ago
0 Votes
2 Answers
60 Views
0 Votes 2 Answers 60 Views
Is there a GCP driver similar to aws_driver.py ? None
28 days ago
0 Votes
4 Answers
560 Views
0 Votes 4 Answers 560 Views
⚠️ ClearML version 1.8.1 had "fix" for the deferred init which introduces a bug btw, I've opened None to highlight this.
one year ago
0 Votes
30 Answers
626 Views
0 Votes 30 Answers 626 Views
What happens to file that are downloaded to a remote_execution via StorageManager? Are they removed at the end of the run, or does it continuously increases ...
2 years ago
0 Votes
18 Answers
534 Views
0 Votes 18 Answers 534 Views
What would be the best way to approach this flow? We have a configuration file that defines e.g. the project name to use in ClearML, alongside other experime...
one year ago
0 Votes
13 Answers
132 Views
0 Votes 13 Answers 132 Views
Can steps be removed from pipelines, and/or can pipelines be generally modified other than adding steps to them?
2 months ago
0 Votes
10 Answers
516 Views
0 Votes 10 Answers 516 Views
Is there any way in the WebUI to search by task name? I see the search bar only goes through projects
2 years ago
Show more results questions
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

I dunno :man-shrugging: but Task.init is clearly incompatible with pytest and friends

one year ago
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

This is with:
Task.set_offline_mode(True) task = Task.init(..., auto_connect_streams=False)

one year ago
0 Is There An Autoscaler Equivalent For K8S? That Is, A Service That Will Launch Pods Based On Incoming Requests?

Maybe @<1523701827080556544:profile|JuicyFox94> can answer some questions then…

For example, what’s the difference between agentk8sglue.nodeSelector and agentk8sglue.basePodTemplate.nodeSelector ?
Am I correct in understanding that the former decides the node type that runs the “scaler” (listening to the given agentk8sglue.queue ), and the latter for any new booted instance/pod, that will actually run the agent and the task?
Read: The former can be kept lightweight, as it does no...

10 months ago
0 Can I Somehow Change A Project Settings Via The Python Sdk?

I can also do this via Mongo directly, but I was hoping to skip the K8S interaction there.

one year ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

UPDATE: Apparently the quotation type matters for furl ? I switched the ' to \" and it seems to work now

one year ago
0 Is There An Autoscaler Equivalent For K8S? That Is, A Service That Will Launch Pods Based On Incoming Requests?

We’re using karpenter (more magic keywords for me), so my understanding is that that will manage the scaling part.

10 months ago
0 Is There An Autoscaler Equivalent For K8S? That Is, A Service That Will Launch Pods Based On Incoming Requests?

Yes, I’ve found that too (as mentioned, I’m familiar with the repository). My issue is still that there is documentation as to what this actually offers.
Is this simply a helm chart to run an agent on a single pod? Does it scale in any way? Basically - is it a simple agent (similiar to on-premise agents, running in the background, but here on K8s), or is it a more advanced one that offers scaling features? What is it intended for, and how does it work?

The official documentation are very spa...

10 months ago
0 Is There An Autoscaler Equivalent For K8S? That Is, A Service That Will Launch Pods Based On Incoming Requests?

Perfect, thanks for the answers Valeriano. These small stuff are missing from the documentation, but I now feel much more confident in setting this up.

10 months ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In
  • in the second scenario, I might have not changed the results of the step, but my refactoring changed the speed considerably and this is something I measure.
  • in the third scenario, I might have not changed the results of the step and my refactoring just cleaned the code, but besides that, nothing substantially was changed. Thus I do not want a rerun.Well, I would say then that in the second scenario it’s just rerunning the pipeline, and in the third it’s not running it at all 😄
    (I ...
one year ago
0 Is There A Gcp Driver Similar To

I see that the GUI AutoScaler is only in the paid version, wonder why the GCP driver is not open source?

28 days ago
0 Another Thing, Can I Get The Git Repo/Commit Info Of My Experiment Through The Sdk? Couldn'T Find It In The Docs.

Follow-up question/feature request (out of interest) - could the WebUI show the matching commit message?

2 years ago
0 Hi, I'M Trying To Access/Use Experiment'S Model+Data+Params My Model And Data Are Stored In S3 And I'M Not Sure What Is The Practice Of Getting Them Reddy To Use. When I Use

IIRC, get_local_copy() downloads a local copy and returns the path to the downloaded file. So you might be interested in e.g.
local_csv = pd.read_csv(a_task.artifacts['train_data'].get_local_copy())
With the models, you're looking for get_weights() . It acts the same as get_local_copy() , so it returns a path.

EDIT: I think also get_local_copy() for a model should work 👍

2 years ago
0 What Is Being Stored Exactly In

AgitatedDove14

hmmm... they are important, but only when starting the process. any specific suggestion ?
(and they are deleted after the Task is done, so they are temp)

Ah, then no, sounds temporary. If they're only relevant when starting the process though, I would suggest deleting them immediately when they're no longer needed, and not wait for the end of the task (if possible, of course)

2 years ago
0 Clearml 1.3.2 I'M Running

It's a small snippet that ensures identically named projects are still unique'd with a running number.

one year ago
0 Clearml 1.3.2 I'M Running

From the traceback ( backend_interface/task/task.py, line 178, in __init__ ), notice it's not Task.init

one year ago
2 years ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

So basically what I'm looking for and what I have now is something like the following:
(Local) I have a well-defined aws_autoscaler.yaml that is used to run the AWS autoscaler. That same autoscaler is also run with CLEARML_CONFIG_FILE=.... (Remotely) The autoscaler launches, listens to the predefined queue, and is able to launch instances as needed. I would run a remote execution task object that's appended to the autoscaler queue. The autoscaler picks it up, launches a new instanc...

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

Ah, you meant “free python code” in that sense. Sure, I see that. The repo arguments also exist for functions though.

Sorry for hijacking your thread @<1523704157695905792:profile|VivaciousBadger56>

one year ago
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

Yeah I managed to work around those former two, mostly by using Task.create instead of Task.init . It's actually the whole bunch of daemons running in the background that takes a long time, not the zipping.

Regarding the second - I'm not doing anything per se. I'm running in offline mode and I'm trying to create a dataset, and this is the error I get...
There is a data object it, but there is no script object attached to it (presumably again because of pytest?)

one year ago
0 We'Re Running Into Errors Such As This:

Thanks LazyFish41 , that explains it 👍

one year ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

If I set the following:
"extra_clearml_conf": "sdk.aws.s3.credentials = [\n{\nhost: 'ip:9000'\nkey: 'xxx'\nsecret: 'xxx'\nmultipart: false\nsecure: false\n},\n{\nhost: 'ip2:9000'\nkey: 'xxx'\nsecret: 'xxx'\nmultipart: false\nsecure: false\n}\n]"I run into a weird furl error:
ValueError: Invalid port '9000''.

one year ago
0 Is There Any Testing Suite That Ships With Clearml? If We'D Like To Make Some Unit Tests For Our Code?

Yes exactly that AgitatedDove14
Testing our logic maps correctly, etc for everything related to ClearML

one year ago
0 Any Plans To Add Support For Multiindex Tables? That Comma-Separation Feels A Bit Odd. (Also This Shows Up In

Of course Im using report_table in the above; it seems the support for Pandas DataFrame does not include support for MultiIndex other than by concatenating the indices together

That's fine (as in, it works), but it looks a bit weird and defies the purpose of a MultiIndex 🤔 Was wondering if there are plans to add better support for it

2 years ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

I would expect the service to actually implicitly inject it to new instances prior to applying the user's extra configuration 🤔

one year ago
0 Clearml Pipelines Can Be Build From Tasks, Functions, And Decorated Functions, According To The Examples In

Heh, my bad, the term "user" is very much ingrained in our internal way of working. You can think of it as basically any technically-inclined person in your team or company.

Indeed the options in the WebUI are too limited for our use case, so we're developed "apps" that take a yaml configuration file and build a matching pipeline.
With that, our users do not need to code directly, and we can offer much more fine control over the pipeline.

As for the imports, what I meant is that I encounter...

one year ago
0 How Would I Go About Adding Multiple Credentials In The Autoscaler? (I.E. Specify Multiple

-ish, still debugging some weird stuff. Sometimes ClearML picks ip and sometimes ip2 , and I can't tell why 🤔

one year ago
Show more results compactanswers