Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
ItchyJellyfish73
Moderator
6 Questions, 12 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

12 × Eureka!
0 Votes
2 Answers
992 Views
0 Votes 2 Answers 992 Views
Hello. From Time To Time We Need To Upgrade Our Docker Workers (Usually Changing Configuration). We Have ~15 Of Them And Some (Or All) Of Them Could Be Busy At Time Of Upgrade. Probably You Know Some Solution To Avoid Force Stopping Tasks For Upgrading Ag
Hello. From time to time we need to upgrade our docker workers (usually changing configuration). We have ~15 of them and some (or all) of them could be busy ...
3 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hello! Can You Clarify, How We Can Support Following Scenario With Clearml. We Have Single Clearml Server With Multiple Workers In Docker Mode. We Also Have Multiple Teams. They Work On Different Projects Stored In Different Repositories(Public/Private Gi
Hello! Can you clarify, how we can support following scenario with ClearML. We have single ClearML server with multiple workers in docker mode. We also have ...
3 years ago
0 Votes
10 Answers
1K Views
0 Votes 10 Answers 1K Views
Hello Periodically Under High Load, We Are Facing Too Long(>1 Sec) Processing Times For Requests Such As: Workers.Status_Report Events.Add_Batch Queues.Get_Next_Task. Also There Are Warnings "Connection Pool Is Full, Discarding Connection: Elasticsearch-S
Hello Periodically under high load, we are facing too long(>1 sec) processing times for requests such as: workers.status_report events.add_batch queues.get_n...
3 years ago
0 Votes
2 Answers
919 Views
0 Votes 2 Answers 919 Views
Hello. Recently Installed Packages Behavior Has Been Changed. Previously It Was Following: 1. If Installed Packages Is Empty, Packages Should Be Installed From Requirements.Txt. 2. If Installed Packages Is Not Empty, They Should Be Installed. Now It'S Fol
Hello. Recently INSTALLED PACKAGES behavior has been changed. Previously it was following: 1. If INSTALLED PACKAGES is empty, packages should be installed fr...
3 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hello. I'M Interested In Dynamic Gpu Feature. But I Can'T Find Any Information On How It Works. Can You Help Me With It? Is It Possible To Try It Somewhere ?
Hello. I'm interested in dynamic gpu feature. But I can't find any information on how it works. Can you help me with it? Is it possible to try it somewhere ?
3 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
Hello! I Got The Idea Of Publishing Model/Task. But There Could Be Scenarios When It Still Should Be Archived/Deleted. For Instance Death Of Project. Is It Possible To Archive/Delete/Change Status Of Published Task/Model Via Api? Thanks.
Hello! I got the idea of publishing model/task. But there could be scenarios when it still should be archived/deleted. For instance death of project. Is it p...
3 years ago
0 Hello. I'M Interested In Dynamic Gpu Feature. But I Can'T Find Any Information On How It Works. Can You Help Me With It? Is It Possible To Try It Somewhere ?

Yeah this is my case. But we have multiple machines with different number of gpus (from 1 to 8 )

3 years ago
0 Hello. I'M Interested In Dynamic Gpu Feature. But I Can'T Find Any Information On How It Works. Can You Help Me With It? Is It Possible To Try It Somewhere ?

Thanks, this helps.

3 years ago
0 Hello. I'M Interested In Dynamic Gpu Feature. But I Can'T Find Any Information On How It Works. Can You Help Me With It? Is It Possible To Try It Somewhere ?

Thanks!

3 years ago
0 Hello. Recently Installed Packages Behavior Has Been Changed. Previously It Was Following: 1. If Installed Packages Is Empty, Packages Should Be Installed From Requirements.Txt. 2. If Installed Packages Is Not Empty, They Should Be Installed. Now It'S Fol

AgitatedDove14 Thanks, it looks like issue does not reproduce with v1.0.1

3 years ago
0 Hello Periodically Under High Load, We Are Facing Too Long(>1 Sec) Processing Times For Requests Such As: Workers.Status_Report Events.Add_Batch Queues.Get_Next_Task. Also There Are Warnings "Connection Pool Is Full, Discarding Connection: Elasticsearch-S

As I discovered, this was ES overload due to incorrect ClearML usage: report_scalar was called 100 times per sec(developer reported each sample from wav file). This didn't affect apieserver, because events were batched. Probably there should be some protection against overload on clearml package or apiserver level, as developers could do any crazy stuff in their code 🙃

3 years ago
0 Hello Periodically Under High Load, We Are Facing Too Long(>1 Sec) Processing Times For Requests Such As: Workers.Status_Report Events.Add_Batch Queues.Get_Next_Task. Also There Are Warnings "Connection Pool Is Full, Discarding Connection: Elasticsearch-S

~30rps

3 years ago
0 Hello Periodically Under High Load, We Are Facing Too Long(>1 Sec) Processing Times For Requests Such As: Workers.Status_Report Events.Add_Batch Queues.Get_Next_Task. Also There Are Warnings "Connection Pool Is Full, Discarding Connection: Elasticsearch-S

AgitatedDove14 are you sure ? Api server has low CPU load( < 10% ). Moreover only requests related to ES are affected, other requests (like tasks.get_all or queues.get_all) are < 10ms

3 years ago
0 Hello. From Time To Time We Need To Upgrade Our Docker Workers (Usually Changing Configuration). We Have ~15 Of Them And Some (Or All) Of Them Could Be Busy At Time Of Upgrade. Probably You Know Some Solution To Avoid Force Stopping Tasks For Upgrading Ag

Hi SuccessfulKoala55 I mean upgrading workers.

3 years ago
0 Hello Periodically Under High Load, We Are Facing Too Long(>1 Sec) Processing Times For Requests Such As: Workers.Status_Report Events.Add_Batch Queues.Get_Next_Task. Also There Are Warnings "Connection Pool Is Full, Discarding Connection: Elasticsearch-S

It's apiserver logs.

3 years ago
0 Hello! Can You Clarify, How We Can Support Following Scenario With Clearml. We Have Single Clearml Server With Multiple Workers In Docker Mode. We Also Have Multiple Teams. They Work On Different Projects Stored In Different Repositories(Public/Private Gi

So, did I understand you correctly? I create single ssh key and place it to ~/.ssh dir of all workers. After that anyone, who wants to run task on their repo, should add this key to their user in their repo.

3 years ago
0 Hello! Can You Clarify, How We Can Support Following Scenario With Clearml. We Have Single Clearml Server With Multiple Workers In Docker Mode. We Also Have Multiple Teams. They Work On Different Projects Stored In Different Repositories(Public/Private Gi

Thanks! This works for me except one thing. This work only with keys wit standard names. If keys have non-standard names should I deal with starting ssh-agent and ssh-add inside docker or there is simple way ?

3 years ago
0 Hello! I Got The Idea Of Publishing Model/Task. But There Could Be Scenarios When It Still Should Be Archived/Deleted. For Instance Death Of Project. Is It Possible To Archive/Delete/Change Status Of Published Task/Model Via Api? Thanks.

AgitatedDove14 Thanks! This works

3 years ago