Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
AmiableSeaturtle81
Moderator
17 Questions, 87 Answers
  Active since 14 July 2023
  Last activity one day ago

Reputation

0

Badges 1

86 × Eureka!
0 Votes
0 Answers
386 Views
0 Votes 0 Answers 386 Views
Tested on two PCs now
7 months ago
0 Votes
2 Answers
90 Views
0 Votes 2 Answers 90 Views
What could be the reason that im not getting any scalars reported to clearml using example script? None
one month ago
0 Votes
25 Answers
121 Views
0 Votes 25 Answers 121 Views
one month ago
0 Votes
0 Answers
334 Views
0 Votes 0 Answers 334 Views
4 months ago
0 Votes
29 Answers
471 Views
0 Votes 29 Answers 471 Views
Hello, please dont tell me I just deleted something: I wanted to do two things: - Update clearml server- I followed the guilde at None - I deleted my old con...
7 months ago
0 Votes
1 Answers
382 Views
0 Votes 1 Answers 382 Views
well, FAQ helped: None I had to clear my cookies and session storage, problem solved
7 months ago
0 Votes
1 Answers
78 Views
0 Votes 1 Answers 78 Views
Hello, can anyone please give us update on our self hosted Ceph server not working with your clearml.conf because we dont have "region" field? None
26 days ago
0 Votes
3 Answers
477 Views
0 Votes 3 Answers 477 Views
im downloading clearml dataset with: dataset.get_local_copy(use_soft_links=True) Somwtimes datasets are really huge, how to get progressbar or any callback o...
7 months ago
0 Votes
2 Answers
156 Views
0 Votes 2 Answers 156 Views
How can I move clearml dataset to another directory? Change its URL
2 months ago
0 Votes
1 Answers
462 Views
0 Votes 1 Answers 462 Views
how to make sure that clearml agents are running queue after server restart? Sometimes our worker dies and we manually have to run settuping for queue, we ru...
6 months ago
0 Votes
0 Answers
409 Views
0 Votes 0 Answers 409 Views
deleted /opt/clearml deleted /home/ma/.clearml deleted /home/ma/clearml.conf Nothing helps
7 months ago
0 Votes
0 Answers
84 Views
0 Votes 0 Answers 84 Views
S3 Setup is confusing for me, so this is how clearml.conf looks like on my local machine where I run test scripts
one month ago
0 Votes
1 Answers
120 Views
0 Votes 1 Answers 120 Views
2 months ago
0 Votes
1 Answers
131 Views
0 Votes 1 Answers 131 Views
Another annoying problem we have is that task deletion doesnt delete any files at all. I have selected deletion of images But nothing is deleted. I have to w...
one month ago
0 Votes
1 Answers
283 Views
0 Votes 1 Answers 283 Views
4 months ago
0 Votes
68 Answers
1K Views
0 Votes 68 Answers 1K Views
Why is async_delete not working? - bucket is not right in logs - This is really misleading in web ui, because it says "success" although async_delete failed ...
one month ago
0 Votes
1 Answers
485 Views
0 Votes 1 Answers 485 Views
Is there a reason that im not seeing any side panels? Following this tutorial: None
7 months ago
0 Why Is Async_Delete Not Working?

ok, slight update. It seems like artifacts are uploading now to bucket. Maybe my folder explorer used old cache or something.
However, reported images are uploaded to fileserver instead of s3

here is the script im using to test things. Thanks
image
image

16 days ago
0 Why Is Async_Delete Not Working?

Yes, credetials seems to work
Im trying to figure out not why I dont see the uploaded files / folders

  • I checked maybe clearml task uses fileserver instead but i dont see any files in fileserver folder
  • Nothing is uploaded in bucket (i will ask IT guy to check if im uploading any files in logs)
    image
16 days ago
0 Why Is Async_Delete Not Working?

But there are stil some wierd issues, i cannot see the files uploaded in bucket

17 days ago
0 Why Is Async_Delete Not Working?

good morning, I tried the script you provided and Im getting somewhere
image

17 days ago
0 Why Is Async_Delete Not Working?

The problem is that clearml.conf s3 config doesnt support empty region field, even empty strings crashes it

18 days ago
0 What Could Be The Reason That Im Not Getting Any Scalars Reported To Clearml Using Example Script?

I solved the problem.
I had to add tensorboard loggger and pass it to pytorch_lightning trainer logger=logger
Is that normal?

one month ago
0 Why Is Async_Delete Not Working?

we do, yes. Changing it to https in settings doesnt help

15 days ago
0 Why Is Async_Delete Not Working?

Hi, ok im really close now to working system
Debug image is uploading to s3, im seeing the files, all ok there

Problem now is viewing these images in web UI
Going to Debug Samples panel in Task drops me a popup to fill in s3 credentials

I cant figure out what the right setup is for the creds to work
This is what I have now (Note that we dont have region)
image

15 days ago
0 Why Is Async_Delete Not Working?

This is the link generated
image

15 days ago
0 Why Is Async_Delete Not Working?

unable to see the images with that link tho

12 days ago
0 Why Is Async_Delete Not Working?

@<1523701435869433856:profile|SmugDolphin23> Any ideas how to fix this?

11 days ago
0 Why Is Async_Delete Not Working?

I know these keys work, url and everything else works because I use these creds daily

18 days ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

I also have noticed that this incident usually happens in the morning at around 6-7AM
Are there maybe some clearnup tasks or backups running on clearml server at those times?

one month ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

from docker inspect I can see that allegorai/clearml uses:
"CLEARML_SERVER_VERSION=1.11.0",
"CLEARML_SERVER_BUILD=373"

Image hash:ed05631045c4237f59ad48f477e06dd72274ab67e70d2f9adc489431d1ce75d7

one month ago
0 Why Is Async_Delete Not Working?

It looks like im moving forward

Setting url in clearml.conf without "s3" as suggested works (But I dont add port ther, not sure if it breaks something, we dont have a port)
host: " our-host.com "

Then in test_task.py
task: clearml.Task = clearml.Task.init(
project_name="project",
task_name="task",
output_uri=" None ",
)

I think connection is created
What im getting now is bucket error, i suppose I have to specify it so...

18 days ago
0 Why Is Async_Delete Not Working?

py file:
task: clearml.Task = clearml.Task.init(
project_name="project",
task_name="task",
output_uri=" None ",
)

clearml.conf:
{
# This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
host: " our-host.com "
key: "xxx"
secret: "xxx"
multipart: false
...

18 days ago
0 Why Is Async_Delete Not Working?

maybe someone on your end can try to parse such a config and see if they also have the same problem

18 days ago
0 Why Is Async_Delete Not Working?

removing it doesnt fix the problem

18 days ago
0 Why Is Async_Delete Not Working?

Bump, still waiting, closing in on a month since we are unable to deploy. We have team of 10+ people

18 days ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

hi, thanks for reaching out. Getting desperate here.
Yes, its self hosted
No, only currently running experiments are deleted (task itself is gone, but debug images and models are present in fileserver folder)

What I do see is some random elastisearch errors popping up from time to time

[2024-01-05 09:16:47,707] [9] [WARNING] [elasticsearch] POST None ` [status:N/A requ...

one month ago
0 Why Is Async_Delete Not Working?

we use Ceph Storage Cluster, interface to it is the same as S3
I dont get what I have misconfigured.
The only thing I have not added is "region" field in clearml.conf because we literally dont have, its a self hosted cluster.
You can try and replicate this s3 config I have posted earlier.

18 days ago
0 Why Is Async_Delete Not Working?

I do have write permissions

18 days ago
0 Why Is Async_Delete Not Working?

clearml.conf is a fresh one i did clearml-init to make sure

18 days ago
0 Why Is Async_Delete Not Working?

we might as well have "s5" there but it is needed there

18 days ago
Show more results compactanswers