Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
AmiableSeaturtle81
Moderator
35 Questions, 135 Answers
  Active since 14 July 2023
  Last activity one month ago

Reputation

0

Badges 1

123 × Eureka!
0 Why Is Async_Delete Not Working?
  • Here is how client side clearml.conf looks like together with the script im using to create the tasks. Uploads seems to work and is fixed thanks to you guys 🙌
    image
    image
    image
10 months ago
0 Why Is Async_Delete Not Working?

@<1523703436166565888:profile|DeterminedCrab71> Thanks for responding
It was unclear to me that I need to set 443 also everywhere in clearml.conf
Setting s3 host urls with 443 in clearml.conf and also in web UI made it work

Im now almost at the finish line. The last thing that would be great is to fix archived task deletion.
For some reason i have error of missing S3 keys in clearml docker compose logs, the folder / files are not deleted in S3 bucket.

You can see how storage_credentials.co...

10 months ago
0 Why Is Async_Delete Not Working?
  1. This is how web UI configurations looks like
    image
10 months ago
0 Hello, For Some Reason My Upload Speed To S3 Is Insanely Slow, I Noticed In Logs That It Upoads To /Tmp Folder. What Does That Mean? Why Tmp?

im also batch uploading, maybe thats the problem?

  • The dataset is about 1TB containing 1 million files
  • I dont have the SSD space locally to do the upload
  • So i download a part of the dataset, use add_files() and then upload() to that batch
  • Upload the dataset
    I noticed that each batch is slower and slower
8 months ago
0 Why Is Async_Delete Not Working?

good morning, I tried the script you provided and Im getting somewhere
image

11 months ago
0 Why Is Async_Delete Not Working?

@<1523701070390366208:profile|CostlyOstrich36> Still unable to understand what im doing wrong.
We have self hosted S3 Ceph storage server
Setting my config like this breaks task.init
image

one year ago
11 months ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

we are cleaning, but there is a major problem
When deleting a task from web UI, nothing is deleted elsewhere
Debug images are not deleted, models are not deleted. And I suspect that scalars and logs are not deleted too
Im not sure why is that so

one year ago
0 Why Is Async_Delete Not Working?

there is a typing in clearm.conf i sent you on like 87, there should be "key" not "ey" im aware of it

11 months ago
0 Why Is Async_Delete Not Working?

helper is returned as None for some reason

11 months ago
0 Why Is Async_Delete Not Working?

Adding bucket in clearml.conf causes the same error: clearml.storage - ERROR - Failed uploading: Could not connect to the endpoint URL: " None "
image
image
![image](https://clearml-web-assets.s3.amazonaws.com/scoold/images/TT...

11 months ago
0 Why Is Async_Delete Not Working?

So from our IT guys i now know that
"s3" part of url is subdomain, we use it in all other libs like boto3 and cloudpathlib, never had any problems
This is where the crash happens inside the clearml Task
image

11 months ago
0 Why Is Async_Delete Not Working?

I tried it with port, but still having the same issue
Tried it with/without secure and multipart
image
image
image

11 months ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

from docker inspect I can see that allegorai/clearml uses:
"CLEARML_SERVER_VERSION=1.11.0",
"CLEARML_SERVER_BUILD=373"

Image hash:ed05631045c4237f59ad48f477e06dd72274ab67e70d2f9adc489431d1ce75d7

one year ago
0 Why Is Async_Delete Not Working?

This is the link generated
image

11 months ago
0 Why Is Async_Delete Not Working?

we do, yes. Changing it to https in settings doesnt help

11 months ago
0 I'Ve Had This Bug Where Every Few Weeks All My Current Running Experiments Are Stopped And Then Deleted. This Has Now Happend Like 3-4 Times. I Dont Understand What Is Causing It. Model Files, Debug Images Are Saved In Fileserver Folder, But The Task Itse

hi, thanks for reaching out. Getting desperate here.
Yes, its self hosted
No, only currently running experiments are deleted (task itself is gone, but debug images and models are present in fileserver folder)

What I do see is some random elastisearch errors popping up from time to time

[2024-01-05 09:16:47,707] [9] [WARNING] [elasticsearch] POST None ` [status:N/A requ...

one year ago
0 Why Is Async_Delete Not Working?

i need clearml.conf on my clearml server (in config folder which is mounted in docker-compose) or user PC? Or Both?
Its self hosted S3 thats all I know, i dont think it s Minio

one year ago
0 Why Is Async_Delete Not Working?

maybe someone on your end can try to parse such a config and see if they also have the same problem

11 months ago
0 Why Is Async_Delete Not Working?

unable to see the images with that link tho

11 months ago
Show more results compactanswers