only GPU and machine

Martin you didn't get me right. We have 1 million small files which we upload in chunks of 512 mb

the files are uploaded but metadata is absent 😞

all metadata that standard for clearml dataset: hashes , tempstamps and names of the 1M uploaded files

it is selfhosted

nothing special in developer tools

This happened after we backed up and restored the server

@<1523701070390366208:profile|CostlyOstrich36> yes. empty

clearml-data create --name [Dataset Name] --project [Project Name] --output-uri clearml-data add --files [FILE_PATH] --id [Id] clearml-data close

clearml==1.8.3

it means they are not in ES?

Thanks. We solved it by copying all ES data again.

yes

Might it be that we lost it in migration process?

{"type": "server", "timestamp": "2023-06-21T08:26:03,816Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "${sys:es.logs.cluster_name}", "node.name": "clearml", "message": "uncaught exception in thread [process reaper (pid 218)]", "cluster.uuid": "9l7JSn6ES0upH0UZK2C8Tw", "node.id": "x3JpeJiKSWSNsz8s7Y0Kuw" ,
uncaught exception in thread [process reaper (pid 218)]
java.security.AccessControlException: access denied ("java.lang.RuntimePermi...

restarted ES container. same. no scalars

or we create parent - child 2 datasets splitting the set to two parts

what I meant is that we have 1,000,000 small files in the dataset

so correct numbers are:

AgitatedDove14

as we see it the only way is to split this dataset to smaller sub-datasets

256GB in total of data

78GB

500 chunks in total

can upgrading to 1.11.0 help?

~2000 files in each chunk

chunksize: 512 Mb