ClearML results page: 

ClearML dataset page: 

2025-02-11 13:43:01,001 - clearml.Metrics - ERROR - Action failed <500/100: events.add_batch/v1.0 (General data error: err=2 document(s) failed to index., extra_info=[events-log-d1bd92a3b039400cbafc60a7a5b1e52b][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[events-log-d1bd92a3b039400cbafc60a7a5b1e52b][0]] containing [2] requests and a refresh])>
not compressing
Generating SHA2 hash for 33 files
100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:02<00:00, 14.85it/s]
Hash generation completed
Compressing X:/testcache2/G/GENERIC/part.1.parquet
Compressing X:/testcache2/C/CC_M/part.3.parquet
Uploading dataset changes (1 files compressed to 1.09 GiB) to 

Uploading dataset changes (1 files compressed to 1.07 GiB) to 

2025-02-11 13:43:10,477 - clearml.storage - INFO - Uploading: 1120.35MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.h9p40zta.zip
                                   0% | 1134.60/? MB [00:04<00:00, 233.14MB/s]: 
Compressing X:/testcache2/G/GENERIC/part.0.parquet
Uploading dataset changes (1 files compressed to 1 GiB) to 

2025-02-11 13:43:19,209 - clearml.storage - INFO - Uploading: 1097.54MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.df4qjb48.zip
2025-02-11 13:43:19,209 - clearml.storage - INFO - Uploading: 1097.54MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.df4qjb48.zip
                                         0% | 0.00/1097.54 MB [00:00<?, ?MB/s]: 2025-02-11 13:43:19,210 - clearml.storage - INFO - Uploading: 5.25MB / 1097.54MB @ 54.69MBs to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.df4qjb48.zip    
                                   0% | 1105.54/? MB [00:05<00:00, 220.24MB/s]: 
Compressing X:/testcache2/C/CC_M/part.2.parquet
Uploading dataset changes (1 files compressed to 891.92 MiB) to 

2025-02-11 13:43:27,694 - clearml.storage - INFO - Uploading: 1027.01MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.7h79j91k.zip
██████████████████████████ 100% | 1027.01/1027.01 MB [00:04<00:00, 211.85MB/s]: 
Compressing X:/testcache2/C/GANG_M/part.1.parquet
Uploading dataset changes (1 files compressed to 856.56 MiB) to 

2025-02-11 13:43:35,751 - clearml.storage - INFO - Uploading: 891.92MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.wq4j4b57.zip
                                    0% | 906.67/? MB [00:04<00:00, 214.99MB/s]:
....
Compressing X:/testcache2/G/part.0.parquet
...
Uploading dataset changes (15 files compressed to 489.35 MiB) to 

2025-02-11 13:44:50,826 - clearml.storage - INFO - Uploading: 489.35MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.gwhd88on.zip
2025-02-11 13:44:50,826 - clearml.storage - INFO - Uploading: 489.35MB to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.gwhd88on.zip
                                          0% | 0.00/489.35 MB [00:00<?, ?MB/s]: 2025-02-11 13:44:50,827 - clearml.storage - INFO - Uploading: 5.25MB / 489.35MB @ 58.98MBs to C:/Users/UZMAR~1.GOM/AppData/Local/Temp/dataset.b2ef22f8fca0496386b130d1f289c268.gwhd88on.zip     
                                    0% | 498.60/? MB [00:01<00:00, 253.56MB/s]: 
File compression and upload completed: total size 12.72 GiB, 17 chunk(s) stored (average size 766.24 MiB)

Had to delete some info, but I think it's showing the important parts? basically it zips everything, and checking in my s3 bucket I see the compressed files. I was checking this issue
None
and seems to be still opened

Can you share the log?

sure, just a sec

hey SuccessfulKoala55 , wondering if you had any ideas around this??