and are you sure you mongodb respawned on same node?

this is strange, I have a lot of clusters that went trough nodes issue but I never lost data

Yep, because of selector

This is pretty weird. If pv containing mongodb data is still the same data must be there. what storageclass provider are you using?

I suspect no for some reason that is related pvc management, at least for what I know

Yeah, understandable

not sure if provisioner did something unexpected

Local Storage

It shows newly created projects (I mean examples , which are created at the start of clearml)

ok so data is not there anymore in mongodb

ok next step is to exec into mongodb pod and check the various docscollections(tables) to see if data is still there

it’s also true I never use local storage since it’s not production reliable

Ok, willl check that

Well, it happened unexpectedly so no, but data must be saved on corresponding persistance volume

ReassuredArcticwolf33 did you back up mongodb before this process?
JuicyFox94 any idea?