tbh i have no experience with mongodb. from what I can see, its a nested schema. smth like:
execution -> artifacts -> { hash1_output: {uri: ...}, hash2_output: {uri: ... }, ... }
cant compose a compelling find for it
now i cant download neither of them
would be nice if address of the artifacts (state and zips) was assembled on the fly and not hardcoded into db.
The idea is this is fully federated, the server is not actually aware of it, so users can manage multiple storage locations in a transparent way.
if you have any tips how to fix it in the mongo db that would be great ....
Yes that should be similar, but the links would be in artifact property on the Tasks object
not exactly sure on how to do that though ... maybe something like db.task.find({artifact ?!
now i cant download neither of them 😕 would be nice if address of the artifacts (state and zips) was assembled on the fly and not hardcoded into db. if you have any tips how to fix it in the mongo db that would be great. I found this tip on model relocation: None . I think I need smth really similar but for datasets
okay. I think I see the pattern. datasets that I added from storage server itself have "localhost" in uri of the files. because clearml.conf on the server has it like that. datasets that I added remotely - have old IP address
on the server itself there is clearml.conf with:
# ClearML SDK configuration file
api {
# Notice: 'host' is the api server (default port 8008), not the web server.
api_server:
web_server:
files_server:
link with "localhost" in it Oo
Hmm I think this is the main issue, for some reason the dataset default upload destination is "localhost", what do you have configured in your clearml.conf under files server?
@<1547390422483996672:profile|StaleElk72> when you go to the dataset in the UI, and press on "Full Details" then go to the Artifacts tab, what is the link you see there?
any docs where I can learn a bit more on structure of database? I managed to connect to MongoDB container. databases:
> show dbs
admin 0.000GB
auth 0.000GB
backend 0.027GB
config 0.000GB
local 0.000GB
I assume backend..so
> use backend
> show collections
company
model
project
queue
settings
task
task__trash
url_to_delete
user
versions
nothing related to dataset. I would assume dataset is a task, but not sure
@<1547390422483996672:profile|StaleElk72> the registered URLs are properties of the artifact metadata stored in the task object in the server's mongodb database - to change that, you ill need to exec into the mongodb container and use the mongo CLI to edit the URL
added couple of prints to dataset object. it seems cleaml hardcodes IP for state.json URL. The problem is that server migrated to a new IP. Is there a way to change IP that is hardcoded?