@<1702492411105644544:profile|YummyGrasshopper29> , I suggest you take a look here - None
I solve problem by adding container argument
--network host
I have GCP instance with official clearml image.
from clearml import StorageManager, Dataset
dataset = Dataset.create(
dataset_project="Project", dataset_name="Dataset_name"
)
files = [
'file.csv',
'file1.csv',
]
for file in files:
csv_file = StorageManager.get_local_copy(remote_url=file)
dataset.add_files(path=csv_file)
# Upload dataset to ClearML server (customizable)
dataset.upload()
# commit dataset changes
dataset.finalize()
@<1702492411105644544:profile|YummyGrasshopper29> , how did you save the dataset? Where was the data uploaded to?
I am running clearml server on gcp, but I didn't exposed ports instead I ssh to machine and do port forwarding to localhost. The problem is localhost on my machine is not same as localhost inside docker on worker. If I check dataset, files are stored in localhost, but actually it is not localhost. Didn't fond the solution yet how to properly setup hostname for dataserver. Any ideas?
Hi @<1702492411105644544:profile|YummyGrasshopper29> , clearml registers the uploaded artifacts (including datasets) with the URLs used to upload them, which is why the day is registered under localhost in your case. I think the solution in your case is to use url substitution
In the webUI, when you go to the dataset, where do you see it is saved? You can click on 'full details' in any version of a dataset and see that in the artifacts section