Unanswered
Hello Everyone, While Calling Get_Local_Copy Of The Dataset From The Fileserver, I Get The Path To The Local Copy, But The Files Are Not Downloaded And The Folder Is Empty. Tell Me What Could Be The Problem. I Don'T Get Any Additional Errors Or Warnings.
@<1523701070390366208:profile|CostlyOstrich36> Yes, sure
import pandas as pd
import yaml
import os
from omegaconf import OmegaConf
from clearml import Dataset
config_path = 'configs/structured_docs.yml'
with open(config_path) as f:
config = yaml.full_load(f)
config = OmegaConf.create(config)
path2images = config.data.images_folder
def get_data(config, split):
path2annotation = os.path.join(config.data.annotation_folder, f"sample_{split}.csv")
data = pd.read_csv(path2annotation)
return data
data_train = get_data(config, 'train')
data_val = get_data(config, 'val')
data = pd.concat([data_val, data_train])
files = [os.path.join(path2images, file) for file in data['filename'].values]
dataset = Dataset.create(
dataset_name="test OCR dataset",
dataset_project="Text Recognition"
)
for file in files:
dataset.add_files(path=file)
dataset.upload()
dataset.finalize()
With this script, I uploaded data to the server. You can also see the final status of the dataset in the screenshot.
176 Views
0
Answers
one year ago
one year ago