Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Unanswered
Hello Everyone, While Calling Get_Local_Copy Of The Dataset From The Fileserver, I Get The Path To The Local Copy, But The Files Are Not Downloaded And The Folder Is Empty. Tell Me What Could Be The Problem. I Don'T Get Any Additional Errors Or Warnings.


@<1523701070390366208:profile|CostlyOstrich36> Yes, sure

import pandas as pd
import yaml
import os
from omegaconf import OmegaConf
from clearml import Dataset

config_path = 'configs/structured_docs.yml'

with open(config_path) as f:
    config = yaml.full_load(f)

config = OmegaConf.create(config)
path2images = config.data.images_folder


def get_data(config, split):
    path2annotation = os.path.join(config.data.annotation_folder, f"sample_{split}.csv")
    data = pd.read_csv(path2annotation)
    return data


data_train = get_data(config, 'train')
data_val = get_data(config, 'val')
data = pd.concat([data_val, data_train])

files = [os.path.join(path2images, file) for file in data['filename'].values]


dataset = Dataset.create(
    dataset_name="test OCR dataset",
    dataset_project="Text Recognition"
)
for file in files:
    dataset.add_files(path=file)

dataset.upload()
dataset.finalize()

With this script, I uploaded data to the server. You can also see the final status of the dataset in the screenshot.

  
  
Posted 11 months ago
103 Views
0 Answers
11 months ago
11 months ago