Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Has Anybody Used

Has anybody used albumentations with pytorch in remote execution on clearml? The task is freezing during DataLoader initialization every time I put any augmentations there.

  
  
Posted one year ago
Votes Newest

Answers 6


We use albumentations with scripts that execute remotely and have no issues. Good question from CostlyOstrich36

  
  
Posted one year ago

never tried to start it locally in clearml

  
  
Posted one year ago

we normally do something like that - not sure what why it's freezing for you without more info

  
  
Posted one year ago

When you run it locally it doesn't freeze?

  
  
Posted one year ago

When I run locally w/o clearml, it doesn't freeze

  
  
Posted one year ago

` # dataset_class.py
from PIL import Image
from torch.utils.data import Dataset as BaseDataset

class Dataset(BaseDataset):

def __init__(
    self,
    images_fps,
    masks_fps,
    augmentation=None,
):
    self.augmentation = augmentation
    self.images_fps = images_fps
    self.masks_fps = masks_fps
    self.ids = len(images_fps)


def __getitem__(self, i):

    # read data
    img = Image.open(self.images_fps[i])
    mask = Image.open(self.masks_fps[i])

    # apply augmentations
    if self.augmentation:
        sample = self.augmentation(image=img, mask=mask)
        image_aug, mask_aug = sample["image"], sample["mask"]

    return image_aug, mask_aug

def __len__(self):
    return self.ids

training_script.py

from dataset_class import Dataset
import albumentations as albu
from torch.utils.data import DataLoader

usual clearml config etc

train_albs = [
albu.HorizontalFlip(p=0.5),
]
augs = albu.Compose(train_albs)

train_dataset = Dataset(images_fps, masks_fps, augmentations=augs)

train_loader_l = DataLoader(
train_dataset,
batch_size=16,
shuffle=True,
num_workers=0,
) `

  
  
Posted one year ago
404 Views
6 Answers
one year ago
10 months ago
Tags