Hi,
I wanted to ask whether using ClearML StorageManager class downloads the whole dataset from the remote_url or uses incremental downloading?
I've created a dataset from an S3 bucket, but when I'm trying to train a YOLOV8 model I get:
TypeError: expected str, bytes or os.PathLike object, not NoneType
MWE:
from ultralytics import YOLO
from clearml import StorageManager
# Load the model
model = YOLO(model="yolov8n.pt")
# Get dataset from S3
data_path = StorageManager.get_local_copy(remote_url="
")
# Pass the dataset path to YOLO training
results = model.train(data=data_path, epochs=100, imgsz=640, batch=1, pretrained=True, classes=1)
I don't understand what is "the DataOps feature" in this context.
My question is simple: is it possible to use clearml to train a YOLOV8 model without downloading the whole dataset. The example I provided is not required for answering, just for convenience and reproducibility of the issue.
Hi @<1750327614469312512:profile|CrabbyParrot75> , why use the StorageManager module and not the Datasets to manage your data?
The DataOps feature will abstract your usage of data - None
