Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey All, Is There A Way To Upload A Fiftyone Dataset As An Artifact In A Clearml Pipeline? I Am Getting The Following Error When I Try To Upload It

Hey all, is there a way to upload a fiftyone dataset as an artifact in a clearml pipeline? I am getting the following error when I try to upload it
Can't pickle <class 'mongoengine.base.metaclasses.samples.6626e9ff3a168bc30c5624b2'>: attribute lookup samples.6626e9ff3a168bc30c5624b2 on mongoengine.base.metaclasses failed

  
  
Posted 7 months ago
Votes Newest

Answers 6


Hi @<1610083503607648256:profile|DiminutiveToad80> ! You need to somehow serialize the object. Note that we try different serialization methods and default to pickle if none work. If pickle doesn't work then the artifact can't be uploaded by default. But there is a way around it: you can serialize the object yourself. The recommended way to do this is using the serialization_function argument in upload_artifact . You could try using something like dill which can serialize more objects than pickle .
Note that you need to specify a deserialization_function when pulling the artifact.

  
  
Posted 7 months ago

Thanks!

  
  
Posted 7 months ago

Hi @<1610083503607648256:profile|DiminutiveToad80> , I'd suggest using the Datasets feature. But you can however of course upload it as artifacts.

Where are you trying to upload it? Can you provide the full log? Also, a code snippet would help.

  
  
Posted 7 months ago

File "/opt/conda/envs/bumlo/lib/python3.10/site-packages/clearml/binding/artifacts.py", line 745, in upload_artifact
pickle.dump(artifact_object, f)
_pickle.PicklingError: Can't pickle <class 'mongoengine.base.metaclasses.samples.6627e5ecc60879fe5e49cee6'>: attribute lookup samples.6627e5ecc60879fe5e49cee6 on mongoengine.base.metaclasses failed

  
  
Posted 7 months ago

@<1523701070390366208:profile|CostlyOstrich36>

  
  
Posted 7 months ago

dataset = fo.Dataset.from_dir(
labels_path=labels_path,
dataset_type=fo.types.COCODetectionDataset,
label_field="ground_truth",
use_polylines=True
)
task.upload_artifact(
name="Dataset",
artifact_object=dataset,
)

  
  
Posted 7 months ago
548 Views
6 Answers
7 months ago
7 months ago
Tags
Similar posts