Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Guys, In A Fresh Self-Hosted Clearml Server I Have The Issue That I Cannot See Any Preview Images Of The Files In A Dataset. It Seems Like A Bug, But Maybe I Am Doing Something Wrong?

Hey guys, in a fresh self-hosted ClearML server I have the issue that I cannot see any preview images of the files in a dataset. It seems like a bug, but maybe I am doing something wrong? https://github.com/allegroai/clearml/issues/879 - does someone have an idea what I might be doing wrong?

  
  
Posted one year ago
Votes Newest

Answers 42


And here the same in Chrome:
image

  
  
Posted one year ago

I don't think datasets don't have visualization out of the box, you need to add these previews manually. Only HyperDatasets feature from the Scale & Enterprise versions truely visualizes all the data.

According to your code snippet there isn't any visualization add on top of the dataset

  
  
Posted one year ago

Actually, datasets should have an automatic preview...

  
  
Posted one year ago

This is how I usually add visualization

#Report data preview
ds.get_logger().report_table(title="Data Sample", series="First Ten Rows", table_plot=data1[:10])
ds.upload()
ds.finalize()
  
  
Posted one year ago

What if you add images to the dataset? Can you see them being previewed? @<1523701168822292480:profile|ExuberantBat52>

  
  
Posted one year ago

@<1523701070390366208:profile|CostlyOstrich36> It looks like this:
image

  
  
Posted one year ago

` from clearml import Dataset

IMG_PATH = "/home/mfb/Temp/sample-ds/50-ok.jpg"

Create dataset and add sample image

ds = Dataset.create(dataset_name="Test", dataset_project="Dataset-Test")
ds.add_files(path=IMG_PATH)
ds.upload()

Add and report image

logger = ds.get_logger()
logger.report_image("image", "sample image", iteration=0, local_path=IMG_PATH)
logger.flush()

Finalize the dataset

ds.finalize() `

  
  
Posted one year ago

How about when you view it in the datasets view? Also what version of clearml package do you have?

  
  
Posted one year ago

CostlyOstrich36 Do you have any idea how I could debug that?

  
  
Posted one year ago

Above is the response for the events.debug_images

  
  
Posted one year ago

CostlyOstrich36 I have used ds.get_logger().report_image() in a loop in order to upload all the images from the dataset as previews. Then I called flush() , which returned True . However, the images are still not shown. Any ideas - how could I debug that?

  
  
Posted one year ago

It isn't a bug, you have to add the previews manually through reporting. For example:

ds = Dataset.create(...) ds.add_files(...) ds.get_logger().report_media(...)

  
  
Posted one year ago

I am using the latest version clearml server and I am using version 1.9.1 for the sdk.

Here is the code that I am currently using:

if __name__ == "__main__":
    # create clearml data processing task
    dataset = Dataset.create(
        dataset_name="palmer_penguins",
        dataset_project="palmer penguins",
        dataset_tags=["raw"]
    )
    dataset_path = "data/raw/penguins.csv"
    # add the downloaded files to the current dataset
    dataset.add_files(path=dataset_path)
    # upload data to clearml server
    dataset.upload(verbose=True)
    # close the dataset
    dataset.finalize(verbose=True)

This is a simple dummy example that I use for testing deployments of clearml server

Here is the output that I would expect:
image

  
  
Posted one year ago

@<1523701168822292480:profile|ExuberantBat52> Do you have pandas installed on your machine?

  
  
Posted one year ago

CostlyOstrich36 I do see the new "image" in the metric drop down, so something has changed - but there are no preview images there.

  
  
Posted one year ago

Thanks a lot, I will have a look at that!

  
  
Posted one year ago

Might need to refresh page after opening dev tools 🙂

  
  
Posted one year ago

ProudElephant77 , can you please add a code snippet of what you did?

  
  
Posted one year ago

CostlyOstrich36 Unfortunately that didn't helped 😞

  
  
Posted one year ago

CostlyOstrich36

  
  
Posted one year ago

This is strange. Exactly same code works for me. What version of clearml are you using? What version is your server?

  
  
Posted one year ago

CostlyOstrich36 WebApp: 1.9.1-312 • Server: 1.9.1-312 • API: 2.23

  
  
Posted one year ago

Also when in this view, open developer tools (F12) and see what calls you get back for debug samples

  
  
Posted one year ago

CostlyOstrich36 The latest clearml package, installed these days - 1.9.0

The dataset view is also empty:

  
  
Posted one year ago

@<1523701070390366208:profile|CostlyOstrich36> Hi John, I have posted that in the screenshot above. Isn't that what you meant?

  
  
Posted one year ago

@<1523701168822292480:profile|ExuberantBat52> , did you add debug samples in a similar fashion? What version of the clearml sdk are you using? Also what server?

  
  
Posted one year ago

Yes, I am using a virtualenv that has pandas and clearml installed.

  
  
Posted one year ago

@<1523701070390366208:profile|CostlyOstrich36> I am facing the same issue:

{"meta":{"id":"90841d05dfb1431a8d9dfc6bfdb39f9e","trx":"90841d05dfb1431a8d9dfc6bfdb39f9e","endpoint":{"name":"events.debug_images","requested_version":"2.23","actual_version":"2.7"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"metrics":[]}}
  
  
Posted one year ago

I am currently running the scripts on WSL ubuntu

  
  
Posted one year ago

The above output is on the clearml community server

  
  
Posted one year ago