22.04

CostlyOstrich36 I have used ds.get_logger().report_image() in a loop in order to upload all the images from the dataset as previews. Then I called flush() , which returned True . However, the images are still not shown. Any ideas - how could I debug that?

` from clearml import Dataset

IMG_PATH = "/home/mfb/Temp/sample-ds/50-ok.jpg"

Create dataset and add sample image

ds = Dataset.create(dataset_name="Test", dataset_project="Dataset-Test")
ds.add_files(path=IMG_PATH)
ds.upload()

Add and report image

logger = ds.get_logger()
logger.report_image("image", "sample image", iteration=0, local_path=IMG_PATH)
logger.flush()

Finalize the dataset

ds.finalize() `

CostlyOstrich36 I do see the new "image" in the metric drop down, so something has changed - but there are no preview images there.

My server is hosted on AWS Fargate

I am using the latest version clearml server and I am using version 1.9.1 for the sdk.

Here is the code that I am currently using:

if __name__ == "__main__":
    # create clearml data processing task
    dataset = Dataset.create(
        dataset_name="palmer_penguins",
        dataset_project="palmer penguins",
        dataset_tags=["raw"]
    )
    dataset_path = "data/raw/penguins.csv"
    # add the downloaded files to the current dataset
    dataset.add_files(path=dataset_path)
    # upload data to clearml server
    dataset.upload(verbose=True)
    # close the dataset
    dataset.finalize(verbose=True)

This is a simple dummy example that I use for testing deployments of clearml server

Here is the output that I would expect:

@<1523701070390366208:profile|CostlyOstrich36> I am facing the same issue:

{"meta":{"id":"90841d05dfb1431a8d9dfc6bfdb39f9e","trx":"90841d05dfb1431a8d9dfc6bfdb39f9e","endpoint":{"name":"events.debug_images","requested_version":"2.23","actual_version":"2.7"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"metrics":[]}}

I don't think datasets don't have visualization out of the box, you need to add these previews manually. Only HyperDatasets feature from the Scale & Enterprise versions truely visualizes all the data.

According to your code snippet there isn't any visualization add on top of the dataset

And here the same in Chrome:

Also when in this view, open developer tools (F12) and see what calls you get back for debug samples

I have been using them for a while, they've always had autopreviews.

Might need to refresh page after opening dev tools 🙂

What OS are you running the scripts on, Abed?

This is how I usually add visualization

#Report data preview
ds.get_logger().report_table(title="Data Sample", series="First Ten Rows", table_plot=data1[:10])
ds.upload()
ds.finalize()

The thing is, even on the community server, not all the datasets have automatic previews. So for the same code/dataset, some of the runs have previews and some of them don't.

@<1523701168822292480:profile|ExuberantBat52> , did you add debug samples in a similar fashion? What version of the clearml sdk are you using? Also what server?

@<1523701070390366208:profile|CostlyOstrich36> Hi John, I have posted that in the screenshot above. Isn't that what you meant?

Looping in @<1523703436166565888:profile|DeterminedCrab71> & @<1523701435869433856:profile|SmugDolphin23> for visibility

How about when you view it in the datasets view? Also what version of clearml package do you have?

ProudElephant77 , can you please add a code snippet of what you did?

CostlyOstrich36 Unfortunately that didn't helped 😞

What if you add images to the dataset? Can you see them being previewed? @<1523701168822292480:profile|ExuberantBat52>

This is strange. Exactly same code works for me. What version of clearml are you using? What version is your server?

Yes, I am using a virtualenv that has pandas and clearml installed.

Can you please paste the response from events.debug_images ?

CostlyOstrich36 Do you have any idea how I could debug that?

Actually, datasets should have an automatic preview...

The above output is on the clearml community server

Hmm my bad, I wasn't aware of this 🙂

CostlyOstrich36 WebApp: 1.9.1-312 • Server: 1.9.1-312 • API: 2.23