CostlyOstrich36

` from clearml import Dataset

IMG_PATH = "/home/mfb/Temp/sample-ds/50-ok.jpg"

Create dataset and add sample image

ds = Dataset.create(dataset_name="Test", dataset_project="Dataset-Test")
ds.add_files(path=IMG_PATH)
ds.upload()

Add and report image

logger = ds.get_logger()
logger.report_image("image", "sample image", iteration=0, local_path=IMG_PATH)
logger.flush()

Finalize the dataset

ds.finalize() `

Might need to refresh page after opening dev tools 🙂

Hmm my bad, I wasn't aware of this 🙂

Actually, datasets should have an automatic preview...

@<1523701070390366208:profile|CostlyOstrich36> It looks like this:

Thanks a lot, I will have a look at that!

How about when you view it in the datasets view? Also what version of clearml package do you have?

ProudElephant77 , can you please add a code snippet of what you did?

CostlyOstrich36 I have used ds.get_logger().report_image() in a loop in order to upload all the images from the dataset as previews. Then I called flush() , which returned True . However, the images are still not shown. Any ideas - how could I debug that?

Can you share a screenshot of what you see?

I am using the latest version clearml server and I am using version 1.9.1 for the sdk.

Here is the code that I am currently using:

if __name__ == "__main__":
    # create clearml data processing task
    dataset = Dataset.create(
        dataset_name="palmer_penguins",
        dataset_project="palmer penguins",
        dataset_tags=["raw"]
    )
    dataset_path = "data/raw/penguins.csv"
    # add the downloaded files to the current dataset
    dataset.add_files(path=dataset_path)
    # upload data to clearml server
    dataset.upload(verbose=True)
    # close the dataset
    dataset.finalize(verbose=True)

This is a simple dummy example that I use for testing deployments of clearml server

Here is the output that I would expect:

CostlyOstrich36 WebApp: 1.9.1-312 • Server: 1.9.1-312 • API: 2.23

It isn't a bug, you have to add the previews manually through reporting. For example:

ds = Dataset.create(...) ds.add_files(...) ds.get_logger().report_media(...)

This is how I usually add visualization

#Report data preview
ds.get_logger().report_table(title="Data Sample", series="First Ten Rows", table_plot=data1[:10])
ds.upload()
ds.finalize()

Looping in @<1523703436166565888:profile|DeterminedCrab71> & @<1523701435869433856:profile|SmugDolphin23> for visibility

Above is the response for the events.debug_images

My server is hosted on AWS Fargate

Yes, I am using a virtualenv that has pandas and clearml installed.

Can you please paste the response from events.debug_images ?

CostlyOstrich36 The latest clearml package, installed these days - 1.9.0

The dataset view is also empty:

What OS are you running the scripts on, Abed?

I don't think datasets don't have visualization out of the box, you need to add these previews manually. Only HyperDatasets feature from the Scale & Enterprise versions truely visualizes all the data.

According to your code snippet there isn't any visualization add on top of the dataset

This is strange. Exactly same code works for me. What version of clearml are you using? What version is your server?

I have been using them for a while, they've always had autopreviews.

@<1523701168822292480:profile|ExuberantBat52> , did you add debug samples in a similar fashion? What version of the clearml sdk are you using? Also what server?

The above output is on the clearml community server

What if you add images to the dataset? Can you see them being previewed? @<1523701168822292480:profile|ExuberantBat52>

22.04

@<1523701168822292480:profile|ExuberantBat52> Do you have pandas installed on your machine?