Actually, datasets should have an automatic preview...
@<1523701070390366208:profile|CostlyOstrich36> It looks like this:
I am using the latest version clearml server
and I am using version 1.9.1
for the sdk.
Here is the code that I am currently using:
if __name__ == "__main__":
# create clearml data processing task
dataset = Dataset.create(
dataset_name="palmer_penguins",
dataset_project="palmer penguins",
dataset_tags=["raw"]
)
dataset_path = "data/raw/penguins.csv"
# add the downloaded files to the current dataset
dataset.add_files(path=dataset_path)
# upload data to clearml server
dataset.upload(verbose=True)
# close the dataset
dataset.finalize(verbose=True)
This is a simple dummy example that I use for testing deployments of clearml server
Here is the output that I would expect:
@<1523701070390366208:profile|CostlyOstrich36> I am facing the same issue:
{"meta":{"id":"90841d05dfb1431a8d9dfc6bfdb39f9e","trx":"90841d05dfb1431a8d9dfc6bfdb39f9e","endpoint":{"name":"events.debug_images","requested_version":"2.23","actual_version":"2.7"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"metrics":[]}}
CostlyOstrich36 The latest clearml package, installed these days - 1.9.0
The dataset view is also empty:
Looping in @<1523703436166565888:profile|DeterminedCrab71> & @<1523701435869433856:profile|SmugDolphin23> for visibility
Above is the response for the events.debug_images
What OS are you running the scripts on, Abed?
Also when in this view, open developer tools (F12) and see what calls you get back for debug samples
What if you add images to the dataset? Can you see them being previewed? @<1523701168822292480:profile|ExuberantBat52>
The above output is on the clearml community server
Yes, I am using a virtualenv that has pandas and clearml installed.
I am currently running the scripts on WSL ubuntu
Might need to refresh page after opening dev tools 🙂
So I added the snippet above to the code,
and now the preview for the first 10 rows shows up. However, the automatic preview is still not working.
@<1523701168822292480:profile|ExuberantBat52> Do you have pandas installed on your machine?
How about when you view it in the datasets view? Also what version of clearml
package do you have?
It isn't a bug, you have to add the previews manually through reporting. For example:
ds = Dataset.create(...) ds.add_files(...) ds.get_logger().report_media(...)
Thanks a lot, I will have a look at that!
Can you please paste the response from events.debug_images
?
CostlyOstrich36 I have used ds.get_logger().report_image()
in a loop in order to upload all the images from the dataset as previews. Then I called flush()
, which returned True
. However, the images are still not shown. Any ideas - how could I debug that?
ProudElephant77 , I think you might need to finalize the dataset for it to appear
` from clearml import Dataset
IMG_PATH = "/home/mfb/Temp/sample-ds/50-ok.jpg"
Create dataset and add sample image
ds = Dataset.create(dataset_name="Test", dataset_project="Dataset-Test")
ds.add_files(path=IMG_PATH)
ds.upload()
Add and report image
logger = ds.get_logger()
logger.report_image("image", "sample image", iteration=0, local_path=IMG_PATH)
logger.flush()
Finalize the dataset
ds.finalize() `
ProudElephant77 , can you please add a code snippet of what you did?
CostlyOstrich36 Unfortunately that didn't helped 😞