Reputation
Badges 1
7 × Eureka!Hi @<1523701070390366208:profile|CostlyOstrich36>
here is the full code:
import os
import sys,shutil
import clearml
from clearml import Task, Dataset, Logger
from clearml import PipelineDecorator, PipelineController
project_name = "Titanic Project"
dataset_name = "titanic_data"
datasets = Dataset.list_datasets()
for dataset in datasets:
if dataset["project"] == project_name and dataset["name"] == dataset_name:
parent_datasets_id = dataset["id"]
print(parent_datasets_id)
dso...
Hi @<1523701205467926528:profile|AgitatedDove14>
Thanks alot for pointing out "dso" variable 🙂 . I did not realize that same name i was using for create and get dataset function.
It was my mistake, i changed the variable and everything is functional as expected.
Thank you again.
Please see this code snippet:
reviews_dataset_path = Dataset.get(
dataset_id=config["reviews_data_file_id"],
only_completed=True,
only_published=False,
). get_local_copy()
is there a way we can still read the data without using ". get_local_copy()"
Thanks a lot @<1523701070390366208:profile|CostlyOstrich36> your comment boosted my confidence and i'll deep dive into clearML from now on.
What I wanted is already there in clearML, my worry was why to waste time in learning other tool if clearML is already serving the whole purpose.
May be I can start posting few enhancements which might be good for whole clearML community.
Hi @<1523701205467926528:profile|AgitatedDove14>
first of all many thanks to you for always replying promptly 🙂
Initially I thought function is defined outside that is why I was getting that error. However I am using this way and that also gives error:
@PipelineDecorator.component(return_values=["op"])
def step1():
op = "I am in step-1"
return op
@PipelineDecorator.component(return_values=["op"])
def step2(op):
return ( lambda op:step1(op))
While executing step2, I g...
Hi @<1523701205467926528:profile|AgitatedDove14>
What I mean is, dataset is loaded in server, I want to call that dataset in local PC without downloading its copy and do the analysis.
Thats nice to hear.
Thanks a lot @<1523701205467926528:profile|AgitatedDove14> :-)