Reputation
Badges 1
6 × Eureka!batches = [batch_1, batch_2, batch_3]
if name == 'main':
print("Create the dataset in ClearML")
dataset = Dataset.create(dataset_name="general_ner_es",
dataset_project='general_ner',
output_uri=' None ')
for batch in batches:
df = pandasDF_from_annotations(bucket_name, batch_1)
[df.to](http://df.to) _pickle('df.pkl')
...
thanks a lot Eugen, that was the issue
Traceback (most recent call last):
File "/root/ehread-playgrounds/bbiescas/NER-ES/clearml_pipelines/./step_1_clearml_dataset.py", line 38, in <module>
dataset = Dataset.create(dataset_name="general_ner_es",
File "/root/.clearml/venvs-builds/3.10/lib/python3.10/site-packages/clearml/datasets/dataset.py", line 1248, in create
parent_datasets = [cls.get(dataset_id=p) if not isinstance(p, Dataset) else p for p in (parent_datasets or [])]
File "/root/.clearml/venvs-builds/3.10/lib/py...
I'm working in an interactive session
can you give me an example? And can you give an example of the vault configuration in case I need to run the pipeline remotely? Thanks in advance