Reputation
Badges 1
109 × Eureka!let me know if you need any help/ have issues trying to reproduce...thanks!
@ https://app.slack.com/team/U01J3C692M8 where you able to come up with a solution?
if I list_files the new dataset, I see the same files π
but I don't see any change...where is the link to the file removed from?
How are these two datasets different?
Thanks π
sounds like you need to run a service to monitor for new commits in PROJ_1, to trigger the pipeline
I think so, but I'm not an expert here, I started using this a few weeks ago
take a look at the cleanup service for reference:
https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py
` from clearml import Task
import argparse
only create the task, we will actually execute it later
task = Task.init(project_name='examples', task_name='pipeline demo',
task_type=Task.TaskTypes.controller, reuse_last_task_id=False)
task.execute_remotely()
args = {'dataset_path' : ''}
task.connect(args, section='Args') `like this?
ah, I see master is the same as 0.17.5rc3
if I change line 217 in dataset.py
:fnmatch(k, '*/'+ wildcard)
forfnmatch(k, wildcard)
then it works :-)
I don't understand what I'm doing wrong..how are you supposed to call the remove_files
function?
I don't see anything in the CONFIGURATION section:
clearml == 0.17.5rc5
google_cloud_storage == 1.36.1
joblib == 1.0.1
matplotlib == 3.3.4
numpy == 1.20.0
object_detection == 0.1
opencv_python_headless == 4.5.1.48
pandas == 1.2.3
scikit_learn == 0.24.1
tensorflow == 2.4.0
This is only with Detect API ?
I only tested it with the Detect API
I'll need to see how to extract only the part that we care about
It's kind of a pain to setup Tensorflow Object Detection API
Hi! were you able to reproduce the issue CostlyOstrich36 ?
and is this image just a VM with a docker?
I've never created an image before..is it normal to take this long? 15 minutes?
all is good π
great, thanks!
I did, but I still have the same issue..
tglema@mvd0000xlrndtl2Β clearml-src
git:(28b8502) β
git status
HEAD detached at 0.17.5rc3
I did a python setup.py develop, and ran the script:
` from clearml import Dataset
dataset = Dataset.create(dataset_project='test', dataset_name='example')
dataset.add_files('/home/tglema/example.jpeg')
dataset.add_files('/home/tglema/logo.png')
print(dataset.list_files())
dataset.upload()
dataset.finalize()
dataset_new = Dataset.create...
If I download each file in the folder using StorageManager.get_local_copy(remote_url=gs_url)
, it works OK
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/compute/api/create_instance.py I think this is a nice place to start
it would be completed right after the upload
your example with absl package worked