I'll do that.

1.1.5

trigger_scheduler.add_dataset_trigger(schedule_task_id=TASK_ID, schedule_queue='default',
trigger_project='Cassava Leaf Disease Classification', name='start task on data - End Game')

because those spawned processes are from a file register_dataset.py , however I'm personally not using any file like that and I think it's a file from the library.

Ok so update. It works now. last steps I did I can remember to fix it are.

Yeah

Another question, in the parents sequence in pipe.add_step, we have to pass in the name of the step right?

I know how to enqueue in using the UI. I'm trying to do it programatically.

Oh oh oh. Wait a second. I think I get what you're saying. When I'm originally creating clearml-task, since I'm not passing the argument myself, so it just uses the value False.

the storage is basically the machine the clearml server is on, not using s3 or anything

As I go through the model.py file, I get what you're saying. Only problem is in the case of AutoLogging, I don't have the model id, for the model being saved.

It was working fine for a while but then it just failed.

So right now, I'm creating an OutputModel and passing the current task in the constructor. Then I just save the tensorflow keras model. When I look at the details, model artifact in the ClearML UI, it's been saved the usual way, and no tags that I added in the OutputModel constructor are there. From which to me it seems that ClearML is auto logging the model, and the model isn't connected to the OutputModel object that I created.

You're saying that the model should get connected if I call up...

Yeah

I shared the error above. I'm simply trying to make the yolov5 by ultralytics part of my pipeline.

My draft is View Only but the cloned toy task one is in normal Draft mode.

Collecting idna==3.3
Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting importlib-metadata==4.8.2
Using cached importlib_metadata-4.8.2-py3-none-any.whl (17 kB)
Collecting importlib-resources==5.4.0
Using cached importlib_resources-5.4.0-py3-none-any.whl (28 kB)
ERROR: Could not find a version that satisfies the requirement jsonschema==4.2.1 (from -r /tmp/cached-reqsm1gu3664.txt (line 19)) (from versions: 0.1a0, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8.0, 1.0.0, 1.1.0, 1.2.0, 1.3.0, 2.0...

It says command not found.

Well I'm still researching how it'll work. I'm expecting it to not be very good and will make the model learning very stochastic in nature.

I plan to instead at the training stage, instead of just getting this model, use Dataset.squash, to get previous M datasets merged together.

This should introduce stability in the dataset.

Also this way, our model is trained on a batch of data multiple times but only for a few times before that batch is discarded. We keep the training data fresh for co...

https://github.com/ultralytics/yolov5

As of yet, I can only select ones that are visible and to select more, i'll have to click on view more, which gets extremely slow.

so when I run the task using clearml-task --repo and create a task, it runs fine. It runs into the above error when I clone the task or reset it.

Can you please share the endpoint link?

I understand that storing data outside ClearML won't ensure its immutability. I guess this can be built in as a feature into ClearML at some future point.

This is the original repo which I've slightly modified.

However cloning it uses it from the clearml args, which somehow converts it to string?

before pipe.add_step(train_model)?

Anyway I could apparently delete things in the dataset from the local copy. Isn't it supposed to be immutable?

I did what you said, and got the pipeline DAG and then the executed of the step to use as ID. Thank you it worked fine.

shouldn't checkpoints be uploaded immediately, that's the purpose of checkpointing isn't it?