I'm getting a response 401

However when i'll reset or clone the task, now it won't just accept the default value but clearml will pass the arg directly

I'm not using decorators. I have a bunch of function_steps followed by a normal task step, where I've passed a base_task_id.

I want to check the value of one of the functional steps, and if it holds true, I want to execute the task step otherwise I want the pipeline to end there, since the task step is the last one.

Well I'm still researching how it'll work. I'm expecting it to not be very good and will make the model learning very stochastic in nature.

I plan to instead at the training stage, instead of just getting this model, use Dataset.squash, to get previous M datasets merged together.

This should introduce stability in the dataset.

Also this way, our model is trained on a batch of data multiple times but only for a few times before that batch is discarded. We keep the training data fresh for co...

Thank you, I'll start reading up on this once I've finished setting up the basic pipeline

Thank you.

Set up is on a single machine, I have a nas mounted where I'm watching a folder, if there are sufficient images, it should publish the data but since I was using start_remotely, the code was running somewhere else and couldn't access folder.

Additional Error info

Launching job: ScheduleJob(name='Watch Checkbox Detector Buffer Folder', base_task_id='', base_function=<function watch_folder at 0x7f6e308b6840>, queue=None, target_project=None, single_instance=False, task_parameters=None, task_overrides=None, clone_task=True, _executed_instances=['140111227815680', '140111227815680', '140111227815680', '140111227815680'], execution_limit_hours=None, recurring=True, starting_time=datetime.datetime(2021, 11, 25, 9, 45, 41, 175873), min...

I'll create a github issue. Overall I hope you understand.

Still unsure between finalize and publish? Since upload should upload the data to the server

AgitatedDove14 Can you help me with this? Maybe something like storing the returned values or something in a variable outside the pipeline?

When I try to access the server with the IP I set as CLEARML_HOST_IP, it looks like this. I set that IP to the ip assigned to me by the network

There's a whole task bar on the left in the server. I only get this page when i use the ip 0.0.0.0

Anyway I could apparently delete things in the dataset from the local copy. Isn't it supposed to be immutable?

I got to that conclusion I think yeah. Basically, can access them as artifacts.

I use that model in another step.

Well yeah, you can say that. In add function step, I pass in a function which returns something. And since I've written the name of the returned parameter in add_function_step, I can use it, but I can't seem to figure out a way to do something similar using a task in add_step

is this the correct way to upload an artifact?

checkpoint.split('.')[0] is the name that I want it assigned and the second argument is the path to the file.

That makes sense. But doesn't that also hold true for dataset.get_local_mutable_copy()?

Also the repository is on bitbucket which is why I set git_host to that.

Thus I wanted to pass the model id from the prior step to the next one.

Honestly anything. I tried looking up on youtube but There's very little material there, especially which is up to date. It's understandable given that ClearML is still in beta. I can look at courses / docs. I just want to be pointed in the right direction as to what I should look up and study

This is the simplest I could get for the inference request. The model and input and output names are the ones that the server wanted.

I'll take a look.

Basically want to be able to serve a model, and also send requests to it for inference.

Have never done something like this before, and I'm unsure about the whole process from successfully serving the model to sending requests to it for inference. Is there any tutorial or example for it?

I just followed the instructions here at https://github.com/allegroai/clearml-serving

In the end it says I can curl at the end point and mentions the serving-engine-ip but I cant find the ip anywhere.

I've tried the ip of the ClearML Server and the IP of my local machine on which the agent is also running on and none of the two work.

Yeah I think I did. I followed the tutorial on the repo.

AnxiousSeal95 I just have a question, can you share an example of accessing an artifact of a previous step in the pre execute callback?