link to the line please 🙂

if the first task failed - then the remaining task are not schedule for execution which is what I expect.

agreed

I'm just surprised that if the first task is

aborted

instead by the user,

How is that different from failed? The assumption is if a component depends on another one it needs its output, if it does not then they can run in parallel. What am i missing?

How did you define the decorator of "train_image_classifier_component" ?
Did you define:
@PipelineDecorator.component(return_values=['run_model_path', 'run_tb_path'], ...Notice two return values

Yes, as long as the client is served from http://app.something.com it will look for the api server at http://api.something.com

We should probably change it so it is more human readable 🙂

Or am I forced to do a get, check if the latest version is fainallyzed,

Dataset Must be finalized before using it. The only situation where it is not is because you are still in the "upload" state.

, then increment de version of that version and create my new version ?

I'm assuming there is a data processing pipeline pushing new data?! How do you know you have new data to push?

Correct 🙂

Come to think about it, maybe we should have "parallel_for" as a utility for the pipeline since this is so useful

SmarmySeaurchin8 it could be a switch, the problem is that when you have automatic stopping flows, they will abort a task, which is legitimate (e.g. should not considered failed)
How come you have aborted tasks in the pipeline ? If you want to abort the pipeline, you need to first abort the pipeline Task then the tasks themselves.

I located the issue, I'm assuming the fix will be in the next RC 🙂
(probably tomorrow or before the weekend)

For example HPO, early stopping. It would mark the Task as aborted. Make sense ?

SoreDragonfly16
btw: The difference between the two graphs is the ratio pf the graph display , that it 🙂

I think it should be treated as failed,

I'm not sure where I stand on default behavior, it it could easily be an argument for the pipeline controller

None

WackyRabbit7
Long story short, yes, only by name (hashing might be too slow on large files)
The easiest solution, if the hash is incorrect, delete the local copy it returns, and ask again, it will download it.
I'm not sure if the hashing is exposed, but if it is not, we can add it.
What do you think?

are you referring to the same line? 47 in cache.py?

😊

I aborted the task because of a bug on my side

🙂

Following this one, is treating abort as failed a must feature for the pipeline (in your case) or is it sort of a bug in your opinion ?

Where is darts reporting scalars ?

Where do you have your Task.init ?

Oh that's definitely off 🙂
Can you send a quick toy snippet to reproduce it ?

I found something btw, let me check...

Well I guess you can say this is definitely not self explanatory line 😉
but, it is actually asking whether we should extract the code, think of it as:
if extract_archive and cached_file: return cls._extract_to_cache(cached_file, name)

That is actually what you are looking for:
https://github.com/allegroai/trains/blob/04b3fa809bb73d7101d1995327684ebe5b2911e3/trains/storage/cache.py#L46

Hmm interesting, will pass it along to FE 🙂 3. That is nice! I wonder if this is built into the graph library

Do you want to PR it? should be a quick fix

RC you can see on the main readme, (for some reason the Conda badge will show RC and the PyPi won't)
https://github.com/allegroai/clearml/

Let me know if it solved it, if it did I'll make sure we push the RC

Hi MagnificentPig49 I suspect it's the ResnetCIFAR10Model wrapper, is it your module ?

Wait, so the pipeline step only runs if the pre execute callback returns True? It'll stop if it doesn't run?

Only if you have a Callback function, and that callback function returns False, then it will skip it (otherwise it will process it)

Another question, in the parents sequence in pipe.add_step, we have to pass in the name of the step right?

Correct, the step name is a unique identifier for the pipeline

how would I access the artifact of a previous step within the pre ...