AgitatedDove14 I really don't know how is this possible... I tried upgrading the server, tried whatever I could

About small toy code to reproduce I just don't have the time for that, but I will paste the callback I am using to this explanation. This is the overall logic so you can replicate and use my callback

From the pipeline task, launch some sub tasks, and put in their post_execute_callback the .collect_description_tables method from my callback class (attached below) Run t...

Yes, I have the controller task in hand

As a part of a repo

The weirdest thing, is that the execution is "completed" but it actually failed

💪

now I'm curious what is the effect of _allow_extra_fields_

if I set it to 24 does it explode?

you want to see its contents?

I was refering to what is the returned object of Task.artifacts['...'] - when I call .get I understand what I get, I'm asking because I want to see how the object I'm calling .get on behaves

One sec I'll paste the relevant pieces of code

Cool - what kind of objects are returned by .artifacts. getitem ? I want to check their docs

How do I get all children tasks given a parent?

EagerOctopus10

checking and will let you know

SuccessfulKoala55 , I get a list of 1 value. This value is a dictionary which has the keys type header and cells . cells is a dictionary in itself which has the keys height values align line font

Yeah I got what I needed

AgitatedDove14 just a reminder if you missed this question 😄

SuccessfulKoala55 here it is

SuccessfulKoala55

👍

I prefer we debug on my machine (tell me what you want to check) than create a snippet

after you create the pipeline object itself , can you get Task.current_task() ?

AgitatedDove14 no I can't... Just checked this. This is a huge problem for us, it used to work before and it just stopped working and I can't figure out why.

It's a problem for us because we made it a methodology of running some tasks under a pipeline task and saving summary iunfo to the pipeline task - but now since Task.current_task() doesn't work on the pipeline object we have a serious problem

I'll check the version tomorrow, about the current_task call, I tried before and after - same result

worked

This is a part of a bigger process which times quite some time and resources, I hope I can try this soon if this will help get to the bottom of this

AgitatedDove14 sorry for the late reply,

It's right after executing all the steps. So we have the following block which determines whether we run locally or remotely

if not arguments.enqueue: pipe.start_locally(run_pipeline_steps_locally=True) else: pipe.start(queue=arguments.enqueue)
And right after we have a method that calls Task.current_task() which returns None

It's kind of random, it works sometimes and sometimes it doesn't

In the larger context I'd look on how other object stores treat similar problems, I'm not that advanced in these topics.

But adding a simple force_download flag to the get_local_copy method could solve many cases I can think of, for example I'd set it to true in my case as I don't mind the times it will re-download when not necessary as it is quite small (currently I always delete the local file, but it looks pretty ugly)

BTW is the if not cached_file: return cached_file is legit or a bug?

I know tasks have a name and an ID - but projects?