i believe this is because of this code
None

Which initialized the task if clearml is installed… but a task already exists (because of the pipeline), it will replace it

yeah, it gets to that error because the previous issue is saved…i’ll try to work on a new example

that makes more sense 🙂
would this work now as a workaround until the version is released?

Good to hear 🙏

Noting one difference i do is using TensorBoardCallback , because i believe the clearml docs use an outdated fastai 1 version…

when u say use Task.current_task() you for logging? which i’m guessing that the fastai binding should do right?

but that means that there is no way to work with clearml+fastai2+multi gpu

SmugDolphin23 SuccessfulKoala55 ^

reduced to a small snippet
` from fastai.vision.all import *
from fastai.distributed import *
from clearml import Task
from fastai.callback.tensorboard import TensorBoardCallback
from wwf.vision.timm import timm_learner

task = Task.init(project_name='LIOR_TEST', auto_connect_arg_parser={'rank': False})
path = untar_data(URLs.PETS)

size = 460
batch_size = 32

dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
get_y=lambda x: ...

hi, yes we tried with the same result

looks like it’s working 🙂 tnx

right, it works on 1.7.1

which part?

CostlyOstrich36 This is for a step in the pipeline

If nothing specific comes to mind i can try to create some reproducible demo code (after holiday vacation)

regarding what AgitatedDove14 suggested, i’ll try tomorrow and update

How does this work in the context of a pipeline? One of the steps is a multi gpu training that requires accelerate.

https://github.com/allegroai/clearml/blob/master/examples/frameworks/fastai/fastai_with_tensorboard.py

when i did this with a normal task it worked wonderfully, with pipeline it didn’t

AgitatedDove14 . so if i understand correctly, what i can possibly do is copy paste the https://github.com/fastai/fastai/blob/master/fastai/launch.py code and add the Task.init there?

Yes tnx for clarifying 😁

I'm getting really weird behavior now, the task seems to report correctly with the patch... but the step doesn't say "uploading" when finished... there is a "return" artifact but it doesn't exist on S3 (our file server configuration)

i get for one of the tasks, but then it fails because it seems that the fastai2 tensorboardcallback isn’t fit for distributed training (which i’m opening an issue for them now)

and this is for a normal task

I have but i believe i found the issue

yes

that’s what i started with, doesn’t work in pipelines

and use that instead of the -m fastai.launch part ofcourse

This is the next step not being able to find the output of the last step

ValueError: Could not retrieve a local copy of artifact return_object, failed downloading 

Traceback (most recent call last):
  File "/tmp/tmpxlf2zxb9.py", line 31, in <module>
    kwargs[k] = parent_task.get_parameters(cast=True)[return_section + '/' + artifact_name]
KeyError: 'return/return_object'
Setting pipeline controller Task as failed (due to failed steps) !
Traceback (most recent call last):
  File "/usr/src/lib/clearml_test.py", line 69, in <module>
    pipeline()
  File "/opt/conda/lib/python3.10/site-packages/clearml/automation/controller.py", line 3914, in intern...