No I want to put them inside the pipeline.py file where I config all steps, like this:

from clearml import PipelineDecorator
from train_helpers.common import params
from dataset import DataModule
from train import Trainer


@PipelineDecorator.component(return_values=['_args'], cache=True)
def init_experiment():
    _args = params.parse_args()
    return _args


@PipelineDecorator.component(return_values=['data'], cache=False)
def data_preparation(args):
    data = DataModule(args)
    r...

Thanks for the response, I tried to creat Logger.current_logger in each step but I got -

RuntimeError: can't create new thread at interpreter shutdown

How can we send objet from step1 to step2 without creating it again?

yes

Thank you

ClearML results page: 

step 1
ClearML Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
Traceback (most recent call last):
  File "/var/folders/1c/vz6m8k653j1_xpmpfgynshc00000gn/T/tmp4myzvumt.py", line 61, in <module>
    task.upload_artifact(
  File "/Users/almoghitelman/.pyenv/versions/sunbit-ai-312/lib/python3.12/site-packages/clearml/task.py", line 2341, in upload_artifact
    raise exception_to_raise
  File "/Users/almoghitelman/.pyenv/ve...

the first step init Logger and return it object

Logger.current_logger()

I’m getting error although logger_setup not calling ClearML logger, it’s just an inner class we use. cam we pass object between steps? (our objects) @<1523701070390366208:profile|CostlyOstrich36> @<1523701435869433856:profile|SmugDolphin23>

ValueError: Could not retrieve a local copy of artifact setup_logger

@PipelineDecorator.pipeline(name='Pipeline_Trail', project='Pipeline_Trail', version='0.1')
def main():
    _args = params.parse_args()
    setup_logger = init_experiment(_args)
    data = load_dataset(_args, setup_logger)
    train_model(_args, data, setup_logger)

Hi @<1523701435869433856:profile|SmugDolphin23> , yes since I want to use it next steps

util class that write logs to txt file and holds the paths for data,model etc. @<1523701435869433856:profile|SmugDolphin23>