CostlyOstrich36
hello, I tried some stuff, namely creating the pipeline with hydra with functions, based on https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_functions.py .
Before the task starts remotely, I can access the hydra config no problem, but once in remote and the task starts, I got :
Starting Task Execution: Primary config directory not found. Check that the config directory '/root/.clearml/venvs-builds/3.8/task_repository/clearml-example.git/src/config' exists and readable Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.This is my function for the pipeline:
` @hydra.main(config_path="config", config_name="config")
def executing_pipeline(cfg):

model_settings = cfg.model_settings
dataset_settings = cfg.dataset_settings
print(model_settings, dataset_settings)


# create the pipeline controller
pipe = PipelineController(
    project='clearml-example2',
    name='pipeline demo',
    version='1.1',
    add_pipeline_tags=False,
)

# set the default execution queue to be used (per step we can override the execution)
pipe.set_default_execution_queue('default')

# Use the pipeline argument to start the pipeline and pass it ot the first step
print('launch step 1')
pipe.add_parameter(
    name='model_settings',
    description='model_settings',
    default=model_settings
)
pipe.add_parameter(
    name='dataset_settings',
    description='dataset_settings',
    default=dataset_settings
)

pipe.add_function_step(
    name='basic_training',
    function=basic_training,
    function_kwargs=dict(dataset_settings='${pipeline.dataset_settings}', model_settings='${pipeline.model_settings}'),
    function_return=['task_id'],
    cache_executed_step=True, 
    execution_queue="default",
)
pipe.add_function_step(
    name='HPO',
    # parents=['step_one'],  # the pipeline will automatically detect the dependencies based on the kwargs inputs
    function=run_hpo,
    function_kwargs=dict(base_task_id='${basic_training.task_id}'),
    function_return=['best_model_settings'],
    cache_executed_step=True, 
    execution_queue="default",
)
pipe.add_function_step(
    name='train_best_model',
    function=train_best_model,
    function_kwargs=dict(data='${HPO.best_model_settings}'),
    function_return=['model', 'config_pbtxt'],
    cache_executed_step=True, 
    execution_queue="default",
)

# For debugging purposes run on the pipeline on current machine
# Use run_pipeline_steps_locally=True to further execute the pipeline component Tasks as subprocesses.
# pipe.start_locally(run_pipeline_steps_locally=False)

# Start the pipeline on the services queue (remote machine, default on the clearml-server)
pipe.start()
print('pipeline completed') `

YummyLion54 , Hi

What versions of clearml/clearml-agent are you using?

When running it without the agent, do the hydra configurations show up properly in the UI?

+1?

Could you guys have a sample for pipelines using hydra? I think quite a few people would interested in that, and it isn't as straightforward as I thought it would be

YummyLion54 , let me take a look 🙂

Hello, I am using clearml 1.1.4 and clearml-agent 1.1.0. I used hydra before without pipelines locally and it worked well and showed up in the UI properly yes

+1 for this 🙂