Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Configured An On-Prem File Server For Clearml Which Is Mounted On My Pc.

HI, i configured an on-prem file server for clearml which is mounted on my pc. 🙂

Here is a simple example of code I run:

from clearml import PipelineController


import clearml
clearml.config.verify_ssl = False  # Disable SSL verification
clearml.config.http_timeout = 300

print(clearml.__version__)
# ===================================================
def step_one_sp(kwargs):
    print(kwargs)
    print("AAAAA ")
    pass
    return


# ===================================================
def step_two_sp(kwargs):
    print(kwargs)
    print("AAAAA ")
    pass
    return

if __name__ == '__main__':

    # create the pipeline controller
    pipe = PipelineController(
        project='examples',
        name='Pipeline Spanier 1',
        version='1.3'    )

    # set the default execution queue to be used (per step we can override the execution)
    pipe.set_default_execution_queue('default')

    pipe.add_function_step(
        name='step_one_sp',
        function=step_one_sp,
        cache_executed_step=True,
    )

    pipe.add_function_step(
        name='step_two_sp',
        function=step_two_sp,
        cache_executed_step=True,
    )

    pipe.start_locally()

    print('pipeline completed')

And the output is

Launching the next 2 steps
Launching step [step_one_sp]
Launching step [step_two_sp]
Launching step: step_one_sp
Parameters:
None
Configurations:
{}
Overrides:
{}
Launching step: step_two_sp
Parameters:
None
Configurations:
{}
Overrides:
{}

And I never get the 'pipeline completed' output....

in the "clearml" web. the to taksts are set as "QUEUED"..

any idae ?

  
  
Posted one year ago
Votes Newest

Answers 6


What happens if you comment or remove the pipe.set_default_execution_queue('default') and use run_locally instead of start_locally ?

Because in the current setup, you are basically asking to run the pipeline controller task locally, while the rest of the steps need to run on an agent machine. If you do the changes I suggested above, you will be able to run everything on your local machine.

  
  
Posted one year ago

Ok, then launch an agent using clearml-agent daemon --queue default that way your steps will be sent to the agent for execution. Note that in this case, you shouldn't change your code snippet in any way.

  
  
Posted one year ago

AttributeError: 'PipelineController' object has no attribute 'run_locally'

  
  
Posted one year ago

@<1537605940121964544:profile|EnthusiasticShrimp49> I don't have start_locally

  
  
Posted one year ago

Hi, thanks of the reply

when I remove pipe.set_default_execution_queue('default') i get

ValueError: Node 'step_one_sp' missing execution queue, no default queue defined and no specific node queue defined
  
  
Posted one year ago

The line before the last in your code snippet above. pipe.start_locally .

  
  
Posted one year ago