Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Everyone, I'M New To Clearml Pipelines And My Tasks Are Always In Pending. 1) I Have Created Clearml Agent And It Is Connected To The Main Clearml Server. 2) Agent Configured To Use Docker With Mounted Volumes, Does Listen To The "Services" Queue And

Hey everyone, i'm new to clearml pipelines and my tasks are always in pending.

  1. I have created clearml agent and it is connected to the main clearml server.
  2. Agent configured to use docker with mounted volumes, does listen to the "services" queue and all tasks are displayed in this queue in webui.
  3. Pipeline starts and launches all tasks, but they all just pending.

Could somebody please help me understand what i might be doing wrong here.
Here is my code, i'm trying to track a bunch of existing triton models in specific folders.
I replaced path to models with "path_to_model".

from clearml import OutputModel, InputModel, Task
from clearml import PipelineDecorator


@PipelineDecorator.component(cache=True, execution_queue="services")
def load_model(abs_path: str, path_to_model: str):
    from os import path, listdir
    from clearml import InputModel

    name = path_to_model.strip("/").split("/")[0]
    model_file = listdir(path.join(abs_path, path_to_model, "1"))[0]
    input_model = InputModel.import_model(
        name=name,
        weights_url=path.join(abs_path, path_to_model, "1", model_file),
        # Set label enumeration values
        framework="PyTorch",
        create_as_published=True,
        # Set model tags
        tags=path_to_model.split("-"),
    )
    input_model.set_metadata("version", "1.1", "str")


@PipelineDecorator.pipeline(
    name="Load Models to Registry", project="Model Registry", version="1.0"
)
def pipeline_logic(path_to_model_repo: str):
    from os import listdir

    for elem in listdir(path_to_model_repo):
        load_model(path_to_model_repo, elem)


if __name__ == "__main__":
    pipeline_logic(
        path_to_model_repo="path_to_model"
    )
  
  
Posted 11 months ago
Votes Newest

Answers 2


Ah i see thank you so much!

  
  
Posted 11 months ago

Hi @<1635088270469632000:profile|LividReindeer58> , you should do a separation. The pipeline controller should run on the services queue. Pipeline steps should run on different queues. This is why they are sitting in pending - there is no free worker to pick them up.

  
  
Posted 11 months ago
773 Views
2 Answers
11 months ago
11 months ago
Tags