Hi @<1635088270469632000:profile|LividReindeer58> , you should do a separation. The pipeline controller should run on the services queue. Pipeline steps should run on different queues. This is why they are sitting in pending - there is no free worker to pick them up.
Answered
Hey Everyone, I'M New To Clearml Pipelines And My Tasks Are Always In Pending.
1) I Have Created Clearml Agent And It Is Connected To The Main Clearml Server.
2) Agent Configured To Use Docker With Mounted Volumes, Does Listen To The "Services" Queue And
Hey everyone, i'm new to clearml pipelines and my tasks are always in pending.
- I have created clearml agent and it is connected to the main clearml server.
- Agent configured to use docker with mounted volumes, does listen to the "services" queue and all tasks are displayed in this queue in webui.
- Pipeline starts and launches all tasks, but they all just pending.
Could somebody please help me understand what i might be doing wrong here.
Here is my code, i'm trying to track a bunch of existing triton models in specific folders.
I replaced path to models with "path_to_model".
from clearml import OutputModel, InputModel, Task
from clearml import PipelineDecorator
@PipelineDecorator.component(cache=True, execution_queue="services")
def load_model(abs_path: str, path_to_model: str):
from os import path, listdir
from clearml import InputModel
name = path_to_model.strip("/").split("/")[0]
model_file = listdir(path.join(abs_path, path_to_model, "1"))[0]
input_model = InputModel.import_model(
name=name,
weights_url=path.join(abs_path, path_to_model, "1", model_file),
# Set label enumeration values
framework="PyTorch",
create_as_published=True,
# Set model tags
tags=path_to_model.split("-"),
)
input_model.set_metadata("version", "1.1", "str")
@PipelineDecorator.pipeline(
name="Load Models to Registry", project="Model Registry", version="1.0"
)
def pipeline_logic(path_to_model_repo: str):
from os import listdir
for elem in listdir(path_to_model_repo):
load_model(path_to_model_repo, elem)
if __name__ == "__main__":
pipeline_logic(
path_to_model_repo="path_to_model"
)
660 Views
2
Answers
9 months ago
9 months ago
Tags