Reputation
Badges 1
26 × Eureka!Hey @<1537605940121964544:profile|EnthusiasticShrimp49> I updated clearml but now the issue is my pipeline is stuck here.
Previously it was working fine till the above mentioned issue and I made no change except the mentioned.
Hey, I seem to have resolve this issue, but stuck in another.
Apparently even after all the tasks got completed of a pipeline, the pipeline is still running, I had to abort it manually. Am I missing any code to stop it after all tasks execution?
@<1523701205467926528:profile|AgitatedDove14>
Clearml version - 1.12.1
In the pipeline example None this specifically, at line 83parameter_override={'General/num_boost_round': 250,
'General/test_size': 0.5,
'General/random_state': random_state}
these are fixed and cannot be changed by user when the pipeline is cloned, I am trying to make thes...
Another issue that I am facing is I am unable to call different functions from other modules present in the same directory.
clearml==1.9.1
clearml-agent==1.5.2
I am not self hosting the server, using the one provided by clearml side
It was stuck here. I had to abort manually. All the tasks completed though.
[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpip install --upgrade pip[0m
1687953244763 si-sajalv:0 ERROR User aborted: stopping task (3)
1687953245766 si-sajalv:0 DEBUG Current configuration (clearml_agent v1.5.2, location: /tmp/clearml.conf):
----------------------
sdk.storage.cache.default_base_dir = /clearml_agent_cache
sdk.storage.cache.size.min_free_bytes = 10GB
sdk.storage.direct_access.0.url = file://*
`sdk.metrics.file_history_si...
Ignore default, I am trying to run with another docker, but it is also stuck as same
Image 1 shows original pipeline, image 2 show cloned pipeline and image 3 show parameters in cloned pipeline's run
It resolved.
I was doing 2 things wrong, defining params before task.init
and while using task.connect(params)
I was naming it and trying to set params using General/param_name
thanks for the support
I ll explain you what happened, I ran " None " this code, so all the steps of pipeline ran
so the individual part of pipeline ran, but in dashboard when I am seeing the pipeline it is running continuously and not ending even after all the tasks are completed.
the above part is from the console of the pipeline
Noo, in general when I am running this command clearml-agent daemon --queue services --docker clearml-pipeline:0.6
everything is expected to happen inside docker container. But what is happening is when I am reaching my task eg task1, inside pipeline task1 console is giving me this error. Which means it is expecting docker inside the container in which it is already running.
Hope I am clear now
from clearml import Task
from clearml.automation import PipelineController
pipe = PipelineController(name='PIPE_TEST_3',project='PIPE_TEST_3',version="0.0.1",add_pipeline_tags=False)
pipe.add_parameter("url",
"
None ",
"dataset_url"
)
pipe.set_default_execution_queue('services')
`pipe.add_step(name="stage_da...
@<1523701205467926528:profile|AgitatedDove14>
I am using pipeline by task and using pipe.add_parameter method to add the parameter through ui
pipe.add_parameter('random_state',2024) #model training
pipe.add_parameter('epochs',10)
pipe.add_parameter('learning_rate',0.001)
and then overriding the parameters using parameter_overide
pipe.add_step(
name='model_training',
parents=['preprocess_data'],
base_task_project=global_config.PROJECT_NAME,
` bas...
It ran, thanks.. but that original problem persisits. Pipeline is running once all the tasks completed.
@<1523701205467926528:profile|AgitatedDove14> I am a bit lost, can you elaborate?
@<1537605940121964544:profile|EnthusiasticShrimp49> is this a code issue or some bug?
Also it is returning empty list if I am using tagsmodel = Model.query_models(project_name = global_config.PROJECT_NAME, model_name="model training", tags = ['$all','best'])