Reputation
Badges 1
46 × Eureka!From what I've seen, another fix would be to pass a default value to your parameters, for example: def executing_pipeline(start_date=0, end_date=0):
Yes sure, I will do that
But I've got /opt/clearml/data/fileserver/examples/.pipelines/custom pipeline logic
which has a bunch of folders of old tasks
I'm not sure how I broke it 😅
This is what I've found, and there's no error that seem to come up
If I refresh, the project is still there 😕
I'm going to try deleting it using the APIClient
Yep I'm dumb, it worked. However I've launch a couple of tasks with name
='custom pipeline logic',
project
='examples'
and I have to delete them manually. When I try through the UI it just waits forever
But this works strangely:
` @PipelineDecorator.component(cache=False, execution_queue="default")
def get_param():
return 'hello'
@PipelineDecorator.component(cache=False, execution_queue="default")
def get_best_model(task_ids):
import ...
print('task_ids:', task_ids, type(task_ids)) # task_ids: None <class 'NoneType'>
...
@PipelineDecorator.pipeline(
name='...',
project='...',
version='0.1'
)
def pipeline_entry(task_ids: List[str], ...):
print(task_ids, typ...
Yes sure CostlyOstrich36 , I'm just trying to pass some arguments from my __main__
to my pipeline_entry()
to my component get_best_model()
. But for some reason, I'm getting None
into get_best_model
instead of what I've given it in pipeline_entry
Okay, thanks
CostlyOstrich36 This looks like a bug? Here's a simpler version of it and what I'm getting:
` from clearml.automation.controller import PipelineDecorator
@PipelineDecorator.component(cache=False)
def step_one(my_arg):
print('step_one/my_arg:', my_arg) # step_one/my_arg: None
# I should not get None here! At least that's what I'm expecting
@PipelineDecorator.pipeline(name='custom pipeline logic', project='examples', version='0.0.5')
def executing_pipeline(my_arg):
print('my_ar...
No sorry, I found the where the logs are. And it doesn't seem to have any errors in the logs:
` [2022-10-14 17:22:50,771] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.get_all in 3ms
[2022-10-14 17:22:50,784] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.get_by_id in 7ms
[2022-10-14 17:22:50,853] [9] [INFO] [clearml.service_repo] Returned 200 for events.add_batch in 182ms
[2022-10-14 17:22:50,874] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.edit in 28ms
[202...
Not really, it's an Ubuntu desktop machine that I'm just updating times to times. I've also got a few pipelines running during my trainings. Do you know any tools that I could use to analyze network errors?
Do you know where I can find the logs for that?
Nothing strange in dmesg
at least 😕
We've updated everything now, launched a new experiment and we're tracking the logs. I'll tell you if I find anything
My computer just did an automatic suspend, as simple as that
Sooo for the SHA2 generation, I've tested 2 very different CPUs, and it makes a HUGE difference 😅 I probably have to upgrade my server
I wasn't able to reproduce it with a simple piece of code, I'll try to see later if I can. But what I've seen is that I was logging too many images and it was somehow missing my last reports. With fewer image logs it seems like it's working normally
Yeah I had the same issue: https://clearml.slack.com/archives/CTK20V944/p1664887550256279