Reputation
Badges 1
46 × Eureka!Nope sorry I don't have that, but this was our discussion about it: https://clearml.slack.com/archives/CTK20V944/p1665477602050259
Thanks for trying to help me out! Here's some code that should reproduce the error (at least, it did for me): https://github.com/allegroai/clearml-agent/issues/111
Just dropping this here but I've had some funky compressions with very small datasets! It's not a big issue though, since it's still small and doesn't really affect anything
I'm still in 1.7.0 because of the None arguments thing for now, but I'll test with the latest version if I ever find any other issue
Yep, everything works now, thanks!
Alright, thanks! I can confirm that it works in 1.7.1
Hmm okay, I'm doing a hyper parameter search by launching multiple processes of my train
function. I've got a main task runing the search to log the final results, and a bunch of training tasks running in parallel. It would've been nice to be able to come back to each one individual training task, but I guess I'll do without
This however works fine:
` Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import numpy as np
a = np.ones((100, 100, 3))
a[0:50, 0:50].shape
(50, 50, 3)
`
Yep I'm dumb, it worked. However I've launch a couple of tasks with name
='custom pipeline logic',
project
='examples'
and I have to delete them manually. When I try through the UI it just waits forever
It's a bit strange, my pipeline is called "custom pipeline logic" (which I renamed to "blop" later). This api_client.projects.get_all(name=exact_match_regex("pipeline_project/.pipelines/blop"), search_hidden=True)
returns nothing and this api_client.projects.get_all(name=exact_match_regex("pipeline_project/.pipelines/custom pipeline logic"), search_hidden=True)
returns nothing either
I'm going to try deleting it using the APIClient
Do you know where I can find the logs for that?
I'm not sure how I broke it 😅
This is what I've found, and there's no error that seem to come up
If I refresh, the project is still there 😕
This is from the console by the way
But I've got /opt/clearml/data/fileserver/examples/.pipelines/custom pipeline logic
which has a bunch of folders of old tasks
No sorry, I found the where the logs are. And it doesn't seem to have any errors in the logs:
` [2022-10-14 17:22:50,771] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.get_all in 3ms
[2022-10-14 17:22:50,784] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.get_by_id in 7ms
[2022-10-14 17:22:50,853] [9] [INFO] [clearml.service_repo] Returned 200 for events.add_batch in 182ms
[2022-10-14 17:22:50,874] [9] [INFO] [clearml.service_repo] Returned 200 for tasks.edit in 28ms
[202...
Yes sure CostlyOstrich36 , I'm just trying to pass some arguments from my __main__
to my pipeline_entry()
to my component get_best_model()
. But for some reason, I'm getting None
into get_best_model
instead of what I've given it in pipeline_entry
Weeell it seems to work with version 1.7.0 and not with 1.7.1
Here are the versions: WebApp: 1.7.0-232 • Server: 1.7.0-232 • API: 2.21
From what I could see, generating SHA2:
i7-10700K: ~ 10 - 15 minutes Xeon E3-1240: 4 - 5 hours!Then in both cases I still have about an 1h30 to upload the images to the fileserver. Which I also find quite a bit slow, but the ClearML fileserver is on my old Xeon. I plan to upgrade my server and to test it again
But this works strangely:
` @PipelineDecorator.component(cache=False, execution_queue="default")
def get_param():
return 'hello'
@PipelineDecorator.component(cache=False, execution_queue="default")
def get_best_model(task_ids):
import ...
print('task_ids:', task_ids, type(task_ids)) # task_ids: None <class 'NoneType'>
...
@PipelineDecorator.pipeline(
name='...',
project='...',
version='0.1'
)
def pipeline_entry(task_ids: List[str], ...):
print(task_ids, typ...
Sooo for the SHA2 generation, I've tested 2 very different CPUs, and it makes a HUGE difference 😅 I probably have to upgrade my server
Apparently, the issue is that it will always pass None
if there's no default value