Did a small update: added a workaround and renamed the issue to include more client_facing condition
limit_execution_time is presentinstead of an implementation detail condition
timeout_jobs are present

Many thanks! 😍

For me - workaround is totally acceptable, thus scheduler is once again usable for me.

FiercePenguin76 , maybe add it as PR 🙂

I thought that I can keep self._timeout_jobs = {} # Dict[datetime, str] intact and fix the part where we need to extract keys instead of values.

if a provide a PR, there I don’t see any CI processes in place that will verify the correctness of my code.

and this can break a lot of things, when somebody start the scheduler with an older version of clearml, saves the state, then upgrades and new clearml expects the state in another format

Also added implementation thought to the issue

Thanks FiercePenguin76 , I can totally understand your point on running proper tests, and reluctance to break other things.
I suggest to add a comment with the temp fix that solved the problem for you, and we will make sure the team takes it from there. wdyt?

I think I’ll skip with PR: there is a related problem, that makes the fix (and especially its testing much more difficult): https://github.com/allegroai/clearml/issues/648#issuecomment-1102595620

The only thing I found is that I need to run flake8, but it fails even without any changes, i.e. it was not enforced before (see my msg in )

But the second problem hints that we need to change Dict[datetime, str] -> Dict[str, datetime] or do some custom processing before serialization

difficult to verify correctness without a publicly available test suite

What'd you mean?

I already added to the task:
Workaround: Remove limit_execution_time from scheduler.add_task

I can try, but difficult to verify correctness without a publicly available test suite

I’ll make it more visible though