Thanks FiercePenguin76 , I can totally understand your point on running proper tests, and reluctance to break other things.
I suggest to add a comment with the temp fix that solved the problem for you, and we will make sure the team takes it from there. wdyt?

difficult to verify correctness without a publicly available test suite

What'd you mean?

I think I’ll skip with PR: there is a related problem, that makes the fix (and especially its testing much more difficult): https://github.com/allegroai/clearml/issues/648#issuecomment-1102595620

Also added implementation thought to the issue

Many thanks! 😍

Did a small update: added a workaround and renamed the issue to include more client_facing condition
limit_execution_time is presentinstead of an implementation detail condition
timeout_jobs are present

and this can break a lot of things, when somebody start the scheduler with an older version of clearml, saves the state, then upgrades and new clearml expects the state in another format

I can try, but difficult to verify correctness without a publicly available test suite

For me - workaround is totally acceptable, thus scheduler is once again usable for me.

But the second problem hints that we need to change Dict[datetime, str] -> Dict[str, datetime] or do some custom processing before serialization

I thought that I can keep self._timeout_jobs = {} # Dict[datetime, str] intact and fix the part where we need to extract keys instead of values.

if a provide a PR, there I don’t see any CI processes in place that will verify the correctness of my code.

FiercePenguin76 , maybe add it as PR 🙂

I already added to the task:
Workaround: Remove limit_execution_time from scheduler.add_task

I’ll make it more visible though

The only thing I found is that I need to run flake8, but it fails even without any changes, i.e. it was not enforced before (see my msg in )