SuccessfulKoala55 that's good to know. I moved the signal register handles above the call to Task.init() as you suggested. This is what I should be seeing when the script is terminated manually:
I0526 07:46:14.391154 140262441822016 engine.py:837] Engine run starting with max_epochs=100. I0526 07:46:14.542132 140262441822016 train_utils.py:223] Epoch[1] Iter[1] Loss: 0.43599218130111694 I0526 07:46:24.078526 140262441822016 train_utils.py:46] 2 signal intercepted. I0526 07:46:24.078753 140262441822016 engine.py:635] Terminate signaled. Engine will stop after current iteration is finished.
However what I see is the following:I0526 07:44:15.416634 140574824470336 engine.py:837] Engine run starting with max_epochs=100. I0526 07:44:15.517145 140574824470336 train_utils.py:223] Epoch[1] Iter[1] Loss: 0.43599218130111694 2020-05-26 07:44:36 User aborted: stopping task (1)Once the task is aborted there doesn't seem to be any more log output from the script. That might be because trains is cutting off the log, but I also don't see the model file saved anywhere. I'll keep looking, but thank you for the suggestion!