I think Time Cop answers your request the best.
It is on f-droid (ca.hamaluik.timecop).
Unfortunately, even tho it works well for your use case (you would have to turn on the option to have only one task at a time), it makes very basic use of notifications, and has no widget... It could be so much more.
Edit: I discovered it yesterday while perusing this thread (I have been searching for a time management software for a long time, so I checked all the suggestions one by one on f-droid, and when searching for "a time tracker", I found it), and I have been using since. Very cool software. Works well, very usable. I highly recommend it.