Linux

If you use Mettle, the phone based bank, you get FreeAgent for free. FreeAgent is a really good web based accounting package that works in Firefox. They gave a useful accompanying API and can do payroll, VAT, end of year and director self assessment. It's great.

Maybe you can use some German bank. They allow one to interact with their API for free. I use GNU cash for it. Though I doubt that you can file your taxes via GNU cash and be aligned with current UK law. You would need to check it for your own.

Always use a separate Firefox profile for banking needs.

I've found HSBC to be ok using Firefox on Linux. I don't know if they have integrations with any accounting software, but the web access works well, and you can export your transactions for processing locally.

ETA: I've run small business accounting on Gnucash, I found the learning curve a bit steep, but once you 'get it' it's handy.

I've been Linux in the desktop for years. You really don't have any choice other than to be a little bit flexible.

More times than not it turns out to be a plugin that screws over the site. Here's my general path:

Won't load in Firefox? Disable privacy badger and ublock origin

Still won't load? Try it in a private window with no extensions loaded

Still won't load? Move over to brave.

Still won't work? Disable Shields

Still won't work? Straight to a vanilla copy of edge, (a vanilla copy of Vivaldi would also be reasonable)

Just last night I ran into a problem with my ADP work portal. Things worked fine for ages, All of a sudden my password wouldn't work. I went into private mode My password now works but loading the actual page netted me a blank page

I opened it up in brave and it just worked outright.

Governments should not require the use of proprietary software. I personally would refuse to use it. I don't live in the UK but in the US I think you could easily make the case.

try winapps

Tangential:

I‘m running my own IT company since recently and am transitioning to exclusively using FOSS. I still have some things I need to work around like my iOS phone. It already has a linux successor but its not finished yet. Pretty promising though. My plan is to put a fixed percentage of profits to open source projects.

I am not in the UK, but wound up biting the bullet and using QubesOS for my business machine. It's kind of like a more straightforward to use everyday set of VMs. I have the windows qube there for running CAD/CAM and the sadly sometimes necessary Chrome install. I know this isn't an ideal solution, but it is the best that I personally have been able to come up with without going through the headache of dual booting, especially when dealing with either govt stuff, need Chrome for crappy websites my clients sometimes force me to use, or actually needing proprietary software that I have licensed for my business (MasterCam in my case).