I'm in the process of getting my Home Assistant environment up and running, and decided to run a test: it turns out that my gaming PC (custom 5800X3D/7900XTX build) uses more power just sitting idle, than both of my storage freezers combined.
Background: In addition to some other things, I bought two "Eightree" brand Zigbee-compatible plugs to see how they fare. One is monitoring the power usage of both freezers on a power strip (don't worry, it's a heavy duty strip meant for this), and the other is measuring the usage of my entire desktop setup (including monitors and the HA server itself, a Lenovo M710q).
After monitoring these for a couple days, I decided that I will shut off my PC unless I'm actively using it. It's not a server, but it does have WOL capability, so if I absolutely need to get into it remotely, it won't be an issue.
Pretty fascinating stuff, and now my wife is completely on board as well; she wants to put a plug on her iMac to see what it draws, as she uses it to hold her cross-stitch files and other things.
I discovered a similar issue. PC desk was using 8-9W when the PC was turned OFF! My power strip was taking a bit under 1W (the little light, old), two smart bulbs as well but I'll allow those losses. An older Logitech speaker setup (2+1) was taking 6-7W, turned off! Crazy.. and illegal if it were made today (in EU). So this is completely wasted energy in my opinion.. started disconnecting the whole desk now.
For comparison, my home server is averaging 7-8W, turned on all the time:
I also learned that PC's draw a lot of power lol. I used to sit on my PC all day, now I know how much it cost. Even the monitor turning off splits the power draw by half.
Older speakers like that use always on transformers, constantly wasting energy to keep the core energized. You're correct those cannot be made any more, they must use efficient switch mode supplies.
What are you running your server on?