I keep track of parts of such a solution at https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence namely TTS, STT, LLM, etc. There is also a recent HomeAssistant article https://www.home-assistant.io/blog/2024/06/07/ai-agents-for-the-smart-home/ which is quite interesting... but also concludes that they don't believe it's ready for prime time for most.
If you have specific use cases in mind, happy to give more pointers for solutions I believe might fit. That being said I'm not personally convinced as I don't use any assistant on a daily basis. Still I believe FLOSS alternatives are interesting to consider for any topic.