Spend $500, buy a beefy used GPU and run Whisper + Mistral small. It'll be a bit slow but now you can maintain your privacy on your own hardware! Best of both worlds.
Privacy
A place to discuss privacy and freedom in the digital world.
Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.
In this community everyone is welcome to post links and discuss topics related to privacy.
Some Rules
- Posting a link to a website containing tracking isn't great, if contents of the website are behind a paywall maybe copy them into the post
- Don't promote proprietary software
- Try to keep things on topic
- If you have a question, please try searching for previous discussions, maybe it has already been answered
- Reposts are fine, but should have at least a couple of weeks in between so that the post can reach a new audience
- Be nice :)
Related communities
much thanks to @gary_host_laptop for the logo design :)
Do not use AI for Therapy. Just don't do it. Don't. Idk how much more clear I can be. That shit is NOT private. Please do not do this.
What are you even using GPT for? Is it literally just dictating your thoughts into typed words? There are so money speech-to-text things out there that are not AI... On a cursory search for something: https://fosspost.org/open-source-speech-recognition#Top_Open_Source_STTTTS_Systems
If you feel bad about it, then stop using it, period. Life isn't easy, and the way that modern society tries to make convenience the number one goal is a crock of shit. Plan to put effort into your life, you will 1) get more out of it and 2) be less unprepared when situations arise. As a start, maybe practice typing while you talk and dictate your own thoughts into an offline app like Notepad++ or LibreOffice. You'll have the exact same, identical end-result, except you will have done it all using your own abilities. Remember, the typewriter existed long before the computer, and certainly much longer than LLMs. Even Speech-to-Text systems have existed much longer than LLMs...
Or just do like you mentioned, fully embrace wilful ignorance and suppress whatever makes you feel uneasy about using GPT. Eventually, you'll condition yourself into a perfect crop.
Either way, you need to pick a side and lean into it. Being in a tormented limbo will only make everything much worse.
But lately I've been in a rough state mentally (I say lately but it's always been with me) and having GPT guide me and being able to just dictate what I think helps me a lot on various levels, for various reasons.
You should consider journaling and/or a support group and/or professional help. You say it's always been with you, and relying on an LLM to help you parse your thoughts is only going to lead to bad places. It does not have the knowledge or skill of a psychologist, even with its vast database. It doesn't "know" the correct way to apply psychological concepts or know when you're engaging in destructive behavior, and it could even reinforce those same behaviors. AI can feel like a friend to talk to, but it's an illusion.
But at this point, using ChatGPT with a mic, isn't this basically cancelling every effort I've made? (using it in the first place anyway)
If you're telling it your deepest secrets and current problems, yes. You're putting your personal information out there for a profit-driven company to use at their discretion. Several people have suggested self-hosting, and I would agree, but ultimately I think you need to seek human help for the root cause of why you're leaning so heavily upon AI.
But lately I’ve been in a rough state mentally (I say lately but it’s always been with me) and having GPT guide me and being able to just dictate what I think helps me a lot on various levels, for various reasons.
Do you mean you use it to brain dump your thoughts and things like that?
If so, have you considered simply... writing stuff down with a pen on paper? Aka, journaling.
I've been doing that for, well, almost all all my live (started as little boy, I'm now well into my 50s) and it has always been tremendous help to better understand whatever is going on in my head/happening around me/with other people/the world.
Pen and paper journaling is also 100% not online, unless you want it to be. And it's cheap, when it's not completely free ;)
I really, really don't understand how people see LLMs as the only options for stuff like this...
Literally, written language and even typewriters have been a part of human history for a relatively long amount of time compared to how long simple computers have been a part of human history. I wonder if OP is very young and part of a generation that doesn't even know what landlines are.
The obsession with privacy can be taken to an extreme where it's a negative for your mental health. You can sacrifice a little privacy for another important benefit. 100% privacy is either not possible or practical, so don't let it stress you out. I'd be more concerned that AI might lead you down a wrong path, but you're an adult, use your own judgement.
This. It's all incremental to me. No I can't be 100% free of big tech and surveillance, etc etc - but how much of my data has Google not gotten since I started paying for private email 5 years ago or so?
(Not much cause everyone else refuses to leave Gmail behind 😂 but all I can do is keep moving myself in that direction.)
100% is not possible. Everything else is an illusion.
I don't think the AI is the problem here. I think you should see a real professional therapist.
Word, Riggs. 😆
On top of a therapist, a group session that works on Cognitive Behavioral Therapy principles or Dialectical Behavioral Therapy principles also helps because it puts you in a situation around other real people who are dealing with similar issues as yourself.
The problem (in the USA at least) is affording access to either of those things might be more complicated.
This is the actual best suggestion in the thread, though. Self-hosting an LLM is nice and all but what this poor soul really needs is therapy with a human.
First off, I can't personally cast any stones: I sometimes use chatGPT so summarize a text or help me debug a code. I resisted the idea at first, but many of the other students of my promotion recommended it. Almost any time I asked someone for help, they either told me to use chatGPT or themselves prompted it my question. I try to do without it as much as I can, and I never prompt a queston without having spent several minutes looking for an answer online written by a human, but I have difficulties in several subjects and I've already failed my first semester, so it's not easy to scorn a possible source of help when all else fails... I've installed locally a light weight version of Deepseek on my computer to get some of the benefits with a smaller climate footprint and staying in the Open Source side of the force, but so far I haven't found it satisfactory, perhaps I'll try a heavier version of the model.
But now, it seems you're using it for something way different. You say it dictates what you think, do you have difficulty parsing your own thoughts? I don't think you should feel guilty for it, if you need the help, but I do feel somewhat concerned. Large Language Models only learn and repeat patterns, I don't think it's a good tool for introspection, because it's giving you more generic thoughts and only making it seem personal. It is common for people who enjoy reading to find in the text things they've thought themself without being able to word it and to feel a connection with the author. But in this case, you still know these words are from someone else's mind. You see where the connection starts and where it ends. I think reading helps being good at putting one's thoughts in words, and is healthier than using an llm for it. You should probably also write, even if you keep it for yourself. That way, you'll be certain that these thoughts are your own.
Large Language Models only learn and repeat patterns, I don't think it's a good tool for introspection, because it's giving you more generic thoughts and only making it seem personal.
Worse, it's doing that lazily. People think that just because it has a gigantic database of information that it's able to parse all that information and put together summaries of that sum total of information, but what it does in reality is find the first solution it can cobble together from anonymous internet sources and present that as an "answer."
Asking a generic code question about data structures is less likely to produce bad answers, since there's little disagreement in how to implement them. They're close to brute facts, and a wrong answer is rarely catastrophic. Asking it about psychology or philosophy, however, is prone to terrible answers, not just because there's vast disagreement in those fields, but there's also a lot of personal consideration when it comes to applying the information to individuals.
Privacy issues aside, don’t allow yourself to become too comfortable with leaning on AI for so much. Aside from the obvious things like AI info being flat out wrong sometimes and hallucinations, it’s going to train you into some bad habit holes during a vulnerable time. Look at how quickly we reach to map software for travel. It causes us to get mentally lazy.
If you are focused on using it and worried about privacy, you can host your own model like someone else mentioned, but you need a pretty beefy computer for it, and you could potentially host a model on the cloud (I know that breaks privacy and self-hosting rules a bit), but that can get expensive.
Edit: You could give Jan a try.
I’m a programmer and I’ve had to discipline myself with how I use Copilot. I try to lean on it for troubleshooting code I’ve written, and for doing tedious tasks that I know how to do but want to save time on.
You don't actually have to have a beefy computer to self host. I know with deepseek there's a 1.5b model which is optimized for low end hardware. I self hosted it on a desktop from 2012 and it worked alright. 8gb RAM, i5 processer, quad core. Was the GPU usage high? Yep. Still worked tho.
Obligatory: you're not running deepseek 1.5B. You're running some other distilled model which was finetuned by the deepseek folks
Oh wow, that’s awesome. I’m gonna have to try that model out. I have an old XFX 7950 I can try burning out lmao.
host your own AI using: https://ollama.com/
this paired with "AnythingLLM" is pretty powerful, and you dont have to worry about data being sent anywhere
I actually got it to run reasonably on a mini PC in a docker container, but my setup also lets me use CPU power as a psuedo GPU.
It is possible to use GPT privately. Look up nano GPT. For example, that's one way, and there are others. I want to say Eric Voorhees has a way of using GPT privately as well, but I cannot remember the name of his thing right off hand.