So what about open source self hosted search engines? If it requires some hardware I'd gladly team up with a small group of people to finance a bigass server that just gets us our personal search engine
Any good ones out there?
this post was submitted on 09 Oct 2024
608 points (96.8% liked)
Technology
58689 readers
4041 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
There's stuff like Searxng or whoogle, but these aren't "real" search engines, merely "search aggregators" - they relay requests to a bunch of actual search engines, like bing or google, and aggregate the results. That's why they don't require tons of compute and scraping, and also why they often fail to work (since the search engines in question don't like or allow this). I believe it's not feasible to run a "real" search engine alone or even as a small group of people - according to this comment you need a powerful server with terabytes* of drive, hundreds of gigabytes of RAM and a lot of compute - and all of this will just let you crawl some top domains, nowhere near a good chunk of the internet.
*which sounds low actually, I would have expected more for this
Searxng, but there are plenty of instances already
Perplexica is interesting too, but it uses a moderate amount of ram because of elastic search.
And of course you need to have ollama running
Very cool
I just use chatGPT to search now. I have a super-prompt in its memory telling it how to search and to cite sources and provide links and it is so much better than Google even though it's using AI, too.
*The future is now, old men!
Brave has their own search. There is also meta searches such as metager, searx and mojeek. I hope more search engines enter the market
There are no search engines besides Google and Bing, because everyone else just uses Bing under the hood.
It's intentional.
Obviously, Google makes money showing ads during search. But they have finally bit the bullet and starting tarpitting users in search in order to show more ads.
A quick, useful, and accurate search means that you're on their site for the least amount of time, perhaps mere seconds. That's not what's best for revenue growth.
PS: Go try Kagi and be reminded what good clean search results look like. I use it because my time has value. It's very good.
You only tested Google and Bing.
Qwant and DDG both use the Bing architecture.
I agree though, search engines have become noticeably worse the last 2 years.
I’m pretty sure they discovered in the google monopoly case that google realized a couple years ago that a worse search experience would not negatively impact their bottom line. So makes sense
Kagi is great. It’s a paid service but you can try 100 searches for free.
You can use the Orion browser on iOS and set Kagi as the search engine.
The day i pay for search engines is the day i finally finish my 2020 new year's resolution
Enjoy your war against ads then. I’m not against supporting content, I just wanted a better model than invasive ads.
Infinitely worse. I barely use search engines for issues these days and no longer recommend that people "Google" things.
What do you do instead?
Besides be sad and hit my head against a wall? Depending on the query, I'll sometimes use ChatGPT or find an associated Discord, subreddit or somewhere and hope for the best.
I'm disappointed a lot.
I bang my head against search engines. Normally I can find something decent after multiple searches, but it never used to be this way.
view more: next ›