2mo ago

Will LLMs make finding answers online a thing of the past?

As LLMs become the go-to for quick answers, fewer people are posting questions on forums or social media. This shift could make online searches less fruitful in the future, with fewer discussions and solutions available publicly. Imagine troubleshooting a tech issue and finding nothing online because everyone else asked an LLM instead. You do the same, but the LLM only knows the manual, offering no further help. Stuck, you contact tech support, wait weeks for a reply, and the cycle continues—no new training data for LLMs or new pages for search engines to index. Could this lead to a future where both search results and LLMs are less effective?

59 comments

No. It hallucinates all the time.
- Sure does, but somehow many of the answers still work well enough. In many contexts, the hallucinations are only speed bumps, not show stopping disasters.
  
  It told people to put glue in their pizza to make the dough chewy. It's pretty fucking awful.
And where does LLM take the answer? Forum and socmed. And if LLM don't have the actual answer they blabbering like a redditor, and if someone can't get an accurate answer they start asking forum and socmed.
So no, LLM will not replace human interaction because LLM relies on human interaction. LLM cannot diagnose your car without human first diagnose your car.
- The problem is that the LLMs have stolen all that information, repackaged it in ways that are subtly (or blatantly) false or misleading, and then hidden the real information behind a wall of search results that are entire domains of ai trash. It's very difficult to even locate the original sources or forums anymore.
  
  I've even tried to use Gemini to find a particular YouTube video that matches specific criteria. Unsurprisingly, it gave me a bunch of videos, none of which were even close to what I'm looking for.
- That’s true. There could be a balance of sorts. Who knows. If LLMs become increasingly useful, people start using them more. As they loose training data, quality goes down, and people shift back to forums etc. Could work that way too.
LLMs are awesome in their knowledge until you start to hear its answers to stuff you already know and makes you wonder if anything was correct.
What they call hallucinations in other areas was called fabulations, to invent tales or stories.
I'm curious about what is the shortest acceptable answer for these things and if something close to "I don't know" is even an option.
- Sound similar to betteridges law of headlines.
  Im sure there are tricks like adding 'fact check your response' but I suspect there is something intrinsic to these models that makes it a super difficult problem.
- LLMs are awesome in their knowledge until you start to hear its answers to stuff you already know and makes you wonder if anything was correct.
  This applies equally well to human-generated answers to stuff.
  
  True, the difference is that with humans it's usually more public, it is easier for someone to call bullshit. With LLMs the bullshit is served with the intimacy of embarrassing porn so is less likely to see any warnings.
- I get the feeling that LLMs are designed to please humans, so uncomfortable answers like “I don’t know” are out of the question.
  This thing is broken. How do I fix it?
  Don’t know. 🤷
  Seriously? I need an answer? Any ideas?
  Nope. You’re screwed. Best of luck to you. Figure it out. I believe in you. ❤️
  
  Not designed, but trained. Training involves rewarding finding answers, so they WILL give you something. "I don't know" is not going to fare well in the training development, so it naturally gets filtered out, while very creative (but wrong) LLMs do well.
  
  This is mostly just a matter of proper prompting.
Trouble is that 'quick answers' mean the LLM took no time to do a thorough search. Could be right or wrong - just by luck.
When you need the details to be verified by trustworthy sources, it's still do-it-yourself time. If you -don't- verify, and repeat a wrong answer to someone else, -you- are untrustworthy.
A couple months back I asked GPT a math question (about primes) and it gave me the -completely wrong- answer ... 'none' ... answered as if it had no doubt. It was -so- wrong it hadn't even tried. I pointed it to the right answer ('an infinite number') and to the proof. It then verified that.
A couple of days ago, I asked it the same question ... and it was completely wrong again. It hadn't learned a thing. After some conversation, it told me it couldn't learn. I'd already figured that out.
- Trouble is that 'quick answers' mean the LLM took no time to do a thorough search.
  LLMs don't "search". They essentially provide weighted parrot-answers based on what they've seen elsewhere.
  If you tell an LLM that the sky is red, they will tell you the sky is red. If you tell them your eyes are the colour of the sky, they will repeat that your eyes are red. LLMs aren't capable of checking if something is true.
  Theyre just really fast parrots with a big vocabulary. And every time they squawk, it burns a tree.
- Is your abuse of the ellipsis and dashes supposed to be ironic? Isn't that a LLM tell?
  I'm not even sure what the ('phrase') construct is even meant to imply, but it's wild. Your abuse of punctuation in general feels like a machine trying to convince us it's human or a machine transcribing a human's stream of consciousness.
- Math problems are a unique challenge for LLMs, often resulting in bizarre mistakes. While an LLM can look up formulas and constants, it usually struggles with applying them correctly. Sort of, like counting the hours in a week, it says it calculates 7*24, which looks good, but somehow the answer is still 10 🤯. Like, WTF? How did that happen? In reality, that specific problem might not be that hard, but the same phenomenon can still be seen in more complicated problems. I could give some other examples too, but this post is long enough as it is.
  For reliable results in math-related queries, I find it best to ask the LLM for formulas and values, then perform the calculations myself. The LLM can typically look up information reasonably accurately but will mess up the application. Just use the right tool for the right job, and you'll be ok.
There have been enough times that I googled something, saw the AI answer at the top, and repeated it like gospel. Only to look like a buffoon when we realize the AI was completely wrong.
Now I look right past the AI answer and read the sources it's pulling from. Then I don't have to worry about anything misinterpreting the answer.
- True, but soon the sources will be AI generated too, in a big GIGO loop.
  
  That’s exactly what I’m worried about happening. What If one day there are hardly any sources left?
My 70 year old boss and his 50 year old business partner just today generated a set of instructions for scanning to a thumb drive on a specific model of printer.
They obviously missed the "AI Generated" tag on the Google search and couldn't figure out why the instructions cited the exact model but told them to press buttons and navigate menus that didn't exist.
These are average people and they didn't realize that they were even using ai much less how unreliable it can be.
I think there's going to be a place for forums to discuss niche problems for as long as ai just means advanced LLM and not actual intelligence.
- When diagnosing software related tech problems with proper instructions, there’s always the risk of finding outdated tips. You may be advised to press buttons that no longer exist in the version you’re currently using.
  With hardware though, that’s unlikely to happen, as long as the model numbers match. However, when relying on AI generated instructions, anything is possible.
  
  Not so simple with hardware also. Although less frequent, hardware also has variants, the nuances of which are easily missed by LLMs
LLMs are the big block V8 of search engines. They can do things very fast and consume tons of resources with subterranean efficiency. On top of that, they are privacy invasive, easy to use for manipulation and speed up the problem of less mature users being spoon fed. General purpose LLMs need to be outlawed immediately.
- Right. How about csam, incest, cannibalism?
People will use whatever method of finding answers that works best for them.
Stuck, you contact tech support, wait weeks for a reply, and the cycle continues
Why didn't you post a question on a public forum in that scenario? Or, in the future, why wouldn't the AI search agent itself post a question? If questions need to be asked then there's nothing stopping them from still being asked.
- If you cut a forum's population by 90% it will die.
  This is one of the biggest problems with AI. If it becomes the easiest way to get good answers for most things, it will starve the channels that can answer the things it can't (including everything new).
  
  Depends which 90%.
  It's ironic that this thread is on the Fediverse, which I'm sure has much less than 10% the population of Reddit or Facebook or such. Is the Fediverse "dead"?
  This is one of the biggest problems with AI. If it becomes the easiest way to get good answers for most things
  If it's the easiest way to get good answers for most things, that doesn't seem like a problem to me. If it isn't the easiest way to get good answers, then why are people switching to it en mass anyway in this scenario?
- That is an option, and undoubtedly some people will continue to do that. It’s just that the number of those people might go down in the future.
  Some people like forums and such much more than LLMs, so that number probably won’t go down to zero. It’s just that someone has to write that first answer, so that eventually other people might benefit from it.
  What if it’s a very new product and a new problem? Back in the old days, that would translate to the question being asked very quickly in the only place where you can do that - the forums. Nowadays, the first person to even discover the problem might not be the forum type. They might just try all the other methods first, and find nothing of value. That’s the scenario I was mainly thinking of.
  
  I did suggest a possible solution to this - the AI search agent itself could post a question in a forum somewhere if has been unable to find an answer.
  This isn't a feature yet of mainstream AI search agents but I've been following development and this sort of thing is already being done by hobbyists. Agentic AI workflows can be a lot more sophisticated than simple "do a search summarize results." An AI agent could even try to solve the problem itself - reading source code, running tests in a sandbox, and so forth. If it figures out a solution that it didn't find online, maybe it could even post answers to some of those unanswered forum questions. Assuming the forum doesn't ban AI of course.
  Basically, I think this is a case of extrapolating problems without also extrapolating the possibilities of solutions. Like the old Malthusian scenario, where Malthus projected population growth without also accounting for the fact that as demand for food rises new technologies for making food production more productive would also be developed. We won't get to a situation where most people are using LLMs for answers without LLMs being good at giving answers.
If the tech matures enough , potentially !
Not wrong about LLMs (currently )? bad with tech support , but so are search engines lol
to an extent, yes, but not completely

59 comments