Why Big Tech's bet on AI assistants is so risky

It's risky but the risk is towards the users, and the profits are towards the companies.

I asked OpenAI, Google, and Meta what they are doing to defend against prompt injection attacks and hallucinations. Meta did not reply in time for publication, and OpenAI did not comment on the record.

Discourse analysis tip: what is not said is sometimes more important than what is said. The fact that they refused to reply hints that the reply would be against their best interests, either lying in a liable way or saying the truth and potentially ruining their investment.

The reason why Google actually answered it ("Google confirmed it [prompt injection] is not a solved problem[...]") is likely related to saying "it's an experiment" -

Regarding AI’s propensity to make things up, a spokesperson for Google did say the company was releasing Bard as an “experiment,” and that it lets users fact-check Bard’s answers using Google Search. “If users see a hallucination or something that isn’t accurate, we encourage them to click the thumbs-down button and provide feedback. That’s one way Bard will learn and improve,” the spokesperson said.

Can we [people in general] stop pretending that those models "learn"? Giving it feedback is like telling my cat "don't scratch it!" - it might work for that specific case, but it won't solve the underlying issue, so the model/cat will keep hallucinating/scratching something else. The hallucinations are not individual flaws, they're issues surfacing from the underlying tech: language associates morphemes (tokens) with meaning, not just a token with another! Linguists have been talking about this for at least a century, but those "tech bros" are still trying to model language without it. (Microsoft is apparently doing some progress in this regard though. I can look for the quote if anyone wants.)

TL;DR? Tech companies shouldn’t be so complacent about the purported “inevitability” of AI tools. Ordinary people don’t tend to adopt technologies that keep failing in annoying and unpredictable ways, and it’s only a matter of time until we see the hackers using these new AI assistants maliciously. Right now, we are all sitting ducks.

I don’t know about you, but I intend to wait a little longer before letting this generation of AI systems snoop around in my email.

This is probably the longest TL;DR I've ever read, and that tells something about my reading impression of this text.

New technology brings new risks. Sell stuff via the internet? Are you crazy?

I assume they simply expect that the risks can be addressed.l. and generally I tend to agree with them. We somehow managed the fact that Google search results are not 100% accurate or Wikipedia articles can be hijacked.