I'm not sure if this bit of news is a direct consequence of A.I. but it's concerning (and also reassuring at the same time, that the government is doing something about it):
2 Likes
Unlikely AI is at fault. I remember my school days, being pained whenever many students were asked to read aloud and it was obvious they were sounding out words on the spot. I'm talking '90s. Not pre-AI, pre-consumer Internet, when most of the few who were online didn't use the Internet proper, but stayed in the curated bubble provided by AOL, Prodigy, or Compuserve.
That reading levels have diminished further in a world where elementary schoolers watch Tiktok and YouTube on their OWN phones is not something for which I need to scapegoat AI.
2 Likes
I really must agree with this.
I am quite sure the ancient Romans complained about their deviant youth...
A funny thing I was looking into the other day... A.I. results are comprehensively web expansive.
What this means is - it breaks the social media bubbles. Google, Facebook, etc... the alhorithms are ineffective on A.I.
It is not sheltered into the narrow defined parameters of that users search results. So as a benefit - A.I. can draw many people out of their algorithm defined feed.
When I think back to the 1880's (Mind you, I was a strapping young man, back then. Even had my own horse... Cost me twenty dollars.), people were largely exceptionally ignorant and uninformed. It took a great deal of time for information to spread. By the time it got to where it was going, many people had stopped caring. To many more immediate issues at hand, like cholera.
1 Like
1880's
I tell you, I am old.
Back in my day, all cars were one horsepower.
3 Likes
I am not convinced that this is so. It is true that it's trained on more or less everything it can get its hands on, but with the well documented tendency for AI models to be sycophantic, once a user's bubble is known, it's prone to reinforce it. AI models really don't push back unless you hit something they're explicitly trained to push back on.
Edit: Additionally, as is also well-documented, what it's trained on isn't especially high quality either. When you train on Reddit, you're not eliminating bubbles, you're scooping them in indiscriminately. When the user's prompts match patterns that fall into certain bubbles, it seems likely that AI's tendencies will reinforce them.
1 Like
Very good points.
But I will point out that it is not just Reddit; Scientific Journals and peer reviewed research are in the training models, as well.
A.I. is not exceptionally accurate, but it is nowhere near the cesspool of the internet, either.
1 Like
They are, but the biggest LLMs are what the vast majority use, and those go for absolute volume, which is why Cloudflare has been developing spider baffles to block AI from hosing servers. When you're trained on volume and don't know what a scientific journal is, or how to confirm that a paper is peer reviewed and not just proclaiming itself so, you're still more likely to return chaff than wheat.
Potentially even worse is that AI doesn't understand anything it spits out. (See my earlier remarks on thumbs.) Between AI's authoritative tone and convincing responses, when it returns nonsense woven together from those journals (or confabulates an entirely new, non-existent one), a layman isn't likely to recognize the problem.
I've mentioned before that math and I don't get along. I asked GPT a probability question related to a complex dice roll once and got a result with an explanation that seemed plausible... but the probability it gave me seemed much too high. If I weren't a tabletop RPG player used to multiple sets of dice rules with a bit of intuition born of experience, I never would have realized it was full of it. I had to turn to a friend of mine who's far better at math to disprove the AI.
In that respect, I think training it on high quality materials actually makes it more dangerous in the absence of actual reasoning capacity--which no model I've heard of actually has.
(There are "reasoning" models, but all they're really doing is populating their own context by doing the same thing they always do in two passes, the first of which has an artificial "interpret the prompt in detail" system prompt. This also clutters the context, meaning locally run models will break down faster.)
2 Likes
I think in this post, you really narrowed down exactly why businesses are failing to find solutions with A.I.
Initially, it seemed "smart" and could help them "trim excess"... But while the answers it gives sound confident, were not thought out in anyway.
A.I. output is based on recognizing a prompt pattern, then producing a pattern to match and validate the prompt (I believe, based on my own testing and reviews). It is not based on thinking.
The strongest descriptive word that comes to my mind is
illusory.
2 Likes
It's also why they're just successful enough to keep people using them until they're badly burned. If the parrot actually is spitting out the words you need, it's saved you time. Until it hasn't. My best friend occasionally uses CoPilot to generate code. They've got a degree and more than 20 years of experience coding professionally. When CoPilot does a stupid, they notice quickly and fix it. When some of the less experienced people they work with use CoPilot, said friend ends up catching serious mistakes in code review.
...and let's not forget the time my friend asked for a script and CoPilot decided to also run it. Thank goodness it didn't have create table privileges. 
2 Likes
The following article provides a very high level summary of just what an LLM is actually doing, which I feel is a good complement to this conversation. It uses the words "understand" and "reasoning" where I don't feel it should, for the sake of avoiding linguistic contortions, but it does close a relevant paragraph by clearly stating that it's pattern based, not human-like reasoning. @moderators, please nuke this post if the link runs afoul of our advertising link policy, as the article is from Proton and they do, toward the end, point out their own AI service, but the first half at least is very much on topic.
I don't think that is a bad link by any means. Realistically any article could "advertise" another LLM and be under the same scrutiny. As long as it's not blatantly just an ad, I think this is fine. The article seems like a pretty decent read with nothing standing out as "buy me".
1 Like
Most links do contain some form of advertising as those sites generate revenue in order to provide service.
For the Forum, spamming advertisment is narrowed to certain parameters:
- That the advertisement is directly benefiting the posting member.
- That the posting member is injecting this advertisement in order to advertise, not in order to contribute to the discussion.
1 Like
I agree with a lot of what's been said here, but this article was clearly put through an LLM at some stage of production, which is hilarious to me. "It's not about X. It's about Y," sections between subheaders of roughly identical length (and pretty much never longer than 3 paragraphs of roughly the same length), repetition of subheaders, bullet point lists that aren't really necessary for readability and just repeat stuff already in the bulk of the text, etc. - there are a lot of LLM tells here.
1 Like
The Proton article or one of the others? If it's the Proton article, it shouldn't surprise anyone if an LLM was used at some point as long as it was checked for accuracy. That article isn't on the evils or dangers of LLMs; it was on how they work. Proton has two AI products; it should surprise no one that they use them.
As for the use of unnecessary bullet points though, I wouldn't necessarily trust that that indicates the use of AI. I use bullet points fairly similarly in work e-mails because I find management pays attention to bulleted lists and their eyes glaze over at paragraphs more than two sentences long, forcing me to remind them of the most critical information.
1 Like
Ah, no, it's the one from Forbes in the OP. I don't necessarily doubt the accuracy, but it's kinda funny to me given the tone of the article.
Ah, I see. Forbes... I wouldn't be surprised if they use AI heavily. At some point they decided they cover everything instead of just their core content, and one can only make content on everything by regurgitating it. Some of their gaming articles (I've worked in the industry for more than 20 years) have just left me wondering what Paul Tassi was smoking.
That said, I can't prove it, and for all I know they have humans churning out articles on things they aren't experts on.
This thread warms my heart.
LLMs are just fancy autocomplete with a lot of data. Hallucinations are an integral part of them, not reasoning.
Anything resembling reasoning is just sufficiently verbatim in a large text database.
Companies that believe otherwise and fire real employees are going to find out the hard way.
Anything using only LLMs that claims to use reasoning is either a hilarious "Actually Indians" version of AI (like what happened with Amazon) or an outright lie.
Both are lies, one is just outsourcing with extra steps.
Just received this in my Proton mailbox, Meet Lumo.
Dear Proton community member,
AI is already transforming society in big and small ways. But Big Tech companies aren't using it to help people — they're using it to accelerate their surveillance capitalism business model while leaving our data vulnerable to hackers or governments. And they're doing it without our consent, integrating AI into our lives to build intimate profiles about us whether we like it or not.
There is an urgent need for a private and ethical AI alternative. That's why we built Lumo, the first AI assistant that respects your privacy.
- Lumo doesn't keep any logs of your activity
- Zero-access encryption means nobody but you can see your saved chats
- Lumo is based in Europe, protected by strong privacy laws
- Your information cannot be shared with anyone or used to train AI
- Lumo is open source, so you can verify that it's private
Lumo is AI where you can safely ask anything, because it's truly confidential.
I guess it was only a matter of time... I'm not opposed to it, but we'll have to see if they live to their reputation of being private and responsible with user collected data. Which, to be clear, I have no reason to doubt.
There are a couple of interesting points I found after a quick glance at their ToS:
You may provide input to the Services (“Input”), and receive output from the Services based on the Input (“Output”). Input and Output are collectively “Content.” You are responsible for Content, including ensuring that it does not violate any applicable law or these Terms.
So, if the AI hallucinates something illegal, it's on you? I suppose they're doing this to covert themselves. Is this normal for other, similar services?
you agree that these Terms shall be governed in all respects by the substantive laws of Germany
The company is based in Switzerland, which also has some pretty good privacy laws, and has been one of the main selling points of Proton itself. So, why go by Germany's laws? Again, not sure if this is normal, but I found it strange.