Yes, this is the new battlefield. Previously, there was a lot of talk about jailbreaking models, but seeding the internet with trojan data is the next frontier.
We see it already happening with research papers, I mentioned recently here.
And there is no solution, as you can't ensure reliable behavior of LLMs. There is no such thing as "AI Safety" when the attack surface is essentially the entirety of human language.
Just to be clear, the Pravda that this article talks about is the Russian one. There is a Ukrainian Pravda (https://www.pravda.com.ua), which is presumably more accurate.
“Even with that knowledge, it nevertheless often repeats propaganda from Pravda.” — that is of course because the models are *text* models, not *language* models, let alone *knowledge* models.
I was wondering when Grok 4 and Elon would inevitably would appear in this conversation. I thought about mentioning them myself, but I personally didn't want to give Elon more attention than he already has.
Humans (the 'gold standard of cognition') are spectacularly bad at this exact task. Millions of people read and share articles from known propaganda outlets every single day. The entire field of media literacy exists because the average person doesn’t naturally make the reasoning leap:
(A) This source is biased + (B) This article comes from that source ⇒ (C) I should be highly skeptical of this content.
Humans also struggle to distinguish satire from real news - as shown by the frequent sharing of Onion articles as fact. Conspiracy theories thrive precisely because people fail to evaluate sources and apply consistent reasoning.
So, while Gary frames the LLM’s failure as a uniquely artificial and dangerous flaw, it’s actually one of the most pervasive and dangerous flaws in human cognition.
That sounds about right. The BBC tries hard to get its reporting as accurate as possible. It doesn't always succeed, and gets hauled over the coals in public spaces when it does, but it tries.
Not true. The BBC always puts a negative spin on stories coming out of countries which don’t toe the Anglo-Saxon line anymore. Not as blatant as the US rags, but very untrustworthy.
Interesting idea. I agree that the BBC has a particular world viewpoint, if that's what you mean. I think the BBC has probably reduced its local journalist coverage over the years, which is going to lead it to make more mistakes, but I believe the organisation remains committed to giving a level-headed analysis of the world's events. Can you give me an example that illustrates your point?
Stopped reading when you said "Russian disinformation".
Also anyone who doesn't think Bucha was a hoax is dumb. Why would retreating Russians return to a town after a few days to kill civilians for no reason? Maybe the AI isn't hallucinating and you're wrong
So there is 'our truth' and 'their truth'. Since we are by God's mercy leading power and democracy with our government and military-industrial complex bearing the highest moral authority and racial and cultural supremacy we, self-proclaimed hegemons and truth-arbiters, we say that 'their truth' now becomes disinformation. Pravda newspapers will be from now on labeled as 'propaganda outlet' so that only checked and verified (by us) truth can be accessible to our sheeps who otherwise do not have critical thinking skills and rely on us, moral economical political elites to say what is true and what is falsehood and rigged narrative.
We should also be aware that propaganda is also common in the West. Exhibit 1. Fox News. However, there are many others. This is why, rather than slurping up as much text as possible for training, only a carefully curated database should be used. It won't be perfect, but it would be better.
Neal Stephenson's alt-reality novel, "Anathem" has an internet for retrieving news and information, but items are given a trust rating so that the search filters out low-trust material. The problem in our reality is who does that rating...
The result is that we are stuck with resorting to "critical thinking". This is not easy and adds an unwanted cognitive load. This needs to be inculcated early in life to make it less of a burden. It still won't be a perfect solution, but it will be better. Of course, the easy solution is just to follow the media and people you trust, but that becomes tribal, as anyone who has argued with a Fox News imbiber knows. (It doesn't help that the GOP seems intent on dumbing down K-12 education, and now controlling what universities can teach, too.)
You’re more likely to get at the truth from non-western media as they aren’t as tightly controlled and integrated as US media. Watch the numerous videos of the talking heads babbling the same talking points.
Our media is still highly competitive and fight each other constantly and the truth always leaks out.
This is not surprising! We have synthetic data. It all was inevitable. I remember my high school electronic engineering teacher. He said to me 35 years ago G.I.G. O = garbage in garbage out. Not surprising to me.
Of course LL’s can be groomed. Truth claims have a moral dimension, one which an LLM cannot “know”. A LLM cannot evaluate a truth claim. How could it “know” that the earth is round? It cannot observe the real world or evaluate that claim in any sense. On what basis would it assign credibility to the claim the earth is round over the flat earther’s claim that it is not? The reason live testimony by witnesses is required in courts (at least in the U.S) so that the judge or jury can evaluate the witness’s credibility. Something an LLM cannot do.
I'm reminded of my first big exposure to LLM grooming: Kevin Roose's sensationalised 2023 "Bing’s A.I. Chat: ‘I Want to Be Alive. 😈’" going viral.
To clarify - you can think of this as LLM *Prompt* grooming as distinct from LLM *Data* grooming (as discussed in this current post). LLM Prompt grooming is a problem in its own right that might be caused either intentionally or unintentionally (the latter through our inherent biases, much as Roose's example).
While the main takeaway that lead to the viral exposure of Roose's experience was "shock and concern" at the A.I. responses, *my* main reaction was to be "creeped out" by Roose's apparent proficiency with the use of LLM-prompt-grooming techniques.
Reading the article and conversation transcript, I was most shocked by Roose's use of conversation patterns that reeked of the grooming attacks used online by adults against naive, unsuspecting minors.
As a mode of calculated attack against an LLM / RNN, LLM-prompt-grooming makes a lot of sense: what I couldn't fathom at the time was Roose's performative shock and concern, when to me the LLM engine was simply providing reasonable / anticipatable responses to well-established patterns of conversational grooming.
I saw something like this effect for historical material, too. ChatGPT didn't mention the trials or war crimes for 5 of 14 defendants in one of the Nuremberg Trials https://blog.zgp.org/llms-and-reputation-management/
Yes, this is the new battlefield. Previously, there was a lot of talk about jailbreaking models, but seeding the internet with trojan data is the next frontier.
We see it already happening with research papers, I mentioned recently here.
https://www.mindprison.cc/p/the-ai-hacking-wars-begin-trojan-data
And there is no solution, as you can't ensure reliable behavior of LLMs. There is no such thing as "AI Safety" when the attack surface is essentially the entirety of human language.
Just to be clear, the Pravda that this article talks about is the Russian one. There is a Ukrainian Pravda (https://www.pravda.com.ua), which is presumably more accurate.
Your credulity that any government “news” organization, especially during wartime, is reliably accurate is amusing.
“Even with that knowledge, it nevertheless often repeats propaganda from Pravda.” — that is of course because the models are *text* models, not *language* models, let alone *knowledge* models.
And some of the bad actors are the AI executives themselves, as in Grok 4 looking to see what Elon thinks before answering.
I was wondering when Grok 4 and Elon would inevitably would appear in this conversation. I thought about mentioning them myself, but I personally didn't want to give Elon more attention than he already has.
Incidentally, speaking of Grok and Elon making an appearance.
Have we ever seen Elon and Grok in the same room together?
Thank you so much for this work.
Humans (the 'gold standard of cognition') are spectacularly bad at this exact task. Millions of people read and share articles from known propaganda outlets every single day. The entire field of media literacy exists because the average person doesn’t naturally make the reasoning leap:
(A) This source is biased + (B) This article comes from that source ⇒ (C) I should be highly skeptical of this content.
Humans also struggle to distinguish satire from real news - as shown by the frequent sharing of Onion articles as fact. Conspiracy theories thrive precisely because people fail to evaluate sources and apply consistent reasoning.
So, while Gary frames the LLM’s failure as a uniquely artificial and dangerous flaw, it’s actually one of the most pervasive and dangerous flaws in human cognition.
Pravda is propaganda and CNN/NYT/WSJ/BBC are not?
Okay then.
Update: https://www.opindia.com/2025/07/bbc-gaza-documentary-hamas-leaders-son-narrator/
That sounds about right. The BBC tries hard to get its reporting as accurate as possible. It doesn't always succeed, and gets hauled over the coals in public spaces when it does, but it tries.
Not true. The BBC always puts a negative spin on stories coming out of countries which don’t toe the Anglo-Saxon line anymore. Not as blatant as the US rags, but very untrustworthy.
The Brits are still sore about losing India and the rest of their empire.
BBC negative spin is just their pathetic way of getting back.
Interesting idea. I agree that the BBC has a particular world viewpoint, if that's what you mean. I think the BBC has probably reduced its local journalist coverage over the years, which is going to lead it to make more mistakes, but I believe the organisation remains committed to giving a level-headed analysis of the world's events. Can you give me an example that illustrates your point?
https://thecommunemag.com/5-times-when-bbc-peddled-propaganda-against-india/
This really doesn’t help your case but I understand better where you are coming from. Thank you.
I can’t and won’t educate the wilfully ignorant.
And therein lies the difference: When newspapers or journalists print mistakes, they're held accountable and subject to lawsuits and public scrutiny.
Stopped reading when you said "Russian disinformation".
Also anyone who doesn't think Bucha was a hoax is dumb. Why would retreating Russians return to a town after a few days to kill civilians for no reason? Maybe the AI isn't hallucinating and you're wrong
So there is 'our truth' and 'their truth'. Since we are by God's mercy leading power and democracy with our government and military-industrial complex bearing the highest moral authority and racial and cultural supremacy we, self-proclaimed hegemons and truth-arbiters, we say that 'their truth' now becomes disinformation. Pravda newspapers will be from now on labeled as 'propaganda outlet' so that only checked and verified (by us) truth can be accessible to our sheeps who otherwise do not have critical thinking skills and rely on us, moral economical political elites to say what is true and what is falsehood and rigged narrative.
We should also be aware that propaganda is also common in the West. Exhibit 1. Fox News. However, there are many others. This is why, rather than slurping up as much text as possible for training, only a carefully curated database should be used. It won't be perfect, but it would be better.
Neal Stephenson's alt-reality novel, "Anathem" has an internet for retrieving news and information, but items are given a trust rating so that the search filters out low-trust material. The problem in our reality is who does that rating...
The result is that we are stuck with resorting to "critical thinking". This is not easy and adds an unwanted cognitive load. This needs to be inculcated early in life to make it less of a burden. It still won't be a perfect solution, but it will be better. Of course, the easy solution is just to follow the media and people you trust, but that becomes tribal, as anyone who has argued with a Fox News imbiber knows. (It doesn't help that the GOP seems intent on dumbing down K-12 education, and now controlling what universities can teach, too.)
Almost every word out of western media is propaganda. Westerners can’t see it because they grew up schooled with lies.
As opposed to non-Western nations? Did you live in your own national bubble?
You’re more likely to get at the truth from non-western media as they aren’t as tightly controlled and integrated as US media. Watch the numerous videos of the talking heads babbling the same talking points.
Our media is still highly competitive and fight each other constantly and the truth always leaks out.
You’re the ones living in a bubble.
The Truman Show was funny because it was true.
I watch the BBC in the UK, which has a global reputation for impartiality. Which country's media do you watch?
Don’t make me laugh. The BBC is state funded propaganda, just the old USSR Pravda with a posh British accent.
https://thecommunemag.com/calling-out-bbcs-holier-than-thou-attitude/
This is not surprising! We have synthetic data. It all was inevitable. I remember my high school electronic engineering teacher. He said to me 35 years ago G.I.G. O = garbage in garbage out. Not surprising to me.
AIIGO AI In Garbage Out
thank you
I now have an image of LLMs eating their own crap which I can't get out of my head...
I wonder if models are particularly prone to being tripped up by Pravda because pravda means "truth" in Russian.
“Truth” (fake or real) does not enter into the equation of LLM relativity
Output = LLMc^2
Of course LL’s can be groomed. Truth claims have a moral dimension, one which an LLM cannot “know”. A LLM cannot evaluate a truth claim. How could it “know” that the earth is round? It cannot observe the real world or evaluate that claim in any sense. On what basis would it assign credibility to the claim the earth is round over the flat earther’s claim that it is not? The reason live testimony by witnesses is required in courts (at least in the U.S) so that the judge or jury can evaluate the witness’s credibility. Something an LLM cannot do.
I'm reminded of my first big exposure to LLM grooming: Kevin Roose's sensationalised 2023 "Bing’s A.I. Chat: ‘I Want to Be Alive. 😈’" going viral.
To clarify - you can think of this as LLM *Prompt* grooming as distinct from LLM *Data* grooming (as discussed in this current post). LLM Prompt grooming is a problem in its own right that might be caused either intentionally or unintentionally (the latter through our inherent biases, much as Roose's example).
While the main takeaway that lead to the viral exposure of Roose's experience was "shock and concern" at the A.I. responses, *my* main reaction was to be "creeped out" by Roose's apparent proficiency with the use of LLM-prompt-grooming techniques.
Reading the article and conversation transcript, I was most shocked by Roose's use of conversation patterns that reeked of the grooming attacks used online by adults against naive, unsuspecting minors.
As a mode of calculated attack against an LLM / RNN, LLM-prompt-grooming makes a lot of sense: what I couldn't fathom at the time was Roose's performative shock and concern, when to me the LLM engine was simply providing reasonable / anticipatable responses to well-established patterns of conversational grooming.
I saw something like this effect for historical material, too. ChatGPT didn't mention the trials or war crimes for 5 of 14 defendants in one of the Nuremberg Trials https://blog.zgp.org/llms-and-reputation-management/