AI Under Fire: Investigating the Use of ChatGPT in Covert Messaging and Disinformation

AI Under Fire: Investigating the Use of ChatGPT in Covert Messaging and Disinformation

The internet has always been a weird place. But lately it’s developed a new talent: it can look you dead in the eyes and lie with perfect grammar.

Today, the conversation around ChatGPT and other generative AI tools has shifted from “wow, it writes emails” to “wait… who’s using this, and for what?” On February 25–26, 2026, reporting around OpenAI’s latest disruption work put fresh fuel on the fire: romance scams, fake law firms, and attempts at political smears—all allegedly supported, in part, by AI-generated text and operational scaffolding. (Reuters)

This isn’t a “robots are taking over” story. It’s a “humans are doing human things—deception, influence, manipulation—using better tools” story. And that’s more unsettling, because it’s also more realistic.

What follows is a grounded, human-readable investigation into how tools like ChatGPT can be pulled into covert messaging and disinformation campaigns, what that actually looks like in practice, what defenses are emerging, and how we keep a functioning public reality when synthetic text can be produced faster than critical thinking.


The new battleground isn’t “AI vs humans” — it’s attention vs truth

Disinformation doesn’t need to convince everyone. It just needs to confuse enough people long enough to create noise, cynicism, and division. And the modern attention economy is basically a jet engine pointed directly at our frontal lobes.

Generative AI changes the scale of that problem. Not necessarily by creating magical propaganda that hypnotizes the masses—but by making it cheap to generate:

  • Endless variations of the same narrative

  • Localized messaging in multiple languages

  • Persona-based replies in comment sections

  • “Community-shaped” content that mimics real people

OpenAI itself has emphasized that influence operations often rely on a mix of AI and traditional infrastructure—fake accounts, websites, social platforms—where AI helps accelerate content production and iteration rather than replacing the entire operation. (OpenAI)

So the fear isn’t that one AI post will “mind-control” you. The fear is industrialized persuasion: thousands of posts, plausible personas, and continuous engagement—day after day—until the false starts feeling familiar.

And humans confuse familiarity with truth all the time. Brains are efficient that way. Sometimes tragically so.


What “covert messaging” really means (and why it’s not always spy-movie stuff)

When people hear “covert messaging,” they imagine secret codes, dead drops, or a villain stroking a cat while embedding Morse code into memes. Reality is less cinematic and more “annoyingly practical.”

Covert messaging generally means communicating in a way that hides intent, origin, coordination, or meaning from observers. That can include:

1) Concealed coordination

Multiple accounts appear independent but are actually orchestrated—posting at specific times, pushing the same themes, responding to critics in the same tone.

2) Plausible deniability content

Language that’s carefully written to suggest something while avoiding statements that are easily fact-checked or legally actionable.

3) Steganography in text

Steganography is hiding a message inside another message. Researchers have explored LLM-based steganography, where subtle choices in wording can encode information while remaining readable. That’s not science fiction; it’s an active research area, with papers proposing frameworks for covert communication using language models. (ScienceDirect)

Important note: most real-world “covert messaging” campaigns don’t need fancy steganography. They need coordination, volume, and distribution. The boring stuff. The effective stuff.


How ChatGPT shows up inside disinformation operations

Here’s the key: generative AI is rarely the whole machine. It’s the accelerant.

Based on public reporting and OpenAI’s own disruption summaries, misuse patterns often include:

Drafting and polishing propaganda-like narratives

Threat actors may try to generate posts, talking points, or “reports” designed to push a storyline. OpenAI has previously described disruptions related to covert influence operations and their typical tactics. (OpenAI)

Translation and localization

Instead of one clumsy English post, you get dozens of versions tailored for specific regions, dialects, or cultural references—making campaigns more believable and harder to spot.

Persona scripting

AI can help write like a “single mom,” a “disillusioned veteran,” a “concerned student,” or any other identity-shaped voice. (Which is exactly why authenticity online is getting melted into soup.)

Comment flooding and engagement farming

Even low-quality generated text can overwhelm moderation systems and drown out real discussion. Quantity becomes a weapon.

Operational support for scams

Recent reporting highlights OpenAI’s disruption work involving romance scams and impersonation-style fraud, where AI can help generate messages, scripts, and “professional” sounding material. (Reuters)

None of this requires superintelligence. It requires speed, scale, and low cost—three things generative AI is very happy to provide.


A timely example: what OpenAI’s February 2026 reporting put on the table

On February 25–26, 2026, multiple outlets summarized a new OpenAI threat report describing how malicious actors attempted to use ChatGPT in scams and influence efforts, including operations linked in reporting to romance scams, fake legal services, and a smear effort involving Japanese political leadership. (Reuters)

Two things matter here:

  1. The operations weren’t “AI-only.” They used platforms, fake accounts, and other tools—AI helped generate or refine content. (OpenAI)

  2. The response wasn’t just hand-wringing. It included disruption actions (like banning accounts) and reporting intended to improve transparency and collective defense. (OpenAI)

That’s a crucial shift: we’re moving from “AI might be misused someday” to “here are case studies, patterns, and countermeasures now.”


Why disinformation works so well: it exploits normal human software

Disinformation isn’t powerful because people are dumb. It’s powerful because people are human.

It exploits:

  • Cognitive load: we can’t verify everything

  • Social proof: “everyone’s saying it” feels convincing

  • Identity protection: information that flatters our group is easier to accept

  • Outrage loops: anger spreads faster than nuance

Generative AI can mass-produce content that presses these buttons—carefully, repeatedly, and at scale.

And there’s a nasty twist: disinformation doesn’t only spread false beliefs. It spreads epistemic exhaustion—the feeling that truth is unknowable and everything is propaganda. That’s how you get people to disengage from civic reality entirely.


The stealthy part: disinformation is evolving into “disinformation-as-a-service”

In the old days, running a campaign took writers, translators, and a lot of time.

Now, a small team can do more—faster—because AI can provide:

  • infinite drafts

  • rewriting for tone (“more angry,” “more empathetic,” “more formal”)

  • translation and cultural adaptation

  • rapid response to news cycles

This is why the phrase information warfare is no longer dramatic. It’s accurate.

The scary bit isn’t just that lies spread. It’s that lies can be continuously optimized, like a marketing funnel—A/B testing narratives until something sticks.


“But can’t we detect AI text?” Sometimes. Not reliably.

A lot of people want a magic filter that says: AI wrote this.

Reality check: detection is messy.

  • People edit AI output.

  • AI output gets paraphrased.

  • Humans also write in “generic internet voice,” which confuses detectors.

  • Attackers can intentionally vary style and structure.

Even OpenAI and other major labs have been cautious about overpromising reliable AI-text detection at internet scale. The most effective approach is usually behavioral and network-based:

  • coordinated posting patterns

  • shared infrastructure

  • synchronized engagement bursts

  • repeated narrative templates across accounts

In other words: catching the campaign, not the sentence.


What platforms, journalists, and researchers are doing about it

The best defenses look less like one big wall and more like many smaller systems working together:

Threat intelligence and disruption

OpenAI’s reporting emphasizes investigation and disruption of abusive networks, including banning accounts linked to malicious activity. (OpenAI)

Provenance and authenticity tooling

There’s growing interest in content provenance—cryptographic or metadata-based methods to show where media came from and whether it was altered. (Not a perfect solution, but useful in high-stakes contexts.)

Friction and verification

Platforms can require extra verification for accounts that behave like bots—especially during elections or crises.

Media literacy that doesn’t insult people

The most effective media literacy isn’t “don’t be stupid.” It’s teaching people how manipulation works: emotional triggers, fake credibility signals, and “too-perfect” narratives.


What you can do today (without becoming a paranoid detective monk)

You don’t need to fact-check every atom of the internet. You do need a few habits that scale.

  1. Slow down at emotional spikes
    If a post makes you instantly furious or euphoric, that’s a sign it’s pushing buttons. Pause.

  2. Check for primary sources
    Screenshots of headlines are not sources. “A friend sent this” is not a source.

  3. Look for coordination cues
    Same phrasing across many accounts. Same hashtags. Same links. Same timing. That’s campaign smell.

  4. Treat certainty as suspicious
    Reality is complicated. Propaganda is confident.

  5. Favor trusted verification channels
    Not “someone with a blue check,” but organizations with transparent correction practices.


The policy angle: where responsibility lands

OpenAI’s published usage policies explicitly prohibit generating or promoting disinformation and related deceptive behaviors, and emphasize compliance with safeguards. (OpenAI)

That matters—but policy alone isn’t enough. The real ecosystem includes:

  • model providers (policies + enforcement + transparency)

  • platforms (distribution controls + moderation + verification)

  • governments (laws + oversight, ideally without censorship abuse)

  • civil society (watchdogs, researchers, journalists)

  • users (yes, us—annoying but true)

It’s a collective action problem. Everyone wants clean information. No one wants to pay the full cost. Meanwhile, attackers are highly motivated and don’t care about ethics.

Welcome to the asymmetry buffet.


Where this is headed: three realistic futures (none of them boring)

Future 1: The “verified web” grows

More content becomes signed, sourced, and traceable—especially for news and high-stakes media. Casual content remains messy, but credibility layers improve.

Future 2: Disinformation gets more personalized

Campaigns move from broad messaging to micro-targeted narratives shaped for niche communities. AI makes that cheap.

Future 3: A culture shift toward “epistemic hygiene”

People start treating information intake like food safety: not perfect, but guided by norms (“wash your sources,” “cook your claims,” “don’t eat raw rumors”).

We’ll probably get a mix of all three, plus some chaos seasoning.


Bottom line: AI isn’t the villain, but it is a power tool

ChatGPT didn’t invent deception. It just makes deception easier to scale—especially when combined with distribution systems that reward engagement over accuracy.

The hopeful part is that the same technology can be used defensively: to detect coordinated campaigns, help researchers triage content, assist fact-checking workflows, and educate users faster than bad actors can adapt.

The war isn’t “AI vs truth.” The war is incentives vs truth.

And incentives are slippery little gremlins.


SEO keywords paragraph (one paragraph, optimized)

AI disinformation, ChatGPT misuse, covert messaging, misinformation campaigns, information warfare, propaganda detection, AI security, LLM security, generative AI risks, coordinated inauthentic behavior, social engineering, online manipulation, influence operations, cyber threats, threat intelligence, OSINT investigation, digital forensics, election disinformation, political propaganda, fake news detection, content authenticity, media literacy, fact-checking tools, AI governance, platform integrity, scam prevention, romance scam tactics, impersonation fraud, chatbot safety, OpenAI threat report 2026, AI policy compliance, content moderation strategies, synthetic text detection, steganography in text, linguistic steganography, AI transparency, trust and safety, algorithmic amplification, disinformation defense, online trust crisis.