When AI Attacks: Taxpayer Data, Beijing's Chatbots & LLM Hallucinations

Forecast = Punxsutawney Phil saw his shadow, so we can expect continued Musk-y days ahead in these remaining DOGE days of Winter.

‍

In this week's episode of GreyNoise Storm⚡️Watch, we have a bit of an AI-theme.

First, the Department of Government Efficiency (DOGE), led by Elon Musk, has sparked significant privacy and security concerns by accessing sensitive federal systems like Treasury databases and Education Department records through AI-driven analysis. Critics highlight undisclosed partnerships with vendors like Inventry.ai, which allegedly introduced algorithmic bias by disproportionately targeting diversity programs and climate initiatives while retaining fossil fuel subsidies. Cybersecurity experts warn about unvetted API integrations and data security risks, as Inventry.ai processed taxpayer information without proper FedRAMP authorization. These issues have led to bipartisan calls for stricter AI procurement rules and transparency mandates to rebuild public trust.

Meanwhile, Chinese AI startup DeepSeek faces scrutiny over its claims of rivaling GPT-4 at lower costs, with analysts questioning its $5.6M training budget and geopolitical alignment. The models show systematic pro-China biases, refusing to answer 88% of sensitive questions about Tiananmen Square or Taiwan while promoting CCP narratives in responses. Security researchers flag its opaque training data—potentially using OpenAI outputs—and anti-debugging features that hinder independent audits. These concerns have triggered bans in Australia, South Korea, and U.S. agencies like NASA, with EU officials noting non-compliance with cybersecurity standards.

On the defense front, Splunk's DECEIVE AI honeypot introduces innovative deception tech by letting users simulate systems via text prompts, democratizing access to advanced threat detection. While it offers dynamic behavioral analysis and safe sandboxing, security professionals caution about LLM hallucination risks that could tip off attackers and ethical questions around logging fabricated credentials. The open-source tool shows promise but remains untested against sophisticated adversaries.

Rounding out the cybersecurity landscape, Censys research exposes the BADBOX botnet's infrastructure and BeyondTrust vulnerabilities, while VulnCheck highlights 2024's exploitation trends and Zyxel's unpatched telnet flaws; and GreyNoise's latest Noiseletter showcases new platform features + upcoming events.

Storm Watch Homepage >>

Learn more about GreyNoise >>