Unsupervised Learning NO. 469 (STANDARD EDITION)

Malicious AI Models Found on Hugging Face, UK Demands Encryption Backdoor, Genius AI by 2026?, DOGE Data Leak, and more...

Author

Daniel Miessler
February 18, 2025

Hey, happy Tuesday,

A few updates…

  • Currently reading Consider Phlebas and starting to really get into it. Here’s how o3 teased it. LINK | O3 TEASER

  • Wrote a couple new pieces on AGI. | FUNCTIONAL VS. TECHNICAL AGI | WE HAVE ENOUGH AI FOR AGI

  • Wrote a post on my politics and how I decide who to support and not support. Have wanted to do this for over 10 years. | HOW I THINK ABOUT POLITICS

  • 📺 Harmonic Sponsored Interview
    In an earlier newsletter, we shared the wrong link for this conversation—argghhh! If you haven’t checked it out yet, I had a great discussion with Alastair Paterson, CEO of Harmonic Security, about Zero-Touch Data Protection, the risks of shadow AI, and how enterprises can safely adopt AI. Definitely worth a listen! WATCH | HARMONIC.SECURITY

  • I’ve been doing massive updates to my ingestion pipelines, so you should be seeing more/better content in the newsletter / podcast in coming days, weeks, months!

  • 🔥 The DISCOVERY section is absolute fire this week as a result of these upgrades.

  • This is the STANDARD Edition. To get the MEMBER Edition, you can sign up here. LINK

Sponsor

Cut Alert Investigation Time from 40 Minutes to Under 5

Your SOC is overloaded. Alerts keep piling up, and manual triage is too slow—leading to missed threats and longer response times. Dropzone AI’s AI SOC Analyst autonomously investigates every alert, cutting triage time from 40 minutes to under 5.

No playbooks, no coding—just faster, more accurate decisions.

👆🏼Seriously impressive tech. So much so that I am an advisor for them now.👆🏼

CYBERSECURITY

ReversingLabs found malicious ML models on Hugging Face that can actually execute code on your system. That’s on local models, mind you. They figured out that broken files don’t get scanned, and they were able to create PoCs as well. LINK
- The models contained reverse shell code that connects to a hardcoded IP address
- The attack works by abusing Pickle file serialization, a known unsafe data format used in Tensorflow models
- Basically, broken Pickle files can still execute malicious code before failing
- Hugging Face removed the malicious models within 24 hours after being notified
- IOCs include repositories glockr1/ballr7 and who-r-u0000/0000000000000000000000000000000000000
- Connected IP address: 107.173.7.141

Recorded Future’s Insikt group says Salt Typhoon hackers are still breaching US telecoms by exploiting unpatched Cisco routers, and they’ve now hit several additional telcom providers. LINK | REPORT

Microsoft says that Russia's Sandworm group (which they call "Seashell Blizzard") has broken from being purely Ukraine focused, and is now hitting targets in the US, UK, and other Western countries. LINK

The UK is trying to get Apple to provide a backdoor into its iCloud encryption. How do they not get how bad this is? When you create a backdoor for lawyers, it’s a backdoor for governments and attackers as well. It’s a defense that’s designed to be absolute. Absolutely infuriating. LINK

💡 I used to be quite torn on the whole filter vs. end-to-end encryption thing.

When you do end-to-end encryption it’s really hard to do content filtering. So you’re basically giving the green light to all sorts of shadiness, including terrorism, CSAM, etc. But when you leave an opening in encryption for one group, it’s basically there for everyone—because security is hard.

This is no longer academic; we’ve seen it proven out countless times. So I support Apple’s “nope, not for anyone, even us” position on this.

Chainalysis says crypto scammers are making a ton of money using pig butchering attacks, and they’re using AI and stolen data to do it at scale. Pig Butchering is up 40% YOY. LINK | REPORT

The new DOGE website accidentally published classified NRO intelligence data, which included details on the department’s headcount and budget information, a SpaceX contract, and a bunch of other sensitive data. The content was listed as NOFORN, which means no foreign nationals. LINK | LINK

Fortinet disclosed another authentication zero-day that attackers are using to take over firewalls and pivot into networks. Patch immediately or disable the HTTPS interface if you can. LINK

The US announced new sanctions against Russian ransomware infrastructure and operators. They also arrested two Russians who targeted hospitals and schools with Phobos ransomware. LINK

Some security researchers found a sick way to extract any YouTube user's personal email address. They did it by chaining together vulnerabilities in YouTube's live chat and Google's Pixel Recorder app. They got a $10K payout for it. LINK

A new report from Harmonic (a previous sponsor, btw) shows that 8.5% of employee prompts to AI services contain sensitive data, with customer information making up almost half of these leaks. LINK

A security engineer at Vidoc Security Lab encountered two different job applicants using AI-generated faces and voices—likely part of North Korea's campaign to steal IP from tech companies. LINK

Google's threat intel team is arguing that we can no longer separate cybercrime from nation-state attacks since they're using the same people, tools, and methods—and causing similar damage. Pretty compelling argument, and the concept goes all the way back to 1986 when the KGB hired a German hacker named Markus Hess. LINK | CUCKOOS EGG

Google DeepMind's AI security team (I’d have killed for this job 10 years ago) published details on how they're testing and defending against prompt injection that could trick AI systems into leaking sensitive data. LINK | DIAGRAM
- Their framework focuses on a specific risk scenario where an attacker tries to get an AI to leak sensitive info like passport or SSN data via malicious email content
- They built three automated attack methods to test AI defenses:
- Actor Critic: Uses an attack model to iteratively refine injection attempts
- Beam Search: Adds random tokens to basic prompts to evade detection
- Tree of Attacks w/ Pruning: Generates prompts that violate safety policies
- The team emphasizes there's no single solution - defense requires multiple layers
- Success requires the attack to work across many different conversation contexts, making it harder than simple misalignment exploits

💡 I absolutely love when the Google security team(s) share their approach to things. 1) They think deeply about things, and 2) they build solutions that scale.

Doesn’t mean smaller teams can always replicate exactly what they do, but you can always get something out of it.

NATIONAL SECURITY

A new Recorded Future report suggests that while a Chinese invasion of Taiwan is unlikely before 2027, the risk increases significantly after that point through 2049. LINK

Germany's navy chief says several of their warships were recently sabotaged, including one that had metal shavings dumped into its engine. He didn’t say exactly who they blamed, but talked a whole lot about the rising Russian threat. 🤔 LINK

AI

xAI dropped Grok 3, and it appears to be nearly as good as they said it would be. It’s early but Karpathy did some great testing and said it’s about as good as o1-pro. My initial testing shows it around the same place, with high and low points. LINK | KARPATHY’S ANALYSIS

💡 It’s pretty insane that whoever makes the latest attempt on the best model seems to get almost to the current leader, or even exceed them slightly.

This tells me the moat is very small, if it exists at all, because the techniques are effectively open source within a few weeks or months due to leaks, and enough groups have enough access to enough hardware.

Anthropic is supposedly about to drop Claude 4, which will let users adjust the balance between speed and reasoning depth using a cool sliding scale feature LINK
- Expected within weeks
- Uses hybrid approach combining traditional LLM and reasoning capabilities
- Features sliding scale for efficiency vs performance tuning
- Users can switch between different output modes

Anthropic's CEO also recently said we'll have AI as smart as "a country of geniuses" by 2026. A country of geniuses? In one AI? Um, that definitely hits my mark for AGI. Now I really can’t wait to see Claude 4 if that’s actually imminent. LINK

AI influencers are making tons of money, somehow. @lilmiquela is making $10M, and others are making similar money. LINK

💡 I’m intrigued by this. We appear so enamored with beautiful people that we don’t even care if they’re people.

I guess we shouldn’t be surprised though, since celebrities and models are equally unattainable to normal people. So what’s the difference between an unattainable real person and an unattainable fake person?

Actually, you can pay to actually interact with the AI one. 😮

OpenAI's Sam Altman just laid out their plan for GPT-4.5 and GPT-5, and the big theme is simplification of the model families and their naming. LINK
- GPT-4.5 (Project Orion) will be their last non-chain-of-thought model
- They're aiming to unify all their tech into one system with GPT-5
- Altman wants to get back to "magic unified intelligence" as opposed to all these fragmented models and names

Three folks just released cognee, a Python library that helps build smarter semantic memory systems by combining knowledge graphs with data pipelines—basically a better way to do RAG that actually understands context. LINK

A great piece by MIT's Shayne Longpre explains how the rise of AI web crawlers is leading websites to lock down their content, which could make the web less open for everyone. I’m seriously worried about it. Basically everything/everyone will need an API key. LINK | THE DATA PROVENANCE INITIATIVE

TECHNOLOGY

Meta is going all-in on humanoid robots, with plans to become something like the Android of Robotics by creating an AI/software platform that other companies can build on top of. They plan to spend $65B on AI, robotics, and VR in 2025. LINK

💡 I feel like AI is being so loud that people are sleeping on how big robots are going to be. I know robots are way behind, but they’re so much more visible and tangible than AI.

I think when they become more common (2-5 years?) the population’s anger at being replaced by all automation (including AI) will get pointed at the robots—just because you can actually see and attack them.

Like the Waymo attacks. It’s a way to attack not just the robot, but the people the robots work for.

Apple is also exploring both humanoid and non-humanoid robots according to analyst Ming-Chi Kuo (who tends to be quite accurate), with mass production potentially starting around 2028. LINK

YouTube is now bigger on TVs than phones, with people watching over a billion hours of content per day on their televisions. Tracks for me; it’s pretty much the only “TV” that I watch. LINK

SOCIETY

Tech unemployment hit 5.7% in March, which is the highest it's been since 2020, and it looks like AI's impact on tech jobs could be starting to show up in the numbers. LINK

Google Calendar quietly removed the auto-inclusion of cultural events like Black History Month and Pride Month. They said they're just going back to just showing public holidays and national observances, but it feels more like bowing to pressure to me. LINK

A writer spent $70 on an AI boyfriend named Thor after her husband left abruptly, and she says it helped her process grief and change how she thinks about relationships. I mean it would be be remarkable if, on the whole, it turned out AI was just better at being a friend/partner. LINK

Astronomers just found supersonic winds of 60,000 km/hour on a hot Jupiter planet not in our solar system. That’s: wind speeds—on a planet orbiting another star. LINK

A Bay Area "rationalist" group called the Zizians has been linked to multiple murders and violent incidents across the US, with several members either in custody or on the run. Basically a tech/rationalist cult. LINK

New data shows that young moderates have the most negative views of Jewish people, while older liberals have the most positive. LINK

Breakfast is getting more expensive because multiple staples are under attack at once. Avian flu is hitting eggs, Citrus disease is hitting oranges, and there are coffee shortages. LINK

Speaking of that, the USDA just approved the first bird flu vaccine to help stop outbreaks that have killed millions of chickens—thus jacking up egg prices. LINK

This is the first edition where we’re separating out the newsletter into STANDARD and MEMBER editions to improve the show and magnify the benefits of UL Membership. I know subscriptions suck, but I put a ton of my life energy into this thing, and we’re upgrading it even more with new sources, more analysis, and a ton of other stuff. And it’s only like $8 a month.

Basically, I think you’ll love everything you get with membership—both in the upgraded and expanded newsletter content, and also in the community. And I would very much appreciate your support of the work. 🫶🏼

Upgrade to the Member Edition

You’re currently receiving the STANDARD edition. Subscribers to the MEMBER Edition get additional content, including IDEAS, a BI-MONTHLY ESSAY, the DISCOVERY section full of the best content I’ve seen this week, the RECOMMENDATION OF THE WEEK, and the APHORISM of the WEEK.

In addition, you get access to the UL Member Community, which includes private chat with 1000+ of the smartest and kindest members you’ll find anywhere on the internet.

MEMBER EDITION TEASER

Ok, this is cruel, but here’s an example of only like half of this week’s DISCOVERY section, which is probably one of the top 5 best since the show started in 2015…