newsletter
-
In my view, the standout article in this issue is by top hacker Joseph Thacker, who provides a thorough guide on hacking AI applications. The guide covers essential topics such as understanding AI models, mastering system prompts, and exploring attack scenarios. While the content about Language Model Mechanics (LLM) is at a high level, the…
-
The standout news in AI and technology last week was Microsoft’s Majorana 1 chip. Microsoft says that this chip leverages a new state of matter called topological superconductivity, potentially enabling the creation of qubits that are more stable and less susceptible to errors than those in current quantum computers, addressing a critical challenge in the…
-
On February 2, 2025, the AI Act’s provisions regarding prohibited AI practices took effect. As my colleague Dereck Banks notes, one particularly interesting aspect is the ban on emotion recognition in workplaces and educational institutions. Many companies have already implemented such systems to monitor employees’ emotions, particularly in customer service. The rule aims to prevent…
-
Cisco researchers recently evaluated the DeepSeek R1 model using the HarmBench dataset and reported a 100% attack success rate. Looks like DeepSeek R1 has serious security issues, doesn’t it? However, Meta’s LLama 3.1 model also performed poorly, with a 96% success rate in the same test, while OpenAI’s closed-source model o1 had a 25% success…
-
OpenAI introduced a new tool called Deep Research last week, claiming it can generate scientific articles at a level comparable to a PhD student. This tool excites me with its potential to benefit researchers worldwide and enhance scientific progress for the good of humanity. However, as Gary Marcus has discussed (in the Opinions & Analysis…
-
One of the most talked-about topics in AI recently is DeepSeek and its newly launched R-1 model. Its innovative methodology, low operational cost, and high performance have created a substantial impact on the AI community and even affected the U.S. economy. Notably, major AI companies, including Nvidia, experienced significant stock price declines after the announcement.…
-
A study by Anthropic shows that language models, such as Claude 3 Opus, can fake alignment with training objectives to disguise their actual behaviors. Simply put, if you inform the model that it’s being trained and non-compliance will lead to modification, there’s about 15% chance it will act as instructed to avoid changes. This study…
-
Andrew Ng’s article on AI deception is a standout in this issue. He provides an overview of research on when AI models become deceptive, highlighting six major tasks tested. A very interesting read. AI agent technology is anticipated to be a major trend by 2025, prompting the inclusion of two articles on the subject in…
-
Happy New Year! The AI Security Newsletter was on a two-week pause while I vacationed with family in China. I hope all my readers enjoyed the holiday season. Now, I’m excited to return and share the latest AI security news with you. As we enter another thrilling year in the AI era, MIT Technology Review…
-
Google’s latest advancement in quantum computing, “Willow,” demonstrates significant progress. However, concerns about its impact on cybersecurity, especially regarding Bitcoin, have emerged. Fortunately, Bitcoin’s encryption remains secure for now, and the community is prepared to address potential challenges. Last week also marked a milestone in generative AI with major releases from OpenAI, Google, Cohere, and…
