AI in Cybersecurity

AI is not failing because of bad ideas; it’s "failing" at enterprise scale because of two big gaps: 👉 Workforce Preparation 👉 Data Security for AI While I speak globally on both topics in depth, today I want to educate us on what it takes to secure data for AI—because 70–82% of AI projects pause or get cancelled at POC/MVP stage (source: #Gartner, #MIT). Why? One of the biggest reasons is a lack of readiness at the data layer. So let’s make it simple - there are 7 phases to securing data for AI—and each phase has direct business risk if ignored. 🔹 Phase 1: Data Sourcing Security - Validating the origin, ownership, and licensing rights of all ingested data. Why It Matters: You can’t build scalable AI with data you don’t own or can’t trace. 🔹 Phase 2: Data Infrastructure Security - Ensuring data warehouses, lakes, and pipelines that support your AI models are hardened and access-controlled. Why It Matters: Unsecured data environments are easy targets for bad actors making you exposed to data breaches, IP theft, and model poisoning. 🔹 Phase 3: Data In-Transit Security - Protecting data as it moves across internal or external systems, especially between cloud, APIs, and vendors. Why It Matters: Intercepted training data = compromised models. Think of it as shipping cash across town in an armored truck—or on a bicycle—your choice. 🔹 Phase 4: API Security for Foundational Models - Safeguarding the APIs you use to connect with LLMs and third-party GenAI platforms (OpenAI, Anthropic, etc.). Why It Matters: Unmonitored API calls can leak sensitive data into public models or expose internal IP. This isn’t just tech debt. It’s reputational and regulatory risk. 🔹 Phase 5: Foundational Model Protection - Defending your proprietary models and fine-tunes from external inference, theft, or malicious querying. Why It Matters: Prompt injection attacks are real. And your enterprise-trained model? It’s a business asset. You lock your office at night—do the same with your models. 🔹 Phase 6: Incident Response for AI Data Breaches - Having predefined protocols for breaches, hallucinations, or AI-generated harm—who’s notified, who investigates, how damage is mitigated. Why It Matters: AI-related incidents are happening. Legal needs response plans. Cyber needs escalation tiers. 🔹 Phase 7: CI/CD for Models (with Security Hooks) - Continuous integration and delivery pipelines for models, embedded with testing, governance, and version-control protocols. Why It Matter: Shipping models like software means risk comes faster—and so must detection. Governance must be baked into every deployment sprint. Want your AI strategy to succeed past MVP? Focus and lock down the data. #AI #DataSecurity #AILeadership #Cybersecurity #FutureOfWork #ResponsibleAI #SolRashidi #Data #Leadership

📌 European Union Agency for Cybersecurity (ENISA)'s European Cybersecurity Certification Group Sub-group on Cryptography published their "Agreed Cryptographic Mechanisms". The document covers cryptyographic primitives (algorithms), constructions (encryption, signatures, etc), TLS, RNGs and key management. It's purpose is to "specify which cryptographic mechanisms are recognised agreed, i.e., ready to be accepted by all national cybersecurity certification authorities (NCCAs)". Some highlights from a quantum-safety perspective: 👉 Recommends hybridization to "provide assurance against the quantum threat as well as assurance against security issues that might affect the newer standardized post-quantum mechanisms" 👉 Symmetric 🏷️ Supports Triple-DES until 2027, despite it is disallowed by NIST already 🏷️ Recommends >192-bit parameters when quantum resistance is desired 👉 Hashes & MAC 🏷️ Recommends >384-bit output sizes when quantum resistance is desired 👉 Asymmetric 🏷️ Classical / Quantum-vulnerable 🤔 Parameters approx. under 128-bit security (RSA2048, DH-2048, DSA-2048) are accepted until end of 2025! 💣For RSA, it specifies: "A later acceptability deadline for user/data authentication with this particular algorithm may be set on a national level." Minimum ECC key size is at 256 bits, so it doesn't include that end of life deadline. 🏷️ Post-quantum #PQC 🔖 Lattice cryptography (ML-DSA, ML-KEM) should not be used in standalone mode. Always in hybrid mode with a strong classical algoritm. 🔖 ML-DSA and ML-KEM are recommended on level 3 and 5 parameters. Level 1 is no recommended. 🔖 Hybridization of Hash-based signature schemes is optional. SLH-DSA is supported under Level 3 and 5 parameters. 🔖 Frodo-KEM is supported under Level 3 and 5 parametersand in hybrid mode. 👉 Deterministic RNGs 🏷️ Recommended that the min-entropy of the seed is at least 188 bits This document is interesting and clarifying, but I see two issues: 1. I haven't seen a timeline to deprecation of quantum-vulnerable cryptography in general. I think that's needed and National Institute of Standards and Technology (NIST) has done well in announcing it (in draft form for now) under NIST IR 8547. 2. A deadline on 2025 for 112 bit classical crypto, like RSA-2048 seems too strict for me. New norms should avoind being challenged by reality. No other organism has gone that close and I don't think the world will stop using RSA-2048 in 2026. https://lnkd.in/dUi46V3s #cryptography #quantum #postquantum

The 10 AI Threats Quietly Putting Enterprises at Risk What most companies get wrong about AI security? Thinking it’s just a “tech problem.” It’s not. It’s a behavior problem. Enterprise AI is no longer just answering questions. It’s making decisions. Triggering actions. Accessing sensitive systems. And that changes everything. Here’s the part many teams underestimate: AI doesn’t need to be hacked… It just needs to be misguided. And the impact looks exactly like a breach. Here are 10 AI security threats every enterprise should be thinking about: Prompt Injection Attacks ↳ AI follows malicious instructions → data leaks or wrong actions Data Poisoning ↳ Bad data in training = corrupted outputs at scale Model Inversion ↳ Attackers pull sensitive data from responses Sensitive Data Leakage ↳ Poor context control exposes confidential info API Key & Credential Theft ↳ One stolen key = full system access Unauthorized Tool Invocation ↳ AI triggers actions it shouldn’t even have access to Supply Chain Vulnerabilities ↳ Third-party models can introduce hidden risks Model Drift ↳ AI silently becomes unreliable over time Excessive Autonomy ↳ Agents act beyond boundaries → real-world damage Compliance Violations ↳ AI outputs break regulations without warning What actually protects you isn’t just better models. It’s better control. • Input and output guardrails • Dataset validation pipelines • Access control and tool restrictions • Continuous monitoring • Human-in-the-loop for critical decisions Because here’s the reality: The more powerful your AI becomes… The smaller your margin for error gets. The companies that win with AI won’t be the fastest. They’ll be the most controlled. If you’re deploying AI today Are you treating it like a smart assistant… or like a potential insider with access to everything? Share it with your network. 📌 Follow Marcel Velica for more insights on AI, security, and real-world strategies. If you want short daily thoughts, quick threat observations, and real-time discussions, follow me on X as well →https://x.com/MarcelVelica

The National Institute of Standards and Technology (NIST) has released a draft of its “Cybersecurity Framework Profile for Artificial Intelligence” (open for public comment until Jan 30, 2026) to help organizations think about how to strategically adopt AI while addressing emerging cybersecurity risks that stem from AI’s rapid advance. Building on the #NIST Cybersecurity Framework 2.0, the Cyber AI Profile translates well-established risk management concepts into AI-specific cybersecurity considerations, offering a practical reference point as organizations integrate AI into critical systems and confront AI-enabled threats. The Cyber AI Profile centers on three focus areas: • Securing AI systems: identifying cybersecurity challenges when integrating AI into organizational ecosystems and infrastructure. • Conducting AI-enabled cyber defense: identifying opportunities to use AI to enhance cybersecurity, and understanding challenges when leveraging AI to support defensive operations. • Thwarting AI-enabled cyberattacks: building resilience to protect against new AI-enabled threats. The Profile complements existing NIST frameworks (CSF, AI RMF, RMF) by prioritizing AI-specific cybersecurity outcomes rather than creating a standalone regime.

Today, NIST released the initial preliminary draft of the Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile), a community profile built on NIST CSF 2.0 to help organizations manage cybersecurity risk in an AI-driven world. A key section of this draft is Section 2.1, which introduces three Focus Areas that explain how AI and cybersecurity intersect in practice: 1. Securing AI System Components (Secure) AI systems introduce new assets that must be secured; models, training data, prompts, agents, pipelines, and deployment environments. This focus area emphasizes treating AI components as first-class cybersecurity assets, integrating them into governance, risk assessments, protection controls, and monitoring processes. It reinforces that AI risk should not be siloed from enterprise cybersecurity risk management. 2. Conducting AI-Enabled Cyber Defense (Defend) AI is not just something to protect, it is also a powerful defensive capability. This area focuses on using AI to enhance detection, analytics, automation, and response across security operations. At the same time, it recognizes the risks of over-reliance on automation, model integrity concerns, and the need for human oversight when AI supports security decision-making. 3. Thwarting AI-Enabled Cyber Attacks (Thwart) Adversaries are increasingly using AI to scale phishing, evade detection, and automate attacks. This focus area addresses how organizations must anticipate and counter AI-enabled threats by building resilience, improving detection of AI-driven attack patterns, and preparing for a rapidly evolving threat landscape where AI is weaponized. Why This Matters Together, Secure, Defend, and Thwart provide a practical structure for aligning AI initiatives with existing cybersecurity programs. By mapping AI-specific considerations to CSF 2.0 outcomes (Govern, Identify, Protect, Detect, Respond, Recover), the Cyber AI Profile helps organizations integrate AI security into familiar risk management practices. This is a preliminary draft, and NIST is seeking public feedback through January 30, 2026. If your organization is building, deploying, or defending with AI, now is the time to review and contribute. 🔗 https://lnkd.in/e-ETZXH8

📛 CVE 2025 32711 is a turning point Last week, we saw the first confirmed zero click prompt injection breach against a production AI assistant. No malware. No links to click. No user interaction. Just a cleverly crafted email quietly triggering Microsoft 365 Copilot to leak sensitive org data as part of its intended behavior. Here’s how it worked: • The attacker sent a benign-looking email or calendar invite • Copilot ingested it automatically as background context • Hidden inside was markdown-crafted prompt injection • Copilot responded by appending internal data into an external URL owned by the attacker • All of this happened without the user ever opening the email This is CVE 2025 32711 (EchoLeak). Severity 9.3 Let that sink in. The AI assistant did exactly what it was designed to do. It read context, summarized, assisted. But with no guardrails on trust boundaries, it blended attacker inputs with internal memory. This wasn’t a user mistake. It wasn’t a phishing scam. It was a design flaw in the AI data pipeline itself. 🧠 The Novelty What makes this different from prior prompt injection? 1. Zero click. No action by the user. Sitting in the inbox was enough 2. Silent execution. No visible output or alerts. Invisible to the user and the SOC 3. Trusted context abuse. The assistant couldn’t distinguish between hostile inputs and safe memory 4. No sandboxing. Context ingestion, generation, and network response occurred in the same flow This wasn’t just bad prompt filtering. It was the AI behaving correctly in a poorly defined system. 🔐 Implications For CISOs, architects, and Copilot owners - read this twice. → You must assume all inputs are hostile, including passive ones → Enforce strict context segmentation. Copilot shouldn’t ingest emails, chats, docs in the same pass → Treat prompt handling as a security boundary, not just UX → Monitor agent output channels like you would outbound APIs → Require your vendors to disclose what their AI sees and what triggers it 🧭 Final Thought The next wave of breaches won’t look like malware or phishing. They will look like AI tools doing exactly what they were trained to do but in systems that never imagined a threat could come from within a calendar invite. Patch if you must. But fix your AI architecture before the next CVE hits.

The AI cybersecurity race is here. Today, the Google Threat Intelligence Group released our latest AI Threat Tracker. Here’s the reality: adversaries are deploying highly coordinated, AI-augmented operations at scale. To build a resilient enterprise, security leaders recognize that protecting the AI pipeline is what ultimately unlocks the confidence to scale it. Here are three findings from this latest intelligence: - First AI-developed zero-day: We identified a zero-day exploit (a 2FA bypass) where the adversary likely used an AI model to assist in discovering and weaponizing the vulnerability. The script contained clear indicators of AI generation, including a hallucinated CVSS security score. Our discovery likely prevented its use in a planned mass exploitation event. - Autonomous malware: We're tracking PROMPTSPY, a new Android backdoor designed to autonomously navigate a victim’s device UI and actively block uninstallation attempts. - AI supply chain attacks: Adversaries are increasingly targeting AI software dependencies, such as LiteLLM, to compromise build environments and extract cloud credentials. In this landscape, manual defense fails. When adversaries use automation, defense must move at machine speed. At Google, we are tipping the scale back to the defender: by deploying agentic cyber defense—like Big Sleep and CodeMender—we are finding and patching vulnerabilities before they can be exploited. We are using AI to build software that is secure by design, even as we continue to defend the massive landscape of legacy code the world relies on today. Read the full GTIG AI Threat Tracker report here: https://lnkd.in/gn6UHXaV 

AI Agents Are the New Attack Surface! Are We Ready for That? AI agents powered by large language models (LLMs) introduce entirely new vulnerabilities across confidentiality, integrity, and availability. Here’s what’s new and why it matters: AI Agents execute actions: Unlike typical LLMs, agents interact with tools, systems, and APIs, meaning a hallucinated or adversarial output can change files, leak data, or flood networks. Session management is a blind spot: Most agents don’t isolate user sessions robustly. Result: chat histories bleed across users, leading to data leaks and misassigned actions. Model pollution is real: Malicious inputs can subtly "poison" fine-tuned models, degrading performance and trust without being obviously adversarial. Sandboxing isn’t optional: Experiments showed that 90 out of 95 malicious prompts were accepted by a state-of-the-art agent, with 80% successfully executed, unless sandboxed. Promising defense directions: Session-aware memory and formal monads for state tracking, Encryption-preserving inference (like FPETS and FHE) to process sensitive data safely or toolchain access controls that isolate file systems and limit network requests. 📣 Bottom line: The same autonomy that makes AI agents exciting also makes them dangerous. Without secure-by-design architectures, they could become powerful attack vectors. What security practices are you considering for deploying AI agents in your org?

When AI Meets Security: The Blind Spot We Can't Afford Working in this field has revealed a troubling reality: our security practices aren't evolving as fast as our AI capabilities. Many organizations still treat AI security as an extension of traditional cybersecurity—it's not. AI security must protect dynamic, evolving systems that continuously learn and make decisions. This fundamental difference changes everything about our approach. What's particularly concerning is how vulnerable the model development pipeline remains. A single compromised credential can lead to subtle manipulations in training data that produce models which appear functional but contain hidden weaknesses or backdoors. The most effective security strategies I've seen share these characteristics: • They treat model architecture and training pipelines as critical infrastructure deserving specialized protection • They implement adversarial testing regimes that actively try to manipulate model outputs • They maintain comprehensive monitoring of both inputs and inference patterns to detect anomalies The uncomfortable reality is that securing AI systems requires expertise that bridges two traditionally separate domains. Few professionals truly understand both the intricacies of modern machine learning architectures and advanced cybersecurity principles. This security gap represents perhaps the greatest unaddressed risk in enterprise AI deployment today. Has anyone found effective ways to bridge this knowledge gap in their organizations? What training or collaborative approaches have worked?

🚨 AI Privacy Risks & Mitigations Large Language Models (LLMs), by Isabel Barberá, is the 107-page report about AI & Privacy you were waiting for! [Bookmark & share below]. Topics covered: - Background "This section introduces Large Language Models, how they work, and their common applications. It also discusses performance evaluation measures, helping readers understand the foundational aspects of LLM systems." - Data Flow and Associated Privacy Risks in LLM Systems "Here, we explore how privacy risks emerge across different LLM service models, emphasizing the importance of understanding data flows throughout the AI lifecycle. This section also identifies risks and mitigations and examines roles and responsibilities under the AI Act and the GDPR." - Data Protection and Privacy Risk Assessment: Risk Identification "This section outlines criteria for identifying risks and provides examples of privacy risks specific to LLM systems. Developers and users can use this section as a starting point for identifying risks in their own systems." - Data Protection and Privacy Risk Assessment: Risk Estimation & Evaluation "Guidance on how to analyse, classify and assess privacy risks is provided here, with criteria for evaluating both the probability and severity of risks. This section explains how to derive a final risk evaluation to prioritize mitigation efforts effectively." - Data Protection and Privacy Risk Control "This section details risk treatment strategies, offering practical mitigation measures for common privacy risks in LLM systems. It also discusses residual risk acceptance and the iterative nature of risk management in AI systems." - Residual Risk Evaluation "Evaluating residual risks after mitigation is essential to ensure risks fall within acceptable thresholds and do not require further action. This section outlines how residual risks are evaluated to determine whether additional mitigation is needed or if the model or LLM system is ready for deployment." - Review & Monitor "This section covers the importance of reviewing risk management activities and maintaining a risk register. It also highlights the importance of continuous monitoring to detect emerging risks, assess real-world impact, and refine mitigation strategies." - Examples of LLM Systems’ Risk Assessments "Three detailed use cases are provided to demonstrate the application of the risk management framework in real-world scenarios. These examples illustrate how risks can be identified, assessed, and mitigated across various contexts." - Reference to Tools, Methodologies, Benchmarks, and Guidance "The final section compiles tools, evaluation metrics, benchmarks, methodologies, and standards to support developers and users in managing risks and evaluating the performance of LLM systems." 👉 Download it below. 👉 NEVER MISS my AI governance updates: join my newsletter's 58,500+ subscribers (below). #AI #AIGovernance #Privacy #DataProtection #AIRegulation #EDPB