Säkerhet för stora språkmodeller

🔍 Kategori: concept 🏷️ Alias: AI Model Security, ML Security, LLM Threat Analysis, Adversarial AI Defense ⚖️ Forensisk standard

LLM-säkerhet omfattar skyddsåtgärder och analysmetoder för att identifiera och mitigera hot mot stora språkmodeller, inklusive dataintegritet, modellförgiftning och exfiltrering av känslig information.

🔬 Forensisk Betydelse & Utredningsmetodik

I digitala utredningar används LLM-säkerhetsanalys för att spåra och attribuera attacker mot språkmodeller, identifiera komprometterade träningsdata eller modeller, samt återställa exfiltrerad information från incidenter som involverar AI-system.

🛠️ Praktiska Triage- & Analyskommandon (Cheat Sheet)

⚡ Scan Hugging Face Model for Malicious Payloads

huggingface-cli scan --model-path /path/to/model --output /path/to/report.json

Scans a Hugging Face model repository for known malicious payloads, backdoors, or data exfiltration patterns using the official Hugging Face security scanner.

⚡ Detect Model Poisoning with TensorFlow Privacy

python -m tensorflow_privacy.privacy.privacy_tests.membership_inference_attack --input_examples=/path/to/data --model=/path/to/model

Runs a membership inference attack to detect potential model poisoning or unauthorized data inclusion in a trained LLM.

⚡ Audit LLM API Logs for Exfiltration Patterns

tshark -r /path/to/llm_api_traffic.pcap -Y "http.request.uri contains 'exfil' or http.response.code == 200 and http.content_type contains 'json'" -w /path/to/suspicious_traffic.pcap

Filters network traffic for patterns indicative of data exfiltration via LLM API calls, such as unusual JSON payloads or high-frequency requests.

⚡ YARA-LLM Rule Scan for Backdoors

yara -r /path/to/llm_yara_rules.yar /path/to/model_directory

Applies custom YARA rules to detect known backdoors or malicious modifications in LLM model files (e.g., PyTorch .pt or TensorFlow .pb files).

⚙️ Tekniska Parametrar & Verktygsstöd

Forensiska verktyg:
Hugging Face Model Scanner LLM Guard TensorFlow Privacy PyTorch Privacy Meter Microsoft Counterfit IBM Adversarial Robustness Toolbox (ART) YARA-LLM (custom rules)
MITRE ATT&CK: T1588.006 (Obtain Capabilities: ML Model) ↗
Standarder: NIST AI 100-2 (Draft), ISO/IEC 23894:2023, OWASP Top 10 for LLM Applications

📊 LLM-Security Threat Model

+---------------------+       +---------------------+       +---------------------+
|                     |       |                     |       |                     |
|   Training Data     |------>|   Model Training    |------>|   Deployed LLM      |
|                     |       |                     |       |                     |
+----------+----------+       +----------+----------+       +----------+----------+
           |                               |                               |
           | (Poisoning)                  | (Backdoor)                   | (Exfiltration)
           v                               v                               v
+----------+----------+       +----------+----------+       +----------+----------+
|                     |       |                     |       |                     |
|  Compromised Data   |       |  Malicious Model    |       |  Data Leakage       |
|                     |       |                     |       |                     |
+---------------------+       +---------------------+       +---------------------+

🔍 Förekomst i Forensix Intelligence & Fallstudier

📰 Adam Shostack Talks Hugging Face Breach & PHANTOM-B 2026-08-17

📚 Källor & Fördjupning

🏛️ NIST

NIST AI 100-2 (Draft): Adversarial Machine Learning: A Taxonomy and Terminology

Läs specifikation →

🏛️ OWASP

OWASP Top 10 for Large Language Model Applications

Läs specifikation →

🏛️ MITRE

MITRE ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems

Läs specifikation →

Alla begrepp Sök incidenter i Intelligence →