AI Red Teamer.

A specialist who probes AI systems — LLMs, agents, multimodal models — for harmful behaviors, jailbreaks, safety failures, and security vulnerabilities, blending traditional offensive security with ML-specific adversarial techniques. Относится к категории Роли и карьера в кибербезопасности.

A specialist who probes AI systems — LLMs, agents, multimodal models — for harmful behaviors, jailbreaks, safety failures, and security vulnerabilities, blending traditional offensive security with ML-specific adversarial techniques.

An AI red teamer (sometimes called LLM red teamer or model red teamer) is a newer role created by the rise of large language models and agentic AI. The work blends traditional offensive-security skills with ML-specific adversarial techniques and policy reasoning. Concrete activities include crafting prompt-injection and jailbreak prompts that bypass model safety training; building automated red-team harnesses that scale single-prompt probes into structured eval suites (TextAttack, garak, PyRIT, MAR, Anthropic's HHH evals); probing for harmful-content failures across the operator's policy (dangerous instructions, CSAM, weapons uplift, election interference); testing tool-use agents for tool-use injection, excessive agency, and unintended actions; testing multimodal models for image-, audio-, and video-based prompt injection; probing for training-data extraction and membership inference; and writing the reports that drive both model-level fine-tuning and system-level guardrails. The discipline is codified in frameworks such as the NIST AI RMF GenAI Profile, OWASP LLM Top 10, and MITRE ATLAS. Backgrounds vary widely; many AI red teamers come from offensive security, applied ML, or policy research, and the field is rapidly professionalizing through 2024–2026.

Защита от AI Red Teamer обычно сочетает технические меры и операционные практики, как описано в определении выше.

Распространённые альтернативные названия: LLM red teamer, Model red teamer.