AI Red Teamer.

A specialist who probes AI systems — LLMs, agents, multimodal models — for harmful behaviors, jailbreaks, safety failures, and security vulnerabilities, blending traditional offensive security with ML-specific adversarial techniques. Pertence à categoria Funções e carreiras da cibersegurança.

A specialist who probes AI systems — LLMs, agents, multimodal models — for harmful behaviors, jailbreaks, safety failures, and security vulnerabilities, blending traditional offensive security with ML-specific adversarial techniques.

An AI red teamer (sometimes called LLM red teamer or model red teamer) is a newer role created by the rise of large language models and agentic AI. The work blends traditional offensive-security skills with ML-specific adversarial techniques and policy reasoning. Concrete activities include crafting prompt-injection and jailbreak prompts that bypass model safety training; building automated red-team harnesses that scale single-prompt probes into structured eval suites (TextAttack, garak, PyRIT, MAR, Anthropic's HHH evals); probing for harmful-content failures across the operator's policy (dangerous instructions, CSAM, weapons uplift, election interference); testing tool-use agents for tool-use injection, excessive agency, and unintended actions; testing multimodal models for image-, audio-, and video-based prompt injection; probing for training-data extraction and membership inference; and writing the reports that drive both model-level fine-tuning and system-level guardrails. The discipline is codified in frameworks such as the NIST AI RMF GenAI Profile, OWASP LLM Top 10, and MITRE ATLAS. Backgrounds vary widely; many AI red teamers come from offensive security, applied ML, or policy research, and the field is rapidly professionalizing through 2024–2026.

As defesas contra AI Red Teamer costumam combinar controles técnicos e práticas operacionais, conforme detalhado na definição acima.

Nomes alternativos comuns: LLM red teamer, Model red teamer.