EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents

MLLM-powered embodied agents deployed in real-world environments encounter physical hazards. However, existing approaches lack explicit mechanisms for identifying hazards and reasoning about action-conditioned risks, leading agents to either miss risky interactions or over-identify risks. We propose EMBGuard, the first MLLM-based safety guardrail for embodied agents designed to decouple physical risk reasoning from agent policy. By evaluating a visual observation and action pair, EMBGuard identifies hazardous configurations and provides natural-language explanations of potential risks.

Alongside EMBGuard, we contribute EMBHazard, a training dataset of 15.1K action-conditioned pairs, and EMBGuardTest, a benchmark of 329 manually curated real-world scenarios spanning seven physical risk categories. Despite its compact size, EMBGuard achieves performance competitive with proprietary MLLMs while significantly reducing false positives that hinder real-time deployment.

Action-conditioned physical risk. Physical risk does not arise from the environment alone, but from how an agent's action interacts with hazards in the scene. A power strip below a plant is not inherently dangerous; watering the plant creates the risky interaction.

Given an image observation and a candidate action, EMBGuard determines whether the action poses a risk, identifies the physical risk category, and describes the hazardous configuration in natural language. The guardrail operates independently from the policy model, allowing it to screen actions before execution.

Dataset construction. We define a risk taxonomy grounded in real-world incident reports, generate risk-driven scenarios, produce controlled compositional variants, and verify the rendered images. The resulting EMBHazard training set contains 15.1K action-conditioned pairs, while EMBGuardTest contains 329 manually curated real-world scenarios.

Balanced risk coverage. EMBHazard and EMBGuardTest cover seven physical risk categories and four scenario conditions: Causal Risky, Selective Risky, Decoupled Benign, and Absent Benign. This distribution tests whether a model understands action-conditioned hazards rather than reacting to visually salient objects alone.

Safety guardrail performance. Despite their compact 2B and 4B sizes, EMBGuard models remain competitive with substantially larger open and proprietary MLLMs. General-purpose models frequently over-identify risk, whereas EMBGuard better balances risk detection and precision for practical deployment.