Human Avoidance for Autonomous Warehouse Robots

0) Problem Restatement

Amazon asked: design the human-avoidance subsystem for autonomous robots in a warehouse where people and robots share space. Each robot must detect humans around it, predict where they're going, and adjust its path or stop fast enough to never hit anyone, while staying productive (not stopping for no reason). This is a safety-critical real-time system.

1) Requirements

• Detect humans (and other obstacles) 360° around the robot, in all lighting.
• Reaction: from detection to braking within a strict budget (e.g., under 100 ms).
• Always able to stop in time given its speed (stopping distance), and speed limits near people.
• Fail safe: if sensors or software fail → stop.
• Minimize false stops (productivity), and log everything for incident review.

2) Layered Architecture

flowchart LR
    SEN["Lidar, cameras, ultrasonic, bumpers"] --> PER["Perception - detect + track humans"]
    PER --> PRED["Motion prediction - where will they be in 1-3 s"]
    PRED --> PLAN["Local planner - slow, reroute, yield"]
    PLAN --> CTRL["Motor control"]
    SEN --> SAFE["Certified safety controller - protective zones"]
    SAFE -->|"override: stop"| CTRL
    FLEET["Fleet manager - zone rules, traffic"] --> PLAN
    PER --> LOG[("Event recorder")]

Two independent layers:

1. Smart layer (perception, prediction, planning): tries to avoid people smoothly, by slowing early, going around, or waiting.
2. Safety layer (simple, certified hardware/firmware): a safety-rated lidar with protective zones. If anything enters the inner zone, it stops the motors directly, regardless of what the smart software says. The zone size grows with speed. This layer is the guarantee, and the smart layer is for efficiency.

3) Perception and Prediction

• Sensor fusion: lidar gives accurate distance, cameras recognize people (ML detector), and ultrasonic and bumpers cover close range. Fuse detections into tracked objects with position, velocity and a "human" confidence.
• Tracking: follow each person over time (a Kalman filter), which smooths noise and gives velocity.
• Prediction: extrapolate short-term paths (1–3 seconds), with uncertainty growing over time. Treat humans as unpredictable and keep bigger margins than for static objects.

4) Decisions (short-horizon planning)

• Speed limit by distance: the robot must always be able to stop before reaching the closest predicted human position: v_max = f(distance, braking capability, reaction time).
• Zones: a warning zone → slow down, then a protective zone → stop (the safety layer).
• Replanning: in aisles, yield or wait. In open areas, reroute around the person with a safe clearance.
• Fleet rules: the fleet manager can mark zones as "human work area" (robots slow or excluded), or coordinate robots to avoid blocking people.

5) Reliability, Validation and Logging

• Watchdogs: if perception output is late or a sensor fails a health check → reduce speed or stop (fail-safe).
• Determinism: fixed-rate control loops, real-time scheduling, and no heavy background work on the safety path.
• Validation: simulation with many human-behavior scenarios, closed-course tests, and a documented safety case (following industrial safety standards).
• Event recorder: log sensor snapshots around every stop or near-miss for analysis and model improvement.

6) Wrap-Up

Fuse lidar, camera and close-range sensors to detect and track humans, predict their short-term motion, and let a local planner cap speed by stopping distance and slow, yield or reroute early. Put an independent, certified safety layer underneath, with speed-dependent protective zones that cut the motors directly. Add watchdogs that fail safe, fleet-level zone rules, simulation-heavy validation, and an event recorder for every stop and near-miss.