Behavior Planning — The Decision Layer of Autonomous Driving

0. What this article explains

The job of behavior planning between perception, route planning, and control.
The differences between finite-state machines, rules, and cost-based candidate selection.
How prediction uncertainty, HD maps, MPC, and safe fallback behavior connect.

1. Bottom line: what is behavior planning?

Behavior planning is the decision layer that chooses a discrete driving intention—follow, stop, yield, change lane, or merge—over a short horizon from tracked objects, traffic rules, destination, and ego state. Perception answers “what is where,” trajectory generation answers “which curve,” and control answers “which steering and acceleration”; behavior planning connects them.

2. Why a separate layer?

The same lane centerline can imply following after a lead vehicle brakes, a lane change when the adjacent lane is open, or a stop at a crosswalk. Passing an object list directly into MPC forces every cycle to reinvent meaning and priority. Pure rules, in contrast, become hard to combine across lanes, uncertain predictions, and work zones.

Behavior planning creates candidates that satisfy safety constraints in a short horizon, compares them with costs that include prediction error, and sends an intention to trajectory generation. It does not assert a certain future position; it states switching conditions and conservative conditions for not switching.

3. Inputs and state

Inputs include tracked object position, velocity, class, and confidence; occupancy grids and HD-map lane topology; signals and signs; ego pose and speed; and the route. If target $j$ has state $\hat{s}_j=(p_x,p_y,v_x,v_y,\Sigma_j)$ , the predictor returns covariance $\Sigma_j$ as well as a mean. A large covariance should widen the safety margin even when the mean trajectory is unchanged.

4. Output: intention and constraints

Outputs may be FOLLOW_LANE, STOP, YIELD, CHANGE_LEFT, or MERGE, together with target lane, speed limit, stop position, and constraints for trajectory generation. A candidate trajectory $\tau$ can be scored by collision, traffic-rule, comfort, and progress costs:

J(\tau)=w_c C_{collision}+w_r C_{rule}+w_j C_{jerk}+w_p C_{progress}

Collision cost should be separated as a hard constraint (“if it happens, reject”), while progress must never buy safety violations. The ordering of constraints is part of the specification.

5. Basic pipeline

Figure 1 — Perception and map feed future prediction and multiple intentions. An independent safety filter remains after candidate selection.

6. FSM: make states and transitions explicit

A finite-state machine is easy to audit. For example, FOLLOW transitions to STOP for a stop line, red signal, or time-to-collision (TTC) below a threshold. FOLLOW transitions to CHANGE_LEFT only when the target lane is on the route, predicted neighboring occupancy is safe, and there is enough distance to complete the maneuver.

States can still explode in number, leaving uncovered situations that match no state. Maintain transition tables, priority, timeouts, and re-entry conditions as executable test cases.

7. Comparing candidates and costs

Approach	Mechanism	Strength	Weakness
Rules/FSM	Transition on conditions	Explainable and testable	State and exception growth
Cost function	Score candidates on common scale	Combines objectives	Weights are hard to justify
Sampling	Generate many predictions and trajectories	Nonlinear, multimodal behavior	Compute time and repeatability
Learned policy	Infer intention from data	Compresses rich context	OOD, explanation, safety proof

Production systems often use FSM to remove illegal candidates, cost functions to rank the remainder, and trajectory generation/MPC for continuous values. Even with a learned policy, speed, acceleration, lane boundaries, and stopping distance should be independently monitored.

8. Difficult environments

At an intersection, whether another driver yields cannot be determined from one observation. Pedestrian gaze, bicycle wobble, and a construction worker's gesture make the predictive distribution broad. If map and sensing disagree, do not ignore an object because the map is old: reduce speed and prefer a candidate that can stop. In weather or occlusion, use variance and worst-case quantiles, not only mean TTC.

9. Practical choices and fallback

Motorway lane change: use FSM to constrain rules and target lane; score predicted trajectories and minimum TTC.
Urban intersections: make signals, stop lines, and crosswalks hard constraints; stop when yielding uncertainty is high.
Delivery robots: model pedestrian interaction conservatively and pass speed and stopping-distance limits to control.
Sensor dropout or map conflict: transition to DEGRADED_STOP; do not keep replaying the last trajectory.

Log per-candidate cost components, rejection reasons, prediction covariance, source and destination state, and selection time. A planner that cannot replay why it changed lanes cannot be improved after an incident.

10. Three-line recap

Behavior planning turns perception's object list into driving intention and constraints.
It makes rules explicit with an FSM, compares candidates using predictive distributions and costs, and hands the result to trajectory generation and MPC.
Uncertainty, stale maps, and missing sensors require an independent safety filter and stop fallback.

References

#behavior planning #autonomous driving #FSM #cost function #HD map #decision making