From Theory to the Floor: Applying IE to Agentic AI Dependability

An Industrial Engineering Approach to Controlled, Measurable, and Continuously Improving AI Operations

How do we engineer Agentic AI so that every runtime deviation strengthens the next operating cycle?

Agentic AI Dependability becomes difficult when an action moves from reasoning toward real-world consequence. An action can be technically correct and properly approved, yet become inadmissible moments later. Evidence may expire, workflow state may change, delegated authority may be revoked, or the realized execution plan may drift beyond what was reviewed.

Dependability therefore requires authority, evidence, scope, and operating assumptions to remain valid when consequences become binding.

Industrial Engineering (IE) offers practical methods for addressing this problem through process control, failure analysis, measurement, and continuous improvement.

The Four Runtime Governance Pillars provide the execution architecture, while the Plan-Do-Check-Act (PDCA) cycle turns runtime evidence into disciplined system improvement. Together, they shift Agentic AI Dependability from a one-time governance design problem to a continuously engineered operating capability.

1. Agentic AI as an Industrial Engineering System

Agentic AI Dependability is a system-level engineering problem. Industrial Engineering examines how people, technology, information, controls, and operating conditions interact to produce dependable outcomes.

From Stochastic Models to Controlled Operating Systems

An enterprise agent operates within a socio-technical system of models, humans, tools, APIs, enterprise data, workflows, policies, memory, authority, and changing system states.

The engineering objective is not to eliminate AI stochasticity, but to prevent variability from becoming uncontrolled enterprise consequence.

This article applies the Four Runtime Governance Pillars:

• Pillar 1: Runtime Controls defines the governed operating envelope.
• Pillar 2: Enforcement and Admissibility determines whether consequential actions may proceed.
• Pillar 3: Alerting, Intervention, and Recovery detects material changes and governs responses.
• Pillar 4: Reconstruction preserves evidence, evaluates control effectiveness, and supports governed learning.

These abbreviated names represent broader framework responsibilities.

The architecture organizes them across three planes: Specification & Control (P1), Runtime Decision & Response (P2/P3), and Assurance & Learning (P4).

From Reliability to Dependability

Reliability asks whether an agent performs consistently and produces acceptable results.

Dependability asks whether the complete human-agent system remains controlled, authorized, observable, recoverable, and accountable during actual operation.

A technically correct action may become inadmissible when authority expires, evidence becomes stale, workflow state changes, or execution scope expands.

This systems perspective establishes PDCA: design dependable conditions, govern execution, examine actual behavior, and improve the operating system.

2. PLAN: Design the Dependable Operating Envelope

PLAN defines the conditions under which Agentic AI may operate dependably before consequential execution begins.

Industrial Engineering contributes a preventive mindset: anticipate failure and convert identified risks into operating controls.

Anticipate Failure Before Execution

Failure Mode and Effects Analysis (FMEA) provides a useful starting point. It examines what can fail, why failure may occur, its possible consequences, and appropriate controls.

Agentic AI failure modes extend beyond incorrect model output:

• Expired authority or stale evidence
• Incorrect workflow state or duplicate actions
• Unauthorized execution-scope expansion
• Unsafe memory or shared-state mutations
• Reviewer unavailability or delayed approval
• Conflicting agent actions or cumulative exposure
• Deployment conditions differing from those evaluated

The objective is not merely assigning risk scores. PLAN determines which conditions require prevention, revalidation, escalation, constraint, or prohibition.

Translate Failure Modes into Runtime Controls

This analysis feeds primarily into Pillar 1, which defines authorized actions, evidence requirements, authority boundaries, validity horizons, governance precedence, human-review rules, and recovery expectations.

For multi-agent workflows, PLAN must also define material interaction paths, shared-state dependencies, concurrency limits, and collective risk exposure.

Individually acceptable actions can produce unacceptable combined consequences. These dependencies must become explicit operating assumptions.

PLAN also preserves assumptions behind pre-deployment evaluations, red-team results, certifications, and previous successful runs. Such evidence supports future decisions but does not provide continuing execution permission.

The engineering sequence becomes:

Failure Mode → Control → Admissibility Condition → Intervention → Evidence

For example, stale approval should lead to an approval-validity horizon, commit-time revalidation, and a governed HOLD or escalation path.

PLAN converts anticipated failures into enforceable conditions for DO.

3. DO: Govern Execution at the Point of Consequence

DO is where planned controls meet real execution.

For Agentic AI Dependability, execution requires verifying that proposed consequences remain legitimate, admissible, and consistent with the authorized operating envelope.

Enforce Admissibility Before Consequence

Pillar 2 governs the execution boundary through a staged determination:

State Verification → Intent Alignment → Authority Continuity → Consequence Assessment → Execution Admissibility

The result is one of five binding dispositions:

PERMIT / REFUSE / CONSTRAIN / HOLD / ESCALATE

Commit-time enforcement verifies current authority, evidence, policy, execution scope, workflow state, and applicable risk constraints.

Evaluation-to-execution continuity also matters. Earlier tests or certifications cannot establish current admissibility when assumptions, tools, interfaces, or operating conditions have materially changed.

Prevent Locally Valid Actions from Creating System-Level Risk

In multi-agent operations, admissibility must account for shared-state consistency, concurrent commitments, and aggregate exposure.

Two agents may independently propose valid payments while their combined commitments exceed an institutional spending limit.

Where material, execution controls must bind decisions to validated shared state and enforce applicable concurrency or aggregate-risk constraints.

This extends admissibility beyond individual actions to collective consequences.

Revalidate at the Execution Boundary

Industrial Engineering emphasizes preventing defects from propagating downstream.

Commit-time enforcement serves a Poka-Yoke-like (mistake-proofing) function: critical conditions are embedded in the process so inadmissible actions cannot proceed merely because an agent generates plausible outputs.

For higher-consequence actions, independent safeguards reduce reliance on any single enforcement mechanism.

HOLD and ESCALATE provide an Andon-like function, stopping consequential execution and routing uncertainty through authorized decision paths.

Pillar 3 can contain disruptions, trigger revalidation, and initiate recovery. Recovery actions are themselves governed actions requiring fresh admissibility through Pillar 2.

After execution, the system reconciles expected and actual outcomes. Partial, inconsistent, or unconfirmed effects remain unresolved until verified through authoritative or independently observable evidence.

DO therefore converts planned safeguards into active operational control.

4. CHECK: Turn Runtime Behavior into Engineering Evidence

CHECK examines whether actual operation remained within the dependable conditions established during PLAN.

This requires more than monitoring final outputs. The system must capture what executed, which controls operated, how conditions changed, and what consequences followed.

From Runtime Events to Engineering Evidence

Signals generated by Pillar 3 during execution, including detections, interventions, containment, and recovery events, become inputs to CHECK.

Pillar 4 reconstructs decisions, execution paths, interventions, outcomes, and supporting evidence.

Reconstruction includes execution provenance: model and agent versions, runtime instances, orchestration systems, tools, APIs, policies, and material dependencies.

Before assigning a root cause, the measurement and observation system must be examined.

The diagnostic progression becomes:

Runtime Event → First Operational Deviation → Measurement-System Check → Failure Characterization → Root Cause

The First Operational Deviation identifies where observed behavior initially departed from the expected path. Localization does not establish the underlying cause.

Deviation ≠ Root Cause ≠ Corrective Action ≠ Validated Improvement

These are distinct engineering evidence states.

Verify Measurement Quality

A declining incident rate may indicate improvement. It may also result from missing telemetry, changed event classifications, duplicated records, or failed collectors.

Industrial Engineering applies Measurement System Analysis (MSA) principles to qualify measurement paths.

Relevant checks include event coverage, repeatability, timing accuracy, consistency, drift, and uncertainty.

Missing or unqualified evidence must not be interpreted as zero failures or proof that controls worked.

Learn from Successful Protection

CHECK must distinguish failed operations from successful protective interventions.

A justified HOLD is a successful control event, not automatically an operational failure or near miss.

Repeated HOLD events may indicate effective protection, avoidable workflow friction, or both. Their frequency alone cannot establish control weakness.

Similarly, eventual PERMIT does not mean an earlier HOLD was unnecessary.

This extends learning beyond failures to successful prevention, consistent with Safety-II thinking.

Runtime evidence becomes valuable when it supports trustworthy measurement, bounded causal conclusions, and meaningful engineering improvements.

CHECK transforms observations into qualified engineering findings rather than treating every intervention as a defect.

5. ACT: Convert Evidence into Governed Fortification

ACT turns findings from CHECK into controlled changes to the operating system.

For Agentic AI Dependability, improvement must not become an ungoverned learning loop in which agents automatically rewrite operating rules after incidents.

From Root Cause to System Change

Recurring problems may originate outside the model.

Stale evidence may reveal inadequate synchronization. Repeated authority failures may expose delegation weaknesses. Excessive escalation delays may reflect insufficient review capacity rather than unnecessarily strict safeguards.

Corrective actions may revise evidence requirements, workflows, escalation paths, operating thresholds, recovery mechanisms, monitoring, or test scenarios.

However, proposed improvements must themselves be governed.

The Pillar 4 to Pillar 1 Change Loop

Pillar 4 produces bounded findings with supporting evidence, uncertainty, limitations, and corrective objectives.

These findings do not directly modify runtime policy.

Pillar 1 reviews and tests proposed changes, verifies their consistency with governance requirements, and authorizes a new operating baseline.

The new baseline receives an authorized activation epoch. Previous baseline records remain preserved, while affected standing is revalidated or invalidated as required.

Pillars 2 and 3 operate under the newly activated baseline. Pillar 4 subsequently verifies whether the change achieved its intended improvement without introducing unacceptable new failure modes.

The governed improvement sequence is:

Finding (P4) → Review & Test (P1) → Authorized Activation (P1) → Operate (P2/P3) → Verify Improvement (P4)

This makes PDCA continuous governance rather than uncontrolled adaptation.

PDCA and the Three Governance Planes

The mapping is not one-to-one. All four pillars remain active at runtime, while PDCA operates as the improvement loop around the governed system.

6. The Industrial Engineering Toolbox for Agentic AI Dependability

Industrial Engineering provides methods for preventing failure, measuring operating behavior, managing capacity, and improving processes.

These methods support the Four Runtime Governance Pillars rather than creating additional governance layers.

Failure Analysis and Process Mapping

FMEA anticipates potential failure modes during PLAN and incorporates field evidence from CHECK.

Process mapping traces consequential actions through the operating system:

Intent → Review → Authorization → Admissibility → Execution → State Change → Outcome

Root-cause and Pareto analysis identify recurring sources of failure or friction, including stale evidence, authority expiration, dependency problems, and workflow-state mismatches.

Measurement System Analysis and Statistical Process Control

MSA examines whether observations are sufficiently reliable for their intended decisions.

Measurement paths should be qualified before their metrics justify changes to runtime controls.

Statistical Process Control (SPC) can identify meaningful shifts in HOLD frequency, escalation delays, recovery time, or execution variability.

Control limits must reflect actual operating distributions and context, not serve as universal admissibility thresholds.

Apparent improvement must also be checked against changes in event coverage, denominators, and classification rules.

Human Review Capacity and Governance Friction

ESCALATE capacity must be engineered rather than assumed.

Queueing principles, including Little’s Law and Kingman’s approximation, help analyze review demand, processing variability, capacity, and waiting times within their applicable assumptions.

Review performance must also be evaluated against tail waiting times, consequence tiers, and escalation deadlines, since acceptable averages may conceal missed deadlines.

When demand exceeds capacity, organizations should improve routing, staffing, evidence presentation, or workflow design. Congestion must not justify bypassing controls or reusing expired approvals.

A governance friction budget distinguishes justified protection from avoidable delays and demonstrably unnecessary interruptions.

Exceeding that budget should trigger analysis and governed improvement, not automatic relaxation of safeguards.

Simulation and Learning from Prevention

Simulation and fault injection test missing evidence, reviewer unavailability, state conflicts, degraded safeguards, and recovery failures.

CHECK also learns from successful HOLDs, blocked unsafe actions, and effective human interventions.

Together, these methods support an IE sequence:

Design → Anticipate → Control → Measure → Improve

The Four Runtime Governance Pillars make that sequence operational for Agentic AI Dependability.

7. From Theory to the Floor: A PDCA Cycle in Action

Consider an enterprise agent preparing a supplier payment.

The challenge is not simply calculating the correct amount. Authority, supplier evidence, bank details, workflow state, and payment conditions must remain valid at commitment.

PLAN and DO

During PLAN, the organization defines payment limits, supplier approval, invoice validation, bank-account verification, approval validity horizons, and duplicate-payment prevention.

During DO, the agent prepares a $32,000 payment.

The invoice is valid and within the authorized limit. However, the supplier’s bank information has changed since approval.

Evidence must remain valid throughout the interval from approval to commitment, including queueing, human review, and execution delays. Earlier approval cannot substitute for current bank-account verification.

Commit-time revalidation prevents immediate execution and places the payment on HOLD.

If the discrepancy cannot be resolved within the permitted validation path, the case is ESCALATED to an authorized reviewer.

Payment remains blocked until a fresh admissibility decision permits execution.

The HOLD is a successful protective action, not a payment-processing failure.

CHECK and ACT

Over several weeks, CHECK identifies recurring bank-change escalations.

Measurement records are checked for completeness and consistency. Reconstruction reveals that many cases result from delays between supplier-master updates and payment workflows.

The engineering finding distinguishes:

• Effective protection: Bank-change HOLDs correctly prevented execution under unresolved evidence.
• Avoidable friction: Synchronization delays generated repeated review demand.

ACT addresses the process without weakening safeguards.

The enterprise improves supplier-master synchronization, retains mandatory bank-change verification, introduces event-triggered revalidation, and updates test scenarios.

Pillar 1 reviews, tests, and authorizes revised controls before activating the new baseline.

Pillars 2 and 3 operate under those controls. Pillar 4 subsequently verifies whether unnecessary escalations declined while protective effectiveness and evidence quality remained acceptable.

PLAN → DO → CHECK → ACT → New PLAN

The example demonstrates how Agentic AI Dependability improves when successful protection becomes engineering evidence for reducing avoidable process friction.

Verified operational learning informs the next PLAN, while every new operating baseline remains subject to governed authorization and activation.

8. Conclusion: From Runtime Governance to Continuous Dependability Engineering

Agentic AI Dependability is not achieved by eliminating uncertainty from AI models. It requires an engineered operating system in which consequential actions remain controlled, measurable, recoverable, and accountable.

Industrial Engineering provides methods for anticipating failure, qualifying measurements, managing review capacity, analyzing variation, and improving processes.

The Four Runtime Governance Pillars make these methods operational through runtime controls, admissibility, intervention, reconstruction, and governed learning.

The framework extends these principles across multi-agent interactions, shared-state concurrency, collective admissibility, and dependency invalidation.

PDCA closes the loop by converting trustworthy runtime evidence into authorized operational improvements.

Govern → Execute → Observe → Diagnose → Improve → Re-Govern

The result is not static governance, but a continuously engineered capability in which successful protection is preserved, avoidable friction is reduced, and every improvement strengthens the next operating cycle of Agentic AI Dependability.