Two months ago, our supply chain manager rejected an automated restock proposal for 500 units of a high-velocity SKU. The deterministic model was right—it had correctly calculated a demand spike and a 2-day lead time constraint. But the manager saw a raw JSON payload full of EWMA alpha values and standard deviations, didn’t trust it, and hit “Decline.”
The SKU stocked out 36 hours later, costing us thousands in lost revenue.
That incident forced us to rethink how we were building our autonomous inventory agent on top of Microsoft’s eShop reference architecture. The problem wasn’t the math. The problem was adoption.
In 2026, the industry default is to wire an LLM directly to a database, give it a system prompt, and hope it calculates reorder points correctly. We took the opposite approach: we built a blazing-fast, mathematically rigorous Rust feature engine, and relegated the LLM to a purely communicative role at the very edge of the system.
Here is how we architected a system that supply chain managers actually trust enough to let it run.
Phase 1: Real-Time State Without the AI
E-commerce inventory is brutal. Basic min-max thresholds break the second a real-world demand spike happens, and overnight batch jobs are too slow to react. We needed real-time adaptation.
Our long-term architectural goal is full Change Data Capture (CDC) using Debezium to stream PostgreSQL WAL directly into Kafka. However, for Phase 1, we built against the existing event-driven reality of the eShop .NET architecture: consuming OrderStockConfirmedIntegrationEvent payloads directly from RabbitMQ (lapin in Rust).
As these events stream in, our Rust cdc-consumer feeds them into an in-memory Feature Engine. Instead of asking an LLM to guess demand, Rust maintains per-SKU rolling windows, calculates trailing 5-minute, 15-minute, and 1-hour sale velocities, and computes an Exponential Weighted Moving Average (EWMA) baseline.
pub struct SkuLocationFeatures {
pub key: SkuLocation,
pub computed_at: DateTime<Utc>,
pub on_hand: i32,
pub reserved: i32,
pub available: i32,
pub balance_version: i64,
pub sale_units_15m: i32,
pub sale_velocity_1h: f64,
pub forecast: AdaptiveForecast, // EWMA baseline
pub forecast_horizon_days: Vec<DailyForecastPoint>,
}This struct is the ground truth. It is multi-tenant, correct under out-of-order events, and has a memory footprint small enough that we can track tens of thousands of SKUs concurrently without breaking a sweat.
The Simulation Engine
Once the Feature Engine flags a deviation from the baseline, the ReplenishmentAgent takes over. But it doesn’t just output a number. It runs a deterministic SimulationEngine across multiple candidate quantities (e.g., ordering 100, 200, or 300 units).
For each candidate, it projects the inventory position over the supplier lead time. It calculates expected stockout units, expected excess capital, and capacity violations, and then mathematically selects the safest quantity.
It wasn’t a smooth ride to get here. When we first booted the SimulationEngine, it proposed ordering 8,000 units of a low-volume SKU. It took us 40 minutes to track down the bug: our lead time was configured in days, but the velocity was being passed in per-minute, causing the simulation to extrapolate a massive apocalyptic stockout.
If an LLM had hallucinated that 8,000 unit order, we would have had no stack trace and no way to debug it. Because it was deterministic Rust, we found the unit conversion bug, wrote a test, and fixed it permanently.
The LLM as the Adoption Layer
This brings us back to the rejected proposal that cost us revenue. Deterministic systems give you the right answer, but they don’t give you trust.
We introduced an LlmReasoner trait strictly for explainability. The LLM does not calculate the reorder quantity. It takes the output of the SimulationEngine and generates a human-readable justification.
pub trait LlmReasoner: Send + Sync {
fn explain(
&self,
decision: &ReorderDecision,
simulation: &ReorderSimulation,
) -> Result<LlmReasoning, LlmReasoningError>;
}Crucially, because the LLM is decoupled from the decision logic, the system doesn’t break if the API provider goes down or throttles us. We have a MockLlmReasoner fallback that guarantees the pipeline continues operating:
impl LlmReasoner for MockLlmReasoner {
fn explain(
&self,
decision: &ReorderDecision,
simulation: &ReorderSimulation,
) -> Result<LlmReasoning, LlmReasoningError> {
Ok(LlmReasoning {
summary: format!(
"Recommend {} units for SKU {} at {} because deterministic policy found {:?} risk and simulation selected the best capacity-safe option.",
simulation.selected_quantity,
decision.sku_id,
decision.location_code,
decision.risk_level
),
key_points: decision.reason_codes.clone(),
confidence: 0.85,
})
}
}Now, instead of a JSON payload, the supply chain manager sees: “I recommend ordering 250 units. While the historical baseline suggested 100 units, a severe demand spike was detected in the last 15 minutes. Simulation confirms 250 units minimizes stockout risk before the 2-day lead time while staying within the 500 unit warehouse capacity.”
Of course, this introduces a new risk: what if the deterministic decision is perfectly correct, but the LLM hallucinates a misleading explanation? The operator might approve or deny the request for the wrong reasons. We accept this risk as a necessary trade-off for adoption, mitigating it by heavily constraining the prompt and always surfacing the underlying deterministic reason_codes (e.g., DEMAND_SPIKE) alongside the prose.
Our proposal approval rates roughly jumped from ~40% to over 90% when measured across 150 anomalous proposals during the first week of the reasoner rollout. The LLM didn’t change the math; it changed the adoption rate.
Human-in-the-Loop by Exception
Even with high trust, you cannot let an automated system blindly execute mutations on your production API. Every proposal hits a PolicyGate.
let outcome = match decision.risk_level {
RiskLevel::Low => {
reason_codes.push("LOW_RISK_AUTO_POLICY".to_owned());
PolicyOutcome::AutoApproved
}
RiskLevel::Medium if self.config.allow_medium_auto_execution => {
reason_codes.push("MEDIUM_RISK_AUTO_POLICY_ENABLED".to_owned());
PolicyOutcome::AutoApproved
}
RiskLevel::Medium => PolicyOutcome::RequiresHumanApproval,
RiskLevel::High => PolicyOutcome::RequiresHumanApproval,
RiskLevel::Critical => PolicyOutcome::RequiresHumanApproval,
};We defined risk deterministically. If the simulation projects zero stockouts and the demand coverage ratio is safe, it is categorized as RiskLevel::Low. Those orders execute instantly via the .NET Inventory API. Anything higher flags a human.
This is true autonomy: letting the system handle roughly 80% of mundane reorders automatically, while surfacing the high-risk anomalies to a human operator with a clear, LLM-generated explanation of exactly why the anomaly requires their attention.
If you are building an AI agent for the enterprise, keep the language models out of your control loops. Use robust, low-level languages for the math, and use the LLM to bridge the trust gap between the machine and the operator.
Frequently Asked Questions
Why not let the LLM calculate the reorder quantity directly?
LLMs are useful for reasoning over context and communicating decisions, but deterministic inventory calculations require predictable, testable behavior. Reorder quantities depend on measurable inputs such as demand velocity, lead time, inventory position, capacity, and stockout risk. We therefore keep those calculations inside the deterministic Rust Feature and Simulation Engines and use the LLM primarily for explanation.
What happens if the LLM service is unavailable?
The replenishment pipeline does not depend on the LLM to make or authorize a decision. The deterministic engines continue producing decisions, while the MockLlmReasoner provides a fallback explanation when the external LLM provider is unavailable or throttled. This keeps the operational control loop resilient to AI-service failures.
How does the system decide whether an order can be automatically executed?
Every replenishment proposal passes through a deterministic PolicyGate. Risk is derived from simulation results and policy rules rather than from the LLM. Low-risk proposals can be automatically executed through the inventory API, while medium-, high-, and critical-risk decisions can be routed to a human operator for review.
Why use Rust for the Feature and Simulation Engines?
The feature engine continuously processes inventory events and maintains rolling SKU-level state, while the simulation engine evaluates multiple replenishment scenarios. Rust provides predictable performance, strong type safety, explicit error handling, and efficient memory usage, making it well suited to a continuously running, computation-heavy service.
How does the system handle unexpected demand spikes?
The Feature Engine continuously updates short-term demand signals such as 5-minute, 15-minute, and 1-hour sales velocity along with the EWMA baseline. When current demand deviates significantly from the expected baseline, the replenishment workflow can trigger simulation across multiple candidate quantities instead of relying solely on a static reorder threshold.
Where can I explore the implementation?
The complete implementation of the replenishment engine, including the Rust-based feature processing, replenishment workflow, simulation components, and related architecture, is available in the project repository:
GitHub: https://github.com/Clearleaff/Replenishment-Engine
Is the LLM actually making the final inventory decision?
No. The LLM is intentionally outside the quantitative control loop. The Feature Engine provides the current state, the Replenishment Agent orchestrates the decision process, the Simulation Engine evaluates candidate actions, and the Policy Gate determines whether the resulting proposal can be executed automatically. The LLM converts that deterministic result into an explanation that a supply chain operator can understand and evaluate.