In our previous blog post, we sketched a “what if” vision of the automated enterprise in 2030, where business strategy is expressed as policy and executed as software for the mid-sized European utility.
This blog post steps back from that narrative to examine what could make that future practical. We look at a set of emerging capabilities taking shape across research, product development, and early enterprise deployments.
Together, these components point toward a machine-first operating model, one that can observe with nuance, rehearse decisions with fidelity, and act with accountability.
1. Long-Form Video Collection
What It Is
Think of a system that watches long stretches of video and turns them into short, reviewable evidence tied to a rule, plan, or task. Instead of motion alerts or single-frame labels, it understands sequences over time: who did what, in what order, for how long, and with what outcome.
It produces a small set of time-stamped clips with plain-language notes and structured fields: event type, duration, confidence, “before/after,” and the checklist item or policy it maps to. The output is an evidence bundle that can be attached to a ticket, a care plan, a safety record, or a contract.
Value It Provides
Long-form comprehension reduces manual review, makes small trends visible early, and provides artifacts that stand up to audit or dispute. In operations, it becomes condition assessment and work verification. In customer contexts, it turns “journeys” into concrete sequences that explain friction.
In regulated environments, it anchors compliance in verifiable clips rather than narrative reports. The practical gain is faster, better-justified decisions and fewer reworks.
What It Could Look Like
Picture a nursing home where oversight software watches hours of hallway and common-area footage. It notices Mrs. Alvarez has begun pausing at the same handrail longer each morning, and Mr. Ito skipped breakfast twice this week.
A caregiver opens a weekly prep screen and sees a short summary for each resident. For one, the system flags “increasing pause time on standing.” For the other, “meal completion down 25% over 10 days.”
Two clips are attached: one showing subtle instability near the handrail over several days, another compressing a week of dining-room footage into a clear pattern of early tray returns.
The caregiver schedules a physio check and a nutrition consult; the attending physician receives a weekly roll-up in their inbox with the same evidence and a short rationale. Nothing dramatic happens that same day. But over the next month, a vitamin D adjustment and a walking aid refit prevent a fall that would otherwise have set back recovery by weeks, and a nutritional deficiency that could have become problematic in the long run.
The “AI” isn’t the hero in this case. The team is supported by a system that turns long stretches of video into actionable insight. The shift is from reactive interventions to proactive health maintenance.
2. 3D Object Decomposition
What It Is
Software that turns images, video, or a quick 3D scan into a structured model of an object: its parts, how those parts fit together, and the tolerances and dependencies between them. It identifies the exact variant, measures geometry, materials, and wear, and then outputs a deliverable a team can use: an assembly tree, a parts list, or a step-by-step procedure for service or adjustment.
Value It Provides
3D decomposition moves work from “whole unit” decisions to part-level decisions. In maintenance, it avoids unnecessary swaps by showing the minimal fix and the correct sequence to perform. In commerce, it lets a system reason about products at the component level, matching compatibility, estimating fit, or proposing upgrades, rather than guessing from SKU tags.
In personalization, it aligns a person’s actual geometry and usage patterns with the product’s geometry to suggest specific adjustments (collar width, outsole compound, fin setup, lens curvature) rather than generic recommendations. The practical gains are lower returns, shorter service times, tighter inventory, and configurations that fit first time.
What It Could Look Like
A specialty running store adds a small scanning bay next to the gait treadmill. A customer jogs for 30 seconds while a depth camera captures the ankle, arch, and strike pattern.
The system decomposes the current shoe into upper, midsole, plate (if present), and outsole, measures lug wear and flex, and recognizes the exact model and size. It overlays those parts against the customer’s 3D foot and gait data and generates a fit card: the collar needs two millimeters more room on the lateral side; the traction pattern is too aggressive for their usual surfaces; the midsole stiffness needs to step down one notch to reduce late-stance overpronation.
It proposes two in-stock shoes that meet those constraints, plus an insole profile that it can print on-site. On screen, the customer sees an “optimized model” of the recommended setup and the changes that drive each suggestion. If they stick with their current shoe, the system outputs an adjustment kit (alternative laces, a softer heel clip, or a different insole) and a reminder to rotate to a road outsole for summer.
The system helps customers choose better-fitting gear faster, with fewer trial-and-error returns and fewer avoidable discomfort issues.
3. Neuro-Symbolic Vision
What It Is
Software that pairs modern visual perception with explicit rules. It doesn’t label frames; it recognizes sequences and evaluates them against codified policies. The system detects objects and temporal relations (“X entered Zone 3 within four seconds of Y”), then a rule layer classifies the state as acceptable, risky, or prohibited and explains why.
The output is an interpretable decision with a short note, a clip, the rule reference, and the suggested next step.
Value It Provides
Most sites don’t lack cameras; they lack fast, consistent judgment. Neuro-symbolic vision turns seeing into accountable action. Rules are explicit, so you can tune thresholds, audit decisions, and show why a stop, nudge, or pass occurred.
False positives fall when rules target what actually matters. Trust rises when every flag carries a reason a supervisor can read. Over time the rule layer becomes an operational asset: a living translation of safety standards, SOPs, and quality criteria applied the same way, shift after shift.
What It Could Look Like
At a mixed-materials recycling facility, overhead cameras monitor the incoming conveyor belts, sorting stations, and baling equipment. The vision system identifies familiar hazards and anomalies: batteries mixed into paper waste, propane canisters entering the plastics stream, or loose film beginning to wrap around a roller. A rule-based layer then determines the appropriate response in each case.
If a battery appears in the paper stream, the system triggers a short, controlled stop, places the item on a nearby monitor for safe removal, and saves a short video clip along with the rule that justified the intervention. If plastic film begins building up around a roller, the line pauses briefly and prompts a guided clean-up before the issue becomes a full jam. If a forklift drifts into a pedestrian-marked zone, the driver receives a warning and the incident is logged without escalation if the correction is made immediately.
By the end of the shift, supervisors can review a clear record of what was flagged, why action was taken, what happened next, and how long each intervention lasted. The result is a facility with fewer panicked stoppages, fewer disputes about what occurred, and a decision trail that reflects the plant’s actual operating rules rather than the logic of a black box.
4. Multi-Agent Reinforcement Learning (MARL)
What It Is
Software agents that learn to act together. Each agent pursues a goal under constraints and adjusts its behavior based on feedback: its own reward, other agents’ moves, and rules you set. Instead of one model making a single prediction, many small policies coordinate; they negotiate, escalate, and retry within clear bounds.
The “learning” isn’t a slogan. It’s the process of improving those policies through experience in simulators and in tightly supervised production, with rewards tied to the outcomes you actually care about.
Value It Provides
MARL is useful when there isn’t one “right” move, only better trade-offs that change with context. It handles competition and cooperation simultaneously: balancing throughput and fairness in a service, smoothing load across shared resources, or personalizing choices without collapsing into a filter bubble.
Agents operate locally but are aligned by shared rules and rewards, so the system stays responsive when conditions shift. The practical benefits are fewer manual reallocations, fewer last-minute rescues, and steady improvement as the agents learn from real outcomes rather than from static playbooks.
What It Could Look Like
A shopping app offers a set of recommendation agents, each designed around a different shopping priority: trendsetter, pragmatist, accountant, best-friend energy, and so on. Each one is trained on a different slice of your past choices and on distinct objectives. One pushes novelty and serendipity; one guards budget and durability; one tracks fit and returns; another captures softer preferences that don’t show up in tags.
When you search “running jacket,” they debate in the background and produce a shortlist with a short, readable rationale. You can adjust their weights for the session: more novelty for gifts, more pragmatism for work. And you see the trade-offs in plain terms: slightly higher price, better weatherproofing, lower return risk.
Over time, the council reduces repetition without forcing novelty for novelty’s sake, and avoids the narrowness of a single algorithmic profile. You discover more without feeling pushed, and when you ask, “Why this?” the answer is a short explanation tied to the agents’ goals rather than a generic, “People like you bought…”
5. “Search by Problem” Signature Canvases
What It Is
A workspace that indexes knowledge by problem signature rather than by document title or team. You describe the situation the way practitioners do: “X after Y under Z constraints,” or “unexpected spike in A when B is true.” The canvas retrieves prior fixes, relevant papers, runnable code, and data slices that match the pattern.
Results come back as a few candidate “plays,” each with assumptions, inputs, steps, expected failure modes, and links to the closest prior. You don’t hunt through folders; you start from comparable cases and adapt.
Value It Provides
Time to a credible first option drops from days to minutes, and decisions carry provenance: every step is tied to sources and code you can rerun. Teams stop reinventing; they adapt proven approaches, identify where assumptions break down, and record what changed.
The canvas broadens the search space, pulling in alternatives from other domains so you avoid confirmation bias and surface methods you wouldn’t normally consider. Over time, it becomes institutional memory you can use: the place where “what worked, when, and why” lives.
What It Could Look Like
A retail bank notices a rise in missed credit-card payments after moving due dates to the first of the month. An analyst opens a problem-solving workspace and enters a plain-language description of the issue: “Late payments increased after the due-date change. Goal: reduce them without introducing new fees or tighter credit rules.”
The system retrieves a small set of comparable cases and practical response options, each with the reasoning, required inputs, likely trade-offs, and implementation steps. In this case, it suggests three plausible interventions: aligning due dates more closely with customers’ pay cycles, adding a short grace period, or allowing customers to change their due date in one tap through the banking app.
The team selects the first option, combined with a brief grace period. The system then generates the rollout plan, draft customer communications, the operational changes required, and a review date to measure impact.
Six weeks later, the results appear in the same workspace. If the intervention works, the case is saved as a reusable response pattern for the next time a similar problem appears.
6. GUI-Use Agents
What It Is
Software that operates existing screens the way a trained staffer would: reading forms, clicking buttons, pasting IDs, downloading files, all while keeping a full, human-readable trace of every step. A GUI-use agent can interpret labels and layouts, adapt to small UI changes, pre-fill fields from context, validate before submitting, and roll back if a guardrail trips.
Value It Provides
Most complex work still lives across legacy systems that won’t be rewritten soon. GUI-use agents cut the swivel-chair time between them, reduce re-keying errors, and standardize multi-step tasks so outcomes are consistent across teams and geographies.
Every action is supervised and logged, so you gain speed without giving up control. Onboarding time shortens, exceptions surface faster, and audits rely on traces rather than recollection.
What It Could Look Like
A community clinic is preparing for a difficult respiratory season. Overnight, a clinical support agent reviews the next day’s appointments and prepares a short briefing for each patient.
For Mrs. Schmidt, a 72-year-old with recurring respiratory complaints, it pulls together the past year’s visits, current medications, recent lab results, the latest chest imaging note, and recent air quality and pollen data for her neighborhood. From that information, the system adds a concise note to her electronic health record: coughing episodes tend to cluster after poor air-quality days, and inhaler technique hasn’t been reviewed in nine months. It suggests two practical follow-ups for the clinician to consider: adding a spacer to improve inhaler use and creating a plan for high-heat or poor-air days.
When Mrs. Reyes arrives later that morning, the nurse opens the same briefing, reviews it in under a minute, and finds the insurer’s prior-authorization form already filled in with the relevant details.
7. World Generation From Prompt
What It Is
Software that turns a short description plus real data into a believable scene or scenario you can explore. You describe the situation: people, place, goal, constraints. The system builds a high-fidelity environment (3D space or storyboard), populates it with plausible events, and links it to the underlying numbers (costs, timings, risks).
It isn’t a pretty render; it’s an interactive “what-if” you can inspect, adjust, and save, with assumptions, mitigations, and outcomes included, so decisions are rehearsed before they’re made.
Value It Provides
World generation compresses weeks of back-and-forth into a shared artifact. Teams see trade-offs early, test human factors, and align on changes they can measure. Scenarios are anchored to real data (maps, prices, schedules, policies), so the conversation moves from opinions to “show me.”
In consumer settings, it builds empathy: people can feel consequences rather than skim a chart. In operations, it reduces rework by surfacing layout, timing, or safety issues before they become expensive.
What It Could Look Like
A retailer is rolling out a new order-management flow just before peak season to test the reverse logistics process with her sales floor staff. A supervisor opens a world generation tool and enters a short scenario brief: “Return-and-exchange with split shipment; one item damaged, one back-ordered; customer is loyalty tier Silver; gift card used; partial refund to original tender; escalate if inventory mismatch over €50.”
The system generates a practice session using the retailer’s real operating data, store layout, OMS rules, inventory positions, refund policies, staffing levels, and masked transaction history. Instead of producing a static training script, the system builds an interactive operational world around the use case.
The system brings the sales floor, service desk, inventory records, customer profiles, order timeline, and exception rules together into a single coherent scene. Staff can move through the scenario as it would unfold in practice: checking stock, inspecting the damaged item, tracing the split shipment, applying refund logic, and deciding whether the mismatch threshold triggers escalation. Each action updates the scenario’s downstream effects in real time: refund totals, replacement timing, customer wait time, shrink risk, and policy compliance.
Managers can inspect the assumptions beneath the generated world, adjust them, and rerun the scenario instantly. They can test variations: What changes if the item is available in a nearby store, the customer is Gold rather than Silver, or peak-season staffing adds 5 minutes to every handoff?
The system regenerates the same situation under those conditions and shows how the outcome shifts across cost, time, customer experience, and operational risk.
The result isn’t a practice exercise but a shared, data-backed “what-if” environment. The retailer can rehearse edge cases, compare process choices, surface bottlenecks before launch, and save the full scenario, including prompt, assumptions, mitigations, and outcomes, as a reusable artifact for operations, training, and process design.
8. Synthetic Stakeholder Agents
What It Is
Agents trained to act on behalf of a specific stakeholder or signal source in a domain, and to participate in decisions on that stakeholder’s behalf. Unlike general copilots, these agents are fluent in the “language” of their niche (order-book microstructure, watershed health, clinical acoustics, tariff rules). They adopt the mental models and working heuristics of human SMEs in that domain, and can either advise or co-sign actions under explicit authority.
They don’t predict; they argue from constraints, show their evidence, and register a position you can audit.
Value It Provides
Important signals are often silent in day-to-day choices: market microstructure drowned out by headlines, patient lifestyle patterns hidden in notes, sustainability constraints buried in large PDFs, or non-standard legal terms trapped in dense documents where manual review becomes the bottleneck.
Domain-specific agents pull those signals forward at the moment they matter, making constraints explicit, surfacing conflicts early, and turning implicit trade-offs into deliberate choices. Their scope, authority, and inputs are defined up front, so their participation is consistent and auditable: recommendations are tied to evidence, clear thresholds are set, and a record is kept of when the agent approved, vetoed, or deferred.
What It Could Look Like
After a night of wind and flooding, a local authority is planning how to restore power. Alongside the human operations team, the system brings in several domain-specific agents, each prioritizing a different operational concern.
One is focused on grid reliability and favors restoring the two feeder lines that would return service to the largest number of customers. Another addresses crew safety and prohibits any fieldwork in standing water unless a safe switching plan is in place. A public health agent prioritizes circuits serving facilities like a dialysis center and vaccine cold storage.
An environmental agent prevents work at a substation until an oil-containment inspection has been logged. A cost agent prefers a single larger switching operation, but only if the other constraints are satisfied.
Within minutes, the system proposes a restoration plan that reflects these competing priorities: isolate two flooded sections of line, restore the hospital feeder and telecom trunk first, send a crew toward a downed primary line with a safety hold in place, and defer work at the substation until the environmental check is complete. A human dispatcher reviews and approves the plan.
The following day, the operations summary shows which areas were restored, in what order, and the reasoning behind each decision. The team then shares the plan with local media, and residents and businesses affected by the restoration actions receive contextual updates for their area: what’s happening, why it’s being prioritized, and when service is expected to return.
What Will It Take to Bring This to Life?
Individually, these use cases are modest. Collectively, they change the morning described in our opening story (see the previous blog post): evidence arrives with the work, rehearsals inform real decisions, routine tasks finish with a trace you can read, and orchestration happens under rules you control.
The pattern to notice isn’t a single breakthrough. It’s the bridges built between observing and acting, and the presence of artifacts (clips, traces, logs, diffs) that make each step auditable.
The technologies described above are only the raw materials. Turning them into operational excellence and a future reality requires a shift in how we design and build systems, and in how we govern and measure their performance. This implies three shifts in the enterprise operating model.
From Static Policy to Compiled Intent
Today, your governance lives in PDFs; your decisions live in software. To bridge the gap, organizations must start treating policy as code. Rules, such as safety thresholds, customer promises, and compliance limits, must be written in a form that systems can execute and explain.
The artifact of the future isn’t a revised handbook. It’s a machine-readable record showing exactly which rule changed, who approved it, and how that change propagated to the agents in the field. When an agent takes an action, it points to the specific line of code that authorized it.
From Headcount to Intervention Economics
Stop measuring AI success by headcount reduction and start measuring it by Interventions Per 1,000 Tasks (IPKT) at a constant risk level. In a healthy AI-native organization, routine work is handled by agents within strict autonomy budgets. Humans step in only when those budgets are breached: when a situation is novel, ambiguous, or high-stakes.
As your capabilities mature, your IPKT declines even as complexity increases. This metric changes the incentives: instead of trying to automate everything perfectly (which creates brittle systems), teams focus on defining clear boundaries where the machine is safe to act, and making the handoff to humans smooth when it isn’t.
From Pilot Success to Simulation Concordance
Testing agentic systems solely in production is dangerous. The new standard for maturity is Concordance: the degree to which your simulated environments match your real-world outcomes.
Before a new pricing agent or grid controller goes live, it must run thousands of scenarios in a high-fidelity “world generation” sandbox. If the simulation predicts a 2% efficiency gain and production delivers 2%, you have a mature system.
If production delivers 10% or -5%, you have a risk gap. Trust comes from knowing the system behaves exactly as it did in rehearsal, not from believing the model is smart.
Looking Ahead
The “AI future” is often sold as a moment of singularity: a sudden leap into magic. The future we see looks more like operational calm when executed well.
Excellence in AI isn’t about replacing human judgment. It’s about clearing the path for it: capturing a messy world with video and sensors, rehearsing decisions in valid simulations, and executing them through agents that follow our rules and explain their work.
When we get this right, the result is an organization that’s less frantic and more intentional. We stop celebrating the heroic save and start celebrating the quiet, boring morning where the system simply worked.
The pieces are here. The transition from “smarter models” to “better operations” is the work of the next decade. It’s time to stop waiting for the magic and start designing the rules.
Selected References
These references are directional signals and supporting sources for the concepts discussed above rather than a literature review:
- Chen, Y. et al. Scaling RL to Long Videos. arXiv:2507.07966, 2025. Available via Hugging Face Papers.
- Ma, C. et al. P3-SAM: Native 3D Part Segmentation. Project page / research release, 2025.
- d’Avila Garcez, A. and Lamb, L. C. Neurosymbolic AI: The 3rd Wave. arXiv:2012.05876, 2020.
- Romera-Paredes, et al. Mathematical Discoveries from Program Search with Large Language Models. Nature, 2023.
- VeriGUI-Team. VeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking. GitHub repository.
