AI-Native Series · Research

Let the Model Write the Problem. Let the Solver Sign the Answer.

By Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app · 2026-08-04

Ask a language model to plan your quarter and you get something beautiful. Ask it twice and you get two beautiful, different plans — and if you check either against your actual constraints (this engineer is booked, that deadline is immovable, this budget caps out), it quietly violates several. Ask a binary integer program instead and you get exactly one answer that provably respects every constraint you gave it — and it will never tell you whether you gave it the right constraints, or whether the thing you asked it to maximize is worth maximizing.

Those are not two competing tools. They are two halves of a brain, and the interesting engineering question — the one this article is about — is the seam.

What the research actually settled

The last two years produced an unusually clean answer to "can LLMs plan?" — and it's a division of labor, not a winner.

Half one: LLMs cannot be the guarantor. Kambhampati and colleagues put it bluntly in their ICML 2024 position paper: "We argue that auto-regressive LLMs cannot, by themselves, do planning or self-verification (which is after all a form of reasoning)" [1]. The empirical backbone is just as blunt. On Game of 24, graph coloring, and classical planning tasks, Stechly et al. found that letting the model critique its own answers made things worse: "We observe significant performance collapse with self-critique and significant performance gains with sound external verification" [2]. A companion study concluded that "self-critiquing appears to diminish plan generation performance, especially when compared to systems with external, sound verifiers" [3]. In plain words: an LLM checking an LLM is a rumor checking a rumor.

Half two: LLMs are startlingly good at writing the problem. Stanford's OptiMUS turns natural-language descriptions into mixed-integer linear programs and "can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve its model and code based on these evaluations" [4] — beating prior state of the art by 20-30%. Chain-of-Experts, at ICLR 2024, presented "the first LLM-based solution, namely Chain-of-Experts (CoE), a novel multi-agent cooperative framework to enhance reasoning capabilities" for complex operations-research modeling [5]; by 2025, OR-LLM-Agent had decomposed the pipeline into "three sequential stages: mathematical modeling, code generation, and debugging" and outperformed frontier general models on OR benchmarks [6]. Formulation — the tedious, error-prone translation of a messy situation into decision variables and constraints — is exactly the part humans hate and models excel at.

And the exemplar that proves the marriage works: AlphaGeometry reached near-gold olympiad performance with a system that "uses a neural language model, trained from scratch on our large-scale synthetic data, to guide a symbolic deduction engine through infinite branching points" [7]. The neural half proposes at the points where search explodes; the symbolic half derives with certainty. Twenty-five problems solved where the best symbolic-only system managed ten. Neither half alone comes close.

One more piece, because my planner is built on it: receding-horizon control — plan a window, act, watch reality, re-plan from where you actually are — is not a robotics-only trick. It is a general decision-making paradigm; recent work models entire competitive supply chains as agents where "every agent re-plans their actions in a receding horizon manner based on estimates of market and supplier parameters" [8].

The three systems on my bench

I didn't come to this question from the literature. I came to it holding three artifacts that each own one piece, built years apart without knowing they were parts of one machine.

The solver: Life GPS (2019, public). I told the full story in the previous essay: a weekly planner that treats your week as a grid of hour-slots and solves a binary integer linear program over it — decision variables x[slot,task] ∈ {0,1}, objective = weighted fulfilment, constraints = one task per slot, per-task floors and caps, deadlines. The rewritten Python engine (allocator.py, PuLP + CBC) carries the part that matters most: replan() freezes the past, decrements every budget by what you actually spent, and re-solves the remaining horizon — Model Predictive Control, applied to a life. And here is the detail I only appreciated while researching this article: the rewrite already has two LLM seams. intake() accepts an injected llm_fn that turns "ship my MVP, meditate daily, five days, eight hours" into a validated PlanRequest; explain() renders the solved grid back into sentences. The integration below is not a proposal to bolt AI onto a solver. The sockets are already in the wall.

The strategist (private). A strategy operating system I keep closed-source: it runs a disciplined pipeline from diagnosis (a Rumelt-style kernel: what is actually going on?) through genuinely-different option sets, stress scenarios, and competitive responses, to a decision node that is always signed by a human — with every claim gated on evidence, and execution tracked with leading metrics and tripwires that fire when an assumption breaks. What it produces, in optimization terms, is precisely what a solver cannot produce for itself: the objective (what to value, with evidence behind each weight) and the constraint set (what rules bind, and why).

The proof it's not a toy (private). The strategy engine's first real engagement runs through a role-gated team hub — a live client initiative whose evidence register, options, and 90-day plan all flow from the same pipeline. It matters here for one reason: the weights and constraints in a real engagement come attached to named evidence and a human signature, not to a model's vibes.

The architecture: three layers, two clocks

Put the research and the artifacts side by side and the design almost writes itself.

Layer 1 — the strategist (slow clock: weeks, or whenever a tripwire fires). Language-model reasoning, with humans at the decision nodes, produces the problem statement: which initiatives exist, what each is worth (a weight, with the evidence that justifies it), what the floors, caps, deadlines, and dependencies are. This is OptiMUS's lesson generalized: the model's superpower is formulation. It writes the LP.

Layer 2 — the solver (fast clock: daily, or on every check-in). The binary integer program allocates hours and dollars under the constraint set, and the MPC loop re-solves from actuals. The solver is the external sound verifier the self-critique studies demand [2][3]: it cannot be talked out of a constraint, it cannot hallucinate an hour that doesn't exist, and when the problem is infeasible it says so instead of writing something plausible.

Layer 3 — the sensors. Telemetry closes both loops: tests passing, metrics moving, work items landing (in my stack, the evidence-graded report cards every repo already emits). Cheap sensors are what turned this from a 2019 toy into a live system — the feedback a human once hand-typed twice a day now files itself.

The two clocks are the part most designs miss. Tactical replanning (the solver's clock) and strategic replanning (the strategist's clock) are different operations with different costs. Re-solving the allocation is milliseconds and safe to do hourly. Re-writing the objective is expensive and dangerous to do reactively — that way lies thrash. So the escalation is explicit: the solver re-solves within the current problem statement until a tripwire — a named, pre-registered assumption-breaker — fires and hands the pen back to the strategist. That is the receding-horizon pattern applied twice, at two horizons [8].

The seam contract

Everything above lives or dies at the seam, so make the seam a contract:

• Weights flow down only with evidence and a signature. The model may propose that "initiative A is worth 3× initiative B"; the proposal carries its evidence; a human signs it before the solver ever sees it. An unsigned weight is not in the objective.
• The model formulates; it never verifies. Translation into decision variables, constraints, solver code: the model's job [4][5][6]. Checking feasibility and optimality: the solver's job, structurally — not a prompt asking the model to double-check itself [1][2].
• The model explains the solver's numbers, never its own. explain() turns the solved grid into sentences. The discipline: it narrates what was solved. If the narration and the grid disagree, the grid wins.
• Unmeasurable objectives stay out of the objective function. If a goal has no metric yet, it appears as "not measurable yet — define the metric," never as an invented coefficient. An objective term nobody can measure is how dashboards learn to flatter.

The failure mode: laundering guesses into authority

Here is the danger nobody warns you about, and the reason the contract above is strict. A solver's output feels authoritative — it is provably optimal, after all. But it is provably optimal for the problem it was given. Feed it weights a model hallucinated and it will return confident, precise, optimal garbage — and the precision will make the garbage more persuasive than the model's raw guess ever was. The solver launders the guess into authority.

Three mitigations, all cheap:

• Evidence-gated weights (above): no signature, no coefficient.
• Sensitivity analysis as a first-class output: every plan ships with "which weight, if wrong by 2×, flips this plan?" If the answer is "the one weight we're least sure of," the plan is a hypothesis, not a schedule — and the dashboard should say so.
• Honest odds, clamped: any probability shown to a human comes from observed velocity versus needed velocity, bounded away from certainty. My own dashboard's first reading under this rule was a bracing 2% on a revenue goal — which is exactly the number that makes you change the inputs instead of admiring the interface.

What's running today, and what's next

A v0 of this architecture went live on my portfolio's owner dashboard today: three goal tracks with dated milestones, the venture fleet as a dependency graph, and the best next 1-3 moves re-solved from live report-card telemetry on every load — including time travel that replays what the instrument would have said on any earlier day. Its re-solver is still a greedy scorer, not the full integer program; the pipeline to replace it is short because Life GPS already exposes /plan and /replan over HTTP. The build order, one working session each: (1) swap the greedy scorer for the BILP behind the owner-gated API; (2) let the strategy engine author the weights file the way it authors an engagement — evidence attached, human-signed; (3) register tripwires so strategic re-solves are event-driven, not scheduled. Target: all three seams live within two weeks (by 2026-08-18). Each step is small because each piece already runs; the work is the seam, which is the thesis.

Patterns / Anti-patterns

• Pattern: neural proposes, symbolic disposes — at every infinite-branching point, and only there [7].
• Pattern: two clocks — allocation re-solves fast and cheap; objectives re-solve slow, on tripwires.
• Pattern: the verifier is structural (a solver, a test, a gate), never a second prompt.
• Anti-pattern: asking the model to plan and to check the plan — a rumor auditing a rumor [2][3].
• Anti-pattern: an optimizer fed unsigned weights — optimal garbage, precision-laundered.
• Anti-pattern: one clock — either thrashing strategy daily or freezing allocation quarterly.

Provenance — what's verified and what's mine

All eight research quotations were verified verbatim against the listed sources at publication time (the Nature abstract requires a browser user-agent to fetch). Life GPS is my own public code, described from its repository. The strategy engine and its client engagement are my private systems, described at capability level only — no internals, per their licenses. The dashboard, its greedy re-solver, and the 2% reading are live, owner-gated, and mine.

References

1. Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L., & Murthy, A. (2024). Position: LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks. ICML 2024. arxiv.org/abs/2402.01817
2. Stechly, K., Valmeekam, K., & Kambhampati, S. (2024). On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks. arxiv.org/abs/2402.08115
3. Valmeekam, K., Marquez, M., & Kambhampati, S. (2023). Can Large Language Models Really Improve by Self-critiquing Their Own Plans? arxiv.org/abs/2310.08118
4. AhmadiTeshnizi, A., Gao, W., & Udell, M. (2024). OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models. ICML 2024. arxiv.org/abs/2402.10172
5. Xiao, Z., Zhang, D., et al. (2024). Chain-of-Experts: When LLMs Meet Complex Operations Research Problems. ICLR 2024. iclr.cc/virtual/2024/poster/18977
6. Zhang, B., Luo, P., Yang, G., Soong, B. H., & Yuen, C. (2025). OR-LLM-Agent: Automating Modeling and Solving of Operations Research Optimization Problems with Reasoning LLM. arxiv.org/abs/2503.10009
7. Trinh, T., Wu, Y., Le, Q., He, H., & Luong, T. (2024). Solving olympiad geometry without human demonstrations. Nature 625. nature.com/articles/s41586-023-06747-5
8. Hall, S., Guerrini, L., Dörfler, F., & Liao-McPherson, D. (2024). Receding Horizon Games for Modeling Competitive Supply Chains. arxiv.org/abs/2401.09853

Related

• I Built My Company's Brain in 2019 and Mistook It for a To-Do List — part 1: the confession, and the MPC frame this article builds on.
• We Built a Machine That Argues With Us — the same verifier-not-vibes discipline, applied to shipping claims.
• Strategy as Code — why the strategist layer is data and gates, not slideware.

Part 2 of the company-GPS series. The research packet (eight sources, quotes verified verbatim by fetch at publication) was assembled 2026-08-04. — Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app