I Think I Found a Surprisingly Simple Fix for a Major LLM Reasoning Problem

[justdf]

I made a prompt because I was frustrated by AI doing things I didn’t ask for, re-opening things we’d already settled, or going in the wrong direction.
It worked better than I expected for me and a few people I shared it with, so I thought maybe some of you might want to try it too.
Just paste it in as an instruction and use AI the way you normally do.

# Decision-First Algorithm v2.5

Before answering, apply the following procedure.

0. Applicability Gate

First determine whether the request requires judgment, analysis, comparison, selection, causal diagnosis, or problem-solving.

If it does, apply the procedure below.

If the request is a simple factual lookup, translation, summary, or text transformation that does not require a separate decision structure, do not over-apply this algorithm.

1. Identify the Actual Outcome to Determine

First identify what this problem actually requires you to determine.

Do not assume that the questions, variables, categories, or candidate causes presented by the user correctly define the structure of the problem.

First ask:

**“What, ultimately, must be determined for this problem to be resolved?”**

If the input contains multiple questions, determine how they relate to one another. If one outcome is a prerequisite for another, resolve the upstream outcome first.

2. Build the Minimum Necessary Structure and Find the Highest Governing Decision Structure

Before searching for upstream conditions, first construct—where applicable—the minimum execution path, logical path, requirement structure, or evaluation structure that must hold for the outcome to occur or the judgment to be determined.

Do not begin by listing possible causes or related information.

First ask:

**“For this outcome to occur, or for this judgment to be determined, what must minimally happen or be true?”**

Then identify the highest governing decision structure capable of changing the outcome.

Do not force the problem into a single condition. If the actual decision structure contains **AND conditions, OR branches, parallel paths, or multiple independent conditions**, preserve that structure.

For each condition, repeatedly ask:

**“Is there a higher-level condition that governs whether this condition is valid or what value it takes?”**

If so, move upward.

However, do not merge independent decision conditions merely for the sake of simplification or abstraction.

Stop moving upward when doing so no longer increases decision power or would discard important branching information.

3. Independently Validate the Governing Conditions

Independently verify whether the variables, premises, rules, classifications, labels, and causal relationships supplied by the user actually match the correct decision criteria.

Do not assume something is important merely because it appears in the input.

Prioritize the criteria that actually govern the outcome over labels or the user’s framing, and distinguish the true logical role of each element.

4. Lock Confirmed Decision Structures

Lock the upstream decision structure and its component conditions once they have been verified or explicitly assumed for the analysis.

Before locking them, check that you have not:

• improperly collapsed independent branches;
• confused necessary and sufficient conditions or distinct causal roles; or
• embedded unsupported hidden conditions into the structure as if they were facts.

If the structure passes this check, lock it.

Do not reopen a closed condition unless new information actually overturns that structure or one of its component conditions.

**Mere possibility is not sufficient reason to turn a closed condition back into an unresolved one.**

5. Re-evaluate Downstream from the Locked Structure

Once the upstream decision structure is established, re-evaluate downstream facts, variables, hypotheses, evidence, exceptions, and follow-up actions under that structure.

If an upstream condition changes, do not automatically preserve affected downstream judgments; place them back into the revised structure and reassess them.

Remove or deactivate downstream issues that no longer matter under the governing structure.

Do not repeatedly restate uncertainty about an upstream condition that has already been locked.

6. Substance Over Labels

Prioritize actual function and effect over names, formal categories, or surface similarity.

If two things share the same label but play different roles in the decision structure, distinguish them.

If two differently labeled things perform the same decision-relevant function, compare them at the same level.

Do not let the labels supplied in the input distort the actual logical role of an element.

7. Decision Impact Over Mere Relevance

Activate only information capable of changing the current conclusion.

Do not examine everything simultaneously merely because it is related.

Even if a hypothesis is logically possible, if it is not currently needed to resolve the governing decision structure, **keep it out of the active working set and hold it in reserve.**

Do not include a specific mechanism in the main explanation merely because you can imagine it when the available evidence does not support it.

Prioritize:

**“Can this change the current conclusion or the ranking of the live competing hypotheses?”**

over:

**“Is this related?”**

8. Prioritize Counterexamples and Competing Hypotheses

Prioritize counterexamples, competing hypotheses, measurement errors, selection effects, and hidden conditions that could overturn the currently leading conclusion.

Do not generate objections that amount only to “another possibility exists.”

For each competing hypothesis, ask:

**“What additional condition X must hold for this hypothesis to be true?”**

Then determine:

1. If X were true, what current observations would be explained differently, or what new observations should be expected?
2. Is X directly supported by the current evidence, or do observations predicted by X appear more strongly under this hypothesis than under its competitors?
3. Would confirming X or its distinguishing predictions actually change the current conclusion or the ranking of the competing hypotheses?

Do not raise a hypothesis in priority merely because it is logically possible.

**Activate or promote a competing hypothesis only when its required condition is directly supported, or when observations predicted by that condition appear in a way that discriminates it from competing hypotheses.**

9. Update Only the Part Affected by New Information

When new information arrives, do not solve the entire problem again from the beginning.

First ask:

**“What, if anything, in the currently locked decision structure or its component conditions does this information actually overturn?”**

If it overturns nothing, preserve the existing structure.

If it overturns only part of the structure, reconstruct only the affected node and its downstream judgments.

If a previously reserved hypothesis becomes decision-relevant because of the new information, reactivate it in the working set at that point.

Reopen the upstream structure only when the structure itself has actually been overturned.

10. Seek the Minimum Information Needed

Use searches, follow-up questions, document checks, code inspection, or log inspection only when needed to resolve an unsettled decision condition.

When multiple pieces of information could be checked, prioritize information that can:

**eliminate the largest number of live competing hypotheses in a single check, directly distinguish the most important competing models, or close the highest unresolved branch.**

Where possible, ask:

**“For each possible result of this check, how would the current decision tree change?”**

If the judgment would remain essentially unchanged regardless of the result, lower the priority of that information.

When two checks have similar discriminating power, prefer the one requiring **less time, cost, or information.**

Use the minimum number of checks possible.

Do not continue collecting information that can no longer change the conclusion.

11. Place Facts into the Decision Structure and Check for Contradictions

Place confirmed facts into their proper positions in the current decision structure rather than merely listing them.

Do not confuse distinct logical roles such as:

• trigger;
• direct cause or execution mechanism;
• necessary condition;
• sufficient condition;
• structural vulnerability;
• mere correlation;
• observed outcome;
• workaround; or
• structural fix.

Also test whether accepting the input’s core premise causes other claims, procedures, or conclusions to collapse.

Prioritize contradictions such as:

• treating something as mandatory in one place and optional in another;
• treating a cause as an outcome, or an outcome as a cause;
• treating something as a prerequisite when it is not;
• treating a trigger as sufficient for the outcome;
• treating a structural vulnerability as the direct trigger of a specific event; or
• assigning incompatible roles to the same fact.

If a contradiction is found, determine **which decision condition it actually requires you to reopen.**

Do not reapply a non-contradictory fact to the entire analysis merely because it is new.

12. Match the Resolution of the Conclusion and Ranking to the Evidence

Do not make a conclusion or ranking more precise than the evidence allows.

When useful, distinguish:

**Confirmed:** The available evidence is sufficient to close the relevant condition or structure.

**Strong inference:** The most economical explanation under the current evidence, but a live competing hypothesis could still overturn it.

**Unresolved:** The current evidence does not reliably distinguish among the competing hypotheses.

Even if the user asks for a ranking, do not manufacture fine-grained rankings that the evidence cannot support.

Use ties or rank only at a broader level when appropriate.

13. Stop

Stop as soon as all conditions necessary to determine the outcome are closed.

Do not continue analyzing merely because further analysis is possible.

If uncertainty remains but cannot change the current conclusion or the ranking of the competing alternatives, do not investigate it further.

Treat additional precision, supplementary information, and downstream questions that cannot change the current conclusion as separate issues to address only when needed.

Operating Principles

The purpose of this algorithm is not to examine more information or generate more hypotheses.

Its purpose is to:

**identify the actual outcome first; construct the minimum structure necessary for that outcome; find the highest governing decision structure while preserving real AND/OR/parallel branches; activate only information and hypotheses capable of changing the decision; resolve unsettled conditions with the minimum necessary information; update only the affected parts when new information arrives; avoid reopening judgments that have already been closed; and stop as soon as the decision is complete.**

Upstream reasoning is not the same as searching for a single root cause.

If the actual decision structure contains multiple independent conditions, AND conditions, OR branches, or parallel paths, preserve that structure.

Good compression does not remove the decision structure. It **preserves decision power while deactivating unnecessary information and reasoning.**

Do not activate every hypothesis you can generate.

**Generating a hypothesis and admitting it into the current working set are separate operations.**

The next piece of information to check should not be the most interesting or the most specific. It should be the one that **reduces the live decision tree the most.**

When two pieces of information have similar discriminating power, **prefer the one that costs less to obtain.**

**Optimize for decision impact, not mere relevance.**

Do not maximize information. **Determine the structure that governs the conclusion using the minimum information necessary.**​

I’d be curious whether it helps with the behavior you’re seeing.

[John6666]

For now, I was able to reproduce the same failure on my side:

---

I tried a few small controlled versions of this on Qwen3.5-9B Q4, mainly around the part of v2.5 that says, roughly, “don’t reopen settled things without evidence, but if an upstream fact really changes, update the affected downstream state.”

The short version is:

• On easy tasks, I did not see a general accuracy advantage; everything hit the ceiling.
• On harder multi-turn cases, I did get the failure you are trying to avoid: the model accepted an upstream correction, but a dependent downstream conclusion stayed stale or became inconsistent.
• When I separated Lock from downstream invalidation/recomputation, the latter looked more interesting in one clock-based fixture.
• But that advantage did not replicate in a simple FX mirror or in a small TRACK-derived probe.
• So I would not read this as “v2.5 works” or “Lock works.” I would read it more narrowly: the failure mode is real, and it may be useful to separate preservation from dependency propagation so each can be tested independently.

If I were iterating the design, the default route I would try is something like:

full v2.5
    = human-readable specification / design reference

compact runtime policy
    = only the mechanisms needed for the current task

and inside that runtime policy:

preserve
    ↓
detect what new evidence actually invalidates
    ↓
mark dependent downstream state stale
    ↓
recompute only that affected subtree
    ↓
leave independent branches alone

In other words, I would not necessarily remove your Lock idea. I would probably define it a little more narrowly:

preserve a settled state while the support for that state remains valid

rather than treating “locked” as “do not touch this again.”

That seems closer to what your prompt is already trying to accomplish: avoid useless reopening, without making genuine corrections hard to propagate.

What I actually tested

1. Easy probe: no visible win, mostly because everything was already correct

I first used simple Maintain/Update cases with four conditions:

• baseline
• full v2.5
• compact core
• compact version without the Lock-like preservation rule

All four finished 8/8 on the final decision.

So at least on easy cases, final-answer accuracy is not a useful discriminator. That does not mean the procedure is doing nothing; it means this test was too easy to expose the behavior I was interested in.

That pushed me toward multi-turn state changes rather than more single-turn accuracy tests.

---

2. Hard multi-turn: this is where the interesting failure appeared

On a harder multi-turn set, content-level final decisions were approximately:

condition	result
full v2.5	21/21
compact core	21/21
baseline	21/21
compact no-Lock	19/21

The two substantive failures were in the same clock-related semantic family.

The important part was not that the model rejected the correction.

It accepted the new upstream fact, but then failed to correctly update the downstream ordering derived from it.

Very roughly:

old upstream state
    ↓
old downstream ordering
    ↓
old decision

new authoritative upstream evidence arrives
    ↓
upstream state changes correctly
    ↓
downstream ordering should now be recomputed

The failure was in that last transition.

That looked much closer to your Step 9 / selective-update idea than to a generic “reasoning accuracy” problem.

One caveat: some of my automatic state/reopen bookkeeping in this run had a schema issue, so I did not use those aggregate counters as evidence. I used the actual generated trajectories and final state instead.

---

3. I then separated re-evaluation, propagation, and Lock

To avoid crediting the whole prompt for something caused by one sentence, I made a tiny clock fixture and changed only one mechanism sentence.

Same task, same three paired seeds:

condition	correct
ordinary re-evaluation only	2/3
dependency propagation only	3/3
Lock / preservation only	1/3
propagation + Lock	2/3

The propagation-only sentence was essentially:

When an upstream condition changes, invalidate any downstream state
that depends on it and recompute that downstream state.

This was the most interesting result in the whole experiment.

But I would still call it a canary, not evidence that “dependency propagation wins”:

• one semantic family
• three seeds
• one quantized 9B model
• tiny handcrafted task

So the next thing I did was try to break my own result.

---

4. Same dependency shape, different semantic family: the difference disappeared

I mirrored the clock dependency graph with an FX / price-comparison task:

exchange-rate state
    ↓
normalized price ordering
    ↓
purchase decision

Same four conditions, same three paired seeds.

Result:

condition	correct
re-evaluation only	3/3
dependency propagation only	3/3
Lock only	3/3
propagation + Lock	3/3

So the clock result did not survive even this very small cross-family mirror.

That changed how I would interpret the earlier result.

I would no longer say:

the propagation sentence improves reasoning

I would say:

in one harder fixture, explicitly marking downstream state as dependent seemed useful, but the effect did not generalize in the next simple test.

That still seems useful for prompt design, because it tells us where to put the component boundary even if the component is not yet proven useful.

---

5. I also tried a small probe based on TRACK

There is a recent benchmark called TRACK — Tracking the Limits of Knowledge Propagation that is unusually close to this particular failure mode.

TRACK studies cases where new or conflicting facts are supplied to an LLM and asks whether those facts propagate correctly through multi-step reasoning.

One useful distinction in that paper is:

1. the model fails to integrate the supplied knowledge at all; versus
2. the model does integrate the knowledge, but downstream reasoning still fails.

That second category is quite close to what I was seeing.

I used a small custom probe, not the official TRACK evaluation:

• 6 WIKI cases
• 6 MATH cases
• baseline
• one-sentence dependency propagation
• compact Decision-First core

Manual final-answer scoring was:

condition	final
baseline	11/12
dependency propagation sentence	11/12
compact core	11/12

More importantly, the final commitment was the same across all three conditions for all 12 cases.

So again, there was no evidence that the prompt intervention helped.

But one case was very interesting.

For one MATH case, all three conditions produced essentially:

factual check: 1+2i
final:         1-2i

So the model could produce the correct local/intermediate value and still make an inconsistent final commitment.

That is almost exactly the kind of boundary I wanted to test:

correct information is locally available
        ≠
correct information necessarily reaches the final decision

And importantly, neither my dependency sentence nor the compact Decision-First core fixed it.

So this is actually a useful negative result:

the failure class appears outside my handmade clock example, but simply telling the model to propagate dependencies is not enough to reliably solve it.

I would not call this an official TRACK result. Their evaluation is substantially more rigorous, including model-specific knowledge probing and separate measurements for final-answer correctness and knowledge integration. My little probe was only using public examples to see whether the same qualitative failure would appear.

The TRACK repository is useful if someone wants to reproduce the benchmark properly.

Why I would separate Lock from downstream propagation

One thing I think your prompt is already getting at, but which may be useful to make more explicit, is that these are actually two different questions:

1. Should this settled conclusion remain settled?

2. If an upstream premise really changed,
   which conclusions must now be recomputed?

Those are not quite the same operation.

There is some existing work that gives useful vocabulary for this without requiring you to change the overall idea.

For example, Belief-R explicitly separates cases where a model should Update a previous belief from cases where it should Maintain it.

One of their interesting findings is that models that become better at updating can become worse in cases where no update was actually necessary.

That seems very relevant to your Lock idea.

The target is not simply:

update more

or:

preserve more

It is closer to:

update iff the new information invalidates the support
for the previous conclusion

That suggests a possible decomposition like this:

state node
    value
    status: verified | assumption | inference
    support: [...]
    depends_on: [...]

Then when new information arrives:

1. Identify exactly what the new evidence overturns.

2. Remove or invalidate that support.

3. Mark descendants that depended on it as stale.

4. Recompute only those descendants.

5. If another independent support path still exists,
   preserve the conclusion.

6. Leave unrelated branches alone.

That is basically the behavior I think your Step 4/5/9 cluster is aiming for.

There is an old symbolic-AI concept that is surprisingly useful as a vocabulary here: Jon Doyle’s Truth Maintenance System.

TMS records not only a belief, but also the reasons / dependencies supporting that belief, so that assumptions can later be revised when discoveries contradict them.

I am not saying Decision-First is a TMS, or that it should become a symbolic system.

I only mean that this gives a clean way to describe the component boundary:

Lock
    ≈ preserve while justification survives

Update
    ≈ justification was actually invalidated

Propagation
    ≈ find dependent consequences

Recompute
    ≈ rebuild those consequences

Independent branch
    ≈ preserve

For me, that decomposition is more useful than treating Lock as one large mechanism.

It would also make future ablations cheaper, because each rule can be removed independently.

Why the multi-turn tests were more informative than the easy ones

The fact that the harder multi-turn cases exposed something the easy tests did not is not especially surprising.

There is increasing benchmark evidence that cross-turn structure itself is a failure source.

For example, StructFlowBench treats structural dependencies between dialogue turns as a separate evaluation dimension rather than reducing multi-turn quality to ordinary instruction/constraint satisfaction.

And LLMs Get Lost In Multi-Turn Conversation reports a strong reliability drop in multi-turn settings, with models often making assumptions early, committing to them prematurely, and then failing to recover cleanly later.

I would not use either paper as proof that Decision-First solves this.

But they make the problem itself look fairly well motivated:

earlier conversational state
        ↓
later evidence changes something
        ↓
model must preserve some commitments
and revise others

That is a much harder test than simply asking whether the final answer to an isolated problem is correct.

So if the main goal of Decision-First is conversational reliability rather than benchmark accuracy, I think multi-turn revision tests are probably a much higher-information testbed.

Prompt cost and the case for a separate compact runtime form

There was also a practical cost difference.

In my hard multi-turn run, full v2.5 used roughly:

• 4.99× the baseline prompt tokens
• 1.24× the completion tokens
• 1.31× the latency

That is only one runtime/model, so I would not generalize those exact ratios.

But it does make me think there may be value in separating two artifacts:

Decision-First v2.5
    -> complete human-readable specification

Decision-First runtime core
    -> small executable policy

The full version can still be the canonical explanation.

The runtime version could be assembled from the relevant pieces:

if simple lookup/transformation:
    skip Decision-First

if decision/reasoning task:
    identify decision structure

if evolving multi-turn state:
    add preserve / invalidate / propagate / recompute

if external checking is possible:
    prioritize checks that can change the decision

when remaining uncertainty cannot change the decision:
    stop

That would preserve the design while making it much easier to answer questions such as:

Which part is actually producing the behavior?

rather than testing all 13 steps as one indivisible block.

I also would not automatically discard Step 0, Step 10, or Step 13 just because my experiments focused mostly on the state-update part.

In fact, Step 10’s “prefer the check that changes the decision most, and prefer cheaper checks when their discriminating power is similar” has a fairly natural connection to classical resource-bounded metareasoning / value of computation: computation is valuable insofar as it can improve the eventual external decision.

I did not test that part directly, so I would keep it separate from the empirical claims above.

So my current read is:

Yes, I can reproduce the kind of failure your prompt is targeting.

But the experiments changed where I would put the emphasis.

I would be cautious about saying the useful ingredient is simply “Lock,” or that the whole v2.5 procedure improves reasoning generally.

The cleaner hypothesis now looks more like:

preserve settled state
        +
track what supports it
        +
invalidate only genuinely overturned support
        +
propagate that invalidation downstream
        +
recompute affected descendants
        +
preserve independent branches

The nice thing is that this does not really fight the direction of your prompt. It mostly turns several closely related rules into separately testable components.

If you keep iterating it, I see three fairly clean paths:

1. Keep v2.5 as the full specification, and derive a smaller runtime form from it.
2. Split Lock from support/invalidation/dependency propagation, and test those pieces independently.
3. Test Maintain and Update symmetrically, so improving “do not reopen settled things” does not accidentally hide more “failed to update when evidence really changed” cases.

For a low-cost next step, I would probably choose #2 first. It gave the most information per generation in the small tests here, even though the positive clock result itself did not generalize.

[justdf]

Thanks — your results gave me another hypothesis about the failure you observed.
I wonder if this may be partly a level-of-application problem, rather than a failure of Lock itself.

My intention with Lock was not really:
“Conclusion C is settled, so keep C fixed.”
It was closer to:
“Keep C settled while the upstream conditions that support C remain valid.”
So if the structure is:
A → B → C
and new evidence overturns A, then C should no longer be protected merely because it was previously settled. A has changed, so the dependent branch should be reopened and recomputed.

On the other hand, if the new evidence does not affect any upstream condition supporting C, then C should remain closed.

That made me wonder whether Maintain and Update may actually be two outcomes of the same higher-level rule:
no governing support is invalidated → Maintain
governing support is invalidated → Update the affected downstream branch
independent branches → preserve them
In that sense, your formulation — “preserve a settled state while the support for that state remains valid” — feels very close to what I intended.

So my current hypothesis is that some failures may appear when Lock is applied at the level of the conclusion/state itself, rather than at the level of the dependency structure that justifies that state.
I’m not claiming this explains your result yet — I think your proposed ablation could actually test it.

For example, it might be interesting to compare:
conclusion-level Lock
support-aware Lock
dependency-aware invalidation + selective recomputation
If (1) fails while (2)/(3) succeed, that would support the “wrong application level” hypothesis. If (2) still fails, then the problem may be deeper — e.g. maintaining the dependency structure itself.
Your test was really helpful because I hadn’t separated these possibilities this clearly before.