For now, I looked around for a few related cases:
Yes — I think there are some fairly close relatives of this failure mode, although I would be careful not to assume they share the same cause.
The distinction that seems most useful to me is to separate at least these questions:
- Does the latent state change with the input?
- Is task-/answer-relevant information actually decodable from that state?
- Does the model’s native downstream computation causally use that information?
- Does the task/training setup actually require the latent path to carry example-specific information?
- Was the path given a real opportunity to learn at all — i.e. did the relevant parameters actually move?
Those can fail independently.
That seems especially relevant here because your hidden-question condition already removes an important shortcut. If the answer pass can still see the question, it can simply solve from the original tokens and treat the latent path as optional. Once the question is hidden, that explanation becomes much weaker. In your current result, replacing the thoughts with another question’s thoughts is essentially neutral (-0.001 nats on 4.409, with correctness 7/30 -> 7/30), while replacing them with the suite-average vector does have a noticeable cost. So the answer path seems sensitive to something about having realistic latent states there, while showing very little sensitivity to which question produced them.
That pattern has some precedents.
A simple default diagnostic route I would use is:
Did the relevant parameters actually move?
|
+-- no / unclear
| `-> optimization / storage / training-path question first
|
`-- yes
|
+-- Can a small controlled probe decode answer-relevant information?
| |
| +-- no
| | `-> the latent may be input-dependent without yet carrying
| | the information the answer needs
| |
| `-- yes
| |
| +-- Does a matched / minimal-pair latent transplant change
| | the native answer in the donor-expected direction?
| | |
| | +-- no
| | | `-> fairly clean "encoded but not natively used" case
| | |
| | `-- yes
| | `-> the information is being used; next question is
| | where (step / subspace / task / layer)
| |
| `-- whole-trajectory transplant is null but local effects exist
| `-> localize by step or subspace rather than concluding
| that the whole latent path is inert
I would probably put the parameter-motion check first simply because you are already instrumenting it, and it is unusually cheap and decisive in this particular case.
Some related cases / terminology
One older conceptual analogue comes from emergent communication.
Lowe et al., On the Pitfalls of Measuring Emergent Communication, distinguish roughly between a sender producing messages that correlate with its state and a receiver actually changing its behavior because of those messages. They show cases where messages look informative but do not causally influence the other agent.
It is obviously a different architecture and task, but the signaling vs. listening distinction feels very close to the phenomenon you are describing:
- the latent state can vary systematically with the question;
- that alone does not establish that the downstream computation reads the example-specific content.
There is also a fairly direct connection to the probing literature.
Ravichander et al., Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?, show that information can be decodable from a representation even when it is not needed for the model’s task.
Elazar et al., Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals, push the same distinction further: probing tells you what can be extracted, while a behavioral/causal claim needs an intervention on the representation followed by a measurement of downstream behavior.
That is why I would treat
“different questions produce different vectors”
and even
“a probe can recover some question property”
as weaker claims than
“the native answer path uses that property.”
Recent work on latent reasoning makes essentially the same point in a more directly relevant setting.
Dilgren & Wiegreffe, Are Latent Reasoning Models Easily Interpretable?, report that for Coconut/CODI, latent reasoning tokens are often unnecessary for the final prediction on their logical reasoning evaluations. At the same time, in cases where the latent tokens are necessary, they can often decode the gold reasoning trace from them. I think that is a useful demonstration that “information in latent state” and “dependence on latent state” really can separate.
Aswal et al., Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models, make another useful caution: visually/structurally interesting or decodable latent patterns can appear in matched controls and need not causally affect behavior. They also find that latent use can be graded and localized in low-rank directions, rather than simply “used” or “not used.”
That last part seems important for interpreting a null whole-trajectory transplant. A null transplant is strong evidence against a large example-specific effect of the swapped representation, but it does not automatically exclude a smaller effect concentrated in one step or subspace.
There is also a very close methodological connection to Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM, which explicitly separates:
- message presence,
- message identity,
- example-specific content,
- and task value.
Your other-question transplant is already very much in that family of tests, so I would see this more as confirmation that the measurement direction is sensible than as a completely new suggestion.
Two small tests that might sharpen the interpretation
1. Probe a deliberately simple answer-relevant property
I would avoid starting with a powerful probe that could learn too much on its own.
Something low-capacity and easy to control is more informative, for example:
- answer class,
- one operand,
- relation type,
- which of two counterfactual answers is correct,
- or another simple property that is actually needed by the downstream answer.
Then use held-out examples and a label-permutation/control baseline.
The useful branching result is:
Probe near chance
The vectors may certainly be question-dependent, but there is not yet evidence that they contain the answer-relevant semantic variable being tested.
In that case I would describe them conservatively as input-dependent latent states rather than infer too much from cosine separation alone.
Probe clearly above control, but transplant still null
This would be particularly interesting: the information is accessible in the state, but the model’s own answer computation does not behave as though it uses it.
That is much closer to a clean “encoded but unused” result.
2. Use a matched / minimal-pair donor
Instead of only transplanting a completely different problem, I think a useful additional control is a donor where almost everything stays fixed except the fact that should determine the answer.
For example:
- same task family,
- same wording/template,
- preferably same token length,
- one changed operand / relation / relevant feature,
- different correct answer.
Then ask whether the target-vs-donor answer preference moves in the donor direction.
The general intervention idea is similar to Naturalistic Causal Probing for Morpho-Syntax: change the feature of interest while keeping as much of the rest of the input constant as possible.
I would regard this as a clean diagnostic rather than a benchmark. It mainly helps rule out explanations such as the donor effect being washed out by unrelated changes in task/form/length.
If the all-steps transplant remains null, the next inexpensive variant would be step-by-step transplantation before trying anything substantially more elaborate.
A small check on the older public Hyper v1 checkpoint
I also tried a small independent sanity check on the older public Hyper v1 checkpoint.
I would treat this only as a related data point — not as a reproduction of the current Monarch result.
On a small single-digit arithmetic suite, I compared the native latent sequence with several interventions.
For the answer likelihood, relative to the native/own latent:
- same-operation-family foreign latent: about +0.028 nats
- zero latent: about +1.01
- reversed own latent order: about +2.33
- no latent steps: about +4.12
So I saw essentially the same qualitative asymmetry already described in the Hyper model card: the latent computation/order matters a lot more than the identity of the problem that produced the latent.
I then made the donors more tightly matched — same arithmetic operation, same prompt form and token length, one operand changed, and a different answer.
Across those minimal pairs, transplanting the donor trajectory still did not produce a stable pull toward the donor answer. The mean change in the target-minus-donor answer preference was about -0.013, with a bootstrap interval crossing zero.
I also tried to keep the “is information present?” question separate from the causal transplant.
A small held-out low-capacity probe did not reliably decode the exact answer digit, and parity was also weak. However, a much coarser answer property — whether the answer was >= 5 — became more linearly decodable in the later latent states than in the original prompt state.
For that particular synthetic probe:
- prompt-state balanced accuracy: about
0.586
- latent step 4: about
0.710
After re-running the null with 5,000 fact-group-preserving permutations, the step-4 result still survived the permutation control (p ~= 0.0024; max-stat correction over prompt/step3/step4 p ~= 0.0044), and the step-4 improvement over the prompt state itself was also positive (+0.123, permutation p ~= 0.0126).
I would not generalize much from a tiny synthetic arithmetic probe, but I thought the combination was interesting:
- some coarse answer-related structure becomes easier to read later in the latent trajectory;
- exact answer information was not clearly readable with this simple probe;
- changing the problem-specific donor latent did not reliably steer the native answer toward the donor;
- but destroying/reversing/removing the latent path had a large effect.
To me that looks like a miniature version of the distinction above: a latent path can carry useful structure, and can even matter causally in a generic sense, without yet demonstrating that its example-specific semantic content is what the downstream answer is reading.
Again, that is Hyper v1, not the newer checkpoint.
The low-precision update issue seems like a separate axis
The weight-update finding in your negative-result write-up also seems important, but I would keep it logically separate from the representation/decoder question.
The numerical mechanism itself is well established: if low-precision resident weights are updated with nearest rounding, sufficiently small updates can round away and leave the stored weight unchanged. Zamirai et al., Revisiting BFloat16 Training, study exactly this failure mode and discuss stochastic rounding / compensated accumulation as remedies.
So the reported “optimizer produced an update, but the resident parameter did not actually move” mechanism is entirely plausible in general.
What I would not infer externally is that it has already established the complete root cause of the latent-use result. The cleanest evidence will come from the instrumentation you are already adding:
- did the intended parameter groups actually move?
- did the hidden-question own-vs-foreign gap move away from zero at the same time?
- did thought-only answers become question-relevant?
- did correctness start falling under foreign-thought substitution?
That gives a useful natural experiment.
If the relevant weights now move and those causal measurements move with them, the old write/optimization failure was probably a binding limitation.
If the relevant weights clearly move but the hidden-question transplant remains flat, then the search shifts toward the objective, data/task pressure, latent/backbone interface, or architecture rather than the storage issue.
If the weights still do not meaningfully move, I would avoid reading very much into the architecture yet.
One final point: I do not think these results imply that continuous latent reasoning in general is non-functional. The recent literature actually contains both kinds of results — systems where latent states have measurable causal value and systems/tasks where they are largely bypassed. That makes the separation above more useful than a binary “latent reasoning works / does not work” conclusion.
For this particular failure mode, the combination that would convince me most is:
parameter motion verified + answer-relevant information decodable + matched transplant changes the native answer in the expected direction.
And if the middle condition becomes true while the last one stays null, that would itself be a pretty clean and interesting result: the model learned to represent something that its own downstream answer path still did not learn to listen to.