Deep Dive Into Model Personalities

The earlier cross-family read was suggestive but unstable given how the models generate text and reason. Qwen 27B looked cleanly axis-like, while a long-generation Gemma 31B run looked like it had learned a different basis entirely. That was too quick. The new comparison controls the biggest confound directly.

To extract the geometry of the personality vectors, we now compare four matched English-only runs. For each family we take a prompt-last extraction and a response extraction capped at 256 tokens, giving us Qwen 27B prompt-last, Qwen 27B response256, Gemma 31B prompt-last, and Gemma 31B response256.

We note that the smaller models like Qwen 9B generally follow the model family geometry, possibly due to the same training regime. However, geometry can be extracted cleanly from larger models where personalities have distinct vector presence during generation.

The key logic of these runs is straightforward. If prompt-last and response256 already agree, the geometry is present before rollout. If prompt-last' is messy but response256` sharpens, the model is building the control signal during generation. And if a long 4098-token response run drifts far away from the 256 run, the long generations are smearing the extraction and model behaviour on generation thinking tokens rather than revealing a cleaner object.

Qwen 27B is a stable persona model, consistently exhibiting the same geometry.

In both capture modes the practical steering layer lands at L62/64, and the top norm layers all cluster in the late stack. The two key raw oppositions are present at prompt time, with red / green at -0.194 and blue / yellow at -0.099. They remain visible under short response extraction, where red / green comes in at -0.223 and blue / yellow at -0.107.

After subtracting the shared mean, the structure is even clearer. The residual red / green and blue / yellow cosines stay strongly negative in both modes, which means the two-axis picture is already largely formed before the model starts writing.

This is the simplest version of a model personality geometry: a late control surface that is already legible at prompt time and then preserved through rollout. This suggests a strong steerability of personality of the model during inference time with stability throughout generation.

The most important observation is that Gemma 31B is not actually learning a different colour system under the controlled 256-token protocol. Gemma and Qwen both exhibit the same geometry of personalities within the hidden layers. Both families are learning the same broad oppositions of personalities, but the Gemma personality vector space is less concentrated. This could be due to architecture or training regime.

At prompt-last, Gemma looks asymmetric: red / green sits at -0.780, blue / yellow at -0.080, and green / blue at +0.558. That is not yet the clean intended basis. But under response256, the same model reorganises into the expected pair structure, with red / green at -0.625 and blue / yellow at -0.669.

This also means that the short controlled response run changes the interpretation in the case of Gemma. The initial long-generation Gemma results at 4098 tokens were not evidence that Gemma had learned an unrelated set of personalities. It was evidence that Gemma's response-averaged geometry is much more sensitive to rollout length, possibly due to how the models reason on particular questions. For future control and model mapping, shorter extractions are better suited in production.

Qwen 27B is overwhelmingly late-stack. Its top layers are the final band from L55 onward, and both prompt-last and response extraction point to the same intervention region around L62. This is in line with the convention where models learn complex representations at the later parts of the architecture.

Gemma 31B also peaks at the final block, L59/60, but unlike Qwen it keeps a persistent secondary band around L38-42 in both prompt-last and response extraction. This means Gemma personalities start to show in the middle of the model, while the Qwen family is strongly concentrated to the last layers. This could be due to training data and/or architectural differences. In short, Qwen is mostly late and prompt-stable, and Gemma is final-layer readout plus a durable mid-stack control band. This was observed through various generation experiments and rounds with surprisingly stable results, allowing for a future control pane to be automated and/or crafted by family of models. This should also be strong grounds to predict the architectural personality vectors for upcoming models.

The evidence now supports our claim. Both Qwen 27B and Gemma 31B support the same basic two-axis colour geometry under controlled response extraction. Qwen 27B localises that geometry into a late-stack control surface that is already legible at prompt time. Gemma 31B distributes the same geometry across depth, keeps a secondary mid-stack band active, and sharpens the axis picture during rollout.

To be clear, this is not a proof of an architecture-only explanation. Training and post-training still matter and can have a huge impact on the vector distribution through the stack. But I think it is now reasonable to say that architecture affects where the control signal is organised, not only whether the control exists.

That is the story worth keeping. Qwen looks like a late, prompt-stable control surface. Gemma looks like a distributed-depth control system with a late readout and stronger generation-path dependence. The next engineering question is whether steering should target a single late layer in every family, or whether some families want multi-layer or family-specific interventions.