# LLM "curving" via prompting

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllm-curving-via-prompting%2F177166%3E%5C
**Category:** Research\
**Created:** [June 25, 2026, 11:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllm-curving-via-prompting%2F177166 "2026-06-25T11:29:42Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [June 26, 2026, 3:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllm-curving-via-prompting%2F177166%2F2 "2026-06-26T03:35:13Z")

</div>

Hmm… this is difficult, but based on what I could find so far, maybe it looks something like this:

* * *

## Short answer

I would not treat this as a yes/no question yet.

At the **output level** , the safest first interpretation is that the prompt is creating a strong **style / ontology / behavioral regime shift**. The prompt gives the model a very explicit vocabulary: “field,” “gravity well,” “self-organization,” “manifold,” “density,” “hidden dynamics,” etc. So it is not surprising that the model starts speaking and visualizing in that ontology.

But I would not yet call it evidence that the model’s internal manifold has literally been “curved,” unless the visualization is based on something like **token scores/logprobs, hidden states, or causal interventions** , rather than generated text or model self-description.

My tentative answer to the three questions would be:

1. **Manifold perturbation or stylistic variation?**  
Maybe more than ordinary style variation, but “manifold perturbation” is too strong until the measurement layer is specified. A safer current label might be **prompt-induced regime shift** or **prompt steerability**.

2. **Has anyone tried similar “field condition” prompting?**  
Yes, there are nearby prompt-only experiments on this forum using “field,” “attractor,” “semantic tension,” or “global regulator” language. I would treat those as related observations, not evidence of the same internal mechanism.

3. **How to distinguish rhetorical self-organization from something stronger?**  
I would not look for one magic “self-organization score.” I would use an **evidence profile** : output behavior → behavioral stability → token distribution → hidden-state trajectory → interdependence/order-parameter tests → causal patching/ablation/transfer.

For me, the most important missing detail is:

> What exactly are the maps computed from?

Generated text embeddings? Token scores? Hidden states? Attentions? A custom metric? A 2-D projection of some other vector space? The interpretation changes a lot depending on that answer.

* * *

## The main distinction I would make

I would separate “the prompt curves the model” into several increasingly strong claims.

| Level | What is being claimed | Example evidence |
| --- | --- | --- |
| Output style | The output sounds different | More field/manifold/self-organization language |
| Behavioral regime | The prompt reliably shifts behavior | Replication across seeds/tasks; survives style-only controls |
| Token distribution | The next-token landscape changes | Scores, logprobs, entropy, KL divergence |
| Representation | Hidden-state trajectories change | Layerwise hidden-state distances, CKA/RSA, trajectory clustering |
| Self-organization-like profile | There is stable, structured organization, not just vocabulary | Order parameters, perturbation recovery, interdependence/synergy |
| Causal mechanism | The effect can be moved or removed | Activation patching, ablation, activation-difference transfer |

The first two levels are still interesting. They just do not yet justify the stronger internal-mechanism language.

A useful existing term for the second level is **prompt steerability** : how much prompting alone can shift a model’s behavioral distribution away from baseline. See, for example:

- [Evaluating the Prompt Steerability of Large Language Models](https://arxiv.org/abs/2411.12405)
- [ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs](https://arxiv.org/abs/2410.12405)
- [PromptRobust](https://arxiv.org/abs/2306.04528)
- [PromptBench](https://github.com/microsoftarchive/promptbench)

So I would initially describe the current observation as:

> This prompt may induce a distinctive behavioral regime. Whether that regime corresponds to a deeper representation-level or causal mechanism is testable, but not yet shown by the wording of the output alone.

* * *

## What I would ask first

Before interpreting the maps, I would ask:

> What exactly is being plotted?

| If the map is computed from… | Safer interpretation | Unsafe interpretation |
| --- | --- | --- |
| Generated text embeddings | Output-space shift | Internal manifold perturbation |
| Prompt embeddings | Input-space / prompt-space difference | Model dynamics changed |
| Token scores / logprobs | Next-token distribution changed | Causal mechanism found |
| Hidden states | Representation trajectory changed | The difference caused the behavior |
| Attention patterns | Attention allocation changed | The whole model entered a new field state |
| Custom metrics | Difference under that metric | Internal geometry proved |
| UMAP/t-SNE/PCA projection | Exploratory visualization | Literal gravity wells / basin sizes |

For open-weight Hugging Face models, some of these layers are accessible. Relevant docs:

- [Transformers generation utilities](https://huggingface.co/docs/transformers/en/internal/generation_utils)
- [Text generation API](https://huggingface.co/docs/transformers/main_classes/text_generation)
- [Model outputs / hidden states](https://huggingface.co/docs/transformers/en/main_classes/output)

There is also a practical gotcha: hidden states from `model.generate()` can be confusing because prompt-token states and generated-token states are not always obtained in the same way. This HF Forum thread is relevant:

- [The hidden states when I use model.generate()](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fthe-hidden-states-when-i-use-model-generate%2F73169)

So I would ask:

> Are the maps based on generated text, scores/logprobs, hidden states from a specific layer/token position, attentions, or a custom post-processing metric?

That question is not a criticism. It determines what kind of claim the map can support.

> **Related HF Forum context**
>
> I found several nearby forum discussions. I would not treat them as the same mechanism, but they show that this “field / attractor / regulator” vocabulary is not isolated.
> 
> ### Field Tension Lens / Strong Attractors
> 
> This thread uses “Field Tension Lens” and “strong attractor” language to push the model toward deeper analogical structure:
> 
> - [An Example of Using Strong Attractors to Boost LLM Analogical Thinking](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fan-example-of-using-strong-attractors-to-boost-llm-analogical-thinking%2F160279)
> 
> This feels close to your “field condition” framing. But I would still classify it as a related **prompt-only behavioral observation** , not direct evidence of internal manifold dynamics.
> 
> ### Self-Stablize / Hetu-Luoshu Kernel
> 
> This one uses a similar vocabulary: global manifold, collapse trace, curvature, attractors, semantic vectors, etc.
> 
> - [Self-Stablize Hetu-Luoshu Kernel / Strong Attractor Prompt Use Case](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fself-stablize-hetu-luoshu-kernel-strong-attractor-prompt-use-case%2F167211)
> 
> Again, related vocabulary and maybe related prompting style, but not enough to infer the same mechanism.
> 
> ### Axiomatic prompts as global regulators
> 
> This thread is especially useful because it asks how to separate semantic attractor effects from prompt length, few-shot conditioning, or other simpler explanations:
> 
> - [Can Axiomatic Prompts Act as Global Regulators of LLM Trajectories?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcan-axiomatic-prompts-act-as-global-regulators-of-llm-trajectories%2F173740)
> 
> That is very close to the kind of control design needed here.
> 
> ### Hidden-state analysis attempts
> 
> There are also forum posts trying to look at hidden-state signals directly, for example layerwise cosine similarity in iterative repair loops:
> 
> - [Hidden-State Signals in Iterative LLM Repair](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhidden-state-signals-in-iterative-llm-repair-what-cosine-similarity-at-layer-27-actually-tells-you%2F174143)
> 
> This is not the same phenomenon, but it points toward the kind of measurement detail that would make this discussion easier to evaluate.

* * *

## Rhetorical self-organization vs stronger evidence

I do not know a single accepted metric for “genuine self-organization simulation” in LLM outputs.

But adjacent fields do have recurring measurement families. I would treat them as an **evidence profile** , not one score.

| Measurement family | What it asks | Prompt-experiment version |
| --- | --- | --- |
| Order parameters | Is there a low-dimensional collective variable? | What variable summarizes the induced regime? |
| Stability / recovery | Does the system return to a regime after perturbation? | Seeds, paraphrases, distractors, multi-turn recovery |
| Entropy / complexity / homeostasis | Is there a balance of order and variability? | Token entropy, output entropy, complexity profile |
| Fingerprint metrics | Is there randomness, pattern, and interdependence? | Output variance, repeated structure, mutual information |
| Interdependence / synergy | Are parts becoming structurally coupled? | MI/PID among output features or hidden components |
| Information dynamics | Is information stored/transferred over time? | Prompt influence across generation steps |
| Multiscale profile | At what scale does organization appear? | Token, sentence, response, layer, seed ensemble |
| Topological structure | Are there persistent geometric features? | TDA on hidden-state or output-embedding clouds |
| Causal intervention | Can the effect be moved or removed? | Patching, ablation, activation-difference transfer |

Translated to this experiment, I would draw the boundary like this:

| Category | What it would look like |
| --- | --- |
| Rhetorical self-organization | The output uses self-organization / field / attractor vocabulary |
| Behavioral self-organization-like effect | The regime replicates across seeds/tasks and survives style-only controls |
| Distribution-level evidence | Scores, entropy, or logprob profiles shift relative to controls |
| Representation-level evidence | Hidden-state trajectories shift relative to controls |
| Order-parameter-like evidence | A low-dimensional variable predicts many output/hidden-state features |
| Interdependence evidence | Output or hidden-state components become more mutually dependent/synergistic |
| Causal evidence | Patching, ablation, or activation-difference transfer moves or removes the effect |

So for your third question, I would not ask:

> Is it genuine self-organization, yes or no?

I would ask:

> How far up this ladder does the effect survive?

> **Background links for self-organization measurement families**
>
> Some useful background links from adjacent fields:
> 
> - Haken / synergetics: order parameters and the slaving principle are central concepts in self-organization. See [Information and Self-Organization](https://www.mdpi.com/1099-4300/19/1/18).
> - Gershenson & Fernández propose information-theoretic measures of emergence, self-organization, complexity, and homeostasis: [Complexity and Information](https://arxiv.org/abs/1205.2026).
> - Rosas et al. frame self-organisation through interdependencies, redundancy, synergy, and global structural strength: [An Information-Theoretic Approach to Self-Organisation](https://www.mdpi.com/1099-4300/20/10/793).
> - Swarm/ALife work often uses a “fingerprint” rather than one number: randomness, pattern, and interdependence. See [Complexity Measures in Automatic Design of Robot Swarms](https://iridia.ulb.ac.be/~mbiro/paperi/RolLigBir2018wivace.pdf).
> - Persistent homology and topological data analysis can sometimes provide more defensible geometric summaries than visually reading a 2-D projection. Example: [Quantitative and Interpretable Order Parameters for Phase Transitions from Persistent Homology](https://arxiv.org/abs/2009.14231).
> 
> These are not direct LLM prompt-evaluation standards. I would use them as adjacent measurement families, not as proof that the prompt has produced genuine self-organization.

* * *

## Controls that would make the result easier to interpret

I would add controls before interpreting the effect as internal geometry.

| Control | Why it matters |
| --- | --- |
| Baseline prompt | What does the model do normally? |
| Curving / field prompt | The proposed effect |
| Physics-metaphor-only prompt | Tests whether the effect is just physics vocabulary |
| Self-monitoring-only prompt | Tests whether the effect is generic reflective prompting |
| Same-length abstract prompt | Controls for length and abstraction |
| Shuffled-label prompt | Tests whether labels like “self-assembly” and “self-organization” matter |
| No-visual-language prompt | Tests whether the visual map is induced by visualization instructions |
| Multiple seeds | Tests whether the regime is stable |
| Greedy + sampling | Separates deterministic prompt effect from sampling noise |

Example interpretation:

| Result | Safer interpretation |
| --- | --- |
| Curving prompt and physics-only prompt look similar | Style / ontology priming is likely |
| Curving prompt differs from all style controls across seeds | Behavioral regime shift becomes more plausible |
| Token score / entropy profiles differ from controls | Distribution-level claim becomes stronger |
| Hidden-state trajectories differ from controls | Representation-level claim becomes stronger |
| Patching transfers or removes the effect | Causal claim becomes more plausible |

Controls are not just for disproving the idea. They make the interesting part more precise.

* * *

## Small practical next step

If I were trying to make this easy for others to evaluate, I would create a minimal reproducibility package:

1. Pick one open-weight model.
2. Save exact model revision and tokenizer revision.
3. Save the serialized chat-template output.
4. Use baseline / curving / physics-only / same-length abstract controls.
5. Fix generation settings.
6. Run multiple seeds.
7. Save generated outputs.
8. Save token scores / entropy profiles if available.
9. Save hidden-state extraction method if used.
10. Report which layer/token positions are plotted.
11. Save projection method and parameters.
12. If possible, add one small activation-patching or activation-transfer experiment.

The claim can then be stated in tiers:

| Evidence obtained | Claim strength |
| --- | --- |
| Output changes only | Rhetorical/style regime |
| Replicates across seeds and controls | Behavioral regime shift |
| Scores/logprobs differ | Token-distribution shift |
| Hidden states differ | Representation-level shift |
| Order parameter / interdependence appears | Self-organization-like profile |
| Patching/ablation/transfer works | Causal evidence candidate |

> **Implementation details that can easily confound the result**
>
> For Hugging Face chat models, “same prompt” can be ambiguous. The human-readable prompt is not necessarily the exact model input.
> 
> The chat template matters. HF docs explain that chat templates serialize chat messages into the exact token sequence expected by a model, and details like `add_generation_prompt` and special tokens can matter:
> 
> - [Hugging Face chat templates](https://huggingface.co/docs/transformers/en/chat_templating)
> 
> For reproducibility, I would report:
> 
> - exact model name and revision
> - tokenizer revision
> - chat template output / serialized prompt
> - system/user/assistant role formatting
> - `add_generation_prompt` setting
> - special token handling
> - temperature
> - top\_p / top\_k
> - max\_new\_tokens
> - seed
> - greedy vs sampling
> - dtype / quantization
> - hardware if relevant
> - whether hidden states are from prompt tokens, generated tokens, or both
> - which layer and token positions are used
> 
> This is especially important if comparing many models. Gemini / Claude / ChatGPT / Grok / Copilot-style systems are mostly output-level from the outside. Open-weight HF models allow stronger tests because you can inspect scores and hidden states.

> **A staged open-model test**
>
> If open weights are available, I would test this in stages.
> 
> ### Stage 1: output-level test
> 
> Run:
> 
> - baseline prompt
> - curving prompt
> - physics-metaphor-only control
> - self-monitoring-only control
> - same-length abstract control
> 
> Fix decoding settings and run multiple seeds.
> 
> Measure simple output features:
> 
> - length
> - self-reference rate
> - field/manifold vocabulary rate
> - metaphor rate
> - structural markers
> - semantic clustering
> - human-readable examples
> 
> This tests whether the effect is just rhetorical or behaviorally stable.
> 
> ### Stage 2: token-distribution test
> 
> Use generation scores:
> 
> - token scores
> - transition scores
> - entropy profile
> - logprob profile
> - KL divergence between prompt conditions
> 
> Relevant docs:
> 
> - [Transformers generation utilities](https://huggingface.co/docs/transformers/en/internal/generation_utils)
> - [Text generation API](https://huggingface.co/docs/transformers/main_classes/text_generation)
> 
> This tests whether the next-token distribution changes in a stable way.
> 
> ### Stage 3: representation-level test
> 
> Capture hidden states and compare:
> 
> - layerwise cosine distance
> - token-position differences
> - generation-step trajectories
> - CKA / RSA if useful
> - clustering of hidden-state trajectories
> - comparison against style-only controls
> 
> Relevant docs:
> 
> - [Model outputs / hidden states](https://huggingface.co/docs/transformers/en/main_classes/output)
> 
> This tests whether the prompt induces a representation-level shift.
> 
> ### Stage 4: causal test
> 
> Use activation patching or activation-difference transfer.
> 
> Possible setup:
> 
> - clean run = curving prompt
> - corrupted/control run = baseline or style-only prompt
> - patch curving activations into baseline run
> - patch baseline activations into curving run
> - see whether the effect appears, weakens, or disappears
> 
> Useful tools:
> 
> - [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens)
> - [TransformerLens patching module](https://transformerlensorg.github.io/TransformerLens/generated/code/transformer_lens.patching.html)
> - [nnsight activation patching tutorial](https://nnsight.net/tutorials/tutorials/causal_mediation_analysis/activation_patching/)
> 
> Important caveat: activation patching is stronger evidence, but not magic proof. Metrics and corruption methods matter. See [Towards Best Practices of Activation Patching in Language Models](https://arxiv.org/abs/2309.16042).

> **Relation to activation steering**
>
> Activation steering is a very useful comparison point, but I would not treat it as proof of prompt-only curving.
> 
> Relevant work:
> 
> - [Activation Addition / ActAdd](https://arxiv.org/abs/2308.10248): computes activation differences from prompt pairs and adds them during inference.
> - [Contrastive Activation Addition](https://aclanthology.org/2024.acl-long.828/): uses positive/negative examples to derive steering directions.
> - [IBM activation-steering](https://github.com/ibm/activation-steering): tools for extracting and applying steering vectors from contrastive examples.
> 
> A useful bridge experiment would be:
> 
> 1. Run baseline prompt.
> 2. Run curving prompt.
> 3. Extract baseline-vs-curving activation difference at selected layers/token positions.
> 4. Add that difference to a neutral prompt.
> 5. See whether curving-like behavior transfers.
> 6. Compare against a physics-metaphor-only vector.
> 7. Try reverse patching or ablation.
> 
> But I would keep the distinction clear:
> 
> | Method | What it shows |
> | --- | --- |
> | Prompting | Text input changes behavior |
> | Activation steering | Internal activation intervention changes behavior |
> | Activation transfer from prompt A to prompt B | Stronger evidence that the prompt created a reusable internal direction |
> | Patching/ablation | Stronger evidence about where the effect is mediated |
> 
> There is also recent work cautioning that white-box activation states may not be reachable by any textual prompt:
> 
> - [Steered LLM Activations are Non-Surjective](https://arxiv.org/abs/2604.09839)
> 
> So I would say:
> 
> > Activation steering is a useful comparison, but it is not direct evidence that a text prompt reaches the same internal state.

> **Visualization caveat**
>
> The maps are useful, but I would be careful about reading geometric language too literally.
> 
> If the plots use PCA, UMAP, t-SNE, or similar 2-D projections, distances, density, cluster size, and basin shape can be misleading.
> 
> Useful references:
> 
> - [How to Use t-SNE Effectively](https://distill.pub/2016/misread-tsne)
> - [Understanding UMAP](https://pair-code.github.io/understanding-umap/)
> - [UMAP documentation](https://umap-learn.readthedocs.io/en/latest/how_umap_works.html)
> 
> For example, UMAP/t-SNE plots can be useful for hypothesis generation, but I would avoid claims like:
> 
> > This 2-D map proves a gravity well.
> 
> A safer version would be:
> 
> > Under this embedding/projection method, outputs from the curving prompt cluster differently from baseline/control outputs.
> 
> If the goal is really geometric, topological data analysis might eventually be more defensible than visually reading a 2-D plot. Persistent homology has been used to identify phase transitions and order parameters in physical systems:
> 
> - [Quantitative and Interpretable Order Parameters for Phase Transitions from Persistent Homology](https://arxiv.org/abs/2009.14231)
> 
> But again, the input vectors determine the meaning. Hidden-state point clouds and generated-text embedding clouds are not the same object.

> **Model self-description is output data, not mechanism evidence**
>
> I would also separate the model’s self-description from the mechanism.
> 
> If the prompt gives the model a “field / self-organization / gravity well” ontology, the model may fluently describe itself in that ontology. That does not mean the description is a faithful report of the internal mechanism.
> 
> Relevant cautionary background:
> 
> - [Language Models Don’t Always Say What They Think](https://arxiv.org/abs/2305.04388)
> - [Reasoning Models Don’t Always Say What They Think](https://www.anthropic.com/research/reasoning-models-dont-say-think)
> - [Tracing the thoughts of a large language model](https://www.anthropic.com/research/tracing-thoughts-language-model)
> 
> So I would treat self-description as part of the output, not as a mechanism probe.
> 
> A concise rule:
> 
> > The model talking about self-organization is rhetorical evidence. The model showing stable, measurable, control-resistant, and causally traceable organization would be stronger evidence.

> **Terms I would use / avoid**
>
> ### Safer terms
> 
> | Term | Why |
> | --- | --- |
> | prompt-induced regime shift | Describes behavior without overclaiming mechanism |
> | prompt steerability | Connects to existing evaluation vocabulary |
> | behavioral distribution shift | Measurable across seeds/tasks |
> | token-distribution shift | Measurable with scores/logprobs |
> | hidden-state trajectory shift | Measurable in open models |
> | representation-level difference | Safer than “manifold curved” |
> | order-parameter candidate | Makes “gravity well” more testable |
> | evidence profile | Avoids one fake self-organization score |
> | activation-level causal test | Clearer than “the model self-organized” |
> 
> ### Terms I would avoid unless demonstrated
> 
> | Avoid | Why |
> | --- | --- |
> | The model left next-token prediction | Too strong / misleading |
> | The weights changed | Usually false during inference |
> | The manifold was literally curved | Needs measurement layer and controls |
> | Genuine self-organization was proven | Too strong from output alone |
> | The model self-reported its mechanism | Self-report is not mechanism evidence |
> | UMAP/t-SNE showed a gravity well | Projection can distort geometry |
> | ActAdd proves prompt curving | White-box activation steering is not prompt-only evidence |

* * *

## My current position

This looks like an interesting **prompt-induced regime shift**. It may become a stronger **representation-level** or **self-organization-like** claim if the maps are based on well-specified token/hidden-state data, survive controls, and ideally show some causal transfer or ablation behavior.

Until then, I would keep “manifold curving” as a metaphor or hypothesis, not as the measured conclusion.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllm-curving-via-prompting%2F177166)._