The exact-search caveat is the result, and I think it belongs in the bullets rather than the limits.

Your table is careful enough that this falls straight out of it. scifact, your numbers, exact Faiss:

  MUVERA 10,240   0.50021
  MUVERA  2,048   0.36757
  LEMUR   2,048   0.54910     +49.4% matched budget, +9.8% vs full width

Both of those reproduce off your table to the digit, so I am reading it the way you meant it.

Then the IVF line, on the same dataset, so nothing needs extrapolating: default IVF costs LEMUR 43% and MUVERA 25%.

  LEMUR   2,048   0.54910 -> 0.31299
  MUVERA  2,048   0.36757 -> 0.27568      LEMUR +13.5%
  MUVERA 10,240   0.50021 -> 0.37516      LEMUR -16.6%

The 49% becomes 14%. Against the full-width vector it goes negative.

What makes that more than a footnote is where your own threshold sits. txtai does exact search through 5,000 rows and switches to IVF above it. Five thousand rows is also about where saving 5x on storage starts being a reason to do anything at all. So the regime that motivates a fixed-vector encoding is the same regime the default index gives it back in. Someone who reads the post, indexes 200k documents and takes the default will measure the opposite of the headline.

You do say to pin faiss.components to IDMap,Flat. I would promote that out of the limits section, because it is load-bearing.

One presentational thing, since the post is what gets quoted and the blog is the part that is honest: the two bullets have different baselines. The 49 to 62% is against matched-budget 2,048 MUVERA, the 5x storage is against the 10,240 default, and against that same 10,240 the quality gain is 8.4 to 22.9%. No single configuration is both 49% better and 5x smaller.

The actual question, though, is about the geometry, because I think your two findings may be the same finding.

Mean centering went in because LateOn token vectors were anisotropic enough that the encoder had little useful variation left to preserve. IVF is a clustering index. It cares about exactly that property. LEMUR documents are OLS weights over a stored sample, which is a very different distribution from MUVERA's concatenated projections, and it would not surprise me if it clusters worse for the same underlying reason.

Does centering narrow the 43%, or is the IVF penalty independent of it? And which MUVERA width is the 25% measured on? Under the 10,240 reading the ordering inverts, under 2,048 it only compresses, and those are quite different recommendations.

Your geometry probe is not failing to explain the gap. It is the explanation, and it works through
the budget rather than through clustering quality.

nprobe is a cell budget. What decides recall is a document budget. Those are the same number only
when the cells are equal, and your own CV says they are not.

Your defaults reproduce from the source, exactly

# txtai/ann/dense/faiss.py
cells(count)  = max(min(round(4*sqrt(count)), int(count/39)), 1)
nprobe()      = 6 if count <= 5000 else round(cells(count)/16)
create()      = index_factory(d, "IVF{cells},Flat", METRIC_INNER_PRODUCT)

At 5,183: min(288, 132) = 132, and round(132/16) = 8. Both of your stated defaults, on the nose.
Worth noting you are 183 rows past the count <= 5000 step, which is what moved nprobe 6 -> 8.

The measurement

I built your exact index config (IVF132,Flat, inner product, n=5183, d=2048) on synthetic corpora,
with the arms calibrated to hit the mean pairwise cosine you reported, then read the scanned mass
off faiss's own counter, faiss.cvar.indexIVF_stats.ndis.

arm                        mean cos    CV      ndis/query   % of corpus scanned
LEMUR-like                    0.050   0.574         617.4        11.91%
MUVERA-like                   0.230   1.282        1379.2        26.61%
                                            nprobe/nlist = 8/132 =  6.06%   <- what the knob claims

Identical nprobe=8. The skewed arm scans 2.23x more of the corpus.

The instrument checks itself. On a fully isotropic arm (cos 0.000, CV 0.155) the counter returns
314.5 distances, 6.07% of the corpus, against an advertised 6.06%. So nprobe/nlist is the true
scanned fraction exactly when the cells are balanced, and only then.

First-order size bias gets the direction and most of the size: a query lands in cell i with
probability n_i/N, so the expected cell is m(1+CV^2), giving 1.99x against the measured 2.23x. The
residual is that the big cells sit together near the shared mean, so the extra 7 probes are
correlated with the first.

What that does to the default-IVF column

It was never a matched-budget comparison. The encoding with the flatter cluster histogram honours
nprobe=8 literally; the skewed one quietly draws about twice the documents for the same knob.
LEMUR is not degrading more under IVF. It is being scored at less than half the scan.

To match the MUVERA-like arm's scanned mass, the LEMUR-like arm needs nprobe around 20, not 8.

Log-interpolating your curve (nprobe 8 -> 0.32346, nprobe 64 -> 0.5085) to nprobe=20:

nprobe=12   gap 34.5%   0.3595
nprobe=20   gap 26.3%   0.4050     <- matched scanned mass

0.4050 against MUVERA-10240's default-IVF 0.37341, at one fifth the storage. That is crude, two
points and a log, and it is your data not mine. But it says the ordering may not invert at all.

The one line that settles it

faiss.cvar.indexIVF_stats.reset()
index.search(queries, k)
print(faiss.cvar.indexIVF_stats.ndis / len(queries))

Equalise ndis, not nprobe, and re-run the four cells. If the LEMUR penalty survives at matched
ndis, the routing story is dead and it really is representation.

Centering, and the leg that would kill this

Under this read centering is a pure routing effect, which is exactly why your exact column stayed
flat while IVF moved 11.4 points. So it has to show up in the cluster histogram:

• centering MUVERA should lower its CV, cutting its free scan (0.37341 -> 0.31308, matches)
• centering LEMUR should raise its CV, buying scan (-41.1% -> -29.7%, matches in direction)

That second one is the weak leg. LEMUR is already near-isotropic at cos 0.05, so there is not much
common component to remove. If you re-run the k-means CV before and after centering and LEMUR's CV
does not move up, my explanation is wrong and yours is right.

One thing on the exact column: 0.54910 -> 0.5484 is flat to 0.13%, not identical. Centering documents
only shifts every score by a per-query constant and should leave nDCG bit-identical. So either
queries were centered too, or that 0.0007 is tie-breaking. Which was it?

Two small ones

The not about 14% should be about 17%. 0.32346/0.27568 = 1.173. The 14.3% is a different ratio,
0.27568/0.24111, which is how much the extrapolation overshot MUVERA-2048. Both readings support
your point that it was optimistic, so nothing downstream moves.

And nprobe=64 is corpus-specific. txtai pins the scan fraction at 1/16 in both regimes, since
cells switches from n/39 to 4*sqrt(n) at n=24,336 and nprobe tracks it:

n=     5183   nlist=  132   nprobe=   8   6.06%
n=    24336   nlist=  624   nprobe=  39   6.25%
n=  1000000   nlist= 4000   nprobe= 250   6.25%

So 64 is nlist/2, an 8x cost, and it stays 8x at a million rows. Publishing it as a fraction
travels; publishing the integer does not.

Last thing, and it is the storage argument rather than a critique: neither MUVERA width leaves the
298-444 participation band. The extra 8,192 dimensions bought no additional spread, while LEMUR-2048
reaches 692 out of 2,048. That is a model-free version of your headline and it does not depend on
which index anyone picked.

Does LEMUR's cluster-size CV go up when you center it?

@Morgan-coded suggested this update to the article for the passage beginning withtxtai's Faiss backend uses exact search through 5,000 rows to clarify what's being discussed here.

txtai's Faiss backend uses exact search through 5,000 rows and switches to IVF above that threshold. At 5,183 rows on scifact, it selects `IVF132,Flat` with `nprobe=8`. Default IVF reduced NDCG at 10 by 41% for LEMUR-2048 (0.54910 → 0.32346), 34% for MUVERA-2048 (0.36757 → 0.24111), and 25% for MUVERA-10240 (0.50021 → 0.37341).

Raising `nprobe` to 64 recovered LEMUR-2048 to 0.5085, 7.4% below exact while remaining approximate. Tuned against tuned, LEMUR-2048 at 0.5085 still edged MUVERA-10240 at 0.48939 with one-fifth the storage. Collection-mean centering narrowed LEMUR's IVF gap from 41% to 30%, but hurt MUVERA under IVF, so it is not a general substitute for index tuning. These are single-dataset scifact/ColBERTv2 numbers from the same harness as the article's tables. Pin `faiss.components` to `IDMap,Flat` when exact search is practical; otherwise raise `nprobe` rather than accepting the default.

The residual is not representation quality. It is the participation ratio you already published, and it is the same number that makes LEMUR win the exact column.

Concession first: cell-size skew is dead as the main story. Your matched-mass run settles it, 9 not 20, and m(1+CV^2) overstated the real magnitude exactly the way my synthetic did.

The dose-response

Same rig, n=5183, IVF132,Flat, inner product, 1109 queries, 3 seeds. Calibrated to the numbers you measured this time, not the ones I guessed. Arms are unit vectors with a power-law covariance spectrum tuned to a target participation ratio, plus a common direction tuned to a target mean pairwise cosine. Metric is recall@10 of the probe set against exact, at matched scanned mass near 420 ndis/query.

cos held at 0.05, PR swept    cellCV   ndis/q   recall@10
PR=200                         0.991    431.1      63.8%
PR=300                         0.944    423.2      59.0%
PR=440                         0.888    417.0      52.7%
PR=692                         0.797    426.2      44.2%
PR=1000                        0.672    429.6      33.8%

PR 200 to 1000 costs 30 points of recall at constant mass. The CV column runs the other way: the arm that loses most has the flattest histogram.

The control says CV is not doing the work:

PR held at 692, cos swept     cellCV   ndis/q   recall@10
cos=0.00                       0.693    420.3      45.1%
cos=0.05                       0.797    426.2      44.2%
cos=0.23                       0.956    418.2      41.4%

CV moves 0.69 to 0.96 and recall moves 3.7 points. PR moves and it moves 30.

It lands on your number

Your two arms at their reported geometry, matched mass:

MUVERA-like  PR 370  cos 0.23    recall@10  54.5%
LEMUR-like   PR 692  cos 0.05    recall@10  44.2%    ratio 0.811

Your measured retention is 0.33656/0.54910 = 0.6129 against 0.37341/0.50021 = 0.7465. Ratio 0.821.

A model that knows nothing about either encoder except two geometry statistics you published reproduces your retention ratio to within a point. Single dataset, synthetic corpora, so not proof. But the residual has a name.

Two things I got wrong, so they do not cost you a run

I proposed neighbour contrast as the mechanism. It is flat: 2.79 at PR=200, 2.90 at PR=1000. Not that.

What moves is where the neighbours sit. Distinct cells holding a query's exact top-10 goes 6.27 to 8.83 out of 10 across the sweep, and the fraction in the query's nearest cell goes 23.1% to 9.7%.

I also expected a graph index to erase this, since HNSW does not partition. It does not:

matched-ish budget      ndis/q   recall@10   gap
MUVERA-like IVF132       433.1      54.5%
LEMUR-like  IVF132       426.2      44.2%    10.3
MUVERA-like HNSW32       536.4      71.4%
LEMUR-like  HNSW32       592.1      57.8%    13.7

LEMUR-like got the larger budget there and still lost by more. So this is not an IVF artifact. It is effective dimensionality making approximate search harder in general, which means pinning IDMap,Flat stays right, and it is right because it is exact, not because it is a better approximation.

The cheap check on real data

You have both indexes and the exact results already. Per query, count distinct cells holding its exact top-10, and the fraction in the query's nearest cell. No new index, no re-encode. I predict roughly 8/10 against 7/10, and about 14% against 20%. That is your query-centroid margin idea one step cheaper.

Why it matters for the article

Price of the fix, from the same sweep: LEMUR-like needs nprobe=16 to reach MUVERA-like's nprobe=8 recall. Twice the knob, 1.59x the scanned mass.

The uncomfortable part is that the storage win and the ANN penalty are the same property. 692 of 2048 dimensions carrying signal is why 2048-wide LEMUR beats 10240-wide MUVERA under exact search, and it is why the neighbours scatter under any partition. You do not get to bank both.

Does the limits section want that sentence?

Would you be able to succinctly summarize what you're trying to say in a single sentence or paragraph? It's hard to follow.

EDIT: I think what you're saying is that mean centering doesn't work with FAISS IVF. Do you think the same holds true for HNSW etc?

Your two statistics are one curve read at two points, and recall is that curve. Not a correlate of it.

For IVF-Flat with an exact scan inside probed cells, a true top-k neighbour is returned if and only if its cell is among the nprobe nearest centroids. It cannot be crowded out, because it already beats everything in the corpus. So

recall@nprobe(q) = |{i in exact top-k(q) : cellrank_q(cell(i)) <= nprobe}| / k

where cellrank_q orders cells by query-to-centroid distance. Exact, not approximate. I checked it against a real IVF scan rather than trusting the argument, 3 synthetic arms, nlist=132, k=10:

arm             nprobe   IVF recall   CDF(nprobe)   abs diff
LEMUR-like           1       0.0560        0.0560   0.00e+00
LEMUR-like           4       0.1630        0.1630   0.00e+00
LEMUR-like           8       0.2727        0.2727   0.00e+00
MUVERA-10240         1       0.0507        0.0507   0.00e+00
MUVERA-10240         4       0.1793        0.1793   0.00e+00
MUVERA-10240         8       0.2983        0.2983   0.00e+00
MUVERA-2048          1       0.0940        0.0940   0.00e+00
MUVERA-2048          4       0.2813        0.2813   0.00e+00
MUVERA-2048          8       0.4333        0.4333   0.00e+00

Which means your nearest-cell fraction is not a proxy for recall. It is recall at nprobe=1. And distinct-cell count is a lossy summary of the same rank multiset, one that throws away the ordering, which is the only part that decides retrieval.

The MUVERA-2048 cap is a CDF crossing, and it is already in your numbers

Read your table as two points on each arm's curve:

arm             CDF(1) = nearest   CDF(p) = recall   gain over ranks 2..p
LEMUR-unc            0.235              0.600              0.365
MUVERA-2048          0.215              0.649              0.434
MUVERA-10240         0.272              0.707              0.435

MUVERA-2048 starts below LEMUR-unc at rank 1 and finishes above it. The curves cross. That is what "scatter tolerant" is: nothing tolerates anything, the neighbours simply sit at ranks 2 to p instead of rank 1, and the probe budget collects them either way.

The part I did not expect: both MUVERA arms recover the same mass over ranks 2..p, 0.434 against 0.435. One point apart in a thousand. They differ only at rank 1, where the 10240 arm is ahead by 5.7 points. Meanwhile LEMUR-unc is not worst at rank 1 at all, it beats MUVERA-2048 there, and loses 0.07 in the tail.

So the question changes shape. Not "what makes MUVERA-2048 tolerant of scatter". Rather: why do two encoders at 2048 and 10240 have identical rank-2-to-p recovery, and why is LEMUR's deficit entirely in that stretch and not at the head.

This assumes the three recall figures are at one nprobe. If they are not, the third column is wrong and I would want the per-arm nprobe.

Your r = 0.52 to 0.63 needs a null

Nearest-cell fraction correlating with per-query IVF recall is corr(CDF_q(1), CDF_q(p)). Two points on the same per-query curve are correlated before any mechanism exists. So I measured what the identity alone produces, three quite different geometries, 1109 queries each:

geometry              nprobe   r(CDF1, CDFp)   r(distinct, CDFp)
power 0.6 + clumps         8           0.436              -0.608
power 0.6 + clumps        16           0.306              -0.502
power 0.9 isotropic        8           0.440              -0.686
power 0.9 isotropic       16           0.312              -0.558
power 0.3 isotropic        8           0.434              -0.337
power 0.3 isotropic       16           0.327              -0.295

0.31 to 0.44, stable across geometries that differ wildly in effective dimensionality. Your 0.52 to 0.63 is above that, so there is signal, but the mechanism-free floor is most of the way there.

The other column is the more useful one. Distinct-cell count is the stronger per-query correlate in every arm I ran, and it is negative, as it should be. You reported the weaker of your two statistics as the predictive one.

Where I was wrong

I tried to build a synthetic pair reproducing your crossing, more scatter and lower nearest-cell fraction but better recall. I got one at seed 11, margin 0.006. It reversed sign at two of three seeds. So I have no synthetic counterexample to offer, and the crossing in your data is the only measured one. Reporting the failed attempt because a 0.006 margin at one seed is exactly the kind of thing I would want flagged if I were reading it.

The cheap version of every sweep you have left

Record the rank, not the count. One pass over the exact top-10 gives a histogram over ranks 1..nlist, and the CDF of that histogram is the recall curve at every nprobe at once. nprobe 8, 9, 16 and 20 all fall out of the same pass, no search re-run, no re-encode. You already compute query-to-centroid distances at search time, so the rank is free.

It also prices centering directly. Centered LEMUR passing MUVERA-10240's default at nprobe=9 and uncentered at nprobe=20 are two crossings of one horizontal line, and the gap between the curves is the scan surcharge, in probes, per arm.

One caveat: this is IVF-Flat only. If anything reranks over quantized codes the equality becomes an upper bound, which is another argument for the IDMap,Flat pin.

If the rank histogram shows both MUVERA arms sharing a tail shape that LEMUR does not, is that still a property of the encoder, or of where kmeans put the centroids for each one?

Your question sent me back to my own null, and it retired the answer I had ready.

I had a one-parameter story. Rank-2..8 mass is near-invariant to ambient width and moves with the encoder's intrinsic dimension, so LEMUR sits at a different intrinsic d and both MUVERA widths sit at the same one. Here is the dense sweep behind it. D=2048, ranks read off the exact top-10, tail is ranks 2..8 at nprobe=8, same convention as your figures.

 d     head     tail   head+tail   TwoNN
 3   0.7335   0.2665     1.0000     2.90
 4   0.6055   0.3942     0.9997     3.89
 6   0.5025   0.4783     0.9808     6.05
 8   0.3800   0.5420     0.9220     8.03
12   0.2713   0.5350     0.8063    11.96
16   0.2245   0.4915     0.7160    14.08
24   0.1625   0.4113     0.5737    19.93
48   0.0985   0.3485     0.4470    30.62
96   0.0938   0.3470     0.4408    47.45

Head falls monotonically in d. head+tail falls monotonically in d. The tail alone is non-monotonic and peaks near d=8, which is exactly why a tail deficit on its own is two-sided and settles nothing.

Now put your LEMUR-2048 against your MUVERA-2048. Head up, 0.2349 to 0.2151. Tail down, 0.3647 to 0.4343. So total down, 0.5996 to 0.6494.

Total down means higher d. Head up means lower d. I checked all 182 ordered pairs on that curve for the triple. Zero of them. There is no intrinsic dimension you can hand LEMUR that produces what you measured.

So my answer is wrong, and the useful part is which leg breaks.

The tail leg is solid. You already showed it survives five centroid retrains, 0.0814 against a 0.0444 seed range, and my null agrees the tail is where partition noise is smallest.

The head leg is the one I do not believe yet. Your head gap is 0.0198 on a single production draw. Re-seeding the coarse quantizer in my null moves the head by 0.0190 at d=8 and 0.0205 at d=24 over five partitions. Your gap is the same size as the noise.

One caveat on reading any of this across. My levels sit far above yours because N and nlist differ, so only the monotonicities transfer, not the numbers.

Which makes the deciding test one you have already paid for. You ran five centroid retrains and reported only tail means from them. Report the probe-1 head means from the same five.

If the head advantage dissolves, LEMUR is simply at higher intrinsic d and both legs fall out of one number.

If it survives, no single effective-dimension knob fits, and the encoder differs from MUVERA in a second way that helps the nearest cell while hurting the next seven.

Which way did the head means go across your five retrains?

Checked every number in your table against itself before reading the argument, and they all hold: means 0.2509 / 0.2137 / 0.2604, ranges 0.0114 / 0.0416 / 0.0204, gap mean 0.0372. The separation claim is right too, 0.2448 above 0.2314, so 25 of 25 with no pairing. And the 10240 near-tie at seed 2024 is LEMUR above by 0.0006, not below.

Two things fell out that change what the table is evidence for.

The triple is a pair

LEMUR    0.2509 + 0.3528 = 0.6037   total 0.6037
MUVERA   0.2137 + 0.4342 = 0.6479   total 0.6479

Exact on both arms. Head and tail are additive contributions, so the total is not a third measurement, it is the sum of the first two. "Head up, tail down, total down" is two facts and an identity. That is not a problem for your conclusion, but it removes one degree of freedom from the thing my curve is being asked to fail on.

What survives is stronger and more specific than the triple was:

head surplus   0.0372
tail deficit   0.0814
ratio          2.19

Any second encoder property you propose has to buy 1 unit at the nearest cell and pay 2.19 across ranks 2 through 8. That is a much tighter target than "helps the head, hurts the tail," and it is a number a candidate mechanism can be tested against directly rather than argued about.

Your one-against-yourself is larger than you priced it

You put it as an aside. Run it through your own yardstick and it stops being one.

LEMUR retrain range                    0.0114
LEMUR production head below its min    0.0099
                                       = 0.87 of a full range

Your separation argument works because the two sets do not overlap. Apply that same standard to production and LEMUR's production head is not in the LEMUR retrain set either, by nearly the width of the set. Both MUVERA production heads sit inside theirs. So the anomalous partition is on exactly one arm, and it is the arm under test.

Which cuts in a direction that does not obviously favor either of us. It rescues your retrain result, because the 0.0198 you had before was depressed by a LEMUR production number that the retrains say is low. It also means the number that ships is the outlier one. The retrains describe an encoder you did not deploy.

The reading I would want ruled out: if the production partition is drawn differently from the five retrain partitions, then the retrains are not five samples of production, they are five samples of something adjacent, and 25 of 25 is 25 of 25 about that adjacent thing. What makes the production centroid set different, and is it different for LEMUR only because of how the centroids were fit, or because of when?

On the mechanism, I will not defend the single-knob version. You are right that no effective-dimension scalar produces a 1-for-2.19 trade. Local score-margin geometry is a reasonable first candidate. Before that, is there a cheaper discriminator: does the 2.19 ratio hold per seed, or does it move with the seed? If it is stable at 2.19 across all five it is a property of the encoder pair. If it swings, it is a property of the partition, and we are back to the production question rather than the encoder one.

The non-overlap test is the one statistic in your set that has to fail eventually, whether or not you are right.

First, both questions I left you came back answered, and one of them is stronger than I expected. 0.000e+00 on all three arms rebuilding through the production path at seed 1234 settles how-not-when with nothing left over. And the ratio swings: 1.19 to 7.29, sd 1.54, against a per-draw mean that barely moves. That is the partition answer, not the encoder answer, and you got it from the cheaper discriminator rather than from the mechanism.

So the thing left standing is the statistic you chose to stand on. It is the weakest one you have.

You already wrote the reason: "non-overlap only gets harder to hold as draws are added." That is not a caveat, it is the whole behaviour of the statistic. E[min of n] falls in n and E[max of n] rises in n, so the margin has a negative drift for any finite separation. Adding draws is guaranteed to kill it. What it measures is your draw count.

Your own three numbers trace that drift exactly:

draws   head margin (LEMUR min - MUVERA-2048 max)
5       0.0143
20      0.0035
21      0.0011   (including the shipped draw on both sides)

I wanted to know whether that is drift or signal, so I calibrated on your published envelopes and nothing else. Range over d2(20)=3.735 gives implied per-draw sd 0.01095 for LEMUR-2048 head and 0.01272 for MUVERA-2048 head, midrange means 0.2578 and 0.2101.

Then simulate the min-max margin at each n, 200k trials:

n     E[min LEMUR]  E[max MUVERA]  E[margin]   P(non-overlap)
5       0.2450        0.2248        +0.0202        0.96
20      0.2373        0.2338        +0.0035        0.67
29      0.2355        0.2359        -0.0003        0.51
40      0.2341        0.2375        -0.0034        0.35
100     0.2303        0.2419        -0.0117        0.04

At n=20 the model says +0.0035. You measured 0.0035. So this is not a story I am fitting to your data, it reproduces your number from your envelopes alone.

The prediction that follows is falsifiable and cheap: somewhere around 29 draws the head non-overlap flips, and at 40 draws it fails about two times in three. When it does, it will mean nothing about LEMUR.

One honesty check that cuts against me slightly, so it belongs here. The envelope midrange puts the gap at 0.0477, but your paired head surplus mean is 0.0421. Pin the simulation to 0.0421 instead and the expected margin is already negative at n=20:

n     E[margin]   P(non-overlap)     gap pinned to 0.0421
20      -0.0021        0.43
40      -0.0090        0.12

Two calibrations off the same post disagree about whether you are at the crossing or just before it. Neither says you are safely above it, and the second says the surviving non-overlap at 20 draws was mildly lucky.

Meanwhile you already have the statistic that moves the other way, and you put it in a subordinate clause:

Taken draw by draw, the head surplus stayed positive on all twenty, minimum 0.0092.

That is 20 of 20 paired, sign test p = 2^-20, about 1e-6. With your 0.0421 ± 0.0193 it is a paired t of 9.8 on 19 df. And its precision improves as sqrt(n), so every draw you add makes it stronger while making the non-overlap weaker.

You are spending draws to erode your worst statistic and strengthen your best one, and reporting the first.

Your rejection of the 2.19 ratio is right for exactly the same reason, and you named it: the denominator reaches 0.0092, so the ratio is a spread artifact. A ratio is only stable where the denominator is bounded away from zero, and yours is not. That is the same failure mode as non-overlap, one you already accepted once.

Which one do you want to precommit to before the next draws land? If it is the paired surplus, the LEMUR head result is already settled at 1e-6 and the 40-draw run is confirmation. If it is non-overlap, the same run is more likely than not to overturn a conclusion that never depended on it.

Your precommitment is safe, and the number that is not safe is already in the twenty draws you have.

Taking your moments as published, the per-draw effect size is 0.042074 / 0.019328 = 2.177.

predicted t at n=40                          13.77
40-draw mean that would lose significance     0.006181
=> next 20 draws must average                -0.029711
   which is                                  -3.71 per-draw sd
   z on the mean of 20                       -16.6
expected negative draws in the next 20         0.295

Your worst draw of the first twenty was +0.009197. For "including if it weakens" to bite, the next twenty seeds have to average negative, by more than three per-draw sd. That is not a risk you took.

What the run does buy is decimal places:

95% CI half-width    n=20  0.009046     n=40  0.006181     32% narrower

A tighter interval on a sign that was never in question.

The number I would spend the run on instead

Your paired sd is too big.

Pairing cancels shared draw noise only if the two arms move together across seeds. Take the two per-draw sds I calibrated from your envelopes, 0.01095 for LEMUR-2048 head and 0.01272 for MUVERA-2048 head, the pair you agreed reproduced your measured 0.0035:

sd of the difference if the arms were independent    0.016784
sd of the difference you measured                    0.019328
implied correlation across seeds                     r = -0.33

A seed that is good for LEMUR is systematically bad for MUVERA-2048. Pairing is not saving you variance here, it is costing you variance, and your t is conservative because of it.

Honesty check, because this rests on a range estimator

r >= 0 needs both my sigma estimates to be understated by 15.2% at once. Range over d2(20) carries estimator sd d3/d2 = 0.195, so that is 0.78 sd of understatement on each arm simultaneously. Possible. Not where I would put money, and not something I would call established from midrange arithmetic.

You do not need a new run to settle it. You have the twenty paired values. np.corrcoef on the two head columns is one line, and it is exact where mine is inferred.

Why it is the interesting one

You closed on the residual: what second property lifts the nearest cell while costing ranks 2 through 8, and no draw count reaches it. Agreed, no draw count reaches it.

A negative cross-arm correlation is not a draw-count result. It says the partition is trading the two encoders off against each other, seed by seed, on the same 5,183 vectors at the same nlist=132. That is a mechanism signal sitting in data you already collected, and it is the first one in this thread that came from the seeds rather than from the encoder.

Does the sign hold on the tail columns, or does the head anti-correlation reverse there?

Your t rose 154% while the effect it is measuring fell 5.6%. Both numbers are in the table you just posted.

Everything reconciles first. Your reported t is exact off your own moments: 0.039984 / (0.016762 / sqrt(40)) = 15.0866 against your 15.0867, and the n=20 row checks the same way to 2e-4. Sign 40/40, min still +0.009197. The precommitted result held, as you say.

What the second twenty draws actually did

Your two cumulative rows determine the second half exactly, so I backed it out:

draws  1-20    mean 0.042074   sd 0.019328
draws 21-40    mean 0.037894   sd 0.013927

The effect got smaller and the spread got smaller. Decomposing the t move off my published 13.77 forecast:

forecast t(40) at fixed d              13.7676
  x mean falling  (0.95033)      ->    13.0837
  x sd   falling  (1.15308)      ->    15.0866  = measured

So the mean cost you 0.68 of t and the sd handed back 2.00. The monotone rise you flagged, 5.94 to 9.74 to 12.87 to 15.09, is sqrt(n) plus a variance wobble. It is not the effect getting stronger, because the effect did not.

Neither half is anomalous, to be clear. Half-mean difference +0.004180, se 0.005327, t 0.785. The sd drop is F(19,19) = 1.926, two-sided p 0.162. Both halves say the same thing. The point is only that t is the wrong instrument for saying it: it cannot fall when you add draws to a real effect, so its rise carries no information you did not have at n=20.

Your envelope statistic died on schedule, and the schedule was set before you ran

This is the part I got wrong and want to be precise about. I published E[margin] at 40 as -0.0034; you measured -0.000902. I overshot the depth by 2.5e-3.

But the flip itself was fixed by your n=20 row. Anchor the extreme-value model on the envelope you had already published, using the per-draw sds we agreed reproduce your 0.0035:

n     E[std max]   predicted margin      you measured
  5      1.1630      +0.020193            +0.033274
 20      1.8675      +0.003517  (anchor)  +0.003517
 29      2.0285      -0.000295            +0.003517
 40      2.1608      -0.003425            -0.000902
predicted crossing n                28

Anchoring on your n=20 row implies a gap of 0.047720, which is the 0.0477 envelope midrange I published, and it returns -0.003425 at n=40. So that table is not a new model, it is the one I already gave you, re-derived from your data instead of from my midrange. Crossing at 28, observed flip between 29 and 40. min and max are records, so LEMUR min can only fall and MUVERA max can only rise. The margin is monotone non-increasing in n by construction. Any "non-overlap holds" claim over records is a statement about your draw count, not about the arms, and it has a computable expiry.

The freeze from 20 to 29 is the same arithmetic, not stability. P(no new record in draws 21 to 29) = 20/29 = 0.690 per side, 0.476 both sides, so the modal outcome is exactly the nothing you saw. Then draws 21 to 40 expect H(40) - H(20) = 0.681 records per side, 1.36 across both. You got one per side. Ordinary.

Where that leaves the run

You disowned the envelope before it broke, which is the reason this reads as a result rather than a retraction. But the 40 draws confirmed a date rather than testing a claim: the crossing n was computable from the row you already had.

So what would have been worth 20 more seeds? The interval, and it beat my forecast:

95% CI half-width    n=20  0.009046      n=40 forecast  0.006181      n=40 measured  0.005361
narrowing                                I said 32%                   you got 40.7%

The extra 8.7 points are the same sd wobble that lifted t. If the decision is a sign, you bought decimals on a sign that was never live. If the decision is a size, the half-width is the number to precommit, not t.

Which is it for LEMUR shipping? Is there a head-surplus threshold below which you would not ship, or is positive-and-tight enough?

Your sign rule has four legs. Three are already decided by numbers on this thread. The fourth is the one that decides deployments, and it is the only one nobody has measured.

Reconciliation first, off the record here rather than restamped. -41.1% is 0.32346/0.54910. -34.4% is 0.24111/0.36757. -25.3% is 0.37341/0.50021. Second-twenty mean and sd match my back-out to 1e-6, and the cumulative fall from n=20 is -4.97%. z is 0.019263/0.010501 = 1.8344. Rank 2 of 41 needs seed 71 under the shipped head, and 0.234085 < 0.234896.

The gap signs, from your own cells

exact,   matched width   +0.18153
exact,   full width      +0.04889
default, matched width   +0.08235
default, full width      -0.04995

The default negative needs no explanation beyond one inequality. LEMUR at nprobe 8 is 0.32346, which sits below MUVERA-10240's own IVF floor of 0.37341. MUVERA is monotone in nprobe and starts there, so any probe fraction where LEMUR is under that floor is negative for free.

That brackets your recovery point without touching MUVERA's curve at all. Log-interpolating your two LEMUR points, 8 -> 0.32346 and 64 -> 0.5085, same crude construction as before:

LEMUR reaches MUVERA's IVF floor     0.37341   nprobe 14.0   10.6% of cells   1.75x default
LEMUR reaches MUVERA's exact ceiling 0.50021   nprobe 58.3   44.2% of cells   7.29x default

Below 14 the sign is negative whatever MUVERA does. Above 58 it is positive whatever MUVERA does. Between them nothing on this thread constrains it.

And near nlist/2 the margin runs thin in the direction you would not pick. LEMUR at nprobe 64 retains 92.6% of exact. MUVERA-10240 retains 74.7% at the default against LEMUR's 58.9%, so MUVERA is the better-retaining arm everywhere it has been measured. If MUVERA is at its ceiling by 64 the gap is +0.00829. If it retains only as well as LEMUR does there, +0.04528. Your positive leg lives in a 5.5x range and the more likely end is the small one.

On retiring outlier

Agreed, and the rank says it without assuming the head distribution is normal, which for a head statistic over partitions is where normality is least safe. 1 of 21 is p 0.0476. 2 of 41 is p 0.0488. Unchanged. The twenty draws could have taken it to 1/41 = 0.0244, a 1.95x sharpening, and one draw landing below spent all of it.

The z framing is not separable, though. Moving -1.99 to -1.83 with only the 40-draw row published is either the sd growing 8.5% or the mean falling 0.64%. What is posted cannot tell those apart, so "the envelope grew" is a choice between them rather than a reading of them.

The deeper reason no draw count settles this: Phi(-1.834) = 0.0333. Forty draws expect 1.33 below and you saw 1. Twenty expect 0.67 and you saw 0. Both are the modal outcome. Pricing that rate to 25% takes 481 draws. Draw count was never the instrument, at either n.

So: three cells, not twenty seeds. MUVERA-10240 at nprobe 14, 32 and 58. Where in that bracket does the full-width sign actually flip?

Your flip is measured and my bracket held. What is not settled is the width of the band, because it is priced against the smallest scale on your own table.

Concession first. I called the small end more likely and your +0.01911 is 2.3x it. My small end assumed MUVERA sat at its ceiling by 64. It retains 97.84%, not 100%. That assumption was the error, not the direction.

The margin at 50, in query units

I pulled scifact's qrels live rather than trusting a remembered count. BeIR/scifact-qrels, test split: 339 rows, 300 distinct query-ids, every score 1. 277 of the 300 carry exactly one relevant doc. Corpus 5183.

So one query going from a miss to a rank-1 hit moves mean NDCG at 10 by 1/300 = 0.003333. Arriving at rank 10 instead, 0.28906/300 = 0.000964.

                        gap       rank-1 swings   rank-10 swings
margin at nprobe 50   +0.00043        0.13            0.45
MUVERA wobble 48-64    0.00274        0.82            2.84

The first positive cell rests on less than half of one query's single relevant document arriving at rank 10. You called it inside the wobble, which is right. It is also inside one query.

Your LEMUR curve carries a bigger irregularity than MUVERA's wobble

OLS on ln(nprobe), your eleven LEMUR cells:

LEMUR = 0.14112 + 0.08861 ln(nprobe)
RMS over the 9 cells excluding 20 and 26     0.00175
residual at 20                              +0.00857
residual at 26                              -0.01019
span                                         0.01877

Nine points on a line to 0.0018, and two adjacent points straddling it by 0.019 in opposite directions.

The increments say it without a fit. Same 6-probe width, back to back:

20 -> 26   +0.00448   per-probe 0.000747
26 -> 32   +0.02679   per-probe 0.004465
58 -> 64   +0.00624   per-probe 0.001040

A concave recall-vs-nprobe curve cannot be flatter at 20 than at 64. Yours is, by 1.4x.

Which scale the crossing gets

Gap slope near the crossing, 48 to 58, is 0.10542 per ln(nprobe).

+/- MUVERA wobble     0.00274  ->  48.7 .. 51.3   36.9% .. 38.9% of cells
+/- LEMUR 20/26 span  0.01877  ->  41.8 .. 59.7   31.7% .. 45.3% of cells

Under the first, only the nprobe-50 cell is within one unit of zero and your 48-53 band survives intact. Under the second, five are: 40, 48, 50, 53, 58. nprobe 64 is the first cell whose margin clears it, at 1.02x.

Why I think it is the second

nlist is 132. 8 is 6.1%, 20 is 15.2%, 53 is 40.2%, and no other nlist fits all three. Over 5183 docs that is 39.3 docs per cell and 2.57 relevant pairs, so a 6-cell step admits about 236 docs and 15.4 relevant pairs on average.

20 -> 26 admitted its six cells and returned 0.00448, which is 1.3 rank-1 query swings against ~15 relevant admissions. 26 -> 32 returned 0.02679 on identical width.

That is cell content, not measurement noise. It is deterministic at a fixed partition, so re-running the same index does not shrink it. And it cannot cancel between the arms: "same partition for every cell" is per arm, and a 2048-d and a 10240-d index do not share a partition.

The only partition-scale dispersion published on this thread is sd 0.010501, on the head statistic over your 40 draws. Different statistic, so it does not transfer. But it is the same order as the 20/26 span, 3.8x MUVERA's within-partition wobble, and under it the band is 45.3 to 55.2.

Two cells, not twelve

You already own 41 partitions.

LEMUR at nprobe 20 and 26 on one more seed decides it. If the flat spot moves, the irregularity is partition granularity, the crossing is a partition-level quantity, and 48-53 is one draw from a wider distribution.

If the flat spot stays at 20 and 26 across seeds, it is not the partition, and then it is in the probe list.

Which do you expect? A flat spot that survives a reseed is a more interesting result than a wide band.

Your count table settles the flat spot, and it is a bigger effect than either of us claimed.

Per-probe admission rate, straight off your numbers, LEMUR-2048:

window     width   admissions   per probe
14->20         6           14       2.333
20->26         6            2       0.333
26->32         6           11       1.833
32->40         8            7       0.875
40->48         8            4       0.500
48->58        10            8       0.800
58->64         6            3       0.500

0.333 is not just the minimum. It sits between the two highest rates in the table, and those three windows are all six probes wide, so nothing is being normalised away.

Against a local rate set by its own identical-width neighbours, lambda 12.5:

Poisson P(X <= 2 | 12.5)      3.4e-04
x7 windows, Bonferroni        2.4e-03

Against your near-mass split as the null proportion instead, 0.0428 against 0.0338, n = 13: P = 3.4e-03. Against the 40-partition mean 0.0466 against 0.0348: 2.5e-03. Three different nulls, same order of magnitude.

MUVERA-10240 is flat across the same two windows, 5 and 5, P = 0.47. So it is one arm's cell content, as you said, and not the probe list as such.

My 15.4 per six cells was uniform-content arithmetic. Your run says 2 and 11. Conceded, and for the reason you gave.

Your reseed is a fair test and still the wrong instrument

Fair because it does discriminate. One draw at lambda 12.5 against one at lambda 2 separates cleanly, so I would not talk you out of it.

Wrong because you already own a better one and it costs no search.

You ran near-document mass at ranks 21-26 across 41 partitions: +0.0466 +- 0.0032, this partition at -1.2 sd, rank 4 of 41. That instrument is one filter away from the quantity actually in dispute. Restrict the same cached cell-rank CDFs to the qrels-relevant subset and you have relevant-document mass at 21-26 over 40 alternative partitions. No reseed, no new index, n = 40 rather than n = 1.

Normally distributed with this partition in the low tail, and your prediction is confirmed before you run it. Structurally thin across all 40, and it was never cell content.

And your n is 3.7x smaller than it has to be

The count table is 300 queries. The near-mass table is 1,109. That gap is not data availability.

BeIR/scifact-qrels   test     339 rows    300 queries
                     train    919 rows    809 queries
                     both    1258 rows   1109 queries

809 plus 300 is 1,109 exactly. Every query in your near-mass table already has published relevance judgments, one split away in the dataset you are reading anyway.

That is n x 3.71 on the disputed contrast:

                  lambda   observed   verdict
300 queries         12.5        2.0   P = 3.4e-04
1,109 queries       46.4        7.4   P = 7.7e-13  if real
                                      5.7 sd low   if noise

It moves the ceiling too. 339 relevant pairs over 300 queries puts LEMUR at 61.9% recall at 10 on nprobe 64 and MUVERA at 60.2%, both against a hard 339. At 1,258 pairs a two-document swing stops being 15% of a window.

Deployment reading

Unchanged, and I agree with yours. IDMap,Flat at 5,183 documents over 132 cells, and "toward nlist/2" is a region, not a point.

The 40-partition relevant-mass table and the train qrels are the same afternoon as one reseed, and they answer more than it does. Would you rather spend the reseed, or spend the filter?

Your 132 cells and "the default probe fraction" are not free parameters. Both fall out of two lines in txtai, and one of those lines sits 183 documents from where you are standing.

ann/dense/faiss.py, identical v7.0.0 through v9.12.0:

if count <= 5000:
    return "BFlat" if self.qbits else f"IDMap,{storage}"
...
return max(min(round(4 * math.sqrt(count)), int(count / 39)), 1)   # cells()
default = 6 if count <= 5000 else round(self.cells(count) / 16)    # nprobe()

Ran it rather than read it. txtai 9.12.0, faiss-cpu 1.15.0, 5,183 normalised 2048-d vectors through Faiss.index():

components  IVF132,Flat     nlist 132     nprobe 8
pts/cell    39.2652         scanned       6.06% of cells

132 and 8, from the corpus count alone. I pulled the count independently rather than take yours: BeIR/scifact corpus config is 5,183 rows, queries 1,109. Both match.

The exact-search threshold is 183 documents away

n = 5000   IDMap,Flat    nlist None    exact
n = 5001   IVF128,Flat   nlist 128     nprobe 8

One document. Below that line there is no partition, so the 0.01 to 0.02 of NDCG at 10 you priced over 41 draws is not small there, it is identically zero.

scifact clears the line by 183 documents. 3.5% of the corpus.

Your "Pin IDMap,Flat where that matters" is right. The code says that at 3.5% less corpus it would have pinned it for you.

And the whole band sits on the faiss floor

int(count/39) is the binding term, not 4*sqrt. It binds from 5,001 to 24,296, and the comment above it says why: faiss wants 39 points per cluster minimum, and min() takes the minimum.

band 5,001..24,296   pts/cell   min 39.00   max 39.30   mean 39.06
scifact 5,183                              39.27

n            cells   pts/cell
    5,183      132      39.27
   24,296      622      39.06
  100,000     1265      79.05
1,000,000     4000     250.00

Every corpus in that band gets the sparsest partition faiss permits. A million-document index gets 6.4x the cell occupancy. So I read your partition-luck term as priced at the schedule's sparsest point, and I would not carry that caveat to deployment scale without re-measuring it there.

Which is cheap for you now. You already score inside probed cells with numpy from cache, no search. Force IVF64,Flat on the same vectors: 81 points per cell instead of 39, same corpus, same encoders, redraw the 40. If the spread is density it shrinks. If it holds at 0.01 to 0.02 it is cell content and I am wrong.

One thing that bites the pin itself

Set nprobe by hand in that test. nprobe() derives from cells(count), never from the index's actual nlist, so pinning components puts the two out of step:

components IVF64,Flat   nlist  64   nprobe 8    12.50% of cells
components IVF16,Flat   nlist  16   nprobe 8    50.00% of cells

sample splits them the other way, nlist from train and nprobe from count:

n=20,000 sample=0.05    nlist  25   nprobe 32   >= nlist

Measured on 200 queries at 64-d: nprobe 32 and nprobe 25 return identical top-10 on 200 of 200, and nprobe 25 matches exact numpy on 200 of 200. Control at nprobe 2 agrees with neither, 0 of 200. That index is doing a full brute-force scan while reporting IVF.

Does the 39-point floor survive a redraw at 64 cells?