Experiment 12
Every note points the same way
My search engine kept telling me that any two of my notes were somewhat related. A cooking video and a paper on neural networks: somewhat related. That is either a deep truth about me or a broken ruler, and I wanted to know which.
So I measured the shape of the collection itself. If my 493 notes were spread evenly in every direction, their average direction would cancel out to nearly nothing. Instead it came out eleven times longer than chance allows. Every note leans the same way, like a crowd where everyone faces the stage. Most of what any two notes "have in common" is just that shared lean, and since it is the same for all of them, it carries no information at all.
Subtract the lean and the picture changes. Pairs of notes stop looking uniformly friendly, the space turns out to be genuinely roomy1, and topics that seemed crowded together move nearly apart. Some similarity numbers I had published earlier dropped once the lean was removed, so the record now carries the corrections next to the original claims.
The lesson I keep from it: before trusting that two things are alike, measure the room they are standing in.
What the offset was hiding
The numbers behind the lean. As stored, the mean cosine between two notes is +0.259 and the median cosine between two tag centroids is +0.699; the corpus mean vector has length 0.510 where an even spread would give 0.045. After subtracting the mean, note pairs land at -0.002 and tag centroids at -0.057: near orthogonal. This is the anisotropy described for language-model embeddings by Ethayarajh and by Mu and Viswanath2, showing up in a personal corpus at full strength.
Removing the shared direction does not leave a low-dimensional remnant. The participation ratio3 is 104 of 1024; the top component explains only 4.1% of variance, 30 components reach 41%, and 100 reach 72%. One enormous removable offset, and behind it a genuinely high-dimensional cloud.
Scored against size-matched nulls in each geometry, the mean tag z-score rises from 5.4 to 23.0 after centring, and every one of 69 tags separates from chance where 58 did before. Stripping the leading principal components as well makes things worse, not better: the common mean is the artifact, the top components are signal.
The corrections it forced: a published list of redundant tag pairs survives centring, but the values deflate from about 0.97 to 0.93 and the order reshuffles; the two geometries agree on substance with a rank correlation of only +0.61. One tag flipped its story entirely: reference scored z = -3.4 as stored, apparently anti-coherent, and lands at +0.003 centred. Notes sharing it are simply unrelated: a tag that fails is not one that clusters badly, it is one whose notes have nothing in common.
And the honest downstream result: on the one retrieval-shaped task measured so far, centring gains 1.2 points of tag-match@10 against a registered 2-point bar. It did not clear it. Production search still runs on the stored vectors; the geometry finding stands, the retrieval win is not yet earned.
- Technically: after centering, the collection uses about 104 effective dimensions of the 1024 available, and the average pair of notes is close to unrelated, which is what an honest ruler should say. ↩
- Ethayarajh (2019) and Mu & Viswanath (2018) documented the same anisotropy, a dominant shared direction, in contextual and static word embeddings. ↩
- Participation ratio: an effective-dimension count, roughly how many directions the cloud really uses once the variance spectrum is taken into account. ↩