notes2vec

▶ play demo

The problem

My notes are a folder of Markdown files. grep is great if I remember the exact phrase, and useless if I don’t. I wanted to type something like “how did I handle GPU memory last time” and land on the right paragraph, without uploading my notes anywhere.

How it works

It has two halves. Indexing turns every note into vectors, once. Searching turns your question into a vector and finds the closest ones.

Finding the files

It walks the folder with the ignore crate, the same walker ripgrep uses, so anything in .gitignore gets skipped for free. It picks up .md, .markdown, .mdown, .mkd, .mkdn and .txt.

Chunking

Embed a whole file and everything in it blurs into one vector. So each note is split along its headings with pulldown-cmark, aiming for about 300 characters per chunk (never under 50 or over 500). Every chunk remembers where it lives, like Parallelism › Data parallel › Gradient sync, plus its line numbers, so a result can take you to the exact spot. Front matter goes through serde_yaml for titles and tags.

Embedding

Chunks go through BGE-small-en-v1.5, a small retrieval model from the BGE paper (the weights are about 133 MB), running on Hugging Face’s Candle. That means no Python anywhere. notes2vec init downloads the model once. Each chunk becomes one vector: take the first token’s output (CLS pooling) and normalise it to length 1.

// embeddings: [batch, seq, hidden] → pooled: [batch, hidden]
let pooled = embeddings.narrow(1, 0, 1)?.squeeze(1)?;
let normalized = Self::normalize_l2(&pooled)?;

Searching

Your query goes through the same model, then gets compared with every stored chunk. Since everything has length 1, cosine similarity is just a dot product. I didn’t want to sort thousands of scores to keep ten, so it keeps a min-heap of size k: a new score either beats the weakest one in the heap or gets thrown away. That’s O(n log k) time, and memory stays flat no matter how many notes there are. For a notes folder that’s plenty, so there’s no vector database to run.

Not redoing work

Everything lives in redb, a small embedded database written in Rust. For every file it keeps the modification time and a SHA-256 of the contents, and a file is re-embedded only if one of those changed. It also remembers which model built the index, so swapping models forces a clean rebuild. And notes2vec watch keeps the index fresh as you save, using a file watcher with a 2-second debounce.

Limits

The model is English-only, so notes in other languages won’t match well. Model card And every search compares the query with every chunk. That’s fast for a personal folder, but it grows with the number of notes; a much larger collection would want an approximate index such as HNSW.

Stack

• LanguageRust
• ModelBGE-small-en-v1.5 via Candle, tokenizers, hf-hub
• Storageredb
• Parsingpulldown-cmark, serde_yaml
• Filesignore, notify-debouncer-full, sha2
• Interfaceclap for the CLI, ratatui + crossterm for the TUI
• BuildsGitHub Actions ships binaries for Linux, macOS (Intel and Apple Silicon) and Windows

Further reading

• Semantic search, from the Sentence Transformers docs: the same idea in Python, with examples.
• Xiao et al., C-Pack (2023): how the BGE models were trained.
• Malkov & Yashunin, HNSW (2016): a graph index for fast approximate search, for when checking every chunk gets too slow.

Try it

notes2vec init                 # download the model once
notes2vec index ~/notes        # build the index
notes2vec                      # open the search UI

Built with Astro.