If you are building or evaluating a personal knowledge management (PKM) workflow, you do not need an enterprise benchmark to evaluate retrieval dependability. A small, disciplined test suite of ten synthetic notes can expose fundamental retrieval blind spots in minutes.

The 10 Synthetic Test Notes

Start with the ten invented notes below. These are test fixtures, not records of my infrastructure, incidents, or research. Save each as a separate note with a stable identifier:

1. Note A (API Decision): "2026-03-12: Switched internal event bus from Redis to NATS Core to reduce idle memory."
2. Note B (Reading Note): "Raft consensus simplifies Paxos by enforcing leader-driven log replication."
3. Note C (Hardware Spec): "Workstation runs an M2 Mac mini with 16 GB unified RAM and 512 GB SSD."
4. Note D (Recipe Draft): "Sourdough test #4: 72% hydration with 20% rye flour produced open crumb after 4-hour bulk ferment."
5. Note E (Incident Postmortem): "2026-05-18: Database pool exhaustion caused by unclosed cursor in billing export."
6. Note F (Travel Packing): "Essential travel gear: 65W GaN charger, USB-C to Lightning cable, offline voice recorder."
7. Note G (Architecture Idea): "Decoupling read projections from the primary write store using SQLite replicas."
8. Note H (Reading Summary): "Christopher Alexander on pattern languages: each pattern describes a recurring problem."
9. Note I (Team Retro): "2026-08-04 retro: Standup moved to asynchronous Slack thread to protect morning focus."
10. Note J (Home Network): "Home lab router on 192.168.10.1 with local DNS pointing to AdGuard Home."

The 5 Retrieval Query Categories

Separate retrieval from answer generation: returning useful source notes and composing a faithful answer are different checks. For a search-only tool, inspect the source notes yourself; for an answering tool, also inspect its citations. Test your pipeline against five distinct query categories:

1. Exact Keyword Match: Query: "NATS Core". Target: Note A at rank 1.
2. Paraphrase Match: Query: "Which breakfast bread test yielded an airy crumb?". Target: Note D without requiring an exact phrase match.
3. Source Attribution: Query: "Who wrote about patterns recurring in design?". Target: Note H, extracting the author name.
4. Temporal Ordering: Query: "Did standup move to Slack before or after the billing incident?". Target: Retrieve Notes E and I to resolve chronological sequence.
5. Insufficient Evidence (Unanswerable): Query: "What is our Postgres backup policy?". Target: Explicit negative response without hallucination.

Evaluation Metrics and Verification Checks

Scope and Limitations

This framework represents a proposed evaluation methodology for personal knowledge retrieval, not a report of measured results or a benchmark of any shipped production app. Real-world notes are noisier, longer, and frequently updated. However, establishing ground truth on a small synthetic corpus clarifies whether your retrieval pipeline fails at lexical matching, semantic ranking, or hallucination prevention. One example per category is a smoke test, not a reliable percentage score. Keep the fixtures fixed, record the returned note IDs and ranks, and rerun after changing chunking, embeddings, or ranking. Expand the query set before comparing rates. For a broader research benchmark, see BEIR (Thakur et al., 2021).

About the Project

I build Sanva, an independent product studio. Lumo: AI Memory is listed in the US App Store. If personal note workflows interest you, see Lumo at Sanva. The evaluation above is a proposed method, not a claim that Lumo has passed it; availability and pricing follow the current store listing.