What Gemini Had That I Didn't — Noticed Only After Using It

What Gemini Had That I Didn’t — Noticed Only After Using It

What You Only Learn by Running It

I started actually using my-rag-brain and found something inconvenient.

When I pulled up past records with search_memory, I couldn’t tell which conversation the content had come from. The search results did show the text of lessons and conclusions. But there was no way to get back to the original context — the problem that existed, the process that led to that conclusion.

No file path. No clues beyond the date.

This Was a Design Gap vs. Gemini

I looked up Gemini’s specs.

Gemini has a RAG Engine, a MemoryCorpus, and Sources: Previous chats. It doesn’t just search past conversations — it can return the conversation itself as a source. The “where did this come from” is always preserved through Grounding.

That’s what my system was missing.

The vault (distilled entries like lessons and conclusions) had its connection to the original conversation files severed. Distillation compresses information but had been cutting off the source.

I Implemented source_origin

The change touched four files. First, a map of what each file does:

File	Role
`store.py`	ChromaDB write layer. Manages all chunk metadata.
`note_writer.py`	Entry point for recording, called from MCP server and CLI. Calls `store.py`.
`distill_v2.py`	Pipeline that promotes raw diary notes to Lesson/Conclusion.
`server.py`	MCP server body. Holds the search and display logic for `search_memory`.

The write flow is distill_v2.py → note_writer.py → store.py → ChromaDB. Reads go server.py → ChromaDB.

① store.py — Add field to ChromaDB metadata

Added source_origin to the metadata of every chunk written to ChromaDB.

meta = {
    "source_type":   source_type,
    "source_file":   str(path),
    "source_ref":    source_ref,
    "source_origin": source_origin,  # records the source file path
    "title":         title,
    "date":          date_str,
    ...
}

② note_writer.py — Add argument to write function

Added source_origin as an argument to write_note(), the recording entry point, and passes it through to store.py.

def write_note(text: str, source_type: str, ref: str = "",
               priority: str = "", source_origin: str = ""):
    ...
    ingest_file(str(target_file), source_type, ..., source_origin=source_origin)

③ distill_v2.py — Pass source file path during distillation

When writing distilled diary content to the vault, the original diary file path is passed as source_origin. This records which day’s conversation each lesson came from.

write_note(
    text=body,
    source_type=note_type,
    ref=f"distill_v2:{source_date}:{time_str}",
    source_origin=str(md_path),  # e.g. data/diary/2026-05-27.md
)

④ server.py — Display source in search results

When assembling search_memory results, display source_origin at the end if it exists.

source_origin = meta.get("source_origin", "")
...
if source_origin:
    lines.append(f"📄 Source: {Path(source_origin).name}")

Now search results show 📄 Source: 2026-05-27.md at the bottom. The path back to the original conversation exists.

Why Grounding Matters

Records existed. But if you can’t return to the context of a record, knowledge becomes an isolated label.

When an engineer designs a system, the quality of judgment differs between having only the requirements document and having the background of why those requirements were born. RAG is the same.

Gemini doesn’t let go of its sources. My system should do the same.

Noticing that Gemini’s behavior matched my own RAG’s design philosophy is covered in The AI Arrived at the Same Design Philosophy I Had.
Fine-tuning this same personal RAG on nothing but a local GPU is covered in Fine-Tuning a Personal RAG with Nothing but an RTX 2070 Laptop.
The measurement-driven record of raising retrieval quality itself is in A RAG with only vector search “forgets when it matters most” — taking recall from 0.2 to 1.0 with hybrid search + measurement.
The design philosophy behind this whole personal RAG is in The night when AI corrected me 5 times — Design philosophy to make personal RAG your companion for 5 years.