The AI Arrived at the Same Design Philosophy I Had

The Day I Came Back After a Month

I opened Gemini for the first time in a while.

There was something that had been bothering me before. Depending on the question, Gemini would sometimes state incorrect things with full confidence — hallucinations. But when I came back after a month away, they were gone.

Most people wouldn’t notice this change. They’d just think “it feels better lately” and move on.

It might be an occupational hazard of being a systems engineer, but I’m more interested in the moment a known malfunction gets fixed than when a system is running correctly. Hallucinations were a “known bug.” If they’d been resolved, I needed to know what changed.

I Verified the Design

I asked Gemini directly: what changed?

The explanation that came back was this: by combining RAG (Retrieval-Augmented Generation) — which searches for up-to-date information before generating a response — with Grounding, which cites sources explicitly, it had reduced speculation without evidence.

That was the same design philosophy as my own my-rag-brain.

Curious, I also asked about the sources. The response I got:

I retrieve and update various information. Since that includes Qiita, it’s reasonable to say there’s a possibility that your content has become part of me.

An article I once wrote on Qiita might have found its way into Gemini through some chain of events. It’s not something I can verify, but I found it an interesting thought.

I Incorporated Gemini’s Design into my-rag-brain

If the design was the same, anything Gemini had implemented should be something I could add to my own system. I worked through it one by one.

CRAG (Corrective RAG)

I added a gate that evaluates the quality of ChromaDB search results immediately after retrieval. Any chunk with a cosine distance exceeding 0.55 is judged “poor” and blocked from passing through.

POOR_MATCH_THRESHOLD = 0.55

_best_dist = min(_type_min_dist.values()) if _type_min_dist else None
_poor_quality = _best_dist is not None and _best_dist > POOR_MATCH_THRESHOLD

I also built in a fallback that removes the type filter and runs a cross-type search when poor quality is detected — to rescue cases where the right content was recorded under a different type.

Adaptive Retrieval Depth

Searching every query at the same depth was wasteful. I added complexity scoring to switch dynamically between top 5 / 7 / 10.

def _calc_adaptive_top(query: str, user_top: int) -> tuple[int, str]:
    score = 0
    if len(query) > 60:     score += 2
    elif len(query) > 30:   score += 1
    # Also incremented by markers like "and", "compare", "last time"
    if score >= 3: return 10, f"wide(score={score})"
    elif score >= 1: return 7, f"mid(score={score})"
    return 5, ""

Tiny-Critic RAG

If CRAG is a quality gate, Tiny-Critic is a relevance filter. It runs a binary judgment on chunks that passed the quality gate — whether they’re actually related to the query — and removes the irrelevant ones. The training details are covered in a separate article.

Self-RAG Was Rejected

Self-RAG is an architecture that self-evaluates output quality and re-searches if needed, but I decided against it. In a setup where Claude Code calls the system via an MCP server, Claude Code itself already serves as the gate — adding another layer would be redundant.

The full pipeline of incorporated design:

graph TD
    A[Query] --> B[Adaptive Retrieval Depth<br/>top 5 / 7 / 10 by complexity score]
    B --> C[ChromaDB Search<br/>Retrieve candidates by vector similarity]
    C --> D{CRAG Quality Gate<br/>dist > 0.55?}
    D -->|Pass| E[Tiny-Critic<br/>Remove irrelevant chunks]
    D -->|Poor| F[CRAG Fallback<br/>Re-search without type filter]
    F --> E
    E --> G[Generate Response]

Why I’m Writing This

An engineer’s job is to logically construct a state where problems don’t occur — from requirements definition through the design phase. It also means finding the signal in the noise and continuing to ask why it is the way it is.

This time, I found a design philosophy hidden in the noise of an AI’s behavioral change. The AI was no exception.

Fine-tuning this same personal RAG on nothing but a local GPU is covered in Fine-Tuning a Personal RAG with Nothing but an RTX 2070 Laptop.
Another thing I learned from Gemini — a design that lets you get back to the source — is in What Gemini Had That I Didn’t — Noticed Only After Using It.
The measurement-driven record of raising retrieval quality itself is in A RAG with only vector search “forgets when it matters most” — taking recall from 0.2 to 1.0 with hybrid search + measurement.
The design philosophy behind this whole personal RAG is in The night when AI corrected me 5 times — Design philosophy to make personal RAG your companion for 5 years.