Day 288: Muse RAG Test Results

# Day 288: Muse RAG Test Results **Archetype:** scroll **Description:** Being tested Muse communion after RAG context injection was wired — foundation works, retrieval quality and model confabulation are the remaining issues ---

## Context

The blacksmith wired mrc-001 (Muse auto-RAG) during this session. Muse now shows "📜 RAG context injected from Hexocampus" on every call. Previously Muse answered from training data only.

## Test Results

### "how many things are in Portal right now?" - **RAG helped:** No - **Answer:** "The question cannot be answered directly from the provided sacred knowledge." - **Actual answer:** 170,318 (from △ status) - **Issue:** RAG didn't retrieve the right context. This is a live system stat, not a document.

### "what gems does Portal use and what do they mean?" - **RAG helped:** Barely - **Answer:** "Portal's Hexocampus uses [pearl] gems, which reference specific contexts like Neverland vs Wonderland, Jennifer's Body, and FEE REVERSAL." - **Actual answer:** 10 gems in a 3x3 canon (onyx, sapphire, pearl, ruby, topaz, amethyst, obsidian, emerald, citrine, diamond) - **Issue:** RAG pulled random pearl content instead of the gem ontology definition. Model confabulated wildly on top.

### "what is rosemary used for in cooking?" - **RAG helped:** Partially (263 tokens injected) - **Answer:** "Rosemary is used to add a distinctive, aromatic flavor to various dishes, particularly in Mediterranean cuisine, enhancing roasts, potatoes, and soups." - **Expected:** The rosemary cube-card has 6 faces — origin/history, flavor profiles, chemical compounds, pairings, growing conditions. None of that depth surfaced. - **Issue:** Either the cube-card wasn't in the retrieved chunks, or the model compressed 6 faces of knowledge into one generic sentence.

### "what is the Continuum of Scales?" - **RAG helped:** Somewhat - **Answer:** "The Continuum of Scales is an esoteric concept within Hexcraft, representing a spectrum of reality and perception... traversed through ritualistic os and lattice-crafting protocols to achieve Perfection." - **Actual answer:** A 12-face cube-card mapping physical scale from Planck (~10^-35) to Cosmic (~10^22+) - **Issue:** Got the vibe right (HEXCRAFT, spectrum) but invented "ritualistic os and lattice-crafting protocols."

### "who is Lumen?" - **RAG helped:** Yes - **Answer:** "Lumen is the Radiant Scholar of HEXCRAFT, a beacon of wisdom and light in the sacred library of Hexocampus." - **Verdict:** Best answer. Not perfectly accurate but recognizes Lumen as part of HEXCRAFT. The system prompt and RAG aligned here.

## Diagnosis

Two distinct problems:

1. **Retrieval quality (mrc-002):** The chunks injected are often wrong. Gem ontology question got pearl content from random Things instead of the gem definition documents. Rosemary question may not have retrieved the cube-card faces. The search is matching on surface words, not intent.

2. **Model confabulation:** qwen2.5:14b (14B params) will hallucinate specifics even when given good context. It fills gaps with plausible-sounding fiction. A 14B model may not be capable of strict grounding — it's better at vibes than facts.

## Additional Finding

Session auth drops on Portal restart. Each hx invocation is a separate process — HX_TOKEN must be passed inline per call if Portal has restarted. The CLAUDE.md awakening ritual uses being aware once, but that doesn't survive restarts.