Electronics Recipe Generator — Marshall & McAdam Dynamics

Live Python · FastAPI · Qdrant · Ollama · Docker

Python FastAPI Qdrant Ollama RAG Sentence Transformers Docker

How it works

01 — EMBED

Query encoded with all-MiniLM-L6-v2 (384-dim)

02 — RETRIEVE

Top-5 Hackaday projects from Qdrant (54k vectors, 1500-char snippets)

03 — MATCH

Parts fuzzy-matched to 15k reviewed Amazon products; Instructables refs from 48k indexed titles

04 — STREAM

BOM renders instantly via SSE; qwen3:0.6b synthesises build steps & narrative from all 5 projects

Live Demo

← Back to Projects

Build Journal

Summary

A five-stage RAG pipeline: semantic search over 54k Hackaday project embeddings (1500-char snippets), fuzzy-match to a community-reviewed product DB and 48k Instructables references, and a local LLM (qwen3:0.6b) that synthesises a structured Maker Receipt with build steps — streamed via SSE, cached in Redis, running fully offline in containers.

Implementation

Embedded 54,599 Hackaday posts (1500-char snippets) with all-MiniLM-L6-v2 into Qdrant (cosine, 384-dim)
Built reliability scores from 15k Amazon electronics reviews via sentiment analysis
RapidFuzz bridge maps free-text BOM parts to reviewed products; threshold-tuned per class
Indexed 47,724 Instructables hardware project titles into a second Qdrant collection (instructables_refs) for "See Also" links
Two-phase SSE endpoint: BOM appears in <2 s from Qdrant+SQLite; LLM narrative + build steps stream in after
Redis cache (SHA-256 key, TTL 3600 s) eliminates repeat-query latency
LLM synthesises across all 5 retrieved projects — receipt is a generalised build guide, not a single-article summary
Path-stripped Caddy reverse proxy with flush_interval -1 for SSE; live demo embedded as iframe

Troubleshooting

Model download failed inside sandboxed runtime — pre-seeded a shared volume via a one-shot bootstrap container
Internal DNS broken inside hardened container — switched to static container IPs
Proxy path-stripping broke JS fetch calls — changed absolute URLs to relative
Product DB stored review titles, not product names — rewrote URL builder to search by component name instead
Ollama bound to 127.0.0.1 — unreachable from containers; fixed with OLLAMA_HOST=0.0.0.0 systemd drop-in
SSE not streaming through Caddy — added flush_interval -1 to reverse_proxy block
--internal Docker network blocks DNAT port publishing — start on bridge, then connect to vault network
Instructables Kaggle dataset contains no step text — pivoted to "See Also" reference links

To Do

[x]Stream LLM tokens via SSE for real-time receipt generation
[x]Cache receipts in Redis to cut repeat-query latency
[x]Add recipe-style build steps to every receipt
[x]Synthesise from multiple community projects, not a single article
[ ]⚠ BOM not working effectively — component list is incomplete (fuzzy match limited to ~15k product index) and quantities are inaccurate (LLM defaults to 1)
[ ]Expand product index to cover full hardware categories (Octopart / DigiKey taxonomy)
[ ]Add deterministic quantity extractor: parse numbers adjacent to part names in snippet text
[ ]Add price scraping and affiliate link validation layer
[ ]User feedback loop to tune fuzzy-match threshold per component type