LLM Notes: From Transformer to RAG

LLM learning curves are steep, but the core modules are separable.

Current Framework

1. Transformer: self-attention as foundation
2. Pretrain + fine-tune: where general ability comes from
3. RAG: external knowledge when parametric memory isn’t enough

Open Questions

• As context windows grow, where does RAG’s boundary lie?
• Can small models + good tooling approximate large-model UX?

More detailed notes to follow.