Ai

In the previous post, the important idea was that an editor is not a magical code mind… Visual Studio, VS Code, Neovim, and Emacs know how to display files and react to cursor positions. The deeper code knowledge usually lives in language services: parsers, syntax trees, symbol tables, type checkers, diagnostics, caches, and sometimes language servers speaking LSP. That explains how an editor can answer precise questions: what symbol is under the cursor? where is it defined? what type does this expression have? which references should change during a rename? where should this diagnostic squiggle appear? AI coding assistants start from a different place. ...

I walked out of the cinema expecting to think about Homer. Instead, I couldn’t stop thinking about AI. Not because Christopher Nolan’s The Odyssey is secretly about machine learning. It is not. Nor because every old story must now be converted into a parable about large language models. The connection is more interesting than that. Nolan keeps returning to people who push intelligence past an old boundary and then discover that getting there was only half the problem. ...

I had read about attention more times than I can count. Watched the 3Blue1Brown video, skimmed the Illustrated Transformer, half-read the paper. I could recite the slogan queries, keys, values but when I tried to picture what a single forward pass actually looked like in memory, I got fuzzy. Specifically: I was never sure, in a given diagram, whether each row of the attention matrix corresponded to one query or one key. That tiny ambiguity told me I did not really understand it yet. ...