Lesson 1.4
References & Further Reading
The books, chapters, and blog posts that teach lexing deeper than this course covers. Start with Nystrom, branch out from there.
The essential read.
Crafting Interpreters — Chapter 4: Scanning
by Robert Nystrom
Free online. The best introduction to lexing ever written. Nystrom builds a complete scanner for Lox, which is structurally similar to Monk's. Read this chapter before or alongside building the lexer.
craftinginterpreters.com/scanning.htmlWhy this book? Most compiler textbooks start with formal grammar theory. Nystrom starts with code. You build something that works, then understand why. Same philosophy as this course.
Go lexer implementations to study.
Writing An Interpreter In Go — Ch. 1: Lexing
by Thorsten Ball
Builds a lexer for Monkey (a teaching language) in Go. Almost identical architecture to what we're building. The token types, the NextToken() loop, the keyword table — all directly applicable.
interpreterbook.comGo's own scanner: go/scanner
The Go standard library includes a complete Go lexer. It's real production code — handles Unicode, number literals, string escapes. More complex than what Monk needs, but instructive to browse.
pkg.go.dev/go/scannerIf you want the theory.
Engineering a Compiler — Ch. 2: Scanners
by Cooper & Torczon
The formal approach: regular expressions, finite automata, DFA minimization. You won't need this for Monk (hand-written scanners are simpler), but it explains why tools like lex/flex exist and when they're worth using.
Compilers: Principles, Techniques, and Tools (Dragon Book) — Ch. 3
by Aho, Lam, Sethi, Ullman
The classic. Dense and academic, but comprehensive. Good as a reference, not as a first read.
Blog posts and talks.
Rob Pike — "Lexical Scanning in Go"
A talk about Go's template lexer using a state-function pattern. Elegant alternative to the switch-based approach. Worth watching after you've built the switch version.
Eli Bendersky — "How to write a lexer in Go"
Step-by-step blog post building a lexer for a simple language. Practical, code-heavy, good complement to Nystrom.
Munificent (Nystrom) — "Pratt Parsers: Expression Parsing Made Easy"
Not about lexing — but about what comes next. Read this before Phase 2.
Suggested reading order.
Crafting Interpreters Ch. 4 — read alongside building Monk's lexer. The code is in Java, but the structure maps 1:1.
Writing An Interpreter In Go Ch. 1 — if you want to see the same ideas in Go specifically.
Rob Pike's talk — after your lexer works. Interesting alternative architecture.
Dragon Book / Cooper & Torczon — only if you're curious about the formal theory. Not required for building Monk.