Building Monk Lang — The Learning Journey

Build a programming language from scratch.

You'll go from zero to a working compiler that produces native binaries. Along the way, you'll learn compiler design, C, and how programming languages actually work.

Source

.monk

Lexer

tokens

Parser

AST

Checker

typed AST

Codegen

.c file

Binary

native

Foundations Before you write any compiler code

Unit 0 — The Ground Floor

How compilers work. Project setup. Then we build.

3 lessons

0.1

How Compilers Work

The 10-minute mental model. Source to binary, step by step.

concepts

0.2

Setting Up

Install Go, create the project, run your first test.

exercise

0.3

Reading the Source

Developer's tour: the pipeline in Go code, key structs, and where to start reading.

Phase 1 Build the tokenizer

Unit 1 — The Lexer

Break source code into tokens. Characters in, meaningful chunks out.

4 lessons

1.1

What Is a Lexer?

Characters flow in, tokens flow out. The word splitter.

concepts

1.2

Designing Monk's Token Set

Every keyword, operator, delimiter mapped to a token type.

monk

1.3

Building the Lexer

Step by step: scanning, keywords, literals, line tracking. TDD.

build

1.4

References & Further Reading

Crafting Interpreters Ch. 4, lexer blog posts.

links

Phase 2 Build the parser

Unit 2 — The Parser

Transform tokens into a tree. Understand grammar and precedence.

5 lessons

2.1

What Is a Parser?

Tokens in, tree out. The sentence diagrammer.

concepts

2.2

Operator Precedence

Why 1+2*3 is 7, not 9. 13 levels of precedence.

concepts

2.3

Designing Monk's AST

29 node types. Expressions vs statements. Assignment is a statement.

monk

2.4

Building the Parser

Recursive descent, one grammar rule at a time. TDD.

build

2.5

References & Further Reading

Crafting Interpreters Ch. 5-8, Pratt parsing.

links

Phase 3 Build the C runtime library

Unit 3 — The C Runtime

The small C library inside every Monk binary. Values, builtins, error handling.

6 lessons

3.1

Why a Runtime Library?

The small C library that lives inside every Monk binary.

concepts c

3.2

Values and Types in Memory

MonkValue: the tagged union. Stack vs heap. Type tags.

concepts c

3.3

Value Semantics

Deep copy on assign. No GC. Your data is yours.

monk

3.4

Built-in Functions

40+ functions: math, strings, arrays, file I/O, type checking.

monk c

3.5

Error Handling

guard/against/throw via setjmp/longjmp.

monk c

3.6

References & Further Reading

Tagged unions, value semantics, setjmp/longjmp resources.

links

Phase 4 Generate C from the AST

Unit 4 — Code Generation

Walk the AST, write C. Translation rules, function hoisting, source mapping.

5 lessons c

4.1

What Is Code Generation?

The final step. Walk the tree, write C. Readable output.

concepts c

4.2

Translation Rules

How each Monk construct becomes C. The systematic mapping.

monk c

4.3

Function Hoisting

Why functions are lifted above main(). The two-pass approach.

monk c

4.4

Building the Code Generator

Emitter pattern, temp variables, source mapping, real bugs.

build

4.5

References & Further Reading

Compile-to-C strategies, lcc, Chicken Scheme.

links

Phase 5 Build the CLI

Unit 5 — The CLI

monk build, monk run, monk check. The user-facing tool that ties it all together.

3 lessons

5.1

Building the CLI

Commands, flags, error format. ~255 lines, no framework.

build go

5.2

The Compilation Pipeline

From monk build to native binary. Every step, every file.

monk concepts

5.3

References & Further Reading

CLI design, Go tooling, compilation pipelines.

links

Phase 6 Type System + Scalar Unboxing

Unit 6 — The Type System

Static analysis before codegen. Type inference, structural records, all-paths return, and scalar unboxing to C parity.

5 lessons

6.1

Why Add a Type System?

What breaks without one. What Monk's checker catches before runtime.

concepts

6.2

Monk's Type Model

Nine kinds, optional flag, AssignableTo. The whole model in three parts.

concepts go

6.3

Building the Checker

Scope, hoisting, all-paths return, and the bugs we found building it.

6.4

Scalar Unboxing

Eliminating MonkValue overhead. How Monk reached C parity on scalar benchmarks.

go c

6.5

References & Further Reading

Type systems, static analysis, V8 hidden classes, TAPL.

links

Phase 7 Multi-file programs

Unit 7 — The Module System

Split programs across files. use/export syntax, dependency graphs, cycle detection, topological ordering.

4 lessons

7.1

Why a Module System?

What breaks when every program is one file. The updated compilation pipeline.

concepts

7.2

Monk's Module Syntax

The four import forms. export. How names cross file boundaries.

monk

7.3

Building the Resolver

DFS, the Graph struct, cycle detection, export validation. How src/module/ works.

build go

7.4

References & Further Reading

Module systems, topological sort, DFS cycle detection.

links

Coming Up Built as each phase is completed

C FFI

Calling C from Monk. Linking external libraries.

Linter & Formatter

monk fmt, monk lint. Code style and static analysis tooling.

LSP & Editor Support

Language server protocol. VS Code extension. Syntax highlighting.

Distribution

Homebrew, installers, cross-compilation. Shipping the compiler.

What you need before starting

Any programming language

TypeScript, Python, Go, Java. You know variables, functions, loops.

Terminal basics

You can run commands, navigate directories, use git.

C knowledge

Not needed upfront. Taught in Phase 3 (runtime) and Phase 4 (codegen).

The stack you'll build

Compiler

Written in Go
Lexer + Parser + Type Checker
Generates C source code

Runtime

Small C library (~2-5 KB)
Built-in functions
Value types, error handling

Output

Native binary
Zero runtime dependencies
Compiled via system cc