The Complete Compilation Pipeline, End to End

{todo.title}

For our program, the token stream begins:

Keyword(Entity)   "entity"     1:1
Identifier("Todo") "Todo"      1:8
LeftBrace          "{"         1:13
Newline                        1:14
Identifier("title") "title"   2:5
Colon              ":"         2:10
Keyword(Text)      "text"      2:12
Newline                        2:16
Identifier("done")  "done"    3:5
Colon              ":"         3:9
Keyword(Bool)      "bool"      3:11
Equal              "="         3:16
Keyword(False)     "false"     3:18
Newline                        3:23
RightBrace         "}"         4:1
Newline                        4:2
Identifier("todos") "todos"   6:1
Equal              "="         6:8
Identifier("Todo")  "Todo"    6:10
Dot                "."         6:14
Keyword(All)       "all"       6:15
Newline                        6:18
...

Two observations. First, every token carries a precise position -- line and column for human-readable error messages, byte offset for machine processing. This position information propagates through the entire pipeline and ultimately appears in error diagnostics and debug information. Second, the lexer resolves the ambiguity between < as a comparison operator and < as a tag opener based on context. In code mode, < followed by a letter produces TagOpen. In expression mode, < followed by a digit or space produces Less.

The lexer's output is Vec -- a flat, ordered sequence with no hierarchy. All structure is gone. The lexer has no idea what an entity declaration looks like; it just knows that entity is a keyword, Todo is an identifier, and { is a left brace. Structure is the parser's job.

Phase 2: Syntactic Analysis

The parser consumes the token stream and builds an Abstract Syntax Tree. Where the lexer sees a flat sequence, the parser sees hierarchical structure: statements containing expressions, view elements containing children, blocks containing statements.

The parser uses recursive descent for statements and a Pratt parser for expressions. Statement parsing dispatches on the leading token:

• Keyword(Entity) -- parse entity declaration
• TagOpen -- parse view element
• Keyword(Save) -- parse save statement
• Identifier followed by Equal -- parse variable declaration or assignment
• Identifier followed by Dot -- parse expression (entity query, field access)

For our program, the AST looks like this:

Program
  Stmt::EntityDecl
    name: "Todo"
    fields:
      FieldDecl { name: "title", type: Text, default: None }
      FieldDecl { name: "done", type: Bool, default: Some(Bool(false)) }

Stmt::VarDecl name: "todos" type_ann: None value: Expr::EntityQuery { entity: "Todo", operation: All }

Stmt::View ViewElement tag: "div" attributes: [] children: ViewChild::For variable: "todo" iterable: Expr::Identifier("todos") body: ViewChild::Element tag: "p" attributes: ViewAttribute name: "class" value: Dynamic( Expr::If { condition: Expr::FieldAccess { object: Identifier("todo"), field: "done" }, then: Expr::String("completed"), else: Expr::String("pending") } ) children: ViewChild::Expression( Expr::FieldAccess { object: Identifier("todo"), field: "title" } ) ```

The parser has resolved every ambiguity. Todo.all is not a field access followed by an identifier -- it is an EntityQuery with operation All. The {for todo in todos} block is not a series of identifiers -- it is a ViewFor with a bound variable, an iterable expression, and a body of view children. The {if todo.done "completed" else "pending"} inside the class attribute is an inline conditional expression.

The AST preserves source spans on every node. When the type checker later encounters an error in the conditional expression, it can point to the exact position in the source file because the Expr::If node carries the span from the if keyword to the closing "pending" string.

Phase 3: Semantic Analysis

The type checker walks the AST and verifies that the program is semantically valid. It maintains a symbol table that maps names to types and scopes.

For our program, the type checker performs these steps:

1. Register entity schema. entity Todo { title: text, done: bool = false } adds Todo to the entity registry with two fields.

2. Type the query. Todo.all -- look up Todo in the entity registry, verify it exists, determine that .all returns [Todo] (a list of Todo entities).

3. Type the variable. todos = Todo.all -- infer that todos has type [Todo].

4. Enter the for loop. for todo in todos -- verify that todos is iterable (it is a list), bind todo with type Todo in the loop scope.

5. Type the conditional. if todo.done -- verify that todo has field done, verify that done is Bool (suitable for a condition). The then-branch ("completed") is Text, the else-branch ("pending") is Text, so the conditional expression has type Text.

6. Type the attribute. class={...} -- verify that the dynamic attribute value is Text. It is, because the conditional expression resolves to Text.

7. Type the text binding. {todo.title} -- verify that todo has field title, verify that the result is displayable. Text is displayable.

If the programmer had written {todo.titl} (a typo), the type checker would report:

error[T0005]: entity 'Todo' has no field 'titl'
  --> app.flin:11:14
   |
11 |             {todo.titl}
   |                   ^^^^ unknown field
   |
   = help: Did you mean 'title'?

The type checker does not transform the AST -- it validates it. The output of Phase 3 is the same AST with type annotations attached to expression nodes. The code generator can rely on these annotations to emit correct bytecode without re-analyzing types.

Phase 4: Code Generation

The code generator walks the typed AST and emits bytecode. Each AST node translates to a sequence of opcodes. The output is a Chunk: a constant pool, a byte array of instructions, and a line table mapping instruction offsets to source positions.

For our program, the code generation proceeds top-down:

Entity declaration. The entity Todo declaration does not emit runtime instructions. Instead, it registers the schema in the code generator's entity table, which is later serialized into the .flinc file's entity schema section. At VM startup, the runtime reads this section and initializes FlinDB's schema registry.

Variable declaration with query. todos = Todo.all emits:

QueryAll  [Todo_idx]      ; Push list of all Todo entities
StoreGlobal [todos_idx]   ; Store in global 'todos'

View element.

CreateElement [div_idx]   ; Create <div> element

LoadGlobal [todos_idx]    ; Push the todos list
StartFor [end_addr]       ; Begin iteration, jump to end_addr if empty
NextFor [todo_slot]       ; Bind current item to local 'todo'

CreateElement [p_idx]     ; Create <p> element
LoadLocal [todo_slot]     ; Load 'todo'
GetField [done_idx]       ; Get .done field
JumpIfFalse [else_addr]   ; If false, jump to else
LoadConst [completed_idx] ; Push "completed"
Jump [end_attr]           ; Jump past else
; else_addr:
LoadConst [pending_idx]   ; Push "pending"
; end_attr:
BindAttr [class_idx]      ; Bind to 'class' attribute

LoadLocal [todo_slot]     ; Load 'todo'
GetField [title_idx]      ; Get .title field
BindText                  ; Bind as reactive text

CloseElement              ; Close <p>
EndFor                    ; Loop back to NextFor
CloseElement              ; Close <div>
Halt                      ; End program

pub fn serialize(chunk: &Chunk, entities: &[EntitySchema]) -> Vec<u8> {
    let mut output = Vec::new();

loop {
    let opcode = read_byte(ip);
    ip += 1;