Lesson 8 of 10

Shipping a Parser

A grammar becomes useful when other code can call it. There are three ways to ship one: compile the grammar in your program at start-up, which is what the previous lessons did; generate Go source, so that the program needs neither the library nor the grammar; or save a compiled grammar and load it. This lesson shows the last two and the choice of a backend.

#What You Will Build

A config Go package generated from the configuration grammar, and a compiled grammar file that the command line accepts in place of the source.

#Before You Start

Complete Building Typed Trees: you need config.zp and a Go module (go mod init example.com/hello).

#Generating a Standalone Go Parser

zp gen turns a grammar into Go source, so your program does not need the ZeroParse library or the grammar at run time. The generated package (parser.go, api.go and a copy of the runtime in internal/runtime) uses only the standard library. -module is the import path the package will have, so it starts with your module path. Generate it into a config package:

zp gen -g config.zp -module example.com/hello/config -o config

and call it:

package main

import (
	"fmt"
	"log"

	"example.com/hello/config" // your module path
)

func main() {
	node, err := config.Parse("port = 8080\n")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(node)
}

Config{Entries: [Entry{Key: Key("port"), Value: Number("8080")}]}

The generated Node type mirrors zp.Node, and the generated parser gives the same trees and errors as the library. config.ParseRule(name, input) starts at another rule. The generated entry points accept functional options: config.Parse(input, config.WithUnit(config.Bytes), config.WithMaxDepth(600_000)). zp gen -max-depth 600000 sets the generated default; omitted or zero keeps 100,000.

#Backends and Compiled Grammars

A grammar can run on several backends with identical results: the default closure-compiled engine, a bytecode VM, and a bytecode VM that uses an explicit stack instead of the Go stack for deeply nested input. Choose one on the command line or in Go:

zp parse -g config.zp -f sexpr -backend bytecode < example.conf
zp parse -g config.zp -check -backend bytecode-iterative < example.conf   # validate only, prints ok

n, err := p.Parse("abc=12", zp.WithBackend(zp.Bytecode)) // also zp.WithUnit(zp.Bytes), zp.RecognizeOnly()

zp compile -g config.zp -o config.zpc saves a compiled grammar that zp parse accepts in place of the .zp file (zp parse -g config.zpc ...), and zp.LoadParser loads it without recompiling. The file serves the parses of one mode: -mode parse (the default) for parsing, -mode recognize for -check, and -mode document for documents and traces. zp convert -to json config.zp converts a grammar to JSON, which is handy for tools that generate grammars. The Running Parsers guide describes the backends and options in detail, and the Compiled Grammars guide the .zpc format.

#Next Steps

Handling Big and Live Input covers input that does not fit in memory and text that is edited while parsed. The guides Generating Parsers, Generated TypeScript Parsers, Compiled Grammars and Using the Ready-Made Parsers go further, and the cookbook recipe Ship a Parser and Keep It Up to Date shows how to keep generated code in step with the grammar.