Guide of CPython's Parser

devguide.python.org
3 min read
standard
Article URL: https://devguide.python.org/parser/ Comments URL: https://news.ycombinator.com/item?id=28667469 Points: 1 # Comments: 0
The Parser in CPython is currently a PEG (Parser Expression Grammar) parser. The first version of the parser used to be an LL(1) based parser that was one of the oldest parts of CPython implemented before it was replaced by PEP 617 . In particular, both the current parser and the old LL(1) parser are the output of a parser generator . This means that the way the parser is written is by feeding a description of the Grammar of the Python language to a special program (the parser generator) which outputs the parser. The way the Python language is changed is therefore by modifying the grammar file and developers rarely need to interact with the parser generator itself other than use it to generate the parser.

How PEG Parsers Work¶

A PEG (Parsing Expression Grammar) grammar (like the current one) differs from a context-free grammar in that the way it is written more closely reflects how the parser will operate when parsing it. The fundamental technical difference is that the choice operator is ordered. This means that when writing:

rule: A | B | C

a context-free-grammar parser (like an LL(1) parser) will generate constructions that given an input string will deduce which alternative ( A , B or C ) must be expanded, while a PEG parser will check if the first alternative succeeds and only if it fails, will it continue with the second or the third one in the order in which…
Read full article