Context-sensitive parsing for programming languages

Abstract

Parsing programming languages using context-sensitive rather than context-free grammars is being considered here because a stronger formalism might be beneficial for dealing with increasingly complex programming languages and their syntax, or is more appropriate in some applications. A new deterministic non-backtracking algorithm for parsing deterministic context-sensitive languages is described. It is a significant improvement of the algorithm built into WEAVE and CWEAVE tools for literate programming and requires that the language is described by a context-sensitive reduction system, namely a deterministic formalism similar to a context-sensitive grammar but with strict rules about how reductions are to be applied. The new algorithm uses a reduction automaton for finding the position of the next reduction at each step during parsing rather than a hardcoded trie that is build into the original algorithm of WEAVE and CWEAVE. The new algorithm performs at least twice as few operations per input symbol as the original one. Furthermore, it is shown that parsing a language described by a context-sensitive reduction system need not be limited to typesetting purposes as in literate programming but can be used as a general parsing approach.