# Optimally Streaming Greedy Regular Expression Parsing

Niels Bjørn Bugge Grathwohl, Fritz Henglein, and Ulrik Terp Rasmussen

September, 2014

Theoretical Aspects of Computing – ICTAC 2014

## Abstract

We study the problem of streaming regular expression parsing: Given a regular expression and an input stream of symbols, how to output a serialized syntax tree representation as an output stream during input stream processing. We show that optimally streaming regular expression parsing, outputting bits of the output as early as is semantically possible for any regular expression of size m and any input string of length n, can be performed in time O(2m log m + mn) on a unit-cost random-access machine. This is for the wide-spread greedy disambiguation strategy for choosing parse trees of grammatically ambiguous regular expressions. In particular, for a fixed regular expression, the algorithm’s run-time scales linearly with the input string length. The exponential is due to the need for preprocessing the regular expression to analyze state coverage of its associated NFA, a PSPACE-hard problem, and tabulating all reachable ordered sets of NFA-states. Previous regular expression parsing algorithms operate in multiple phases, always requiring processing or storing the whole input string before out- putting the first bit of output, not only for those regular expressions and input prefixes where reading to the end of the input is strictly necessary.

@inproceedings{GHR14,
author = {Niels Bj{\o}rn Bugge Grathwohl, Fritz Henglein, and Ulrik Terp Rasmussen},
title = {Optimally Streaming Greedy Regular Expression Parsing},
booktitle = {Theoretical Aspects of Computing – ICTAC 2014},
volume = {8687},
year = {2014},
series = {Lecture Notes in Computer Science},
editor = {Ciobanu, Gabriel and Méry, Dominique},
doi = {10.1007/978-3-319-10882-7_14},
url = {http://dx.doi.org/10.1007/978-3-319-10882-7_14},
publisher = {Springer International Publishing},
month = {September},
isbn = {978-3-319-10881-0},
pages = {224-240},
abstract = {We study the problem of streaming regular expression parsing: Given a
regular expression and an input stream of symbols, how to output a
serialized syntax tree representation as an output stream during input
stream processing.
We show that optimally streaming regular expression parsing,
outputting bits of the output as early as is semantically possible for
any regular expression of size m and any input string of length n, can
be performed in time O(2m log m + mn) on a unit-cost random-access
machine. This is for the wide-spread greedy disambiguation strategy
for choosing parse trees of grammatically ambiguous regular
expressions. In particular, for a fixed regular expression, the
algorithm’s run-time scales linearly with the input string length. The
exponential is due to the need for preprocessing the regular
expression to analyze state coverage of its associated NFA, a
PSPACE-hard problem, and tabulating all reachable ordered sets of
NFA-states.
Previous regular expression parsing algorithms operate in multiple
phases, always requiring processing or storing the whole input string
before out- putting the first bit of output, not only for those
regular expressions and input prefixes where reading to the end of the
input is strictly necessary.},
}