diff options
| author | Carl Worth <cworth@cworth.org> | 2010-05-12 13:19:23 -0700 |
|---|---|---|
| committer | Carl Worth <cworth@cworth.org> | 2010-05-12 13:20:31 -0700 |
| commit | 012295f94c4b02d2683072d9aa6ab56f81409507 (patch) | |
| tree | da16e55b4b69d18f6046802af5c0652d434932e1 /glcpp-lex.l | |
| parent | a68e668b17a00ed5714cdb1e7809b7ba4522d89d (diff) | |
Simplify lexer significantly (remove all stateful lexing).
We are able to remove all state by simply passing NEWLINE through
as a token unconditionally (as opposed to only passing newline when
on a driective line as we did previously).
Diffstat (limited to 'glcpp-lex.l')
| -rw-r--r-- | glcpp-lex.l | 41 |
1 files changed, 9 insertions, 32 deletions
diff --git a/glcpp-lex.l b/glcpp-lex.l index 9ec4deb718..18d9050d71 100644 --- a/glcpp-lex.l +++ b/glcpp-lex.l @@ -32,9 +32,6 @@ %option reentrant noyywrap %option extra-type="glcpp_parser_t *" -%x ST_DEFINE -%x ST_UNDEF - SPACE [[:space:]] NONSPACE [^[:space:]] NEWLINE [\n] @@ -46,48 +43,28 @@ TOKEN {NONSPACE}+ %% {HASH}define{HSPACE}* { - BEGIN ST_DEFINE; return DEFINE; } -<ST_DEFINE>{IDENTIFIER} { - yylval.str = xtalloc_strdup (yyextra, yytext); - return IDENTIFIER; -} - -<ST_DEFINE>{TOKEN} { - yylval.str = xtalloc_strdup (yyextra, yytext); - return TOKEN; -} - -<ST_DEFINE>\n { - BEGIN INITIAL; - return NEWLINE; -} - -<ST_DEFINE>{SPACE}+ - {HASH}undef{HSPACE}* { - BEGIN ST_UNDEF; return UNDEF; } -<ST_UNDEF>{IDENTIFIER} { + +{IDENTIFIER} { yylval.str = xtalloc_strdup (yyextra, yytext); return IDENTIFIER; } -<ST_UNDEF>\n { - BEGIN INITIAL; - return NEWLINE; -} - -<ST_UNDEF>{SPACE}+ - - /* Anything we don't specifically recognize is a stream of tokens */ -{NONSPACE}+ { +{TOKEN} { yylval.str = xtalloc_strdup (yyextra, yytext); return TOKEN; } +\n { + return NEWLINE; +} + +{SPACE}+ + %% |