DOC PREVIEW
Berkeley COMPSCI 164 - Scanner Generators

This preview shows page 1 out of 3 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 3 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 3 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Scanner GeneratorsExample of JFlex LexerUsing the LexerScanner Generators• Various tools are able to convert r egular-exp ression descriptions oflexemes into scanners• For this course, flex and jflex (based on a “classic” Unix programcalled lex).• Documentation under “Miscellaneous Tools” on the web page.• JFlex demonstrated in first homework.Last modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5 1Example of JFlex Lexerpackage eg5;import java.io.*;import java.util.*;import static eg5.Token.*;%%%class Lexer%type TokenL = [a-zA-Z _ ]D = [0-9]WHITE = [ \t\r\n]%{public String lexeme;%}%%{WHITE} { /* Ignore */ }"//".* { /* Ignore */ }"=" { return AS S I G N; }"==" { return EQ U A L S; }"+" { return PL U S ; }"*" { return TI M E S ; }{L}({L}|{D}) * { lexeme = yyt e x t ( ) ; r eturn ID; }[-+]?{D}+ { lexeme = yytext (); return INT; }. { System. o ut.println ("<ERROR>"); }enum Token { ID, INT, EQUALS, PLUS, TIMES, ASSIGN }Last modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5 2Using the Lexerpublic static void main (String[] args)throws IOException {Lexer lexer = new Lexer (System.in);while (true) {Token t = lexer.yylex ();if (t == null) {System.out.p r i n tln ("EOF");return;}switch (t) {case ID: ca s e I N T :System.out.p r i n tf ("%s(\"%s\")%n",t, lexer. l exeme);break;default:System.out.p r i n tln (t);}}}Input:// a commen tag12 = 13 * ; -12Output:ID("ag12")ASSIGNINT("13")TIMES<ERROR>INT("-12")EOFLast modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5


View Full Document

Berkeley COMPSCI 164 - Scanner Generators

Documents in this Course
Lecture 8

Lecture 8

40 pages

Load more
Download Scanner Generators
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Scanner Generators and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Scanner Generators 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?