Scanner GeneratorsExample of JFlex LexerUsing the LexerScanner Generators• Various tools are able to convert r egular-exp ression descriptions oflexemes into scanners• For this course, flex and jflex (based on a “classic” Unix programcalled lex).• Documentation under “Miscellaneous Tools” on the web page.• JFlex demonstrated in first homework.Last modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5 1Example of JFlex Lexerpackage eg5;import java.io.*;import java.util.*;import static eg5.Token.*;%%%class Lexer%type TokenL = [a-zA-Z _ ]D = [0-9]WHITE = [ \t\r\n]%{public String lexeme;%}%%{WHITE} { /* Ignore */ }"//".* { /* Ignore */ }"=" { return AS S I G N; }"==" { return EQ U A L S; }"+" { return PL U S ; }"*" { return TI M E S ; }{L}({L}|{D}) * { lexeme = yyt e x t ( ) ; r eturn ID; }[-+]?{D}+ { lexeme = yytext (); return INT; }. { System. o ut.println ("<ERROR>"); }enum Token { ID, INT, EQUALS, PLUS, TIMES, ASSIGN }Last modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5 2Using the Lexerpublic static void main (String[] args)throws IOException {Lexer lexer = new Lexer (System.in);while (true) {Token t = lexer.yylex ();if (t == null) {System.out.p r i n tln ("EOF");return;}switch (t) {case ID: ca s e I N T :System.out.p r i n tf ("%s(\"%s\")%n",t, lexer. l exeme);break;default:System.out.p r i n tln (t);}}}Input:// a commen tag12 = 13 * ; -12Output:ID("ag12")ASSIGNINT("13")TIMES<ERROR>INT("-12")EOFLast modified: Fri Jan 28 10:59:5 5 2005 CS164: Lecture #5
View Full Document