| Refresh | Home EGTry.com

common lexer tokens



1. Identifier

specification

ID  :	('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
    ;


Identifier examples

i
Hello
_action
Class1



2. float point number

specification


FLOAT
    :   ('0'..'9')+ '.' ('0'..'9')* EXPONENT?
    |   '.' ('0'..'9')+ EXPONENT?
    |   ('0'..'9')+ EXPONENT
    ;



fragment
EXPONENT : ('e'|'E') ('+'|'-')? ('0'..'9')+ ;



float number examples

3.14
0.22
1.01e+9
1E+10
11e+3
.23
.24e+0



3. String with escape characters

STRING
    :  '"' ( ESC_SEQ | ~('\\'|'"') )* '"'
    ;

fragment
HEX_DIGIT : ('0'..'9'|'a'..'f'|'A'..'F') ;

fragment
ESC_SEQ
    :   '\\' ('b'|'t'|'n'|'f'|'r'|'\"'|'\''|'\\')
    ;


string examples

"hello"
"he said, \"I am ok\""
"Escape \n and \\"



4. A single comment

antlr lexer grammar for a single comment

COMMENT
    :   '//' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
    ;


a single comment examples

//this is a comment
//////This is another comment/////



5. Multiple line comment

antlr lexer grammar for multline comment

COMMENT  :	'/*' ( options { greedy = false ;} .)* '*/' {$channel=HIDDEN};