diff options
Diffstat (limited to 'org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html')
| -rw-r--r-- | org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html | 214 |
1 files changed, 214 insertions, 0 deletions
diff --git a/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html b/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html new file mode 100644 index 000000000..2b4377520 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html @@ -0,0 +1,214 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> + <head> + <title>How to: Generate the Parser</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> + </head> + <body bgcolor="#FFFFFF" text="#000000"> + <table border=0 cellspacing=5 cellpadding=2 width="100%" > + +<tr> + <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> + JDT Core / HowTo: Generate the Parser </font></b></td> +</tr> + +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Where to get the parser generator</b><br> + <blockquote>The parser files and resources are automatically generated using the LPG parser generator. This tools has + been renamed Jikes Parser generator. You can find more information and latest releases at this <A HREF="http://www-124.ibm.com/developerworks/projects/jikes/">link</A>. + The latest tool is provided in source format. We don't provide any help for compiling these + source files. Refer to the link above if you have trouble to get binaries. + <br>Our grammar is generated using the version 2.30 of LPG. If newer versions fail to generate resources from our + grammar, please send request to the Jikes Parser Generator team. + </blockquote> + </p> + </td> +</tr> +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Where to get the grammar</b><br> + <blockquote>The latest grammar is always located in the <code>grammar()</code> method of the <code>Parser</code> class. Go to the org.eclipse.jdt.core plugins + directory (<font color="#3366FF">eclipse\plugins\org.eclipse.jdt.core</font> where eclipse is the root of your eclipse installation) and open the + <font color="#3366FF">jdtcoresrc.zip</font> file. Then search for the <font color="#3366FF">Parser.java</font> inside the + folder <font color="#3366FF">org\eclipse\jdt\internal\compiler\parser\</font>. You need to look for the method named + <code>grammar()</code>. Then copy its contents from: +<PRE>--main options +%options ACTION, AN=JavaAction.java, GP=java, +.... +$end +-- need a carriage return after the $end +</PRE> + into a file called <font color="#3366FF">java.g</font>. It is important to add a carriage return at the end of the last line. + You can save this file where you want, we will assume from thereon you saved it in <code>d:\temp\</code>. + </blockquote> + </p> + </td> +</tr> +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>What to do with these files and update the parser class...</b><br> + <blockquote> +Assuming, the LPG executable (<code>lpg.exe</code> or <code>jikespg.exe</code>) is located inside <code>d:\lpg</code>. + <ol> + <li>First in a console, run: + <PRE> + d: + cd \temp + d:\lpg\lpg.exe java.g + </PRE> + </li> + <li>You will get an output that looks like this: + <PRE> + LPG Parser Generator (V2.30) Tue Apr 02 12:49:13 2002 +%OPTIONS ACTION, AN=JavaAction.java, GP=java, +%OPTIONS FILE-PREFIX=java, ESCAPE=$, PREFIX=TokenName, OUTPUT-SIZE=125 , +%OPTIONS NOGOTO-DEFAULT, SINGLE-PRODUCTIONS, LALR=1 , TABLE=TIME , +%OPTIONS ERROR_MAPS +%OPTIONS first follow +%OPTIONS TRACE=FULL , +%OPTIONS VERBOSE +Options in effect: + ACTION ACTFILE-NAME=JavaAction.java BLOCKB=/. BLOCKE=./ BYTE CONFLIC + DEFAULT=5 NODEBUG DEFERRED NOEDIT ERROR-MAPS ESCAPE=$ + FILE-PREFIX=java FIRST FOLLOW GENERATE-PARSER=JAVA NOGOTO-DEFAULT + HACTFILE-NAME=javahdr.java HBLOCKB=/: HBLOCKE=:/ LALR=1 LIST + MAX-DISTANCE=30 MIN-DISTANCE=3 NAMES=OPTIMIZED NONT-CHECK ORMARK=| + OUTPUT-SIZE=125 PREFIX=TokenName READ-REDUCE NOSCOPES NOSHIFT-DEFAULT + SINGLE-PRODUCTIONS STACK-SIZE=128 STATES SUFFIX= TABLE=TIME TRACE=FU + VERBOSE WARNINGS XREF + + +This grammar is LALR(1). + +Number of Terminals: 105 +Number of Nonterminals: 202 +Number of Productions: 437 +Number of Single Productions: 162 +Number of Items: 1265 +Number of States: 591 +Number of Shift actions: 3482 +Number of Goto actions: 4061 +Number of Shift/Reduce actions: 369 +Number of Goto/Reduce actions: 687 +Number of Reduce actions: 7736 +Number of Shift-Reduce conflicts: 0 +Number of Reduce-Reduce conflicts: 0 +Number of Reductions saved by default: 4913 +Reallocating storage for TIME table, adding 3603 entries + +Length of Check table: 16836 +Length of Action table: 16608 +Number of entries in Action Table: 12013 +Percentage of increase: 38.2% +Highest symbol in Check Table: 307 +Storage Required for Tables: 66888 Bytes, 66K +Storage Required for Rules: 1308 Bytes + + +Actions in Compressed Tables: + Number of Shifts: 3482 + Number of Shift/Reduces: 369 + Number of Gotos: 4061 + Number of Goto/Reduces: 687 + Number of Reduces: 2823 + Number of Defaults: 390 + +Error maps storage: + Storage required for ACTION_SYMBOLS_BASE map: 1182 Bytes + Storage required for ACTION_SYMBOLS_RANGE map: 1007 Bytes + Storage required for NACTION_SYMBOLS_BASE map: 1182 Bytes + Storage required for NACTION_SYMBOLS_RANGE map: 630 Bytes + Storage required for SYMBOL_INDEX map: 616 Bytes + Storage required for STRING_BUFFER map: 4652 Bytes + +***Warning: Base Check vector contains value > 127. 16-bit words used. +***Warning: Terminal symbol > 127. 16-bit words used. +Escaped symbol $eof is an invalid C variable. + +Escaped symbol $error is an invalid C variable. + </PRE> + It can be quite different if the output changed since the version 2.30 of lpg. The important part is:<br> + <blockquote><b>This grammar is LALR(1).</b></blockquote> + This creates in the current directory some java source files and information files. + <blockquote><table BORDER=1 CELLSPACING=2 CELLPADDING=10> + <tr> + <th VALIGN=top align=left>java.l</th> + <td VALIGN=top>Information generated by lpg/jikespg. Enumarate all the states created for the automaton, etc.</td> + </tr> + <tr> + <th VALIGN=top align=left>JavaAction.java</th> + <td>It contains the method consumeRule(int) of the class org.eclipse.jdt.internal.compiler.parser.Parser that handles all semantic actions dispatches.</td> + </tr> + <tr> +<th VALIGN=top align=left>javahdr.java</th> + <td>You don't need this file. It is actually empty.</td> + </tr> + <tr> +<th VALIGN=top align=left>javadcl.java</th> + <td>This files is used to generate the resources files.</td> + </tr> + <tr> +<th VALIGN=top align=left>javasym.java</th> + <td>This is the contents of the class org.eclipse.jdt.core.compiler.ITerminalSymbols. You need to replace:<br> + <ul> + <li>TokenName$eof with TokenNameEOF</li> + <li>TokenName$error with TokenNameERROR</li> + </ul></td> + </tr> + <tr> +<th VALIGN=top align=left>javadef.java</th> + <td>This is the contents of the class org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation.</td> + </tr> + <tr> +<th VALIGN=top align=left>javaprs.java</th> + <td>You don't need this file. Its contents is already inlined in the Parser class.</td> + </tr> + </table></blockquote> + </li> + <li><blockquote>Now we need to update the different classes and resource files. + </blockquote> + <ol> + <li>Copy the contents of the <font color="#3366FF">JavaAction.java</font> file into the <font color="#3366FF">consumeRule(int)</font> method of the org.eclipse.jdt.internal.compiler.parser.Parser class. + </li> + <li>The definition of the Parser needs to be updated with two tables from <font color="#3366FF">javadcl.java</font>. Those are <font color="#3366FF">rhs[]</font> and <font color="#3366FF">name[]</font>. + The following entries in name[] need to be replaced: + <ul> + <li><font color="#3366FF">$eof</font> with <font color="#3366FF">UNEXPECTED_EOF</font></li> + <li><font color="#3366FF">$error</font> with <font color="#3366FF">"Invalid Character"</font></li> + </ul> +The previous definition of name[] will guide you. + </li> + <li>The class <font color="#3366FF">org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation</font> needs to be updated with the content of the file <font color="#3366FF">javadef.java</font>. Don't copy the + interface name. Simply copy the field declarations. The actual source of this class will guide you.</li> + <li>This is the contents of the class org.eclipse.jdt.internal.compiler.parser.TerminalSymbols. You need to replace:<br> + <ul> + <li><font color="#3366FF">TokenName$eof</font> with <font color="#3366FF">TokenNameEOF</font></li> + <li><font color="#3366FF">TokenName$error</font> with <font color="#3366FF">TokenNameERROR</font></li> + </ul> + </li> + <li>The last step is to update the resource files:<br> + Copy the jdtcore.jar file in d:\temp. Compile this <A HREF="UpdateParserFiles.java">source</A> inside d:\temp. You will have a file UpdateParserFiles.class. + Then run the following command-line: + <PRE> + D:\temp>java -classpath jdtcore.jar;. UpdateParserFiles javadcl.java + </PRE> +Once this done, you will end up with 5 new files inside d:\temp. They are called parser<n>.rsc, with n equals to 1..5. +All these files need to be moved to the org\eclipse\jdt\internal\compiler\parser folder. Now you are ready to execute and test +the new parser. + </li> + </ol> + </li> + </ol> + <P> + <b>NOTE:</b> <blockquote>Changing the parser is a risky operation if you miss one of the steps above. The resulting parser can be completely + unpredictable. It can go from crashing to reporting invalid errors. Be sure that you followed all the steps and that all the + files are updated and recompiled before you run it. </blockquote> + </P> + </blockquote> +</td> +</tr> +</table> +</body> +</html> |