diff options
| author | wisberg <wisberg> | 2002-12-16 17:58:19 +0000 |
|---|---|---|
| committer | wisberg <wisberg> | 2002-12-16 17:58:19 +0000 |
| commit | d842c4f1139629c1f062b74ba818d233b2c31043 (patch) | |
| tree | 842d3871620bc0eb60edcd95e55804d67e0f61fa /org.eclipse.jdt.core/notes | |
| parent | 3ce247199704eae6b2c92c6e38c69584e3250c52 (diff) | |
| download | aspectj-d842c4f1139629c1f062b74ba818d233b2c31043.tar.gz aspectj-d842c4f1139629c1f062b74ba818d233b2c31043.zip | |
initial version
Diffstat (limited to 'org.eclipse.jdt.core/notes')
24 files changed, 8966 insertions, 0 deletions
diff --git a/org.eclipse.jdt.core/notes/howto/batch compile/batchCompile.html b/org.eclipse.jdt.core/notes/howto/batch compile/batchCompile.html new file mode 100644 index 000000000..71dd4b383 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/batch compile/batchCompile.html @@ -0,0 +1,270 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.79 [en] (Windows NT 5.0; U) [Netscape]"> + <title>How to: Run batch compiler</title> +<link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> +</head> +<body text="#000000" bgcolor="#FFFFFF"> + +<table BORDER=0 CELLSPACING=5 CELLPADDING=2 WIDTH="100%" > +<tr> +<td ALIGN=LEFT VALIGN=TOP COLSPAN="2" BGCOLOR="#0080C0"><b><font color="#FFFFFF">JDT +Core / HowTo: Run the Batch Compiler </font></b></td> +</tr> + +<tr> +<td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + +<td WIDTH="98%"><b>Finding the batch compiler</b> +<br>The batch compiler class is located in the internal classes of the +JDT/Core plugin. So it is in the <i>jdtcore.jar</i> file in the directory +<i>plugins/org.eclipse.jdt.core</i>. +The name of the class is <i>org.eclipse.jdt.internal.compiler.batch.Main</i>. </td> +</tr> + +<tr> +<td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + +<td WIDTH="98%"><b>Running the batch compiler</b> +<ul> +<li> +Using the main method.</li> + +<blockquote>Using the main method. The Main class has a main method. This +is the classical way to invoke the batch compiler on a command-line. +<ul> +<li> +For example on a command-line:</li> + +<br><font color="#3366FF">java -classpath jdtcore.jar org.eclipse.jdt.internal.compiler.batch.Main +-classpath rt.jar A.java</font> +<li> +For example in a java source:</li> + +<br><font color="#3366FF">org.eclipse.jdt.internal.compiler.batch.Main.main(new +String[] {"-classpath", "rt.jar", "A.java"});</font></ul> +</blockquote> + +<li> +Using the static compile(String) method.</li> + +<blockquote>The compile(String) method is a convenient method to invoke +the batch compiler in a java application. +<br>Instead of: +<ul> +<li> +<font color="#3366FF">org.eclipse.jdt.internal.compiler.batch.Main.main(new +String[] {"-classpath", "rt.jar", "A.java"});</font></li> + +<li> +you can simply write: <font color="#3366FF">org.eclipse.jdt.internal.compiler.batch.Main.compile("-classpath +rt.jar A.java");</font></li> +</ul> +</blockquote> +</ul> +</td> +</tr> + +<tr> +<td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + +<td WIDTH="98%"><b>Which options are available?</b> +<p> +With the yellow background, these are required options.<br> +With the orange background, these are suggested options. +</p> +<blockquote> +<table BORDER CELLSPACING=2 CELLPADDING=2 COLS=2 WIDTH="100%" > +<tr> +<th>Name</th> +<th>Usage</th> +</tr> +<tr> +<td>-help</td> +<td valign=top>Display the help message</td> +</tr> +<tr> +<td valign=top>-version</td> +<td>Display the build number of the compiler. This is very useful to report a bug.</td> +</tr> +<tr> +<td valign=top bgcolor="#FFFFCC">-classpath <dir 1>;<dir 2>;...;<dir P></td> +<td valign=top bgcolor="#FFFFCC">This is a list of directory or jar files used to compile the source files. There is no default classpath. So this option +is always required to compile source files.</td> +</tr> +<tr> +<td valign=top bgcolor="#FFCCAA">-d <dir 1>|none</td> +<td bgcolor="#FFCCAA">This is used to specify in which directory the generated .class files should be dumped. If it is omitted, no package directory structure is created.<br> +If you don't want to generate .class files, use <font color="#3366FF">-d none</font>.</td> +</tr> +<tr> +<td valign=top>-target 1.1|1.2</td> +<td>This specifies the classfile target setting. The possible value are <font color="#3366FF">1.1</font> or <font color="#3366FF">1.2</font>, default is <font color="#3366FF">1.1</font></td> +</tr> +<tr> +<td valign=top>-1.3</td> +<td>Set compliance level to <font color="#3366FF">1.3</font> (default)</td> +</tr> +<tr> +<td valign=top>-1.4</td> +<td>Set compliance level to <font color="#3366FF">1.4</font>.</td> +</tr> +<tr> +<td valign=top>-source 1.3|1.4</td> +<td>This is used to enable the assertion support of the compiler. The possible value are: <font color="#3366FF">1.3</font> or <font color="#3366FF">1.4</font>, default is <font color="#3366FF">1.3</font> in <font color="#3366FF">-1.3</font> mode and <font color="#3366FF">1.4</font> in <font color="#3366FF">-1.4</font> mode. +In <font color="#3366FF">1.4</font>, <font color="#3366FF"><I>assert</I></font> is treated as a keyword.</td> +</tr> +<tr> +<td valign=top>-warn: <blockquote>constructorName<br>|packageDefaultMethod<br>|deprecation<br>|maskedCatchBlocks<br>|unusedLocals<br>|unusedArguments<br>|unusedImports<br>|syntheticAccess<br>|assertIdentifier</blockquote> +</td> +<td>Set warning level.<br>e.g. <font color="#3366FF">-warn:unusedLocals,deprecation</font><br> +<table> +<tr> +<th align=left>constructorName</th> +<td>warn method with constructor name</td> +</tr> +<tr> +<th align=left>packageDefaultMethod</th> +<td>warn attempt to override package-default method</td> +</tr> +<tr> +<th align=left>deprecation</th> +<td>warn usage of deprecated type or member</td> +</tr> +<tr> +<th align=left>maskedCatchBlocks</th> +<td>warn hidden catch block</td> +</tr> +<tr> +<th align=left>unusedLocals</th> +<td>warn unused local variable</td> +</tr> +<tr> +<th align=left>unusedArguments</th> +<td>warn unused method argument</td> +</tr> +<tr> +<th align=left>unusedImports</th> +<td>When enabled, the compiler will issue an error or a warning for unused import reference +</td> +</tr> +<tr> +<th align=left>syntheticAccess</th> +<td>warn when performing synthetic access for innerclass</td> +</tr> +<tr> +<th align=left>assertIdentifier</th> +<td>warn occurrence of <i>assert</i> used as identifier</td> +</tr> +</table> +</td> +</tr> +<tr> +<td valign=top>-nowarn</td> +<td>No warning (equivalent to <font color="#3366FF">-warn:none</font>)</td> +</tr> +<tr> +<td valign=top>-deprecation</td> +<td>Equivalent to <font color="#3366FF">-warn:deprecation</font>.</td> +</tr> +<tr> +<td valign=top>-g[:none|:lines,vars,source] +</td> +<td>Set the debug attributes level<br> +<table> +<tr> +<th align=left>-g</th> +<td>All debug info (equivalent to <font color="#3366FF">-g:lines,vars,source</font>) +</td> +</tr> +<th align=left>-g:none</th> +<td>No debug info</td> +</tr> +<tr> +<th align=left>-g:[lines,vars,source]</th> +<td>Selective debug info</td> +</tr> +</table> +</tr> +<tr> +<td valign=top>-preserveAllLocals</td> +<td>Explicitly request the compiler to preserve all local variables (for debug purpose). If omitted, the compiler will removed unused locals.</td> +</tr> +<tr> +<td valign=top>-noImportError</td> +<td>The compiler won't report an error for unresolved imports. A warning is issued instead.</td> +</tr> +<tr> +<td valign=top>-encoding <encoding name></td> +<td>Specify default source encoding format (custom encoding can also be specifed on a per file basis by suffixing each input source file/folder name with <font color="#3366FF">[encoding <encoding name>]</font>).</td> +</tr> +<tr> +<td valign=top>-log <filename></td> +<td>Specify a log file in which all output from the compiler will be dumped. This is really useful if you want to debug the batch +compiler or get a file which contains all errors and warnings from a batch build.</td> +</tr> +<tr> +<td valign=top>-proceedOnError</td> +<td>Keep compiling when error, dumping class files with problem methods or problem types. This is recommanded only if you want +to be able to run your application even if you have remaining errors.</td> +</tr> +<tr> +<td valign=top>-verbose</td> +<td>Print accessed/processed compilation units in the console or the log file if specified.</td> +</tr> +<tr> +<td valign=top>-referenceInfo</td> +<td>Compute reference info. This is useful only if connected to the builder. The reference infos are useless otherwise.</td> +</tr> +<tr> +<td valign=top>-progress</td> +<td>Show progress (only in -log mode)</td> +</tr> +<tr> +<td valign=top>-time +</td> +<td>Display speed information</td> +</tr> +<tr> +<td valign=top>-noExit</td> +<td>Do not call <font color="#3366FF">System.exit(n)</font> at end of compilation (<font color="#3366FF">n=0</font> if no error)</td> +</tr> +<tr> +<td valign=top>-repeat <n> +</td> +<td>Repeat compilation process <font color="#3366FF"><n></font> times (perf analysis).</td> +</tr> +</table> +</blockquote> +</td> +</tr> +<tr> +<td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + +<td WIDTH="98%"><b>Examples</b> +<blockquote> +<table> +<td valign=top><font color="#3366FF">d:\temp -classpath rt.jar -time -g -d d:/tmp</font> +</td> +<td valign=top>It compiles all source files in d:\temp and its subfolders. The classpath is simply rt.jar. It generates all debug +attributes and all generated .class files are dumped in d:\tmp. The speed of the compiler will be displayed once the batch process +is completed.</td> +</tr> +<tr> +<td valign=top><font color="#3366FF">d:\temp\Test.java -classpath d:\temp;rt.jar -g:none</font> +</td> +<td valign=top>It compiles only Test.java and it will retrieve any dependant files from d:\temp. The classpath is rt.jar and d:\temp, which means that all necessary classes +are searched first in d:\temp and then in rt.jar. It generates no debug attributes and all generated .class files are dumped in d:\tmp.</td> +</table> +</blockquote> +</td> +</tr> + +</table> + + + +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/howto/flush metadata/flushMetadata.html b/org.eclipse.jdt.core/notes/howto/flush metadata/flushMetadata.html new file mode 100644 index 000000000..f7208fed6 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/flush metadata/flushMetadata.html @@ -0,0 +1,28 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> + <head> + <title>How to: Flush the Metadata</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> + </head> + <body bgcolor="#FFFFFF" text="#000000"> + <table border=0 cellspacing=5 cellpadding=2 width="100%" > + +<tr> + <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> + JDT Core / HowTo: Flush the Metadata </font></b></td> +</tr> + +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Topic1</b><br> + This is the first topic + <ul> + Blabla1 + </ul> + </td> +</tr> + +</table> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/howto/generate parser/UpdateParserFiles.java b/org.eclipse.jdt.core/notes/howto/generate parser/UpdateParserFiles.java new file mode 100644 index 000000000..92f11fb1c --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/generate parser/UpdateParserFiles.java @@ -0,0 +1,18 @@ +import java.io.IOException; +import org.eclipse.jdt.internal.compiler.parser.Parser; + +public class UpdateParserFiles { + + public static void main(String[] args) throws IOException { + if (args.length != 1) { + printUsage(); + return; + } + Parser.buildFilesFromLPG(args[0]); + } + + public static void printUsage() { + System.out.println("Usage: UpdateParserFiles <path to javadcl.java>"); + System.out.println("e.g. UpdateParserFiles c:/javadcl.java"); + } +} diff --git a/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html b/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html new file mode 100644 index 000000000..2b4377520 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/generate parser/generateParser.html @@ -0,0 +1,214 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> + <head> + <title>How to: Generate the Parser</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> + </head> + <body bgcolor="#FFFFFF" text="#000000"> + <table border=0 cellspacing=5 cellpadding=2 width="100%" > + +<tr> + <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> + JDT Core / HowTo: Generate the Parser </font></b></td> +</tr> + +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Where to get the parser generator</b><br> + <blockquote>The parser files and resources are automatically generated using the LPG parser generator. This tools has + been renamed Jikes Parser generator. You can find more information and latest releases at this <A HREF="http://www-124.ibm.com/developerworks/projects/jikes/">link</A>. + The latest tool is provided in source format. We don't provide any help for compiling these + source files. Refer to the link above if you have trouble to get binaries. + <br>Our grammar is generated using the version 2.30 of LPG. If newer versions fail to generate resources from our + grammar, please send request to the Jikes Parser Generator team. + </blockquote> + </p> + </td> +</tr> +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Where to get the grammar</b><br> + <blockquote>The latest grammar is always located in the <code>grammar()</code> method of the <code>Parser</code> class. Go to the org.eclipse.jdt.core plugins + directory (<font color="#3366FF">eclipse\plugins\org.eclipse.jdt.core</font> where eclipse is the root of your eclipse installation) and open the + <font color="#3366FF">jdtcoresrc.zip</font> file. Then search for the <font color="#3366FF">Parser.java</font> inside the + folder <font color="#3366FF">org\eclipse\jdt\internal\compiler\parser\</font>. You need to look for the method named + <code>grammar()</code>. Then copy its contents from: +<PRE>--main options +%options ACTION, AN=JavaAction.java, GP=java, +.... +$end +-- need a carriage return after the $end +</PRE> + into a file called <font color="#3366FF">java.g</font>. It is important to add a carriage return at the end of the last line. + You can save this file where you want, we will assume from thereon you saved it in <code>d:\temp\</code>. + </blockquote> + </p> + </td> +</tr> +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>What to do with these files and update the parser class...</b><br> + <blockquote> +Assuming, the LPG executable (<code>lpg.exe</code> or <code>jikespg.exe</code>) is located inside <code>d:\lpg</code>. + <ol> + <li>First in a console, run: + <PRE> + d: + cd \temp + d:\lpg\lpg.exe java.g + </PRE> + </li> + <li>You will get an output that looks like this: + <PRE> + LPG Parser Generator (V2.30) Tue Apr 02 12:49:13 2002 +%OPTIONS ACTION, AN=JavaAction.java, GP=java, +%OPTIONS FILE-PREFIX=java, ESCAPE=$, PREFIX=TokenName, OUTPUT-SIZE=125 , +%OPTIONS NOGOTO-DEFAULT, SINGLE-PRODUCTIONS, LALR=1 , TABLE=TIME , +%OPTIONS ERROR_MAPS +%OPTIONS first follow +%OPTIONS TRACE=FULL , +%OPTIONS VERBOSE +Options in effect: + ACTION ACTFILE-NAME=JavaAction.java BLOCKB=/. BLOCKE=./ BYTE CONFLIC + DEFAULT=5 NODEBUG DEFERRED NOEDIT ERROR-MAPS ESCAPE=$ + FILE-PREFIX=java FIRST FOLLOW GENERATE-PARSER=JAVA NOGOTO-DEFAULT + HACTFILE-NAME=javahdr.java HBLOCKB=/: HBLOCKE=:/ LALR=1 LIST + MAX-DISTANCE=30 MIN-DISTANCE=3 NAMES=OPTIMIZED NONT-CHECK ORMARK=| + OUTPUT-SIZE=125 PREFIX=TokenName READ-REDUCE NOSCOPES NOSHIFT-DEFAULT + SINGLE-PRODUCTIONS STACK-SIZE=128 STATES SUFFIX= TABLE=TIME TRACE=FU + VERBOSE WARNINGS XREF + + +This grammar is LALR(1). + +Number of Terminals: 105 +Number of Nonterminals: 202 +Number of Productions: 437 +Number of Single Productions: 162 +Number of Items: 1265 +Number of States: 591 +Number of Shift actions: 3482 +Number of Goto actions: 4061 +Number of Shift/Reduce actions: 369 +Number of Goto/Reduce actions: 687 +Number of Reduce actions: 7736 +Number of Shift-Reduce conflicts: 0 +Number of Reduce-Reduce conflicts: 0 +Number of Reductions saved by default: 4913 +Reallocating storage for TIME table, adding 3603 entries + +Length of Check table: 16836 +Length of Action table: 16608 +Number of entries in Action Table: 12013 +Percentage of increase: 38.2% +Highest symbol in Check Table: 307 +Storage Required for Tables: 66888 Bytes, 66K +Storage Required for Rules: 1308 Bytes + + +Actions in Compressed Tables: + Number of Shifts: 3482 + Number of Shift/Reduces: 369 + Number of Gotos: 4061 + Number of Goto/Reduces: 687 + Number of Reduces: 2823 + Number of Defaults: 390 + +Error maps storage: + Storage required for ACTION_SYMBOLS_BASE map: 1182 Bytes + Storage required for ACTION_SYMBOLS_RANGE map: 1007 Bytes + Storage required for NACTION_SYMBOLS_BASE map: 1182 Bytes + Storage required for NACTION_SYMBOLS_RANGE map: 630 Bytes + Storage required for SYMBOL_INDEX map: 616 Bytes + Storage required for STRING_BUFFER map: 4652 Bytes + +***Warning: Base Check vector contains value > 127. 16-bit words used. +***Warning: Terminal symbol > 127. 16-bit words used. +Escaped symbol $eof is an invalid C variable. + +Escaped symbol $error is an invalid C variable. + </PRE> + It can be quite different if the output changed since the version 2.30 of lpg. The important part is:<br> + <blockquote><b>This grammar is LALR(1).</b></blockquote> + This creates in the current directory some java source files and information files. + <blockquote><table BORDER=1 CELLSPACING=2 CELLPADDING=10> + <tr> + <th VALIGN=top align=left>java.l</th> + <td VALIGN=top>Information generated by lpg/jikespg. Enumarate all the states created for the automaton, etc.</td> + </tr> + <tr> + <th VALIGN=top align=left>JavaAction.java</th> + <td>It contains the method consumeRule(int) of the class org.eclipse.jdt.internal.compiler.parser.Parser that handles all semantic actions dispatches.</td> + </tr> + <tr> +<th VALIGN=top align=left>javahdr.java</th> + <td>You don't need this file. It is actually empty.</td> + </tr> + <tr> +<th VALIGN=top align=left>javadcl.java</th> + <td>This files is used to generate the resources files.</td> + </tr> + <tr> +<th VALIGN=top align=left>javasym.java</th> + <td>This is the contents of the class org.eclipse.jdt.core.compiler.ITerminalSymbols. You need to replace:<br> + <ul> + <li>TokenName$eof with TokenNameEOF</li> + <li>TokenName$error with TokenNameERROR</li> + </ul></td> + </tr> + <tr> +<th VALIGN=top align=left>javadef.java</th> + <td>This is the contents of the class org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation.</td> + </tr> + <tr> +<th VALIGN=top align=left>javaprs.java</th> + <td>You don't need this file. Its contents is already inlined in the Parser class.</td> + </tr> + </table></blockquote> + </li> + <li><blockquote>Now we need to update the different classes and resource files. + </blockquote> + <ol> + <li>Copy the contents of the <font color="#3366FF">JavaAction.java</font> file into the <font color="#3366FF">consumeRule(int)</font> method of the org.eclipse.jdt.internal.compiler.parser.Parser class. + </li> + <li>The definition of the Parser needs to be updated with two tables from <font color="#3366FF">javadcl.java</font>. Those are <font color="#3366FF">rhs[]</font> and <font color="#3366FF">name[]</font>. + The following entries in name[] need to be replaced: + <ul> + <li><font color="#3366FF">$eof</font> with <font color="#3366FF">UNEXPECTED_EOF</font></li> + <li><font color="#3366FF">$error</font> with <font color="#3366FF">"Invalid Character"</font></li> + </ul> +The previous definition of name[] will guide you. + </li> + <li>The class <font color="#3366FF">org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation</font> needs to be updated with the content of the file <font color="#3366FF">javadef.java</font>. Don't copy the + interface name. Simply copy the field declarations. The actual source of this class will guide you.</li> + <li>This is the contents of the class org.eclipse.jdt.internal.compiler.parser.TerminalSymbols. You need to replace:<br> + <ul> + <li><font color="#3366FF">TokenName$eof</font> with <font color="#3366FF">TokenNameEOF</font></li> + <li><font color="#3366FF">TokenName$error</font> with <font color="#3366FF">TokenNameERROR</font></li> + </ul> + </li> + <li>The last step is to update the resource files:<br> + Copy the jdtcore.jar file in d:\temp. Compile this <A HREF="UpdateParserFiles.java">source</A> inside d:\temp. You will have a file UpdateParserFiles.class. + Then run the following command-line: + <PRE> + D:\temp>java -classpath jdtcore.jar;. UpdateParserFiles javadcl.java + </PRE> +Once this done, you will end up with 5 new files inside d:\temp. They are called parser<n>.rsc, with n equals to 1..5. +All these files need to be moved to the org\eclipse\jdt\internal\compiler\parser folder. Now you are ready to execute and test +the new parser. + </li> + </ol> + </li> + </ol> + <P> + <b>NOTE:</b> <blockquote>Changing the parser is a risky operation if you miss one of the steps above. The resulting parser can be completely + unpredictable. It can go from crashing to reporting invalid errors. Be sure that you followed all the steps and that all the + files are updated and recompiled before you run it. </blockquote> + </P> + </blockquote> +</td> +</tr> +</table> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/howto/patch jdtcore/patchJdtcore.html b/org.eclipse.jdt.core/notes/howto/patch jdtcore/patchJdtcore.html new file mode 100644 index 000000000..7d42931c5 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/patch jdtcore/patchJdtcore.html @@ -0,0 +1,28 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> + <head> + <title>How to: Patch Jdt/Core</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> + </head> + <body bgcolor="#FFFFFF" text="#000000"> + <table border=0 cellspacing=5 cellpadding=2 width="100%" > + +<tr> + <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> + JDT Core / HowTo: Patch Jdt/Core</font></b></td> +</tr> + +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Topic1</b><br> + This is the first topic + <ul> + Blabla1 + </ul> + </td> +</tr> + +</table> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/howto/set classpath/setClasspath.html b/org.eclipse.jdt.core/notes/howto/set classpath/setClasspath.html new file mode 100644 index 000000000..d90ec7e58 --- /dev/null +++ b/org.eclipse.jdt.core/notes/howto/set classpath/setClasspath.html @@ -0,0 +1,28 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> + <head> + <title>How to: Set the Classpath</title> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> + </head> + <body bgcolor="#FFFFFF" text="#000000"> + <table border=0 cellspacing=5 cellpadding=2 width="100%" > + +<tr> + <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> + JDT Core / HowTo: Set the Classpath </font></b></td> +</tr> + +<tr> + <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> + <td WIDTH="98%"><b>Topic1</b><br> + This is the first topic + <ul> + Blabla1 + </ul> + </td> +</tr> + +</table> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/classpath container/classpathContainer.html b/org.eclipse.jdt.core/notes/r2.0/classpath container/classpathContainer.html new file mode 100644 index 000000000..47b035739 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/classpath container/classpathContainer.html @@ -0,0 +1,308 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta http-equiv="Content-Language" content="en-us"> + <meta name="GENERATOR" content="Microsoft FrontPage 4.0"> + <meta name="ProgId" content="FrontPage.Editor.Document"> + <title>Class Path Container Enhancement</title> +</head> +<body> + +<h1>Class Path Container Enhancement</h1> +<span style="FONT-SIZE: 10pt">Last Modified April 23, 2002</span> + + +<h2> +Background</h2> +JDT supports to switch the JDK that is used for building. It is currently +implemented as follows: +<ul> +<li> +org.eclipse.jdt.launching maintains the following JDK/VM information in +its plugin metadata:</li> + +<ul> +<li> +a set of <i>VM install types</i>: a description of a VM install. It knows +how to find the location of the binary JAR and the source JARs.</li> + +<li> +<i>VM installs: </i>the location/home of a VM install on the file system. +A VM install has an internal ID that is not visible to the user.</li> + +<li> +one of the VM installs is marked as the <i>default VM install</i>.</li> +</ul> + +<li> +org.eclipse.jdt.launching defines a JRE_LIB, JRE_SRC, JRE_SRCROOT variables +that binds to the default VM install:</li> + +<ul> +<li> +JRE_LIB: the binary JAR (e.g. rt.jar)</li> + +<li> +JRE_SRC: the source JAR/zip (e.g. src.jar)</li> + +<li> +JRE_SRCROOT: the prefix in the source JAR (e.g. "src")</li> +</ul> + +<li> +The Java project creation wizard adds a JRE_LIB classpath variable on a +project's build class path.</li> + +<li> +org.eclipse.jdt.debug.ui contributes a preference page to define new VM +installs and to set the default VM install. When the default VM install +changes, then the bindings of the JRE_* variables are changed accordingly.</li> + +<li> +org.eclipse.jdt.ui contributes a class path variable preference page. It +"knows" that the JRE_* variables are reserved and doesn't allow the user +to edit them.</li> + +<li> +The packages view shows the resolved JRE_LIB contents as a referenced library.</li> +</ul> +Characteristics of the current implementation: +<ul> +<li> +Class path stability: changing the default JRE/VM install doesn't affect +the build class path since the JRE_LIB variable is not affected by this +change. In other words, when a user changes the JRE for building then the +.classpath file is not affected.</li> + +<li> +JDK switching at the workspace level for all projects is straightforward +by the user, only the default VM install needs to be changed and all projects +in the workspace switch to use this VM install.</li> + +<li> +Since class path variables can only bind to a single JAR, the JRE_LIB variable +can only bind to a single JAR (for the standard SUN JRE this is rt.jar).</li> + +<li> +There is some magic involved with regard to the handling of JRE_* variables +that is not obvious to the user:</li> + +<ul> +<li> +on the build class path the user sees JRE_LIB but when defining a launch +configuration then the user sees VM Installs. The user has to know that +JRE_LIB is indirectly bound to the VM install via the JRE installed preference +settings.</li> + +<li> +The user also has to understand that the reserved variables cannot be edited +like the other class path variables, etc.</li> +</ul> + +<li> +Build class path ordering - users can control the order of the build class +path in a simple way. For example, to do JCL development, to do so users +can put their source folders in front of the JRE_LIB class path entry.</li> + +<li> +Java Core is not affected by the JDK switching support and is independent +of launching concerns.</li> +</ul> + +<h2> +Motivation for enhancing the current implementation</h2> + +<h3> +New Requirements</h3> +There are new requirements with regard to the handling of the JRE/JDK on +the build class path that need to be addressed by 2.0: +<ul> +<li> +Multiple JARs per JDK</li> + +<br>The JRE_LIB class path variable can only bind to a single JAR file +typically the rt.jar. However, there are JDK installs that have split the +rt.jar into multiple JARs. For example, the JDK on the MacOS X has split +the rt.jar into: classes.jar and ui.jar (contains AWT and Swing). +<br> +<li> +Workspaces with a different JDK per project</li> + +<br>WSDD needs support for having a workspace where different projects +build against different JDKs. The JRE_LIB variable is global and there +is no infrastructure and UI support to have different JDKs for different +projects. WSDD defined their own build description mechanism that bypasses +the JRE_LIB support. This results in a problematic user experience when +switching from WSAD or vanilla Eclipse Java development to WSDD.</ul> + +<h3> +Existing characteristics to be preserved in the new implementation</h3> + +<ul> +<li> +Class path stability, it has to be possible to switch a JDK locally in +a workspace without affecting the .classpath file.</li> + +<li> +Easy JDK switching at the workspace level, i.e., a single setting can be +changed to change the build class path of all projects.</li> + +<li> +Java Core is independent of the VM install infrastructure</li> +</ul> + +<h2> +Proposal</h2> +The proposal affects core, launching, java debug UI, and the Java UI components. +<h3> +JavaCore</h3> +JavaCore provides a new type of classpath entry "CPE_CONTAINER", +which is just a named reference to a set of other classpath entries. +A container entry refers to a container path, which can be resolved by +a <code>ClasspathContainerInitializer</code> through an extension point, +or explicitly assigned using a setter method. + +The actual binding from the CPE_Container entry to +the target classpath entries is implemented in term of an extension point +to keep Java Core independent of VM install concerns: +<p><tt> <!ELEMENT classpathContainerInitializer +EMPTY></tt> +<br><tt> <!ATTLIST classpathContainerInitializer</tt> +<br><tt> id +CDATA #REQUIRED</tt> +<br><tt> class +CDATA #REQUIRED</tt> +<br><tt> ></tt> +<ul> +<li><b>id</b> - the container unique name for which this resolver will be activated.</li> +<li><b>class</b> - the class that implements this container initializer. +The class must implement a public subclass of <code>org.eclipse.jdt.core.ClasspathContainerInitializer</code> +with a public 0-argument constructor.</li> +</ul> +<tt> abstract class ClasspathContainerResolver +{</tt> +<br><tt> +void initialize(IPath containerPath, IJavaProject project) throws CoreException;</tt> +<br> } +<p>The initialize call passes in a project, this enables +to resolve a classpath container in the context of a particular project. +The initialize method should only be called once to resolve the class path +entry (in case of failure, the container will not be considered as having been +resolved). + +<p>It is possible to register an initializer per container ID. The full container path +being passed along to the initializer. Its first segment is +the container ID for which an initializer should be registered. The remaining segments +can be used to provide additional hints for the container expansion. +In case multiple resolvers are registered on the same container ID, the first +registered one will be used). + +<p>JavaCore provides a method to perform explicit modifications of a container: +<pre> +JavaCore#setClasspathContainer( + IPath containerPath, + IJavaProject[] affectedProjects, + IClasspathContainer[] respectiveContainers, + IProgressMonitor monitor) +</pre> +In particular, this method is to be used in the context of a classpath initializer so as to perform +the actual initialization. Note that it allows to modify the value of a container for a set of projects +at once. In reaction to invoking this method, the JavaModel will be refreshed and corresponding Java element +changes will be notified. + +<p> A classpath container implements <code>org.eclipse.jdt.core.IClasspathContainer</code> and can be queried +through a JavaCore API: <code>JavaCore#getClasspathContainer(IPath containerPath, IJavaProject project) </code>. + +There is no assumption that the returned container must answer the exact same containerPath +when requested <code>IClasspathContainer#getPath</code>. +Indeed, the containerPath is just an indication for resolving it to an actual container object. +<p> +Classpath container values are persisted locally to the workspace, but +are not preserved from a session to another. It is thus highly recommended to register a +<code>ClasspathContainerInitializer</code> for each referenced container +(through the extension point "org.eclipse.jdt.core.ClasspathContainerInitializer"). + +<pre> +public interface IClasspathContainer { + + /** + * Kind for a container mapping to an application library + */ + int K_APPLICATION = 1; + + /** + * Kind for a container mapping to a system library + */ + int K_SYSTEM = 2; + + /** + * Kind for a container mapping to a default system library, implicitly contributed by the runtime + */ + int K_DEFAULT_SYSTEM = 3; + + /** + * Answers the set of classpath entries this container is mapping to. + * The set of entries associated with a classpath container may contain any of the following: + * - library entries (<code>CPE_LIBRARY</code>) + * - project entries (<code>CPE_PROJECT</code>) + * A classpath container can neither reference further classpath containers or classpath variables. + */ + IClasspathEntry[] getClasspathEntries(); + + /** + * Answers a readable description of this container + */ + String getDescription(); + + /** + * Answers the kind of this container. Can be either: + * - K_APPLICATION if this container maps to an application library + * - K_SYSTEM if this container maps to a system library + * Typically, system containers should be placed first on a build path. + */ + int getKind(); + + /** + * Answers the container path identifying this container. + * A container path is formed by a first ID segment followed with extra segments. + * which can provide additional hint for resolving. + * This container ID is used in conjunction with the hints for resolving to this container. + * The container ID is also used to identify a ClasspathContainerInitializer + * registered on the extension point "org.eclipse.jdt.core.classpathContainerInitializer", which can + * be invoked if needing to resolve the container before it is explicitely set. + */ + IPath getPath(); +} +</pre> + +<p><font color="#FF0000"><b>Issue</b>: The Mac OS X JDK install is +an interesting case. There the rt.jar is split into two binary JARs (classes.jar, +ui.jar), but there is still a single src.jar. This cases needs to be handled by </font><font color="#FF0000">the +source lookup. For example, when src.jar is attached to classes.jar then when +looking up java.awt.Frame out of ui.jar, the source attachment of classes.jar +needs to searched as well.</font> + +<h2> +Example</h2> +The class path of a project will look as follows: +<p> +<classpath> +<br> +<classpathentry kind="src" path="/src"/> +<br> +<classpathentry kind="container" path="JDK/1.3"/> +<br> +<classpathentry kind="output" path="bin"/> +<br> +</classpath> +<p>In the case where the user didn't override the +VM install at the project level. Then the Java launching contributed container +resolver (registered for container prefixes: "JDK") would resolve "JDK/1.3" using the default +VM install, using "1.3" as an hint, and may expand it into the following: +<p> +<classpathentry kind="lib" path="d:/jdk/1.3.1/jre/lib/rt.jar" +rootpath="d:/jdk1.3.1/lib/src.jar" sourcepath="/src"/> + +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/dom ast/ASTPositions.html b/org.eclipse.jdt.core/notes/r2.0/dom ast/ASTPositions.html new file mode 100644 index 000000000..ae0413a56 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/dom ast/ASTPositions.html @@ -0,0 +1,148 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.75 [en] (Windows NT 5.0; U) [Netscape]"> +</head> +<body> +Hi, +<p>Here are my thoughts regarding positions and AST. The major goal was +to come up with a consistent view, even though this causes some more work +for refactoring and for the implementation of the new AST. But I think +it might be worth if we can come up with a consistent story for positions. +<h3> +Some general statements</h3> + +<ol> +<li> +sourceStart and sourceEnd should always cover the whole node. This is different +to the current implementation where for some nodes declarationSourceStart +and declararionSourcEnd covers the whole node and sourceStart and sourceEnd +only covers the name (examples are: LocalVariableDeclaration, TypeDeclararion, +...)</li> + +<li> +sourceStart and sourceEnd should also cover all subnodes</li> + +<li> +whenever possible we should follow the grammar as defined in The Java Language +Specification book. So if the grammar says that a production includes the +semicolon then the AST node should include it too. For example the grammar +defines a return statement like</li> + +<br> return (expression) ; +<br>So the corresponding AST node should include the ;</ol> + +<h3> +Some statements from earlier discussions (mainly between Jim, Philippe, +and me)</h3> + +<ul> +<li> +There will be an ExpressionStatement node for expressions used as statements. +For example "if (isChecked()) {}" versus "isCheck();". We agreed that the +expression will not include the semicolon and the ExpressionStatement will. +For the isCheck() example this will look like [[isChecked()];]. This is +consistent with the grammer defined in (3). This together with the general +statement (2) leads to the conclusion that statements that have child statements +will include the semicolon if the child statement has one. For the example</li> + +<p><br>for (int i= 0; i < 10; i++) +<br> foo(); +<p>sourceEnd of the for statement will include the semicolon of the expression +statement.</ul> + +<h3> +Open issues</h3> + +<h4> +Multiple local declarations</h4> +Currently multiple local declarations appear in the AST as n separate local +declarations without any relationship to each others. This raises various +questions: +<ul> +<li> +what are the positions of those local declarations</li> + +<li> +how is a visitor of that AST able to figure out that he deals with multiple +local declaration.</li> +</ul> +Since the new AST isn't a 1:1 mapping of the compiler's AST anyway (we +have the ExpressionStatement node) I opt to introduce new nodes as defined +in the grammar. Since the semicolon doesn't belong to the variable declaration, +it should be managed by the parent node that ties together multiple declarations. +Here is an example: +<p>int x= 10, x[]= null, i; +<p>LocalVariableDeclaration node manages: +<br> the type (e.g. int) +<br> the positions of the commas (if needed) +<br> the actual variable declarators +<br> sourceStart= start of the type +<br> sourceEnd= ; +<p>VariableDeclarator node manages: +<br> the variable name and its positions +<br> the initialization +<br> sourceStart= start of variable name +<br> sourceEnd= end of initialization. Doesn't include +the comma. +<p>If we want to do some optimization we could also have a node SingleLocalVariableDeclaration +for declaration like int x; or int y= 10; The node would have the following +fields: +<br> the type +<br> the variable name and its positions +<br> the initialization +<br> sourceStart= start of type +<br> sourceEnd= ; +<h4> +Updates in for statements</h4> +Analogous to the local variable declaration, the comma to separate the +update expressions can not be part of the expression (expressions don't +contain a semicolon so they can't contain a comma either). To know the +positions of the commas the for statement should manage them in a separate +array. +<p><i>The general rule is, that whenever language elements are separate +using a comma (for example an interface list in the implements statement, +arguments of a method declaration, ...) the node containing the separated +nodes should manage the positions of the comma, if they are of any interest. +In a first implementation we could leave these positions out and use the +scanner to find them if they are of interest.</i> +<h4> +Treatment of semicolon</h4> +From our experiences with refactoring it is helpful in some cases to know +where the position of the semicolon is. For example if the user extract +a for statement and he doesn't select the action's semicolon we allow the +extraction. So what can we do in these cases: +<ul> +<li> +simple don't allow the case. To support better selection we can offer some +actions to extend the text selection to spawn valid AST nodes. We have +a running prototype for this.</li> + +<li> +do some parsing of the source code to find the position of the semicolon. +We could use the scanner for this.</li> +</ul> + +<h4> +Answers to explicit questions from Olivier</h4> + +<ul> +<li> +for (;;); : in this case the for statement should cover the semicolon. +The best way to achieve this is to have an empty statement as defined in +the grammar.</li> + +<li> +declaration source start of an argument: yes, Adam uses argument.type.sourceStart +as the start not declarationSourceStart.</li> + +<li> +test: the test we have are the refactoring test. We don't have special +test to check if the AST positions are correct.</li> +</ul> + +<br> +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/dom ast/NodeTypeNames b/org.eclipse.jdt.core/notes/r2.0/dom ast/NodeTypeNames new file mode 100644 index 000000000..d1ed983e1 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/dom ast/NodeTypeNames @@ -0,0 +1,62 @@ +&start;AnonymousClassDeclaration&end;
+&start;ArrayAccess&end;
+&start;ArrayCreation&end;
+&start;ArrayInitializer&end;
+&start;ArrayType&end;
+&start;AssertStatement&end;
+&start;Assignment&end;
+&start;Block&end;
+&start;BooleanLiteral&end;
+&start;BreakStatement&end;
+&start;CastExpression&end;
+&start;CatchClause&end;
+&start;CharacterLiteral&end;
+&start;ClassInstanceCreation&end;
+&start;CompilationUnit&end;
+&start;ConditionalExpression&end;
+&start;ConstructorInvocation&end;
+&start;ContinueStatement&end;
+&start;DoStatement&end;
+&start;EmptyStatement&end;
+&start;ExpressionStatement&end;
+&start;FieldAccess&end;
+&start;FieldDeclaration&end;
+&start;ForStatement&end;
+&start;IfStatement&end;
+&start;ImportDeclaration&end;
+&start;InfixExpression&end;
+&start;InstanceofExpression&end;
+&start;Initializer&end;
+&start;Javadoc&end;
+&start;LabeledStatement&end;
+&start;MethodDeclaration&end;
+&start;MethodInvocation&end;
+&start;NullLiteral&end;
+&start;NumberLiteral&end;
+&start;PackageDeclaration&end;
+&start;ParenthesizedExpression&end;
+&start;PostfixExpression&end;
+&start;PrefixExpression&end;
+&start;PrimitiveType&end;
+&start;QualifiedName&end;
+&start;ReturnStatement&end;
+&start;SimpleName&end;
+&start;SimpleType&end;
+&start;SingleVariableDeclaration&end;
+&start;StringLiteral&end;
+&start;SuperConstructorInvocation&end;
+&start;SuperFieldAccess&end;
+&start;SuperMethodInvocation&end;
+&start;SwitchCase&end;
+&start;SwitchStatement&end;
+&start;SynchronizedStatement&end;
+&start;ThisExpression&end;
+&start;ThrowStatement&end;
+&start;TryStatement&end;
+&start;TypeDeclaration&end;
+&start;TypeDeclarationStatement&end;
+&start;TypeLiteral&end;
+&start;VariableDeclarationExpression&end;
+&start;VariableDeclarationFragment&end;
+&start;VariableDeclarationStatement&end;
+&start;WhileStatement&end;
\ No newline at end of file diff --git a/org.eclipse.jdt.core/notes/r2.0/dom ast/ast.html b/org.eclipse.jdt.core/notes/r2.0/dom ast/ast.html new file mode 100644 index 000000000..cd2b9adba --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/dom ast/ast.html @@ -0,0 +1,2091 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>JDT - Abstract Syntax Trees</title> +</head> +<body> + +<h2> +Abstract Syntax Trees</h2> +<font color="#3366FF">Last revised 14:00 Friday October 26, 2001 (most +recent change in blue)</font> +<p>Original work item: "Exposing the AST API." +<p>Related work item: "Improved Java code manipulation support, address +JDOM limitations; JDOM doesn't preserve markers and isn't document aware; +JDOM finer grained update support (e.g. change method name, type name); +buffer contents is duplicated in Java UI document and needs to be manually +synchronized." +<h3> +Background</h3> + +<ul> +<li>refactoring is a key customer (Dirk B.)</li> +<li>for 2.0: have good support in place so that refactoring can be made an +open API; would need corresponding core APIs for abstract syntax trees</li> +<li>current refactoring appoach uses AST visitor which runs post-resolve on +designated compilation unit</li> +<li>visitor collects info (parent stack) including source positions</li> +<li>uses name environment (bindings, scopes) to validate/locate</li> +<li>all changes are formulated as reified undoable edits in a source buffer</li> +<li>a batch of changes are made at a time</li> +<li>refactoring does not reason hypothetically (changes are undoable)</li> +<li>JDOM is not used by refactoring, but potentially could be (JDOM is used +inside Java Model)</li> +</ul> +Dirk's wish list with regard to the AST support: +<ul> +<li> +consistent positions in AST (e.g. sourceStart and sourceEnd should always +cover the whole node).</li> + +<li> +comment handling in AST. Some node contain preceding comments other don't</li> + +<li> +Parent pointer in AST node.</li> + +<li> +Data pointer in AST node.</li> + +<li> +Expression should provide getType method (cached value of resolveType(Scope)).</li> +</ul> + +<h3> +Summary from Sept. 10-11 meetings</h3> +Dirk travelled to SNZ to discuss refactoring requirements and possible +solutions with Philippe M. and Jeem. +<p>Some of the forms of solutions discussed, but ultimately abandoned: +<ul> +<li> +A vanilla DOM.</li> + +<ul> +<li> +too limiting: difficult to provide for pre-computing bindings.</li> + +<li> +clumsy for clients to use without AST node types represented by different +Java types</li> +</ul> + +<li> +AST plus resolves info in form of Java model elements.</li> + +<ul> +<li> +Java model methods are unsuitable canonical representation for resolved +methods because parameter types are as per source.</li> + +<li> +It is hard to map a type back to a Java element (need to remember package +fragment).</li> + +<li> +Would need additional Java model elements to represent variable declarations.</li> + +<li> +Would need special Java element implementations for local types, methods, +and fields.</li> +</ul> +</ul> +In the end, we agreed on AST plus bindings: +<ul> +<li> +AST is simple tree of typed nodes as determined by parser.</li> + +<li> +A different Java type for each AST node type.</li> + +<li> +TBD: are node types API classes, API interfaces, or both?</li> + +<li> +Simple case: no bindings.</li> + +<li> +Basic AST Lifecycle: client requests AST; compilation unit is parsed and +nodes are created; root is handed back to client; AST is garbage collected +after client lets go of all AST nodes. AST is not affected if underlying +compilation unit is changed (i.e., eager parse, with no residual dependence +on state of file in workspace).</li> + +<li> +Any syntax errors detected need to be reported to client. Problems are +opaque: source character position plus human-readable message. Clients +needs to determine whether an AST is only partial (e.g., busted flags).</li> + +<li> +Predicatable trees with simple correspondence to source code (parser should +not optimize ASTs).</li> + +<li> +Reliable source position information for all nodes.</li> + +<li> +Scratch "data" field on each AST node for client to record an Object.</li> + +<li> +Navigate AST upwards as well as downwards. Parent link.</li> + +<li> +AST walker for convenient traversal.</li> + +<li> +ASTs can be read-only or read-write.</li> + +<li> +Read-write ASTs can be modified in terms of ASTs. AST node factories (no +Java parser required). Cut/copy/paste.</li> + +<li> +Cut/copy/paste between separate ASTs requires fragment cloning to preserve +independence of storage.</li> + +<li> +Client may be interested in cut/copy/pasting comments too.</li> + +<li> +New nodes would not carry source positions.</li> + +<li> +Read-write AST can be serialized back to compilation unit char[].</li> + +<ul> +<li> +Preserve existing comments, whitespace, and use of \u.</li> + +<li> +Control whitespace and use of \u for insertions.</li> + +<li> +Provide edit map for updating markers.</li> +</ul> + +<li> +Resolved ASTs: basic AST annotated with bindings (non-syntactic information).</li> + +<li> +Binding information is derived from AST plus the Java model (relative to +some project).</li> + +<li> +Availability/validity of bindings ceases when Java model changes.</li> + +<li> +Client must request up front that bindings be created.</li> + +<li> +Certain AST nodes get decorated with bindings.</li> + +<li> +Client should have ways to communicate up front what kind of bindings are +required and where.</li> + +<li> +Availability/validity of bindings for a read-write AST ceases when it is +first modified.</li> + +<li> +Bindings (non-syntactic information) includes things such as the following:</li> + +<ul> +<li> +Resolved names - which type, field, method, or local variable does a AST +name refernce resolve to.</li> + +<li> +Resolved types - what is the resolved type of an AST expression or type +reference.</li> + +<li> +Resolved supertypes - what are the resolved supertypes of a resolved type.</li> + +<li> +Resolved declarations - what resolved type, field, method, or local variable +does an AST declaration map to.</li> + +<li> +Thrown exceptions - what are the resolved types of the exceptions thrown +by a given expression or statement.</li> + +<li> +Resolved members - what are the resolved members of a resolved type.</li> +</ul> + +<li> +Problems also should be reported with resolving. Problems are opaque: source +character position plus human-readable message.</li> + +<li> +Space for bindings storage is significant; increases monotonically as more +bindings are accessed.</li> + +<li> +Space for bindings is for lifetime of the AST.</li> + +<li> +Advanced AST Lifecycle: client requests AST with bindings; compilation +unit is parsed, nodes created, and perhaps some bindings created and annotated +on nodes; root is handed back to client; AST is garbage collected after +client lets go of all AST nodes and bindings. AST itself is not affected +if underlying compilation unit is changed (i.e., eager parse, with no residual +dependence on state of file in workspace). Bindings may become stale or +invalid if workspace changes (i.e., possibly lazy and incremental construction +of bindings using Java model).</li> + +<li> +<font color="#000000">Bindings from two ASTs are not generally comparable.</font></li> + +<ul> +<li> +<font color="#000000">For bindings with stable global names, API provides +strings that can be compared between ASTs.</font></li> +</ul> +</ul> +AST will either extend or replace JDOM. In the latter case, JDOM would +be deprecated. +<p>AST will exist along side Java model. +<h3> +API Design Issue: AST Node Types - Classes, interface, or both</h3> +There are on the order of 87 node types for Java ASTs. Bindings will add +some more. There are a couple of way this can be mapped to Java types. +<p>(1) Use org.w3c.DOM interfaces as API. Provide a private concrete implementation +of the DOM. +<p>Pro: Very small, and standard API for read-write documents. +<br>Con: API is not convenient for clients. +<br>Con: API is not amenable to exposing bindings and other non-structural +information. +<p>(2) Concrete API class per AST node type. +<p>Pro: API as small as possible. +<br>Pro: Client can create nodes directly. +<p>Con: Cannot easily hide implementation details; e.g. representation +and mechanism for reassembling compilation unit text after editing; lazy +binding creation. +<p>Clients who create node from scratch only ever need basic constructors +(the nodes they create do not have source positions, bindings, or other +decorations). On the other hand, the parser needs to remember more info +including fine-grained source positions. +<p>(3) API interface per AST node type, along with node factory methods. +Provide a private concrete implementation. Allow clients to reimplement +node types (from scratch) and supply a factory. +<p>Like JDOM (except JDOM does not permit client to reimplement) and org.w3c.dom. +<p>Pro: API as small as possible. +<br>Pro: Easy to tailor different kinds of representation: read-write vs. +read-only ASTs; raw ASTs vs. AST+bindings. +<p>Con: Hidden concrete implementation classes takes more space. +<br>Con: Using factory methods is a bit less direct than using constructors. +<p>We will use API interfaces for bindings, and exposed API classes for +AST nodes. +<h3> +API Design Issue: Statement vs. Statement Lists</h3> +For structured statements, like while, the child statement is grammatically +a single statement. However, since a block is one type of statement, it +is possible to have a list of statements underneath. There are options +for rendering this: +<p>(1) Child is a single statement. +<p>(Like current compiler's internal ASTs.) +<p>Pro: As per the Java grammar. +<br>Con: A client wanting to insert an additional statement into the child +must be prepared to replace by a block if there isn't one. +<p>(2) Child is a list of statements. +<p>(Like the .net codeDOM.) +<p>Pro: More convenient for clients that edit ASTs. Uniform mechanism for +inserting and removing statements from child. +<br>Con: Muddies scopes (enclosing braces around statements introduce a +scope and make declarations syntactically legal). +<p>We will go with (1) and stick closely to the grammar. +<h3> +Usage</h3> +There are a couple different usage scenarios for ASTs: +<ul> +<li> +Analyze an existing compilation unit to discover syntactic structure.</li> + +<li> +Discover relationship between syntactic structure and original text.</li> + +<li> +Discover relationship between syntactic structure and resolved world.</li> + +<li> +Create a new compilation unit from scratch.</li> + +<li> +Edit an existing compilation unit.</li> +</ul> + +<h3> +Source Construct Normalization</h3> + +<ul> +<li> +Most syntactic constructions are rendered in one and only one way.</li> + +<li> +When this is not the case, the AST construction is "lossy".</li> + +<li> +Some forms cannot be distinguised in input (if one cares).</li> + +<li> +Some forms cannot be produced in output.</li> + +<li> +Copying the construct normalizes it.</li> + +<li> +Example: Modifier order</li> + +<ul> +<li> +final static public int X = 1;</li> + +<li> +public static final int X = 1; // preferred form</li> +</ul> + +<li> +Example: Compound variable declarations</li> + +<ul> +<li> +int i = 1, j = 2;</li> + +<li> +int i = 1; int j = 2; // preferred form</li> +</ul> + +<li> +Example: Array type declarators</li> + +<ul> +<li> +int[] x[];</li> + +<li> +int[][] x; // preferred form</li> +</ul> + +<li> +Example: Short ifs</li> + +<ul> +<li> +if (a) f(); else ;</li> + +<li> +if (a) f(); // preferred form</li> +</ul> + +<li> +Can only be done for syntactic nuances that are have no semantic import.</li> + +<li> +Normalization is generally acceptable where unimportant syntactic nuances +are involved.</li> + +<li> +Normal form should follow JLS recommendations and Java coding standards.</li> + +<li> +Note that parentheses and blocks are important to user and should not be +normalized.</li> +</ul> + +<h3> +Source Positions</h3> + +<ul> +<li> +When AST is obtained by parsing a text string, exposing source ranges for +nodes allows clients to navigate back into original string; e.g., for making +text editor selections.</li> + +<li> +AST supports only character-oriented position information; mapping character +positions to lines are handled elsewhere (e.g., text editor).</li> + +<li> +Source ranges are irrelevant for nodes created by other means.</li> + +<li> +Source ranges give original position in original string.</li> + +<ul> +<li> +Editing the AST does not alter positions or anything clever.</li> +</ul> + +<li> +Most constructs occupy contiguous character positions, or ranges.</li> + +<li> +Ranges are represented by 0-based start position and length.</li> + +<li> +Start position begins at first significant character of construct corresponding +to AST node.</li> + +<ul> +<li> +First significant character.</li> + +<li> +Does not include leading whitespace.</li> + +<li> +Does not include preceding comment (except the javadoc comment preceding +a declaration, or the comment preceding a statement - see below).</li> +</ul> + +<li> +End position includes last significant character of construct corresponding +to AST node.</li> + +<ul> +<li> +Last significant character.</li> + +<li> +Includes trailing terminators that are part of construct; e.g., include +trailing semicolon at end of local variable declaration.</li> + +<li> +Does not include separators; e.g., exclude trailing comma in parameter +list.</li> + +<li> +Does not include trailing whitespace.</li> + +<li> +Does not include trailing comment.</li> + +<li> +Statement end-of-line comments are not encompassed by statement.</li> + +<ul> +<li> +<tt>System.out.println("hello"); // $non-nls$</tt></li> +</ul> + +<li> +<font color="#000000">Embedded comments are encompassed if they occur before +end position.</font></li> + +<ul> +<li> +<tt><font color="#000000">System.out.println("hello") /* comment */;</font></tt></li> +</ul> +</ul> + +<li> +Some node types would have source ranges for significant contiguous subconstructs +not readily gleanable from source ranges of the subnodes.</li> + +<ul> +<li> +Additional source ranges would be specified for each node type.</li> + +<li> +E.g., method declaration has additional source range for the method name +and for the method declaration excluding its javadoc comment.</li> + +<li> +Use start and length arrays rather than proliferate API methods for additional +source ranges.</li> +</ul> +</ul> + +<h3> +Unicode Escapes</h3> + +<ul> +<li> +Original source text might contain Unicode escapes (JLS 3.2, 3.3).</li> + +<li> +E.g., void\u0040\u005a(); declares a method named Z.</li> + +<li> +Scanner removes all Unicode escapes and returns a Unicode token stream.</li> + +<li> +Newly created AST nodes are "post" Unicode escaping.</li> + +<li> +Output options:</li> + +<ul> +<li> +Preserve existing Unicode escapes (default); remove all existing Unicode +escapes.</li> + +<li> +Do not introduce Unicode escapes (default); introduce Unicode escapes for +characters in a specified set (e.g., all non-ASCII).</li> +</ul> + +<li> +Initial implementation: support default behavior only.</li> +</ul> + +<h3> +Comments</h3> + +<ul> +<li> +Comments are problematic for ASTs; these lexical items are normally filtered +out of token stream.</li> + +<li> +Comments are significant to user.</li> + +<li> +Editing an existing compilation unit should generally preserve existing +comments.</li> + +<li> +Should be able to include comments for newly created subtrees.</li> + +<li> +Copying a subtree from one place to another should include relevant comments.</li> + +<li> +Most common forms of comments:</li> + +<ul> +<li> +Javadoc comments - on one or more lines preceding field, method, and type +declarations.</li> + +<li> +Boilerplace comments (copyright notices) - one or more lines preceding +the package declaration, or between the package declaration and first import +or type declaration.</li> + +<li> +Statement comments - one or more lines between statements in a block.</li> + +<li> +Statement end-of-line comments.</li> +</ul> + +<li> +VA/ST experience: not worth bending over backwards to accomodate all comments.</li> + +<li> +Determined clients can rescan original string to get at all comments.</li> + +<li> +Expose high value comments:</li> + +<li> +Javadoc comments - treat as attribute of the field, method, and type declarations.</li> + +<ul> +<li> +Clients can extract Javadoc attributes (including @deprecated).</li> + +<li> +Clients can create declarations with Javadoc.</li> +</ul> + +<li> +Statement comments within blocks</li> + +<ul> +<li> +Approach 1: Treat as pseudo-statements with a special AST node type.</li> + +<ul> +<li> +Pro: Clients can include comments with blocks.</li> + +<li> +Con: Only works for comments within genuine blocks. E.g., can't handle</li> + +<li> +<tt>if (test)</tt></li> + +<li> +<tt> // should not happen</tt></li> + +<li> +<tt> throw new RuntimeException();</tt></li> + +<li> +Would work better if we were using statement lists in more places.</li> +</ul> + +<li> +Approach 2: Treat as a property of following statment node.</li> + +<ul> +<li> +Pro: Clients can include comments before any statement.</li> + +<li> +Con: Does not handle trailing comments in blocks. E.g.,</li> + +<li> +<tt>{</tt></li> + +<li> +<tt> throw new RuntimeException();</tt></li> + +<li> +<tt> // can't reach here</tt></li> + +<li> +<tt>}</tt></li> +</ul> + +<li> +Recommend approach 2 since it covers most cases.</li> +</ul> + +<li> +Boilerplate comments would not be exposed, but would be preserved through +edit and output.</li> +</ul> + +<h3> +Whitespace</h3> + +<ul> +<li> +Whitespace (JLS 3.6) includes ASCII SP, HT, and FF characters, and line +terminators.</li> + +<li> +Like comments, whitespace is significant to user.</li> + +<li> +Editing an existing compilation unit should generally preserve whitespace.</li> + +<li> +Whitespace for newly created subtrees automatically generated to produce +output that follows common conventions and blends in with surrounding text +(use the same leading whitespace).</li> + +<li> +Copying a subtree from one place to another should should generally preserve +whitespace.</li> +</ul> + +<h3> +AST Parent Backpointer</h3> + +<ul> +<li> +Each AST node will carry a backpointer to its parent node.</li> + +<li> +ASTNode.getParent() returns ASTNode</li> + +<li> +This permits clients to traverse ASTs in upward as well as downward direction.</li> + +<li> +Bidirectional links must be maintained during editing.</li> + +<li> +Deletion API must unlink child from parent.</li> + +<li> +Insertion API must link child to parent.</li> + +<ul> +<li> +To preserve treeness, automatically clone child subtree if child already +has parent.</li> +</ul> + +<li> +Replace API must unlink old child before inserting new child.</li> + +<li> +Parent backlinks means that hanging on to <i>any</i> node in an AST instance +will prevent any part of the AST instance from being garbage collected.</li> +</ul> + +<h3> +Multiple ASTs</h3> + +<ul> +<li> +Muliple ASTs can exist side by side (and ASTs are potentially for same +compilation unit).</li> + +<li> +Allow insertion of nodes from one AST into another AST.</li> + +<ul> +<li> +Automatically clones child subtree (forgetting source positions and binding +decorations).</li> + +<li> +Ensure memory representation of ASTs remain completely independent.</li> +</ul> +</ul> + +<h3> +<font color="#3366FF">Structural Equality</font></h3> + +<ul> +<li> +<font color="#3366FF">Structural equality predicate on AST nodes.</font></li> + +<li> +<font color="#3366FF">Isomorphic subtrees.</font></li> + +<li> +<font color="#3366FF">Belonging to same or different AST.</font></li> + +<li> +<font color="#3366FF">Considers structural info only; ignores source positions, +bindings, etc.</font></li> + +<li> +<font color="#3366FF">Named something other than "equals" to avoid having +to reimplement hashCode too.</font></li> +</ul> + +<h3> +Syntactic Correctness of Parser-built ASTs</h3> + +<ul> +<li> +For ASTs built by a Java parser, there are issues of syntactic correctness.</li> + +<li> +Syntactic correctness is judged by the Syntactic Grammar (as defined in +JLS2 section 2.3).</li> + +<li> +Java parser <b>must</b> guarantee to produce a faithful AST for any syntactically +correct compilation unit.</li> + +<li> +Java parser <b>may</b> also build ASTs for syntactically incorrect compilation +units.</li> + +<li> +Complicant Java compilers must reject syntactically incorrect compilation +units.</li> + +<li> +What principle do we apply to Java parsers and the ASTs they return?</li> + +<li> +Real Java parsers are invariably more liberal than the Syntactic Grammar, +and rely on post-parse checks to report errors for any syntactically incorrect +constructs that makes it past the parser.</li> + +<ul> +<li> +E.g., conflicting modifiers: public private</li> + +<li> +E.g., field declared with no initializer occurs in an interface</li> + +<li> +E.g., void foo() [];</li> +</ul> + +<li> +In the current Eclipse compiler, many of these checks are done in the course +of type and name resolution. If client only wants AST, we want to avoid +doing expensive name and type analysis.</li> + +<li> +Approach 1: Guarantee that no ASTs are built for syntactically incorrect +compilation units.</li> + +<ul> +<li> +You do not get an AST at all for compilation units with syntax errors.</li> + +<li> +Pro: Client can trust parser to distinguish syntactically correct from +incorrect.</li> + +<li> +Con: Client cannot manipulate syntactically incorrect compilation units +at all.</li> + +<li> +Con: Requires post-parse check to detect residual syntax errors.</li> +</ul> + +<li> +Approach 2: Provide no guarantee about the ASTs for syntactically incorrect +compilation units.</li> + +<ul> +<li> +You might not get a useful AST at all.</li> + +<li> +You might get an AST that had pieces missing; e.g., a malformed method +was excised</li> + +<li> +You might get an AST that is incoherent or self-contradictory; e.g., a +transient class!?</li> + +<li> +Pro: Maximum flexibility for implementation.</li> + +<li> +Pro: Client can get useful ASTs for some syntactically incorrect programs.</li> + +<li> +Con: Client cannot trust parser to distinguish syntactically correct from +incorrect.</li> +</ul> + +<li> +Approach 3: Guarantee that the client examining the resulting AST has some +way to determine whether the compilation units is incorrect.</li> + +<ul> +<li> +Priniciple: Syntactic errors must not be suppressed.</li> + +<li> +AST nodes could carry flags indicating certain syntax problem; e.g., duplicate +modifiers public public</li> + +<li> +A bit on root node could say "unspecified syntax errors".</li> + +<li> +Could be special AST nodes types indicating major problems; e.g., bogus +method body</li> + +<li> +Could be representable configurations of AST node types that are recognizable +as syntactially incorrect; e.g., conflicting modifiers public private; +missing field initializer in interface</li> + +<li> +Pro: Client can trust parser to not hide any syntax errors that are in +the source.</li> + +<li> +Pro: Client can get useful ASTs for syntactically incorrect programs.</li> + +<li> +Con: Client must do extra work to determine whether there are syntax errors.</li> + +<li> +Con: Extra work to include this information if no client really cares about +the difference between syntactically correct and incorrect.</li> +</ul> + +<li> +The first approach is too harsh. It is much more reasonable, and interesting, +to be able to work with some syntactically incorrect compilation units.</li> + +<li> +The second approach feels reasonable if clients never care whether the +source is syntactically correct or not.</li> + +<li> +The third approach feels reasonable if some clients would care whether +the source is syntactically correct or not.</li> + +<li> +The principle difference between the second and third appoaches is that +the former sanctions quietly suppressing syntax errors whereas the latter +precludes it.</li> + +<li> +The nature of the AST nodes inherently makes room to express a wide range +of syntactically malformed programs.</li> + +<li> +An extra flag per node for "unspecified syntax errors" should cover the +bases.</li> + +<li> +The existing compiler's ASTs already carry enough information to enable +the compiler to do thorough post-parse detecting of residual syntax errors.</li> + +<li> +Therefore the third approach is within easy reach.</li> + +<li> +The third approach gives clients more than the second approach.</li> + +<li> +Recommendation: we adopt the third approach.</li> +</ul> + +<h3> +Syntactic Correctness of Non-parser-built ASTs</h3> + +<ul> +<li> +ASTs do not just come from a parser.</li> + +<ul> +<li> +They can be created from scratch.</li> + +<li> +A parser-build AST can be edited.</li> +</ul> + +<li> +These ASTs will need to be serialized to a source compilation unit (why +else would they exist?).</li> + +<li> +What kinds of support and guarantees are in place to ensure that such a +suitable source compilation unit can be generated?</li> + +<li> +Basic guarantee: any AST that could have come from parsing a syntactically +correct compilation unit will serialize to a compilation unit that is</li> + +<ul> +<li> +(a) syntactically correct</li> + +<li> +(b) strongly semantically equivalent to the original compilation unit.</li> + +<li> +and possibly (c) normalized; that is, parse(serialize(x)) is isomorphic +to x</li> +</ul> + +<li> +There are likely many ways to get ASTs that do not correspond to any syntactically +correct compilation unit.</li> + +<ul> +<li> +E.g., use illegal identifiers ("1abc" or "try" or "//").</li> + +<li> +E.g., use illegal modifier combinations with modifier bit masks.</li> +</ul> + +<li> +Post-screening the AST for syntactic correctness would be misguided.</li> + +<li> +Should just go ahead and generate the obvious, syntactically incorrect, +compilation unit.</li> + +<li> +More importantly: ensure semantic equivalence.</li> + +<li> +Operator precedence creates issues:</li> + +<ul> +<li> +E.g., given AST for expression <tt>v1*v2</tt>, replace <tt>v1</tt> node +by expression <tt>v1+v3</tt>.</li> + +<li> +Naive serialization yields <tt>v1+v3*v2</tt> which is not semantically +equivalent to the AST.</li> + +<li> +Result should be (<tt>v1+v3)*v2</tt>.</li> + +<li> +Parentheses may need to be introduced during serialization.</li> +</ul> + +<li> +Nested if statement creates issues:</li> + +<ul> +<li> +E.g., given AST for statement <tt>if (a) f(); else g();</tt>, replace <tt>f();</tt> +by <tt>if (b) h();</tt></li> + +<li> +Naive serialization yields <tt>if (a) if (b) h(); else g();</tt></li> + +<li> +Result should be <tt>if (a) if (b) h(); <b>else </b>; else g();</tt></li> + +<li> +Extra verbiage may need to be introduced during serialization.</li> +</ul> +</ul> + +<h3> +Deep Constructs</h3> + +<ul> +<li> +Some programs involve impossibly deep constructs.</li> + +<li> +Multi-line string concatenation expressions are the main offender.</li> + +<ul> +<li> +For example, <tt>"Line 1\\n"+"Line 2\\n"+...+"Line 5000"</tt></li> +</ul> + +<li> +Runtime stacks blow when recursing over deep ASTs.</li> + +<li> +AST node types should be designed to keep trees reasonably shallow for +reasonably typical programs.</li> + +<li> +Introduce N-ary operator expression node type to deal with multi-line string +concatenation expressions.</li> + +<li> +N.B. Current compiler performs compile-time concatenations during parse +phase to deal with this problem.</li> +</ul> + +<h3> +Editing Protocol</h3> + +<ul> +<li> +What general form should the editing API take?</li> + +<li> +Setters on receiver to manipulate its children (parent never affected)</li> + +<ul> +<li> +E.g., whileStatement.setCondition(newExpression)</li> + +<li> +Use null for optional children</li> +</ul> + +<li> +Treat lists as an array-valued property.</li> + +<ul> +<li> +E.g., block.getStatements() returns Statement[]</li> + +<li> +E.g., block.setStatements(Statement[] statements)</li> + +<li> +Use empty list for no children (rather than null)</li> +</ul> + +<li> +Alternative approach for lists: use Collection-like protocol</li> + +<ul> +<li> +E.g., block.addStatement(pos, newChildStatement)</li> + +<li> +E.g., block.removeStatement(oldChildStatement)</li> + +<li> +Con: Increased number of methods on API; bad when a node type has several +list properties.</li> +</ul> + +<li> +Alternative approach for delete/replace: use parent backpointers to implement +generic delete and replace operation which affect the receiver's relationship +to its parent</li> + +<ul> +<li> +E.g., oldChildStatement.delete()</li> + +<li> +Con: semantics of deletion ugly when node occurs outside of any list</li> +</ul> +</ul> + +<h3> +User Data Field</h3> + +<ul> +<li> +Each AST node has a user data slot reserved for client use.</li> + +<li> +ASTNode.getClientData() returns Object</li> + +<li> +ASTNode.setClientData(Object data)</li> + +<li> +The initial value is null.</li> + +<li> +Client may use for decorations, or whatever.</li> + +<li> +AST nodes created by parser carry no data initially.</li> + +<li> +AST nodes created explicitly carry no data initially.</li> + +<li> +Even read-only ASTs have read-write data slots.</li> + +<li> +Cloning an AST node creates a new node (does <b>not</b> copy or clone data).</li> +</ul> + +<h3> +Lists of Members</h3> + +<ul> +<li> +List of field, method, and type members of a type declaration.</li> + +<li> +This list is syntactically and semantically heterogenous.</li> + +<li> +No syntactic constraints on number and order.</li> + +<li> +Order is significant to user.</li> + +<li> +Within field declarations, relative order is semantically significant.</li> + +<li> +Standard practices:</li> + +<ul> +<li> +Place field declarations before member methods and types.</li> + +<li> +Place types before methods.</li> +</ul> + +<li> +Option (1): expose separate lists for field, methods, and types.</li> + +<li> +Pro: This is way internal AST works.</li> + +<li> +Pro: Convenient for clients to locate member fields, methods, and types.</li> + +<li> +Con: Not flexible for editing; editing will mangle member order.</li> + +<li> +Option (2): expose a single list of members</li> + +<li> +Pro: parser does not normalize; client controls order of members.</li> + +<li> +Con: More work for clients to locate member fields, methods, and types.</li> + +<li> +Option (3): expose a single list of members, with extra getters for locating +member fields, methods, and types.</li> + +<li> +Pro: Combines advantage of (2) with convenience of (1).</li> + +<li> +Recommended approach: (3).</li> + +<li> +For class declarations, treat initializers and constructors as members.</li> + +<ul> +<li> +Lump instance and static initializers in with field declarations.</li> + +<li> +Lump constructor declarations in with method declarations.</li> +</ul> +</ul> + +<h3> +Serialization</h3> + +<ul> +<li> +Clients of read-write ASTs will generally want to serialize to a Java compilation +unit.</li> + +<li> +Serialization via simple AST tree walk.</li> + +<ul> +<li> +Straightforward.</li> + +<li> +Introduce line breaks and whitespace to make it look pretty.</li> + +<li> +Or post-process it with the Java formatter.</li> + +<li> +If AST originated by parsing, the result is likely unacceptable to user:</li> + +<ul> +<li> +Completely reformatted.</li> + +<li> +Constructs are normalized.</li> + +<li> +Some comments may have be lost.</li> +</ul> + +<li> +Could be provided by API that makes use of regular AST API only.</li> + +<li> +Could be written by clients.</li> +</ul> + +<li> +Serialization via source reconstruction.</li> + +<ul> +<li> +Only applicable to ASTs initially constructed by parser.</li> + +<li> +Use source position information in modified AST to reconstruct compilation +unit.</li> + +<li> +Retain passages of original text corresponding to unchanged AST trees.</li> + +<li> +Generates new text only where required.</li> + +<li> +Produce a result that a user will recognize and accept.</li> + +<ul> +<li> +Preserve formatting wherever possible.</li> + +<li> +Preserve source construct normalization wherever possible.</li> + +<li> +Preserve arbitrarily-placed comments wherever possible.</li> +</ul> + +<li> +Requires retaining the original compilation unit, and likely recording +additional information in nodes to allow reconstruction.</li> + +<li> +This is the way the current JDOM implementation works.</li> + +<li> +Could be provided by API that has privileged access to AST nodes and parser-recorded +information.</li> + +<li> +Should also return a list of edit instructions so that markers can be adjusted, +etc.</li> + +<li> +Clients would have a hard time doing this themselves.</li> +</ul> + +<li> +<font color="#000000">Recommend deferring implementation of serializer +that does source reconstruction.</font></li> + +<ul> +<li> +<font color="#000000">In interim, refactoring can apply edits to original +compilation unit text directly.</font></li> +</ul> +</ul> + +<h3> +Node types</h3> +The AST node types are based on the standard grammar for the Java language +given in the JLS2. +<p>Every AST node belongs to a single AST instance. (In DOM terminology, +the AST is the document and the AST nodes are the elements). The AST instance +can serve as a factory for creating new nodes. Nodes point to their owning +AST (fixed for the node's lifetime). The AST points directly to the root +node (a compilation unit). +<p>The AST node types do not define their own notion of equality; they +just inherit the object identity based implementation from Object. +<p>Note: Grammar rules (in comments) are expressed in the Pascal-style +extended BNF used in <tt>section 18</tt> of JLS2. We use C# style property +declarations as a convenient abbreviation for a standard matched pair of +get and set methods. +<p><tt>public class AST</tt> +<br><tt> public AST();</tt> +<br><tt> public property CompilationUnit root;</tt> +<br><tt> public void loadFromSource(char[] source);</tt> +<br><tt> public void setOptions(...);</tt> +<br><tt> public char[] serialize();</tt> +<p><tt>public abstract class ASTNode</tt> +<br><tt> protected ASTNode(AST ast);</tt> +<br><tt> public AST getOwner();</tt> +<br><tt> public property int[] startPositions;</tt> +<br><tt> public property int[] lengths;</tt> +<br><tt> public property boolean isWholeLine;</tt> +<br><tt> public property Object clientData;</tt> +<br><tt> public ASTNode getParent();</tt> +<br><tt> ... other protocol common to all AST node types</tt> +<h4> +Names</h4> +As explained in JLS2 section 6.5, the grammar does not allow names to be +resolved more finely than the following 6 categories by syntactic means +alone: +<p><tt>PackageName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageName . Identifier</tt> +<br><tt>TypeName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageOrTypeName . Identifier</tt> +<p><tt>ExpressionName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p><tt>MethodName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p><tt>PackageOrTypeName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageOrTypeName . Identifier</tt> +<p><tt>AmbiguousName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p>Given that names cannot be resolved definitively to a package, type, +field, or variable at AST node construction time, an open question is how +much of the categorization that could be done should be reflected in the +AST. More categories means more information flow from the parser to the +AST client; on the other hand, a variety of categories is not necessarily +convenient for clients. For example, in <tt>import a.b.c</tt> the name +is a <tt>TypeName</tt> whereas in <tt>import a.b.c.*</tt> the name <tt>a.b.c</tt> +is a <tt>PackageOrTypeName</tt>. If the name category was to be reflected +in the type of the AST nodes, the client would need to know to create the +appropriate type of name nodes when editing the AST. +<p>Proposal: Use two AST node types for names: simple names, and qualified +names. Qualified names are expressed recursively, to facilitate clients +discovering how the qualifier part of a name resolves. Use these for everything +but <tt>MethodName</tt>; for <tt>MethodName</tt>, which can appear only +in a method invocation expression, separate the selector identifier from +any preceding qualifier. +<p>(Note: The current internal AST nodes go beyond making the simple/qualified +distinction: they also have simple & qualified type names (classes +<tt>SimpleTypeReference</tt> +and <tt>QualifiedTypeReference</tt>) in additional to simple & qualified +named (classes <tt>SimpleNameReference</tt> and +<tt>QualifiedNameReference</tt>).) +<p><tt>// Name:</tt> +<br><tt>// +SimpleName</tt> +<br><tt>// +QualifiedName</tt> +<br><tt>// SimpleName:</tt> +<br><tt>// +Identifier</tt> +<br><tt>// QualifiedName:</tt> +<br><tt>// +Name <b><u>.</u></b> Identifier</tt> +<br><tt>public interface IName // "marker" interface</tt> +<br><tt> public IBinding resolvedBinding(); // +optional</tt> +<p><tt>public class SimpleName extends ASTNode implements IName, IExpression</tt> +<br><tt> public SimpleName(AST ast);</tt> +<br><tt> public property char[] identifier;</tt> +<p><tt>public class QualifiedName extends ASTNode implements IName, IExpression</tt> +<br><tt> public QualifiedName(AST ast);</tt> +<br><tt> public property IName qualifier;</tt> +<br><tt> public property char[] identifier;</tt> +<h3> +Compilation Units and Major Declarations</h3> +<tt>// CompilationUnit:</tt> +<br><tt>// +[<b><u>package</u></b> Identifier { <b><u>.</u></b> Identifier } <b><u>;</u></b> +]</tt> +<br><tt>// +{ImportDeclaration}</tt> +<br><tt>// +{TypeDeclaration | <b><u>;</u></b>}</tt> +<br><tt>public class CompilationUnit extends ASTNode</tt> +<br><tt> public CompilationUnit(AST ast);</tt> +<br><tt> public property Name packageName; // optional</tt> +<br><tt> public property ImportDeclaration[] imports;</tt> +<br><tt> public property TypeDeclaration[] types;</tt> +<p><tt>// ImportDeclaration:</tt> +<br><tt>// <b><u>import</u></b> +Identifier { <b><u>.</u></b> Identifier } [ <b><u>.</u></b> <b><u>*</u></b> +] +<b><u>;</u></b></tt> +<br><tt>public class ImportDeclaration extends ASTNode</tt> +<br><tt> public ImportDeclaration(AST ast);</tt> +<br><tt> public property Name importName;</tt> +<br><tt> public property boolean onDemand;</tt> +<br><tt> public IBinding resolveBinding();</tt> +<p><tt>// TypeDeclaration:</tt> +<br><tt>// +{Modifier} <b><u>class</u></b> Identifier</tt> +<br><tt>// +[<b><u>extends</u></b> Type]</tt> +<br><tt>// +[<b><u>implements</u></b> Type { <b><u>,</u></b> Type}]</tt> +<br><tt>// +<b><u>{</u></b> {ClassBodyDeclaration | <b><u>;</u></b> } <b><u>}</u></b></tt> +<br><tt>// +{Modifier} <b><u>interface</u></b> Identifier</tt> +<br><tt>// +[<b><u>extends</u></b> Type { <b><u>,</u></b> Type}]</tt> +<br><tt>// +<b><u>{</u></b> {InterfaceBodyDeclaration | <b><u>;</u></b> } <b><u>}</u></b></tt> +<br><tt>// Modifier:</tt> +<br><tt>// <b><u>public</u></b></tt> +<br><tt>// <b><u>protected</u></b></tt> +<br><tt>// <b><u>private</u></b></tt> +<br><tt>// <b><u>static</u></b></tt> +<br><tt>// <b><u>abstract</u></b></tt> +<br><tt>// <b><u>final</u></b></tt> +<br><tt>// <b><u>native</u></b></tt> +<br><tt>// <b><u>synchronized</u></b></tt> +<br><tt>// <b><u>transient</u></b></tt> +<br><tt>// <b><u>volatile</u></b></tt> +<br><tt>// <b><u>strictfp</u></b></tt> +<br><tt>// ClassBodyDeclaration:</tt> +<br><tt>// +MethodDeclaration</tt> +<br><tt>// +ConstructorDeclaration</tt> +<br><tt>// +FieldDeclaration</tt> +<br><tt>// +ClassDeclaration</tt> +<br><tt>// +TypeDeclaration</tt> +<br><tt>// +Initializer</tt> +<br><tt>// InterfaceBodyDeclaration:</tt> +<br><tt>// +MethodDeclaration</tt> +<br><tt>// +FieldDeclaration</tt> +<br><tt>// +TypeDeclaration</tt> +<br><tt>public class TypeDeclaration extends ASTNode implements IStatement, +IMember</tt> +<br><tt> public TypeDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Name superclass; // optional</tt> +<br><tt> public property Name[] superInterfaces;</tt> +<br><tt> public property IMember[] members;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> // convenience methods</tt> +<br><tt> public FieldDeclaration[] getFields; // includes +constants; excludes initializers</tt> +<br><tt> public AbstractMethodDeclaration[] getMethods; +// includes constructors</tt> +<br><tt> public TypeDeclaration[] getTypes;</tt> +<br><tt> public ITypeBinding resolveBinding();</tt> +<p><tt>// MethodDeclaration:</tt> +<br><tt>// +{Modifier} (Type | <b><u>void</u></b>) Identifier <b><u>(</u></b></tt> +<br><tt>// +[FormalParameter { <b><u>,</u></b> FormalParameter}] <b><u>)</u></b> +{<b><u>[</u></b> <b><u>]</u></b>}</tt> +<br><tt>// +[<b><u>throws</u></b> QualifiedIdentifierList] ( MethodBody | <b><u>;</u></b> +)</tt> +<br><tt>// ConstructorDeclaration:</tt> +<br><tt>// +{Modifier} Identifier <b><u>(</u></b> [FormalParameter { <b><u>,</u></b> +FormalParameter}] <b><u>)</u></b></tt> +<br><tt>// +[<b><u>throws</u></b> QualifiedIdentifierList] MethodBody</tt> +<br><tt>// FormalParameter:</tt> +<br><tt>// +[<b><u>final</u></b>] Type Identifier {<b><u>[</u></b> <b><u>]</u></b>}</tt> +<br><tt>public abstract class AbstractMethodDeclaration extends ASTNode +implements IMember</tt> +<br><tt> protected AbstractMethodDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] selector;</tt> +<br><tt> public property FormalParameter[] parameters;</tt> +<br><tt> public property Name[] thrownExceptions;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> public property Block body; // optional</tt> +<br><tt> public IMethodBinding resolveBinding();</tt> +<p><tt>public class MethodDeclaration extends AbstractMethodDeclaration</tt> +<br><tt> public MethodDeclaration(AST ast);</tt> +<br><tt> public property Type returnType; // includes +void</tt> +<p><tt>public class ConstructorDeclaration extends AbstractMethodDeclaration</tt> +<br><tt> public ConstructorDeclaration(AST ast);</tt> +<p><tt>// FieldDeclaration:</tt> +<br><tt>// +{Modifier} Type Identifier {<b><u>[</u></b> <b><u>]</u></b>} [ <b><u>=</u></b> +Expression]</tt> +<br><tt>// +{ <b><u>,</u></b> Identifier {<b><u>[</u></b> <b><u>]</u></b>} [ <b><u>=</u></b> +Expression] }</tt> +<br><tt>public class FieldDeclaration extends ASTNode implements IMember</tt> +<br><tt> public AbstractMethodDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> public property IExpression initializer; // +optional</tt> +<br><tt> public IFieldBinding resolveBinding();</tt> +<p><tt>// Initializer:</tt> +<br><tt>// +[<b><u>static</u></b>] Block</tt> +<br><tt>public final class Initializer extends ASTNode implements IMember</tt> +<br><tt> public Initializer(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property Block body;</tt> +<p><tt>// LocalVariableDeclaration:</tt> +<br><tt>// +[<b><u>final</u></b>] Type Identifier {<b><u>[]</u></b>} [ <b><u>=</u></b> +Expression ]</tt> +<br><tt>// +{ <b><u>,</u></b> Identifier {<b><u>[]</u></b>} [ <b><u>=</u></b> Expression] +} <b><u>;</u></b></tt> +<br><tt>public class LocalVariableDeclaration extends ASTNode implements +IStatement</tt> +<br><tt> public LocalVariableDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property IExpression initializer; // +optional</tt> +<br><tt> public ILocalVariableBinding resolveBinding();</tt> +<br> +<h4> +Types</h4> +The Type node (= TypeReference) represents a reference to a base type, +a named type, or an array thereof. +<p><tt>// Type:</tt> +<br><tt>// +(BasicType | TypeName ) {<b><u>[]</u></b>}</tt> +<br><tt>// BasicType:</tt> +<br><tt>// <b><u>byte</u></b></tt> +<br><tt>// <b><u>short</u></b></tt> +<br><tt>// <b><u>char</u></b></tt> +<br><tt>// <b><u>int</u></b></tt> +<br><tt>// <b><u>long</u></b></tt> +<br><tt>// <b><u>float</u></b></tt> +<br><tt>// <b><u>double</u></b></tt> +<br><tt>// <b><u>boolean</u></b></tt> +<br><tt>public class Type extends ASTNode implements IExpression</tt> +<br><tt> public Type (AST ast);</tt> +<br><tt> public property int baseType; // either</tt> +<br><tt> public property Name typeName; // or</tt> +<br><tt> public property int dimensions;</tt> +<br><tt> public IBinding resolvedType();</tt> +<h4> +Statements</h4> +There is a different AST node type for each different kind of statement. +Use a "marker" interface (<tt>IStatement</tt>) to bring all constructs +that can appear within a block (nonterminal <tt>BlockStatement</tt>, which +includes local variable and type declarations). +<p><tt>// Block:</tt> +<br><tt>// <b><u>{</u></b> +BlockStatement <b><u>}</u></b></tt> +<br><tt>// BlockStatement :</tt> +<br><tt>// LocalVariableDeclaration</tt> +<br><tt>// TypeDeclaration</tt> +<br><tt>// [Identifier +<b><u>:</u></b> +] Statement</tt> +<br><tt>//Statement:</tt> +<br><tt>// Block</tt> +<br><tt>// <b><u>if +(</u></b>Expression <b><u>)</u></b> Statement [<b><u>else</u></b> Statement]</tt> +<br><tt>// <b><u>for +(</u></b> ForInitOpt <b><u>;</u></b> [Expression] +<b><u>;</u></b> +ForUpdateOpt <b><u>)</u></b> Statement</tt> +<br><tt>// <b><u>while +(</u></b> Expression <b><u>)</u></b> Statement</tt> +<br><tt>// <b><u>do</u></b> +Statement <b><u>while</u></b> <b><u>(</u></b> Expression +<b><u>);</u></b></tt> +<br><tt>// <b><u>try</u></b> +Block [Catches] [ <b><u>finally</u></b> Block ]</tt> +<br><tt>// <b><u>switch +(</u></b> Expression <b><u>)</u></b> <b><u>{</u></b> SwitchBlockStatementGroups +<b><u>}</u></b></tt> +<br><tt>// <b><u>synchronized +(</u></b> Expression <b><u>)</u></b> Block</tt> +<br><tt>// <b><u>return</u></b> +[Expression] <b><u>;</u></b></tt> +<br><tt>// <b><u>throw</u></b> +Expression <b><u>;</u></b></tt> +<br><tt>// <b><u>break</u></b> +[Identifier] <b><u>;</u></b></tt> +<br><tt>// <b><u>continue</u></b> +[Identifier] <b><u>;</u></b></tt> +<br><tt>// <b><u>;</u></b></tt> +<br><tt>// ExpressionStatement</tt> +<br><tt>// Identifier +<b><u>:</u></b> +Statement</tt> +<br><tt>public interface IStatement // "marker" interface</tt> +<p><tt>public class Block extends ASTNode implements IStatement</tt> +<br><tt> public Block(AST ast);</tt> +<br><tt> public property IStatement[] statements;</tt> +<br><tt>public class IfStatement extends ASTNode implements IStatement</tt> +<br><tt> public IfStatement(AST ast);</tt> +<br><tt> public property IExpression test;</tt> +<br><tt> public property IStatement thenPart;</tt> +<br><tt> public property IStatement elsePart; // +optional</tt> +<br><tt>public class WhileStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ForStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class DoStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class TryStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class SwitchStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class SynchronizedStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ReturnStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ThrowStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class BreakStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ContinueStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class NullStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class LabeledStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class AssertStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<h4> +<font color="#000000">Expression Statements</font></h4> +<font color="#000000">Certain types of expressions can also appear as statements. +The ExpressionStatement node wraps an expression up as a statement. The +source range for the ExpressionStatement includes the trailing semicolon.</font><font color="#000000"></font> +<p><tt><font color="#000000">public class ExpressionStatement extends ASTNode +implements IStatement</font></tt> +<br><tt><font color="#000000"> public ExpressionStatement(AST +ast);</font></tt> +<br><tt><font color="#000000"> public property IExpression +expression;</font></tt> +<h4> +Expressions</h4> +There is a different AST node type for each different kind of expression. +Use a "marker" interface (<tt>IExpression</tt>) to bring all constructs +that can appear as expressions. +<p>(Many details TBD). +<p><tt>// Expression:</tt> +<br><tt>// +Identifier</tt> +<br><tt>// +ArrayAllocationExpression</tt> +<br><tt>// +StringLiteral</tt> +<br><tt>// +FloatingPointLiteral</tt> +<br><tt>// +BooleanLiteral</tt> +<br><tt>// +CharacterLiteral</tt> +<br><tt>// +StringLiteral</tt> +<br><tt>// +NullLiteral</tt> +<br><tt>// +( Type | <b><u>void</u></b> ) <b><u>.</u></b> class</tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>this</u></b></tt> +<br><tt>// <b><u>(</u></b> +Expression <b><u>)</u></b></tt> +<br><tt>// +[ Expression <b><u>.</u></b> ] <b><u>new</u></b> Type <b><u>(</u></b></tt> +<br><tt>// +[ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b> [ ClassBody +]</tt> +<br><tt>// +Expression <b><u>.</u></b> Identifier</tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>super .</u></b> Identifier</tt> +<br><tt>// +MethodName <b>(</b> [ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b></tt> +<br><tt>// +Expression <b><u>.</u></b> Identifier <b>(</b> [ Expression { <b><u>,</u></b> +Expression } ] <b><u>)</u></b></tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>super .</u></b> Identifier <b>(</b></tt> +<br><tt>// +[ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b></tt> +<br><tt>// +Expression <b><u>[</u></b> Expression <b><u>]</u></b></tt> +<br><tt>// +Expression InfixOperator Expression</tt> +<br><tt>// +Expression <b><u>instanceof</u></b> Type</tt> +<br><tt>// +Expression PostfixOperator</tt> +<br><tt>// +PrefixOperator Expression</tt> +<br><tt>// <b><u>(</u></b> +Type <b><u>)</u></b> Expression</tt> +<br><tt>// +Expression <b><u>?</u></b> Expression <b><u>:</u></b> Expression</tt> +<br><tt>// +Expression AssignmentOperator Expression</tt> +<br><tt>// ArrayInitializer</tt> +<br><tt>public interface IExpression // "marker" interface</tt> +<br><tt> public IBinding resolvedType(); // optional</tt> +<p><tt>// ArrayAllocationExpression:</tt> +<br><tt>// <b><u>new</u></b> +PrimitiveType <b><u>[</u></b> Expression <b><u>]</u></b> { <b><u>[</u></b> +Expression <b><u>]</u></b> } { <b><u>[</u></b> <b><u>]</u></b> }</tt> +<br><tt>// <b><u>new</u></b> +TypeName <b><u>[</u></b> Expression <b><u>]</u></b> { +<b><u>[</u></b> Expression +<b><u>]</u></b> +} { <b><u>[</u></b> <b><u>]</u></b> }</tt> +<br><tt>// <b><u>new</u></b> +PrimitiveType <b><u>[</u></b> <b><u>]</u></b> { <b><u>[]</u></b> } ArrayInitializer</tt> +<br><tt>// <b><u>new</u></b> +TypeName <b><u>[</u></b> <b><u>]</u></b> { <b><u>[]</u></b> } ArrayInitializer</tt> +<br><tt>public class ArrayAllocationExpression</tt> +<br><tt> extends ASTNode implements IExpression</tt> +<br><tt> public ArrayAllocationExpression(AST ast);</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property Expression[] dimensions; // +optional</tt> +<br><tt> public property Expression arrayInitializer; +// optional</tt> +<p><tt>public class StringLiteral extends ASTNode implements IExpression</tt> +<br><tt> public StringLiteral(AST ast);</tt> +<br><tt> public property String value;</tt> +<p><tt>public class CastExpression extends ASTNode implements IExpression</tt> +<br><tt> public CastExpression(AST ast);</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property IExpression value;</tt> +<p><tt>public class InfixExpression extends ASTNode implements IExpression</tt> +<br><tt> public InfixExpression(AST ast);</tt> +<br><tt> public property int infixOperator;</tt> +<br><tt> public property IExpression leftOperand;</tt> +<br><tt> public property IExpression rightOperand;</tt> +<br><tt> public property IExpression[] extendedOperands; +// L op R op R2 op R3...</tt> +<h3> +Bindings</h3> +The "world of bindings" is an integrated picture of the structure of the +program as seen from the compiler's point of view. The bindings correspond +to named entities (packages, types, fields, methods, local variables). +<p>Clients navigate from AST nodes into the world of bindings to discover +things like: +<ul> +<li> +the entity an identifier resolves to</li> + +<li> +the resolved type of an expression node</li> + +<li> +the resolved binding of a declaration node</li> + +<li> +others?</li> +</ul> +Once in the world of bindings, the client can navigate the web of bindings: +<ul> +<li> +from array type to its component type, and vice versa</li> + +<li> +from field or variable to its declared type</li> + +<li> +from method to its parameter and return types</li> + +<li> +from type to its constructors and its declared method, field, and type +members</li> + +<li> +from constructor, method, or field to its declaring type</li> + +<li> +from nested type to its enclosing type</li> + +<li> +from type to declaring package</li> + +<li> +from type to its supertypes (but, significantly, <i>not</i> to its subtypes)</li> + +<li> +directly to the binding for any base type (int, float, char, etc.)</li> + +<li> +directly to the binding for a handful of well-known types (java.lang.Object, +etc.)</li> +</ul> +Some of the navigations that are not supported (quite intentionally): +<ul> +<li> +from package to its (known) types - very expensive</li> + +<li> +from package to one of its types by name - very expensive</li> + +<li> +from type to its (known) subtypes - very expensive</li> + +<li> +from type or method to the local types it encloses - binding for local +types are only of interest to those with the enclosing type's AST in their +hand</li> + +<li> +from method to the variables declared within it - binding for variables +are only of interest to those with the method's AST in their hand</li> +</ul> +There are no links from the world of bindings back to the world of ASTs. +<p>Other things dealt with in the world of bindings: +<ul> +<li> +synthetic entities stemming from default constructors, abstract method +copy-down from interfaces, and inner class emulation</li> + +<li> +missing bindings for entities that are required (mentioned by name) but +were not found</li> + +<li> +type hierachy circularities</li> + +<li> +internal inconsistencies</li> +</ul> +Other issues: +<ul> +<li> +Compile-time-computed values for constants (public static final fields +with compile-time computable values)</li> +</ul> + +<h4> +Existing Binding classes</h4> +To give an idea of the scope of the existing binding infrastructure, below +is a dump of the type hierarchy of the compiler's binding classes from +package <tt>rg.eclipse.jdt.internal.compiler.lookup</tt>. +<p><tt>public abstract class Binding</tt> +<br><tt> public abstract class TypeBinding</tt> +<br><tt> public final class ArrayBinding</tt> +<br><tt> public final class BaseTypeBinding</tt> +<br><tt> public abstract class +ReferenceBinding</tt> +<br><tt> +public class SourceTypeBinding</tt> +<br><tt> +public class NestedTypeBinding</tt> +<br><tt> +public final class LocalTypeBinding</tt> +<br><tt> +public final class MemberTypeBinding</tt> +<br><tt> +public class ProblemReferenceBinding</tt> +<br><tt> +public class UnresolvedReferenceBinding</tt> +<br><tt> public class PackageBinding</tt> +<br><tt> public class ProblemPackageBinding</tt> +<br><tt> public abstract class VariableBinding</tt> +<br><tt> public class LocalVariableBinding</tt> +<br><tt> +public class SyntheticArgumentBinding</tt> +<br><tt> public class FieldBinding</tt> +<br><tt> +public class SyntheticFieldBinding</tt> +<br><tt> +public class ProblemFieldBinding</tt> +<br><tt> public class MethodBinding</tt> +<br><tt> public class ProblemMethodBinding</tt> +<br><tt> public class SyntheticAccessMethodBinding</tt> +<br><tt> public class ImportBinding</tt> +<br><tt> public class ProblemBinding</tt> +<h4> +Binding API</h4> +The existing binding classes are not immediately suitable for exposing +as a binding API. +<p>However, the Java builder does have an API for the built "image", in +package <tt>org.eclipse.jdt.internal.core.builder</tt>. (This API is a +hold-over from Leapfrog era, and is not exposed in the Eclipse code base). +This API was designed to expose the same kind of integrated picture of +the structure of the program as seen from the compiler's point of view. +This API has a detailed specification that does not expose implementation +details, so the proposal is to use it as the basis for the new binding +API. +<p>Re-purposing this API would entail: +<ul> +<li> +introducing entities for local variables</li> + +<li> +removing protocol for navigations that are not supported (e.g., from package +to its known types)</li> + +<li> +removing unneeded protocol; including states, non-state-specific handles, +deltas, report cards, dependency graph, package references</li> +</ul> +Below is a dump of the relevant interfaces from package <tt>org.eclipse.jdt.internal.core.builder</tt>. +Unnecessary protocol has been omitted. (Note that NotPresentException is +an unchecked exception, and would not be required.) +<p><tt>public interface IHandle</tt> +<br><tt> int K_JAVA_IMAGE = 1;</tt> +<br><tt> int K_JAVA_PACKAGE = 2;</tt> +<br><tt> int K_JAVA_TYPE = 3;</tt> +<br><tt> int K_JAVA_FIELD = 4;</tt> +<br><tt> int K_JAVA_METHOD = 5;</tt> +<br><tt> int K_JAVA_CONSTRUCTOR = 6;</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> int hashCode();</tt> +<br><tt> boolean isFictional() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<br><tt> int kind();</tt> +<p><tt>public interface IMember extends IHandle</tt> +<br><tt> IType getDeclaringClass();</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isBinary() throws NotPresentException;</tt> +<br><tt> boolean isDeprecated() throws NotPresentException;</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<p><tt>public interface IPackage extends IHandle</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getClassHandle(String name);</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isUnnamed();</tt> +<p><tt>public interface IType extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getArrayHandle();</tt> +<br><tt> IType getComponentType();</tt> +<br><tt> IConstructor getConstructorHandle(IType[] parameterTypes);</tt> +<br><tt> IType[] getDeclaredClasses() throws NotPresentException;</tt> +<br><tt> IConstructor[] getDeclaredConstructors() throws +NotPresentException;</tt> +<br><tt> IField[] getDeclaredFields() throws NotPresentException;</tt> +<br><tt> IMethod[] getDeclaredMethods() throws NotPresentException;</tt> +<br><tt> int getDeclaredModifiers() throws NotPresentException;</tt> +<br><tt> String getDeclaredName() throws NotPresentException;</tt> +<br><tt> IType getDeclaringClass() throws NotPresentException;</tt> +<br><tt> IField getFieldHandle(String name);</tt> +<br><tt> IType[] getInterfaces() throws NotPresentException;</tt> +<br><tt> IMethod getMethodHandle(String name, IType[] +parameterTypes);</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> IPackage getPackage();</tt> +<br><tt> String getSimpleName();</tt> +<br><tt> IType getSuperclass() throws NotPresentException;</tt> +<br><tt> boolean isAnonymous() throws NotPresentException;</tt> +<br><tt> boolean isArray();</tt> +<br><tt> boolean isBinary() throws NotPresentException;</tt> +<br><tt> boolean isClass() throws NotPresentException;</tt> +<br><tt> boolean isDeprecated() throws NotPresentException;</tt> +<br><tt> boolean isInnerClass() throws NotPresentException;</tt> +<br><tt> boolean isInterface() throws NotPresentException;</tt> +<br><tt> boolean isLocal() throws NotPresentException;</tt> +<br><tt> boolean isPackageMember() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<br><tt> boolean isPrimitive();</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<br><tt> boolean isTopLevel() throws NotPresentException;</tt> +<p><tt>public interface IMethod extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType[] getExceptionTypes() throws NotPresentException;</tt> +<br><tt> IType[] getParameterTypes();</tt> +<br><tt> IType getReturnType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p><tt>public interface IConstructor extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType[] getExceptionTypes() throws NotPresentException;</tt> +<br><tt> IType[] getParameterTypes();</tt> +<br><tt> boolean isPresent();</tt> +<p><tt>public interface IField extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p>In this vein, the interface for local variables would look something +like: +<p><tt>public interface IVariable extends IHandle</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getDeclaringClass();</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<br><tt> IType getType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p>Also will need to add: +<ul> +<li> +Pseudo-bindings for base types: boolean, int, float, etc.</li> + +<li> +Access to well-known java.lang bindings: Object, String, Throwable, Exception, +RuntimeException, Error, Class.</li> +</ul> + +<h3> +Document History</h3> +18:30 Thursday September 27, 2001 - incorporated first round comments from +PM and DB. +<br><font color="#000000">10:45 Monday October 1, 2001 - incorporated comments +from DB.</font> +<br><font color="#000000">10:45 Tuesday October 2, 2001 - clarify handing +of ExpressionStatement.</font> +<br><font color="#3366FF">14:00 Friday October 26, 2001 - add subtree structural +equality.</font> +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/dom ast/example-ast.txt b/org.eclipse.jdt.core/notes/r2.0/dom ast/example-ast.txt new file mode 100644 index 000000000..d8714fcc4 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/dom ast/example-ast.txt @@ -0,0 +1,110 @@ +Example AST +=========== + +The Java program shown in source form first is followed by a representation +of of its AST. + +================ +package com.example; + +import java.util.*; + +public class HelloWorld { + public static void main(String[] args) { + System.out.println("Hello" + " world"); + } +} +================ +<CompilationUnit + package: + <PackageDeclaration + name: + <QualifiedName + qualifier: <SimpleName identifier: "com"> + name: <SimpleName identifier: "example"> + > + > + imports: + [ + <ImportDeclaration + name: + <QualifiedName + qualifier: <SimpleName identifier: "java"> + name: <SimpleName identifier: "util"> + > + isOnDemand: true + > + ] + types: + [ + <TypeDeclaration + isInterface: false + modifiers: IModifier.PUBLIC + name: + <SimpleName identifier: "HelloWorld"> + superclass: null + superInterfaces: [] + bodyDeclarations: + [ + <MethodDeclaration + isConstructor: false + modifiers: IModifier.PUBLIC | IModifier.STATIC + selector: <SimpleName identifier: "main"> + returnType: + <PrimitiveType primitiveTypeCode: PrimitiveType.VOID> + parameters: + [ + <SingleVariableDeclaration + modifiers: IModifier.NONE + type: + <ArrayType + componentType: + <SimpleType + name: + <SimpleName identifier: "String"> + > + > + > + name: + <SimpleName identifier: "args"> + initializer: null + > + ] + thrownExceptions: [] + body: + <Block + statements: + [ + <ExpressionStatement + expression: + <MethodInvocation + expression: + <QualifiedName + qualifier: + <SimpleName identifier: "System"> + name: + <SimpleName identifier: "out"> + > + name: + <SimpleName identifier: "println"> + arguments: + [ + <InfixExpression + operator: InfixExpression.Operator.PLUS + leftOperand: + <StringLiteral escapedValue: "\"Hello\""> + rightOperand: + <StringLiteral escapedValue: "\" world\""> + extendedOperands: [] + > + ] + > + ] + > + ] + > + > + ] + ] + +
\ No newline at end of file diff --git a/org.eclipse.jdt.core/notes/r2.0/dom ast/knowProblems.txt b/org.eclipse.jdt.core/notes/r2.0/dom ast/knowProblems.txt new file mode 100644 index 000000000..2bfa663c9 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/dom ast/knowProblems.txt @@ -0,0 +1,9 @@ +- setLeadingComment is a problem. The parser doesn't store the comment + +TODO: + +getDeclaredModifiers() + +check that we filter out synthetic methods, fields and parameters. + +SimpleType diff --git a/org.eclipse.jdt.core/notes/r2.0/element deltas/java-element-deltas.html b/org.eclipse.jdt.core/notes/r2.0/element deltas/java-element-deltas.html new file mode 100644 index 000000000..73b60d337 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/element deltas/java-element-deltas.html @@ -0,0 +1,207 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.75 [en] (Windows NT 5.0; U) [Netscape]"> + <title>JDT - Java Element Deltas</title> +</head> +<body> + +<h2> +Java Element Deltas</h2> +Last revised 12:00 Thursday September 6, 2001 +<p>Original work item: "IJavaElementDelta should carry information about +marker changes (ProblemMarkers)." +<h3> +Markers and Java Elements</h3> +The workspace allows markers to be associated with resources. The Java +compiler associates Java problem markers with Java source files; these +markers locate the error within the compilation unit by source character +position. +<p>The Java model does not directly support markers on Java elements. It +does provide a means (<tt>ICompilationUnit.getElementAt</tt>) for mapping +from a source character position within a compilation unit resource to +the nearest enclosing Java element. Since all Java problem markers carry +a source character position, any Java problem marker can be mapped to the +nearest enclosing Java element. In most cases, this element will be a fine-grained +Java element corresponding to a declaration of a Java method, field, type, +or import. In the worst case, it maps to the compilation unit element itself. +Thus it is always possible in principle to associate markers with Java +elements. +<p>The Java UI visually annotates Java elements that have problems (with +a red X). +<p>The cost of mapping from a marker position to a fine-grained Java element +requires opening (and parsing) the compilation unit. Once opened, additional +markers within the same compilation unit can be mapped with negligible +additional cost. +<h3> +Java Element Deltas</h3> +Unlike workspace resource deltas, Java element deltas do not currently +include information about marker changes. +<p>The Java UI needs to maintain the visual annotations on Java elements +that have problems. +<p>There are three kinds of Java element deltas issued: +<ol> +<li> +By the Java model when <tt>IWorkingCopy.commit</tt> is called to save a +compilation unit working copy.</li> + +<li> +By the Java model when <tt>IWorkingCopy.reconcile</tt> is called during +editing.</li> + +<li> +By the Java model in response to notification of a workspace resource delta +affecting a compilation unit resource.</li> +</ol> +The first two kinds of Java element delta are always issued against fine-grained +Java elements. The previous fine-grained structure of the compilation unit +is compared to the current fine-grained structure. These deltas are recognizable +as kind <tt>CHANGED</tt> at the compilation unit level with <tt>F_CHILDREN</tt> +flag set and <tt>F_CONTENT</tt> flag <i>not</i> set. +<p>The last kind of Java element delta is never fine-grained; it goes no +finer than the compilation unit element itself. This is because the fine-grained +structure of the previous state of the compilation unit is not available +(the Java model does not necessarily have a record of the previous structure +(although it <i>might </i>be in an internal cache), and cannot compute +it since it does not a copy of the previous state of the compilation unit). +These deltas are recognizable as kind <tt>CHANGED</tt> at the compilation +unit level with <tt>F_CHILDREN</tt> flag <i>not </i>set and with <tt>F_CONTENT</tt> +flag set. +<h3> +Markers and Java Element Deltas</h3> +Java problems markers on Java compilation units originate when the Java +builder is called. These markers are associated with Java compilation unit +files, and come to the attention of the Java model via a problem delta +associated with the resource delta that follows the build. The Java builder +only deletes or adds Java problem markers; it never changes existing ones. +The Java builder does not decorate Java problem markers with information +other than the source character position. +<p><tt>(IWorkingCopy.reconcile</tt> also returns a set of tranisent markers.) +<p>When the user opens a compilation unit, the Java editor creates and +locates visual markers corresponding to the Java problem markers. These +visual markers are sticky, and move around as the text is edited. +<p>When the user navigates into an already open compilatation unit from +a problem marker in the Tasks view, the Java editor automatically adjusts +the positions to account for recent edits. +<p>When the Java editor saves a compilation unit, it permanently adjusts +the source positions of Java problem markers associated with the corresponding +compilation unit. This is done because the next build may be a ways off +and these markers show up in the Tasks view (the Java editor might be closed). +<p>Java element deltas also carry information about non-Java resources; +this is surfaced by <tt>IJavaElementDelta.getResourceDeltas</tt>. This +works as follows: when a non-Java resource is affected, the corresponding +resource delta is exposed on the nearest parent Java element. For files +directly under the project, the resource delta will be associated with +Java element delta for the <tt>IJavaProject</tt>; for files sitting in +the same folder as a Java source file, the resource delta will be associated +with Java element delta for the <tt>IPackageFragment</tt>; for files sitting +inside a source folder but not in any package fragment, the resource delta +will be associated with Java element delta for the <tt>IPackageFragmentRoot</tt> +(this includes the case where the project itself is a package fragment +root. Thus the <tt>IJavaElementDelta.getResourceDeltas</tt> mechanism is +only of use for discovering non-resource deltas; it is not useful for discovering +marker changes to resources corresponding to Java elements. +<p>Thus the only way a client can find out about most marker deltas is +to register its own resource change listener with the workspace. Since +the client would still need to register a Java model change listener, having +to do raises the question of relative ordering of these two notifications +(there is no guarantee which will comes first). +<h3> +Proposal: Expose corresponding resource deltas on Java element delta</h3> +The proposed change is to improve the Java element delta to expose the +corresponding resource deltas whenever they are available. This would be +done via the following new method on <tt>IJavaElementDelta</tt>: +<p><tt>/**</tt> +<br><tt> * Returns the workspace resource delta for the resource that +corresponds directly to the</tt> +<br><tt> * Java element, or <code>null</code> if either there +is no resource that corresponds</tt> +<br><tt> * to the Java element, or there is a corresponding resource +but there is no resource delta</tt> +<br><tt> * for it.</tt> +<br><tt> * <p></tt> +<br><tt> * Note that a Java element delta that does not arise from +a workspace resource delta will always</tt> +<br><tt> * return <code>null</code>.</tt> +<br><tt> * </p></tt> +<br><tt> * <p></tt> +<br><tt> * If the result is non-<code>null</code>, then</tt> +<br><tt> * <code>getCorrespondingResourceDelta().getResource()</code> +is the same resource as</tt> +<br><tt> * <code>getElement().getCorrespondingResource()</code>.</tt> +<br><tt> * </p></tt> +<br><tt> *</tt> +<br><tt> * @return the corresponding workspace resource delta, or +<code>null</code> if not applicable or none</tt> +<br><tt> */</tt> +<br><tt>public IResourceDelta getCorrespondingResourceDelta();</tt> +<p>Resource deltas will be available when the Java element delta results +from the Java model receiving a resource change notification. When this +happens, the resource delta for the corresponding will be made available +from the Java element delta for the Java element: +<ul> +<li> +Java project: expose resource delta for the corresponding project resource.</li> + +<li> +Java compilation unit: expose resource delta for the corresponding Java +source file.</li> + +<li> +Java class file: expose resource delta for the corresponding Java class +file.</li> + +<li> +Java package fragment root: expose resource delta for the corresponding +source folder, jar or zip file.</li> + +<li> +Java package fragment: expose resource delta for the corresponding package +folder under a source package fragment root.</li> +</ul> +Note that the proposed API change would not break compatibility with Eclipse +1.0. It adds a new method to an interface that is not intended to be implemented +by clients. +<p>With the proposed change, a client of the Java model would be able to +discover marker changes to both Java and non-Java resource by registering +for Java model changes and composing <tt>IJavaElementDelta.getCorrespondingResourceDelta</tt> +and <tt>IResourceDelta.getMarkerDeltas</tt>. These marker changes will +only be present on Java element deltas stemming from workspace resource +deltas. They are no marker deltas associated with fine-grained Java element +deltas arising from working copy reconciliation or saving (no markers have +changes). +<p>Other things we considered: +<ul> +<li> +For the case where a coarse-grained Java element deltas is issued, add +marker delta information to Java elements finer than the compilation unit. +This is not feasible because there would be no way to compute fine-grained +Java element or marker deltas since the Java model has no "before" state +to compare against.</li> + +<li> +For the cases where a fine-grained Java element deltas is issued, add marker +delta information to Java elements finer than the compilation unit. This +is not feasible because the markers aren't changing in this case, and the +Java model itself is not in the business of adjusting markers.</li> +</ul> + +<h3> +Document History</h3> + +<ul> +<li> +13:45 Thursday August 31, 2001 - First proposal send to Erich and Philippe</li> + +<li> +16:30 Wednesday Sepember 5, 2001 - Modified after discussion with Erich +and Martin.</li> + +<li> +12:00 Thursday September 6, 2001 - Modified after discussion with Jerome +who clarified how <tt>IJavaElementDelta.getResourceDeltas</tt> works.</li> +</ul> + +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/extension dir/ext-dirs.html b/org.eclipse.jdt.core/notes/r2.0/extension dir/ext-dirs.html new file mode 100644 index 000000000..36a9f4a56 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/extension dir/ext-dirs.html @@ -0,0 +1,233 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="Author" content="Build"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>JDT - Extension Directories</title> +</head> +<body> + +<h2> +Extension Directories</h2> +Last revised 17:10 Thursday October 25, 2001 +<p>Work item: Add support to Java model for JDK 1.2-style extension directories +(aka optional packages) +<p>An extension directory is a folder containing any number of JAR files +(including 0). Extension directories were originally added in JDK 1.2, +and are described in <a href="http://java.sun.com/j2se/1.3/docs/guide/extensions/index.html">http://java.sun.com/j2se/1.3/docs/guide/extensions/index.html</a>. +<p>The proposal is to allow a library classpath entry to refer to an extension +directory. A library classpath entry (<tt>CPE_LIBRARY</tt>) with a path +whose last segment is "*" indicates an extension directory; for example, +a library entry with the path "D:/jdk1.4/jre/lib/ext/*" means the extension +directory "D:/jdk1.4/jre/lib/ext". Like all library entries, extension +directories can be in folders inside the workspace or in directories in +the local file system outside the workspace. A variable classpath entry +may also resolve to an extension directory following the same rule about +ending in "*". Thus a variable classpath entry like "JRE_LIB/ext/*" would +resolve to an extension directory; and a simple one like "FOO" could resolve +to either a JAR library, a library folder, an extension directory, or a +project. +<p>Each of the JARs in an extension directory on a project's class path +would give rise to a separate <tt>IPackageFragmentRoot</tt> (kind <tt>K_BINARY</tt>); +these package fragment roots are all child elements of the <tt>IJavaProject</tt>. +The Java model would not have a Java element corresponding to the extension +directory itself. A classpath entry that identifies a extension folder +internal to the workspace gives rise to 0, 1, or more binary package fragment +roots for JAR file resources internal to the workspace; similarly, a classpath +entry that identifies a extension folder external to the workspace gives +rise to binary package fragment roots for JAR files outside the workspace. +<p>Every package fragment root that is a child of a project element stems +from some entry on that project's original classpath. Given a package fragment +root, how does one trace it back to the classpath entry that gave rise +to it? In the presence of variable classpath entries and extension directories, +the answer is not obvious. Having a simple answer is important to some +clients. For instance, the Java UI includes the names of classpath variables +in items in its Packages view. +<p><tt>IPackageFragmentRoot</tt> +<br><tt>/**</tt> +<br><tt> * Returns the classpath entry that gives rise to this package +fragment root.</tt> +<br><tt> * This package fragment root must exist. The returned classpath +entry is</tt> +<br><tt> * equivalent to one of the classpath entries of this package +fragment</tt> +<br><tt> * root's project.</tt> +<br><tt> *</tt> +<br><tt> * @return the originating classpath entry</tt> +<br><tt> * @exception JavaModelException if this element does not +exist</tt> +<br><tt> */</tt> +<br><tt>IClasspathEntry getOriginatingClasspathEntry() throws JavaModelException;</tt> +<p>This method would replace <tt>IJavaProject.getPackageFragmentRoots(IClasspathEntry +entry)</tt>, which has problems and would be removed from the API (via +deprecation). +<h3> +Packages View Clutter</h3> +The packages view would show all the JARs from an extension directory as +children of a project. Packages view clutter exists for anyone with a large +number of JAR libraries on their build classpath. Extension directories +exacerbate the problem by making it easy to include a whole set of JAR +libraries with a single classpath entry (WSAD reputedly has 70+ JARs in +its extension directory). +<p>The packages view supports filtering out JAR libraries, meaning they +could be hidden. However, the filter is off by default and not all users +are aware that the filter even exists. +<p>Note that extension folders typically contain a bunch of unrelated JARs, +making it likely that a user will be interested in a particular subset +and uninterested in the rest. The IBM 1.3.0 JRE jre\lib\ext contains indicim.jar +and JawBridge.jar. The Sun JDK 1.4.0-beta jre\lib\ext contains dnsns.jar, +ldapsec.jar, and sunjre_provider.jar. All these JARs are likely pure noise +and do not contain API that a client would write to. In a J2EE JRE, +there would be a diverse collection of JARs in the jre\lib\ext extension +directory (e.g., Java Telephony, JavaMail, Java 2D, Java 3D, Java Media +Frameworks), only a handful of which would be used within any given project. +<p>One idea is to allow a new JavaElement "ExtensionFolder". This would +allow to group the extension JARs and would avoid some clutter. Although +an extension folder element could avoid clutter, it may not provide enough +of an improvement over the existing filtering mechanism to warrant complicating +the Java model API. +<p>[This problem is still open. I recommend living with the currently supported +filtering for now. We could come back later and look at ways to reduce +clutter as a separate work item.] +<h3> +Default Build Classpath</h3> +The standard JRE implicitly has the jre/lib/ext extension directory on +its runtime classpath. By default, the build classpath for a typical project +should include entries for both the standard class library (jre/lib/rt.jar) +and the standard extension directory (jre/lib/ext/*). +<p>These defaults should be a function of the VM launcher for the project. +It should always be possible for the user to override the defaults and +set up their build classpath as they see fit. +<h3> +Build Classpath Ordering</h3> +The UI currently allows users to freely reorder package fragment roots +for the project. This does not really make sense for package fragments +roots that arise from extension directories. +<p>Indeed, the addition of extension directories means that there is no +longer a 1-1 correspondence between package fragment roots and classpath +entries. The UI should allow the user to determine the order of classpath +entries. Here are the guidelines and restrictions on the build classpath +for a project (N.B., these apply to the raw classpath, not necessarily +to a partially or fully "resolved" classpath): +<ul> +<li> +There may be 0, 1, or more source folders on the classpath. The relative +ordering of multiple source folders is significant to the user. Source +code in an earlier source folder hides a source file of the same name in +a later folder. This flexibility is rarely used: multiple source folders +is rare enough, and duplicated source files, even rarer.</li> + +<li> +Source folders must precede all other types of classpath entries. Nothing +should hide source (except another source). This property is implicitly +relied on whenever a source code patch is being applied to a binary library.</li> + +<li> +The relative ordering of non-source classpath entries (library folders, +library JARs, extension folders, required projects, and ones involving +variables) is significant to the user. The main reason it is significant +is to deal with conflicting library versions, where the user needs to control +which version wins. The ones earlier in the list should take precedence +over ones later in the list.</li> + +<li> +For a standard Java runtime, all libraries on the boot classpath (usually +just the standard JRE class library) are consulted, in the order specified, +and before other libraries). Placing the standard JRE class library as +the first non-source entry on the build classpath mirrors the most common +configuration.</li> + +<li> +For a standard Java runtime, all extension directories (usually just the +standard JRE extension directory) are consulted, in the order specified, +after libraries on the boot classpath but before other libraries. Within +an extension directory, the various library JARs are consulted in unspecified +order. Placing the standard JRE extension directory as the second non-source +entry on the build classpath mirrors the most common configuration.</li> +</ul> + +<h3> +Build Classpath Duplicates</h3> +How are duplicates libraries on the build classpath handled? For instance, +if there are two classpath entries on a project's build classpath for exactly +the same JAR, should there be one or two package fragment roots? What if +a JAR classpath entry explicitly mentions a JAR that is included in an +extension folder also on the build classpath? +<p>At the Java model API, package fragment roots are handle objects with +well-known identity criteria. It turns out that this dictates the answer +to how dupicates are handled. +<p>For library package fragment roots based on a resource in the workspace, +the identity criteria includes the resource handle (handle constructor +isJavaProject.getPackageFragmentRoot(IResource)) For library package fragment +roots based on files outside the workspace, the identity criteria includes +the path (handle constructor isJavaProject.getPackageFragmentRoot(String)). +<p>This means the answer is forced: any duplication in the build classpath +necessarily washes out in the mapping to package fragment roots. Two classpath +entries for exactly the same JAR must map to a single package fragment +root. JARs from an extension directory gives rise to the same package fragment +roots one would get if each JAR was mentioned explicitly. +<p>In the case of duplicates, we do have a choice for which classpath entry +is considered the originating one. <tt>IPackageFragmentRoot.getOriginatingClasspathEntry()</tt>, +described above, will be used to locate an entry in the build classpath +carring important source attachment information. A specific classpath entry +should always be preferred to a generic classpath entry (extension folders +entries carry no source attahment info), and an earlier entry should be +preferred over a later one. +<p>For example, given the following (unusual) build classpath: +<br><classpath> +<br> <classpathentry kind="lib" path="/jre/lib/ext/*"> +<br> <classpathentry kind="lib" +<br> path="/jre/lib/ext/servlet.jar" +<br> sourcepath="/src/servletsrc.zip"> +<br> <classpathentry kind="lib" +<br> path="/jre/lib/ext/servlet.jar" +<br> sourcepath="/backup/servletsrc.zip"> +<br></classpath> +<br>the package fragment root for "/jre/lib/ext/servlet.jar" is considered +to originate from the second classpath entry: the second is more specific +that the first, and is equally specific to, but ordered earlier than, the +third. +<h3> +Extension Directories and the Runtime Classpath</h3> +By default, jre/lib/ext is the extension directory at runtime. Overriding +this default entails specifying a command line argument to set the value +of the <tt>java.ext.dirs</tt> system property. This system property specifies +one or more directories to search for installed extensions, each separated +by <tt>File.pathSeparatorChar</tt>. +<p>We need to determine whether the JAR needs to be added to the runtime +class path or not. Since the launcher is JVM-install-type-specific, it +feels like the right place to hardcode the knowledge of how to map a build +classpath to a runtime classpath. The JAR for the standard class library +does not need to be included in the runtime classpath because it is implicitly +on the boot classpath. Similarly, any JAR that is included in an extension +directory does not need to be mentioned explicitly. +<h3> +Attaching Source to Library JARs in an Extension Folder</h3> +JARs from extension directories are not really any different from other +class libraries. It would make debugging difficult if there were no way +to attached source code to a JAR in a extension directory. +<p>How does one attach source to JARs in an extension directory? As you +recall from R0.9, we consciously opted to put the source attachment information +in a place where it would be sharable, rather than leave this information +local to the workspace. The API for attaching source is on the classpath +entry; this allows this information to be shared between developers. Clearly, +this would not work for extension directories, where the names of the JARs +in the directory might not even be known in advance. There is no general +naming scheme for locating the source code for a given JAR (we do not proposing +to invent one). +<p>As discussed in the section on package view clutter, it will be common +for an extension directory to contain a diverse set of JARs, only a handful +of which would be used within any given project. The proposal is to provide +no direct support for attaching source via an extension folder classpath +entry. If the user needs to see source code for a JAR in an extension folder, +they would need to create an additional explicit classpath entry for that +JAR. As discussed above, this would not result in a new package fragment +root; however, it would effectively change the originating classpath entry +to the explicit one where the source attachment information would be found. +This would allow the user to attach source code to JARs on an as-needed +basis. The bet is that this will suffice (and finesses the issue of where +one might find the source for a JAR in an extension folder). +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/extension dir/extensionDir.html b/org.eclipse.jdt.core/notes/r2.0/extension dir/extensionDir.html new file mode 100644 index 000000000..52fc49bdd --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/extension dir/extensionDir.html @@ -0,0 +1,44 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.75 [en] (Win98; U) [Netscape]"> +</head> +<body> + +<h1> +Extension Directory Support</h1> +The WSAD effort has exposed the need for extension directories so as to +simplify their classpath setup. Interestingly enough, the classpath rework +(make classpath less sensitive to project change from source to binary +form) has some nice connections with this project. +<p>The direction followed would allow to contribute all JARs contained +inside a given project to dependent projects. Thus such a project could +act as an extension directory relatively to its dependent projects. This +would however require to materialize such extension directories with true +projects, and could be considered as one way to achieve this, but a true +extension directory support should also exist, in a similar way as we allow +users to directly refer to external JARs if they want. +<p>We could add a new classpath entry kind for these. Now the next issue +is: do we allow classpath variables to denote extension directories ? Presumably, +this would be expected. +<p>A classpath variable is not bound to a particular classpath entry, but +rather to an IPath which is then resolved when the variable is substituted +with a resolved classpath entry. The resolution process can either bind +a variable to a project or a library, depending to what the IPath maps +to. +<p>In the library case, it can either be an archive file (*.jar, *.zip) +or a binary folder (*.class files). The distinction is made again depending +on the nature of what the IPath maps to, is it a JAR/ZIP file or a folder. +Now, if we want to fit the extension directory support in this, it will +be hard to distinguish in between a folder containing JARs or .class files... +<p>We could however imagine that the syntax of the IPath would tell which +one is expected to be resolved. For example, a path of the form "/SomeDirectory" +would continue to indicate a binary folder, whereas "/SomeDirectory/*" +would mean an extension directory. The benefit for this is that we actually +do not need to add a new type of entry. We just add a third flavor of a +library entry, the one mapping to an extension directory. +<p>Thus, extension directories would just be particular cases of library +ones, and be eligible in classpath variables. +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/jdom ast/ast.html b/org.eclipse.jdt.core/notes/r2.0/jdom ast/ast.html new file mode 100644 index 000000000..7296e6794 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/jdom ast/ast.html @@ -0,0 +1,2108 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>JDT - Abstract Syntax Trees</title> +</head> +<body> + +<h2> +Abstract Syntax Trees</h2> +<font color="#3366FF">Last revised 14:00 Friday October 26, 2001 (most +recent change in blue)</font> +<p>Original work item: "Exposing the AST API." +<p>Related work item: "Improved Java code manipulation support, address +JDOM limitations; JDOM doesn't preserve markers and isn't document aware; +JDOM finer grained update support (e.g. change method name, type name); +buffer contents is duplicated in Java UI document and needs to be manually +synchronized." +<h3> +Background</h3> + +<ul> +<li> +refactoring is a key customer (Dirk B.)</li> + +<li> +for 2.0: have good support in place so that refactoring can be made an +open API; would need corresponding core APIs for abstract syntax trees</li> + +<li> +current refactoring appoach uses AST visitor which runs post-resolve on +designated compilation unit</li> + +<li> +visitor collects info (parent stack) including source positions</li> + +<li> +uses name environment (bindings, scopes) to validate/locate</li> + +<li> +all changes are formulated as reified undoable edits in a source buffer</li> + +<li> +a batch of changes are made at a time</li> + +<li> +refactoring does not reason hypothetically (changes are undoable)</li> + +<li> +JDOM is not used by refactoring, but potentially could be (JDOM is used +inside Java Model)</li> +</ul> +Dirk's wish list with regard to the AST support: +<ul> +<li> +consistent positions in AST (e.g. sourceStart and sourceEnd should always +cover the whole node).</li> + +<li> +comment handling in AST. Some node contain preceding comments other don't</li> + +<li> +Parent pointer in AST node.</li> + +<li> +Data pointer in AST node.</li> + +<li> +Expression should provide getType method (cached value of resolveType(Scope)).</li> +</ul> + +<h3> +Summary from Sept. 10-11 meetings</h3> +Dirk travelled to SNZ to discuss refactoring requirements and possible +solutions with Philippe M. and Jeem. +<p>Some of the forms of solutions discussed, but ultimately abandoned: +<ul> +<li> +A vanilla DOM.</li> + +<ul> +<li> +too limiting: difficult to provide for pre-computing bindings.</li> + +<li> +clumsy for clients to use without AST node types represented by different +Java types</li> +</ul> + +<li> +AST plus resolves info in form of Java model elements.</li> + +<ul> +<li> +Java model methods are unsuitable canonical representation for resolved +methods because parameter types are as per source.</li> + +<li> +It is hard to map a type back to a Java element (need to remember package +fragment).</li> + +<li> +Would need additional Java model elements to represent variable declarations.</li> + +<li> +Would need special Java element implementations for local types, methods, +and fields.</li> +</ul> +</ul> +In the end, we agreed on AST plus bindings: +<ul> +<li> +AST is simple tree of typed nodes as determined by parser.</li> + +<li> +A different Java type for each AST node type.</li> + +<li> +TBD: are node types API classes, API interfaces, or both?</li> + +<li> +Simple case: no bindings.</li> + +<li> +Basic AST Lifecycle: client requests AST; compilation unit is parsed and +nodes are created; root is handed back to client; AST is garbage collected +after client lets go of all AST nodes. AST is not affected if underlying +compilation unit is changed (i.e., eager parse, with no residual dependence +on state of file in workspace).</li> + +<li> +Any syntax errors detected need to be reported to client. Problems are +opaque: source character position plus human-readable message. Clients +needs to determine whether an AST is only partial (e.g., busted flags).</li> + +<li> +Predicatable trees with simple correspondence to source code (parser should +not optimize ASTs).</li> + +<li> +Reliable source position information for all nodes.</li> + +<li> +Scratch "data" field on each AST node for client to record an Object.</li> + +<li> +Navigate AST upwards as well as downwards. Parent link.</li> + +<li> +AST walker for convenient traversal.</li> + +<li> +ASTs can be read-only or read-write.</li> + +<li> +Read-write ASTs can be modified in terms of ASTs. AST node factories (no +Java parser required). Cut/copy/paste.</li> + +<li> +Cut/copy/paste between separate ASTs requires fragment cloning to preserve +independence of storage.</li> + +<li> +Client may be interested in cut/copy/pasting comments too.</li> + +<li> +New nodes would not carry source positions.</li> + +<li> +Read-write AST can be serialized back to compilation unit char[].</li> + +<ul> +<li> +Preserve existing comments, whitespace, and use of \u.</li> + +<li> +Control whitespace and use of \u for insertions.</li> + +<li> +Provide edit map for updating markers.</li> +</ul> + +<li> +Resolved ASTs: basic AST annotated with bindings (non-syntactic information).</li> + +<li> +Binding information is derived from AST plus the Java model (relative to +some project).</li> + +<li> +Availability/validity of bindings ceases when Java model changes.</li> + +<li> +Client must request up front that bindings be created.</li> + +<li> +Certain AST nodes get decorated with bindings.</li> + +<li> +Client should have ways to communicate up front what kind of bindings are +required and where.</li> + +<li> +Availability/validity of bindings for a read-write AST ceases when it is +first modified.</li> + +<li> +Bindings (non-syntactic information) includes things such as the following:</li> + +<ul> +<li> +Resolved names - which type, field, method, or local variable does a AST +name refernce resolve to.</li> + +<li> +Resolved types - what is the resolved type of an AST expression or type +reference.</li> + +<li> +Resolved supertypes - what are the resolved supertypes of a resolved type.</li> + +<li> +Resolved declarations - what resolved type, field, method, or local variable +does an AST declaration map to.</li> + +<li> +Thrown exceptions - what are the resolved types of the exceptions thrown +by a given expression or statement.</li> + +<li> +Resolved members - what are the resolved members of a resolved type.</li> +</ul> + +<li> +Problems also should be reported with resolving. Problems are opaque: source +character position plus human-readable message.</li> + +<li> +Space for bindings storage is significant; increases monotonically as more +bindings are accessed.</li> + +<li> +Space for bindings is for lifetime of the AST.</li> + +<li> +Advanced AST Lifecycle: client requests AST with bindings; compilation +unit is parsed, nodes created, and perhaps some bindings created and annotated +on nodes; root is handed back to client; AST is garbage collected after +client lets go of all AST nodes and bindings. AST itself is not affected +if underlying compilation unit is changed (i.e., eager parse, with no residual +dependence on state of file in workspace). Bindings may become stale or +invalid if workspace changes (i.e., possibly lazy and incremental construction +of bindings using Java model).</li> + +<li> +<font color="#000000">Bindings from two ASTs are not generally comparable.</font></li> + +<ul> +<li> +<font color="#000000">For bindings with stable global names, API provides +strings that can be compared between ASTs.</font></li> +</ul> +</ul> +AST will either extend or replace JDOM. In the latter case, JDOM would +be deprecated. +<p>AST will exist along side Java model. +<h3> +API Design Issue: AST Node Types - Classes, interface, or both</h3> +There are on the order of 87 node types for Java ASTs. Bindings will add +some more. There are a couple of way this can be mapped to Java types. +<p>(1) Use org.w3c.DOM interfaces as API. Provide a private concrete implementation +of the DOM. +<p>Pro: Very small, and standard API for read-write documents. +<br>Con: API is not convenient for clients. +<br>Con: API is not amenable to exposing bindings and other non-structural +information. +<p>(2) Concrete API class per AST node type. +<p>Pro: API as small as possible. +<br>Pro: Client can create nodes directly. +<p>Con: Cannot easily hide implementation details; e.g. representation +and mechanism for reassembling compilation unit text after editing; lazy +binding creation. +<p>Clients who create node from scratch only ever need basic constructors +(the nodes they create do not have source positions, bindings, or other +decorations). On the other hand, the parser needs to remember more info +including fine-grained source positions. +<p>(3) API interface per AST node type, along with node factory methods. +Provide a private concrete implementation. Allow clients to reimplement +node types (from scratch) and supply a factory. +<p>Like JDOM (except JDOM does not permit client to reimplement) and org.w3c.dom. +<p>Pro: API as small as possible. +<br>Pro: Easy to tailor different kinds of representation: read-write vs. +read-only ASTs; raw ASTs vs. AST+bindings. +<p>Con: Hidden concrete implementation classes takes more space. +<br>Con: Using factory methods is a bit less direct than using constructors. +<p>We will use API interfaces for bindings, and exposed API classes for +AST nodes. +<h3> +API Design Issue: Statement vs. Statement Lists</h3> +For structured statements, like while, the child statement is grammatically +a single statement. However, since a block is one type of statement, it +is possible to have a list of statements underneath. There are options +for rendering this: +<p>(1) Child is a single statement. +<p>(Like current compiler's internal ASTs.) +<p>Pro: As per the Java grammar. +<br>Con: A client wanting to insert an additional statement into the child +must be prepared to replace by a block if there isn't one. +<p>(2) Child is a list of statements. +<p>(Like the .net codeDOM.) +<p>Pro: More convenient for clients that edit ASTs. Uniform mechanism for +inserting and removing statements from child. +<br>Con: Muddies scopes (enclosing braces around statements introduce a +scope and make declarations syntactically legal). +<p>We will go with (1) and stick closely to the grammar. +<h3> +Usage</h3> +There are a couple different usage scenarios for ASTs: +<ul> +<li> +Analyze an existing compilation unit to discover syntactic structure.</li> + +<li> +Discover relationship between syntactic structure and original text.</li> + +<li> +Discover relationship between syntactic structure and resolved world.</li> + +<li> +Create a new compilation unit from scratch.</li> + +<li> +Edit an existing compilation unit.</li> +</ul> + +<h3> +Source Construct Normalization</h3> + +<ul> +<li> +Most syntactic constructions are rendered in one and only one way.</li> + +<li> +When this is not the case, the AST construction is "lossy".</li> + +<li> +Some forms cannot be distinguised in input (if one cares).</li> + +<li> +Some forms cannot be produced in output.</li> + +<li> +Copying the construct normalizes it.</li> + +<li> +Example: Modifier order</li> + +<ul> +<li> +final static public int X = 1;</li> + +<li> +public static final int X = 1; // preferred form</li> +</ul> + +<li> +Example: Compound variable declarations</li> + +<ul> +<li> +int i = 1, j = 2;</li> + +<li> +int i = 1; int j = 2; // preferred form</li> +</ul> + +<li> +Example: Array type declarators</li> + +<ul> +<li> +int[] x[];</li> + +<li> +int[][] x; // preferred form</li> +</ul> + +<li> +Example: Short ifs</li> + +<ul> +<li> +if (a) f(); else ;</li> + +<li> +if (a) f(); // preferred form</li> +</ul> + +<li> +Can only be done for syntactic nuances that are have no semantic import.</li> + +<li> +Normalization is generally acceptable where unimportant syntactic nuances +are involved.</li> + +<li> +Normal form should follow JLS recommendations and Java coding standards.</li> + +<li> +Note that parentheses and blocks are important to user and should not be +normalized.</li> +</ul> + +<h3> +Source Positions</h3> + +<ul> +<li> +When AST is obtained by parsing a text string, exposing source ranges for +nodes allows clients to navigate back into original string; e.g., for making +text editor selections.</li> + +<li> +AST supports only character-oriented position information; mapping character +positions to lines are handled elsewhere (e.g., text editor).</li> + +<li> +Source ranges are irrelevant for nodes created by other means.</li> + +<li> +Source ranges give original position in original string.</li> + +<ul> +<li> +Editing the AST does not alter positions or anything clever.</li> +</ul> + +<li> +Most constructs occupy contiguous character positions, or ranges.</li> + +<li> +Ranges are represented by 0-based start position and length.</li> + +<li> +Start position begins at first significant character of construct corresponding +to AST node.</li> + +<ul> +<li> +First significant character.</li> + +<li> +Does not include leading whitespace.</li> + +<li> +Does not include preceding comment (except the javadoc comment preceding +a declaration, or the comment preceding a statement - see below).</li> +</ul> + +<li> +End position includes last significant character of construct corresponding +to AST node.</li> + +<ul> +<li> +Last significant character.</li> + +<li> +Includes trailing terminators that are part of construct; e.g., include +trailing semicolon at end of local variable declaration.</li> + +<li> +Does not include separators; e.g., exclude trailing comma in parameter +list.</li> + +<li> +Does not include trailing whitespace.</li> + +<li> +Does not include trailing comment.</li> + +<li> +Statement end-of-line comments are not encompassed by statement.</li> + +<ul> +<li> +<tt>System.out.println("hello"); // $non-nls$</tt></li> +</ul> + +<li> +<font color="#000000">Embedded comments are encompassed if they occur before +end position.</font></li> + +<ul> +<li> +<tt><font color="#000000">System.out.println("hello") /* comment */;</font></tt></li> +</ul> +</ul> + +<li> +Some node types would have source ranges for significant contiguous subconstructs +not readily gleanable from source ranges of the subnodes.</li> + +<ul> +<li> +Additional source ranges would be specified for each node type.</li> + +<li> +E.g., method declaration has additional source range for the method name +and for the method declaration excluding its javadoc comment.</li> + +<li> +Use start and length arrays rather than proliferate API methods for additional +source ranges.</li> +</ul> +</ul> + +<h3> +Unicode Escapes</h3> + +<ul> +<li> +Original source text might contain Unicode escapes (JLS 3.2, 3.3).</li> + +<li> +E.g., void\u0040\u005a(); declares a method named Z.</li> + +<li> +Scanner removes all Unicode escapes and returns a Unicode token stream.</li> + +<li> +Newly created AST nodes are "post" Unicode escaping.</li> + +<li> +Output options:</li> + +<ul> +<li> +Preserve existing Unicode escapes (default); remove all existing Unicode +escapes.</li> + +<li> +Do not introduce Unicode escapes (default); introduce Unicode escapes for +characters in a specified set (e.g., all non-ASCII).</li> +</ul> + +<li> +Initial implementation: support default behavior only.</li> +</ul> + +<h3> +Comments</h3> + +<ul> +<li> +Comments are problematic for ASTs; these lexical items are normally filtered +out of token stream.</li> + +<li> +Comments are significant to user.</li> + +<li> +Editing an existing compilation unit should generally preserve existing +comments.</li> + +<li> +Should be able to include comments for newly created subtrees.</li> + +<li> +Copying a subtree from one place to another should include relevant comments.</li> + +<li> +Most common forms of comments:</li> + +<ul> +<li> +Javadoc comments - on one or more lines preceding field, method, and type +declarations.</li> + +<li> +Boilerplace comments (copyright notices) - one or more lines preceding +the package declaration, or between the package declaration and first import +or type declaration.</li> + +<li> +Statement comments - one or more lines between statements in a block.</li> + +<li> +Statement end-of-line comments.</li> +</ul> + +<li> +VA/ST experience: not worth bending over backwards to accomodate all comments.</li> + +<li> +Determined clients can rescan original string to get at all comments.</li> + +<li> +Expose high value comments:</li> + +<li> +Javadoc comments - treat as attribute of the field, method, and type declarations.</li> + +<ul> +<li> +Clients can extract Javadoc attributes (including @deprecated).</li> + +<li> +Clients can create declarations with Javadoc.</li> +</ul> + +<li> +Statement comments within blocks</li> + +<ul> +<li> +Approach 1: Treat as pseudo-statements with a special AST node type.</li> + +<ul> +<li> +Pro: Clients can include comments with blocks.</li> + +<li> +Con: Only works for comments within genuine blocks. E.g., can't handle</li> + +<li> +<tt>if (test)</tt></li> + +<li> +<tt> // should not happen</tt></li> + +<li> +<tt> throw new RuntimeException();</tt></li> + +<li> +Would work better if we were using statement lists in more places.</li> +</ul> + +<li> +Approach 2: Treat as a property of following statment node.</li> + +<ul> +<li> +Pro: Clients can include comments before any statement.</li> + +<li> +Con: Does not handle trailing comments in blocks. E.g.,</li> + +<li> +<tt>{</tt></li> + +<li> +<tt> throw new RuntimeException();</tt></li> + +<li> +<tt> // can't reach here</tt></li> + +<li> +<tt>}</tt></li> +</ul> + +<li> +Recommend approach 2 since it covers most cases.</li> +</ul> + +<li> +Boilerplate comments would not be exposed, but would be preserved through +edit and output.</li> +</ul> + +<h3> +Whitespace</h3> + +<ul> +<li> +Whitespace (JLS 3.6) includes ASCII SP, HT, and FF characters, and line +terminators.</li> + +<li> +Like comments, whitespace is significant to user.</li> + +<li> +Editing an existing compilation unit should generally preserve whitespace.</li> + +<li> +Whitespace for newly created subtrees automatically generated to produce +output that follows common conventions and blends in with surrounding text +(use the same leading whitespace).</li> + +<li> +Copying a subtree from one place to another should should generally preserve +whitespace.</li> +</ul> + +<h3> +AST Parent Backpointer</h3> + +<ul> +<li> +Each AST node will carry a backpointer to its parent node.</li> + +<li> +ASTNode.getParent() returns ASTNode</li> + +<li> +This permits clients to traverse ASTs in upward as well as downward direction.</li> + +<li> +Bidirectional links must be maintained during editing.</li> + +<li> +Deletion API must unlink child from parent.</li> + +<li> +Insertion API must link child to parent.</li> + +<ul> +<li> +To preserve treeness, automatically clone child subtree if child already +has parent.</li> +</ul> + +<li> +Replace API must unlink old child before inserting new child.</li> + +<li> +Parent backlinks means that hanging on to <i>any</i> node in an AST instance +will prevent any part of the AST instance from being garbage collected.</li> +</ul> + +<h3> +Multiple ASTs</h3> + +<ul> +<li> +Muliple ASTs can exist side by side (and ASTs are potentially for same +compilation unit).</li> + +<li> +Allow insertion of nodes from one AST into another AST.</li> + +<ul> +<li> +Automatically clones child subtree (forgetting source positions and binding +decorations).</li> + +<li> +Ensure memory representation of ASTs remain completely independent.</li> +</ul> +</ul> + +<h3> +<font color="#3366FF">Structural Equality</font></h3> + +<ul> +<li> +<font color="#3366FF">Structural equality predicate on AST nodes.</font></li> + +<li> +<font color="#3366FF">Isomorphic subtrees.</font></li> + +<li> +<font color="#3366FF">Belonging to same or different AST.</font></li> + +<li> +<font color="#3366FF">Considers structural info only; ignores source positions, +bindings, etc.</font></li> + +<li> +<font color="#3366FF">Named something other than "equals" to avoid having +to reimplement hashCode too.</font></li> +</ul> + +<h3> +Syntactic Correctness of Parser-built ASTs</h3> + +<ul> +<li> +For ASTs built by a Java parser, there are issues of syntactic correctness.</li> + +<li> +Syntactic correctness is judged by the Syntactic Grammar (as defined in +JLS2 section 2.3).</li> + +<li> +Java parser <b>must</b> guarantee to produce a faithful AST for any syntactically +correct compilation unit.</li> + +<li> +Java parser <b>may</b> also build ASTs for syntactically incorrect compilation +units.</li> + +<li> +Complicant Java compilers must reject syntactically incorrect compilation +units.</li> + +<li> +What principle do we apply to Java parsers and the ASTs they return?</li> + +<li> +Real Java parsers are invariably more liberal than the Syntactic Grammar, +and rely on post-parse checks to report errors for any syntactically incorrect +constructs that makes it past the parser.</li> + +<ul> +<li> +E.g., conflicting modifiers: public private</li> + +<li> +E.g., field declared with no initializer occurs in an interface</li> + +<li> +E.g., void foo() [];</li> +</ul> + +<li> +In the current Eclipse compiler, many of these checks are done in the course +of type and name resolution. If client only wants AST, we want to avoid +doing expensive name and type analysis.</li> + +<li> +Approach 1: Guarantee that no ASTs are built for syntactically incorrect +compilation units.</li> + +<ul> +<li> +You do not get an AST at all for compilation units with syntax errors.</li> + +<li> +Pro: Client can trust parser to distinguish syntactically correct from +incorrect.</li> + +<li> +Con: Client cannot manipulate syntactically incorrect compilation units +at all.</li> + +<li> +Con: Requires post-parse check to detect residual syntax errors.</li> +</ul> + +<li> +Approach 2: Provide no guarantee about the ASTs for syntactically incorrect +compilation units.</li> + +<ul> +<li> +You might not get a useful AST at all.</li> + +<li> +You might get an AST that had pieces missing; e.g., a malformed method +was excised</li> + +<li> +You might get an AST that is incoherent or self-contradictory; e.g., a +transient class!?</li> + +<li> +Pro: Maximum flexibility for implementation.</li> + +<li> +Pro: Client can get useful ASTs for some syntactically incorrect programs.</li> + +<li> +Con: Client cannot trust parser to distinguish syntactically correct from +incorrect.</li> +</ul> + +<li> +Approach 3: Guarantee that the client examining the resulting AST has some +way to determine whether the compilation units is incorrect.</li> + +<ul> +<li> +Priniciple: Syntactic errors must not be suppressed.</li> + +<li> +AST nodes could carry flags indicating certain syntax problem; e.g., duplicate +modifiers public public</li> + +<li> +A bit on root node could say "unspecified syntax errors".</li> + +<li> +Could be special AST nodes types indicating major problems; e.g., bogus +method body</li> + +<li> +Could be representable configurations of AST node types that are recognizable +as syntactially incorrect; e.g., conflicting modifiers public private; +missing field initializer in interface</li> + +<li> +Pro: Client can trust parser to not hide any syntax errors that are in +the source.</li> + +<li> +Pro: Client can get useful ASTs for syntactically incorrect programs.</li> + +<li> +Con: Client must do extra work to determine whether there are syntax errors.</li> + +<li> +Con: Extra work to include this information if no client really cares about +the difference between syntactically correct and incorrect.</li> +</ul> + +<li> +The first approach is too harsh. It is much more reasonable, and interesting, +to be able to work with some syntactically incorrect compilation units.</li> + +<li> +The second approach feels reasonable if clients never care whether the +source is syntactically correct or not.</li> + +<li> +The third approach feels reasonable if some clients would care whether +the source is syntactically correct or not.</li> + +<li> +The principle difference between the second and third appoaches is that +the former sanctions quietly suppressing syntax errors whereas the latter +precludes it.</li> + +<li> +The nature of the AST nodes inherently makes room to express a wide range +of syntactically malformed programs.</li> + +<li> +An extra flag per node for "unspecified syntax errors" should cover the +bases.</li> + +<li> +The existing compiler's ASTs already carry enough information to enable +the compiler to do thorough post-parse detecting of residual syntax errors.</li> + +<li> +Therefore the third approach is within easy reach.</li> + +<li> +The third approach gives clients more than the second approach.</li> + +<li> +Recommendation: we adopt the third approach.</li> +</ul> + +<h3> +Syntactic Correctness of Non-parser-built ASTs</h3> + +<ul> +<li> +ASTs do not just come from a parser.</li> + +<ul> +<li> +They can be created from scratch.</li> + +<li> +A parser-build AST can be edited.</li> +</ul> + +<li> +These ASTs will need to be serialized to a source compilation unit (why +else would they exist?).</li> + +<li> +What kinds of support and guarantees are in place to ensure that such a +suitable source compilation unit can be generated?</li> + +<li> +Basic guarantee: any AST that could have come from parsing a syntactically +correct compilation unit will serialize to a compilation unit that is</li> + +<ul> +<li> +(a) syntactically correct</li> + +<li> +(b) strongly semantically equivalent to the original compilation unit.</li> + +<li> +and possibly (c) normalized; that is, parse(serialize(x)) is isomorphic +to x</li> +</ul> + +<li> +There are likely many ways to get ASTs that do not correspond to any syntactically +correct compilation unit.</li> + +<ul> +<li> +E.g., use illegal identifiers ("1abc" or "try" or "//").</li> + +<li> +E.g., use illegal modifier combinations with modifier bit masks.</li> +</ul> + +<li> +Post-screening the AST for syntactic correctness would be misguided.</li> + +<li> +Should just go ahead and generate the obvious, syntactically incorrect, +compilation unit.</li> + +<li> +More importantly: ensure semantic equivalence.</li> + +<li> +Operator precedence creates issues:</li> + +<ul> +<li> +E.g., given AST for expression <tt>v1*v2</tt>, replace <tt>v1</tt> node +by expression <tt>v1+v3</tt>.</li> + +<li> +Naive serialization yields <tt>v1+v3*v2</tt> which is not semantically +equivalent to the AST.</li> + +<li> +Result should be (<tt>v1+v3)*v2</tt>.</li> + +<li> +Parentheses may need to be introduced during serialization.</li> +</ul> + +<li> +Nested if statement creates issues:</li> + +<ul> +<li> +E.g., given AST for statement <tt>if (a) f(); else g();</tt>, replace <tt>f();</tt> +by <tt>if (b) h();</tt></li> + +<li> +Naive serialization yields <tt>if (a) if (b) h(); else g();</tt></li> + +<li> +Result should be <tt>if (a) if (b) h(); <b>else </b>; else g();</tt></li> + +<li> +Extra verbiage may need to be introduced during serialization.</li> +</ul> +</ul> + +<h3> +Deep Constructs</h3> + +<ul> +<li> +Some programs involve impossibly deep constructs.</li> + +<li> +Multi-line string concatenation expressions are the main offender.</li> + +<ul> +<li> +For example, <tt>"Line 1\\n"+"Line 2\\n"+...+"Line 5000"</tt></li> +</ul> + +<li> +Runtime stacks blow when recursing over deep ASTs.</li> + +<li> +AST node types should be designed to keep trees reasonably shallow for +reasonably typical programs.</li> + +<li> +Introduce N-ary operator expression node type to deal with multi-line string +concatenation expressions.</li> + +<li> +N.B. Current compiler performs compile-time concatenations during parse +phase to deal with this problem.</li> +</ul> + +<h3> +Editing Protocol</h3> + +<ul> +<li> +What general form should the editing API take?</li> + +<li> +Setters on receiver to manipulate its children (parent never affected)</li> + +<ul> +<li> +E.g., whileStatement.setCondition(newExpression)</li> + +<li> +Use null for optional children</li> +</ul> + +<li> +Treat lists as an array-valued property.</li> + +<ul> +<li> +E.g., block.getStatements() returns Statement[]</li> + +<li> +E.g., block.setStatements(Statement[] statements)</li> + +<li> +Use empty list for no children (rather than null)</li> +</ul> + +<li> +Alternative approach for lists: use Collection-like protocol</li> + +<ul> +<li> +E.g., block.addStatement(pos, newChildStatement)</li> + +<li> +E.g., block.removeStatement(oldChildStatement)</li> + +<li> +Con: Increased number of methods on API; bad when a node type has several +list properties.</li> +</ul> + +<li> +Alternative approach for delete/replace: use parent backpointers to implement +generic delete and replace operation which affect the receiver's relationship +to its parent</li> + +<ul> +<li> +E.g., oldChildStatement.delete()</li> + +<li> +Con: semantics of deletion ugly when node occurs outside of any list</li> +</ul> +</ul> + +<h3> +User Data Field</h3> + +<ul> +<li> +Each AST node has a user data slot reserved for client use.</li> + +<li> +ASTNode.getClientData() returns Object</li> + +<li> +ASTNode.setClientData(Object data)</li> + +<li> +The initial value is null.</li> + +<li> +Client may use for decorations, or whatever.</li> + +<li> +AST nodes created by parser carry no data initially.</li> + +<li> +AST nodes created explicitly carry no data initially.</li> + +<li> +Even read-only ASTs have read-write data slots.</li> + +<li> +Cloning an AST node creates a new node (does <b>not</b> copy or clone data).</li> +</ul> + +<h3> +Lists of Members</h3> + +<ul> +<li> +List of field, method, and type members of a type declaration.</li> + +<li> +This list is syntactically and semantically heterogenous.</li> + +<li> +No syntactic constraints on number and order.</li> + +<li> +Order is significant to user.</li> + +<li> +Within field declarations, relative order is semantically significant.</li> + +<li> +Standard practices:</li> + +<ul> +<li> +Place field declarations before member methods and types.</li> + +<li> +Place types before methods.</li> +</ul> + +<li> +Option (1): expose separate lists for field, methods, and types.</li> + +<li> +Pro: This is way internal AST works.</li> + +<li> +Pro: Convenient for clients to locate member fields, methods, and types.</li> + +<li> +Con: Not flexible for editing; editing will mangle member order.</li> + +<li> +Option (2): expose a single list of members</li> + +<li> +Pro: parser does not normalize; client controls order of members.</li> + +<li> +Con: More work for clients to locate member fields, methods, and types.</li> + +<li> +Option (3): expose a single list of members, with extra getters for locating +member fields, methods, and types.</li> + +<li> +Pro: Combines advantage of (2) with convenience of (1).</li> + +<li> +Recommended approach: (3).</li> + +<li> +For class declarations, treat initializers and constructors as members.</li> + +<ul> +<li> +Lump instance and static initializers in with field declarations.</li> + +<li> +Lump constructor declarations in with method declarations.</li> +</ul> +</ul> + +<h3> +Serialization</h3> + +<ul> +<li> +Clients of read-write ASTs will generally want to serialize to a Java compilation +unit.</li> + +<li> +Serialization via simple AST tree walk.</li> + +<ul> +<li> +Straightforward.</li> + +<li> +Introduce line breaks and whitespace to make it look pretty.</li> + +<li> +Or post-process it with the Java formatter.</li> + +<li> +If AST originated by parsing, the result is likely unacceptable to user:</li> + +<ul> +<li> +Completely reformatted.</li> + +<li> +Constructs are normalized.</li> + +<li> +Some comments may have be lost.</li> +</ul> + +<li> +Could be provided by API that makes use of regular AST API only.</li> + +<li> +Could be written by clients.</li> +</ul> + +<li> +Serialization via source reconstruction.</li> + +<ul> +<li> +Only applicable to ASTs initially constructed by parser.</li> + +<li> +Use source position information in modified AST to reconstruct compilation +unit.</li> + +<li> +Retain passages of original text corresponding to unchanged AST trees.</li> + +<li> +Generates new text only where required.</li> + +<li> +Produce a result that a user will recognize and accept.</li> + +<ul> +<li> +Preserve formatting wherever possible.</li> + +<li> +Preserve source construct normalization wherever possible.</li> + +<li> +Preserve arbitrarily-placed comments wherever possible.</li> +</ul> + +<li> +Requires retaining the original compilation unit, and likely recording +additional information in nodes to allow reconstruction.</li> + +<li> +This is the way the current JDOM implementation works.</li> + +<li> +Could be provided by API that has privileged access to AST nodes and parser-recorded +information.</li> + +<li> +Should also return a list of edit instructions so that markers can be adjusted, +etc.</li> + +<li> +Clients would have a hard time doing this themselves.</li> +</ul> + +<li> +<font color="#000000">Recommend deferring implementation of serializer +that does source reconstruction.</font></li> + +<ul> +<li> +<font color="#000000">In interim, refactoring can apply edits to original +compilation unit text directly.</font></li> +</ul> +</ul> + +<h3> +Node types</h3> +The AST node types are based on the standard grammar for the Java language +given in the JLS2. +<p>Every AST node belongs to a single AST instance. (In DOM terminology, +the AST is the document and the AST nodes are the elements). The AST instance +can serve as a factory for creating new nodes. Nodes point to their owning +AST (fixed for the node's lifetime). The AST points directly to the root +node (a compilation unit). +<p>The AST node types do not define their own notion of equality; they +just inherit the object identity based implementation from Object. +<p>Note: Grammar rules (in comments) are expressed in the Pascal-style +extended BNF used in <tt>section 18</tt> of JLS2. We use C# style property +declarations as a convenient abbreviation for a standard matched pair of +get and set methods. +<p><tt>public class AST</tt> +<br><tt> public AST();</tt> +<br><tt> public property CompilationUnit root;</tt> +<br><tt> public void loadFromSource(char[] source);</tt> +<br><tt> public void setOptions(...);</tt> +<br><tt> public char[] serialize();</tt> +<p><tt>public abstract class ASTNode</tt> +<br><tt> protected ASTNode(AST ast);</tt> +<br><tt> public AST getOwner();</tt> +<br><tt> public property int[] startPositions;</tt> +<br><tt> public property int[] lengths;</tt> +<br><tt> public property boolean isWholeLine;</tt> +<br><tt> public property Object clientData;</tt> +<br><tt> public ASTNode getParent();</tt> +<br><tt> ... other protocol common to all AST node types</tt> +<h4> +Names</h4> +As explained in JLS2 section 6.5, the grammar does not allow names to be +resolved more finely than the following 6 categories by syntactic means +alone: +<p><tt>PackageName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageName . Identifier</tt> +<br><tt>TypeName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageOrTypeName . Identifier</tt> +<p><tt>ExpressionName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p><tt>MethodName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p><tt>PackageOrTypeName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +PackageOrTypeName . Identifier</tt> +<p><tt>AmbiguousName:</tt> +<br><tt> +Identifier</tt> +<br><tt> +AmbiguousName . Identifier</tt> +<p>Given that names cannot be resolved definitively to a package, type, +field, or variable at AST node construction time, an open question is how +much of the categorization that could be done should be reflected in the +AST. More categories means more information flow from the parser to the +AST client; on the other hand, a variety of categories is not necessarily +convenient for clients. For example, in <tt>import a.b.c</tt> the name +is a <tt>TypeName</tt> whereas in <tt>import a.b.c.*</tt> the name <tt>a.b.c</tt> +is a <tt>PackageOrTypeName</tt>. If the name category was to be reflected +in the type of the AST nodes, the client would need to know to create the +appropriate type of name nodes when editing the AST. +<p>Proposal: Use two AST node types for names: simple names, and qualified +names. Qualified names are expressed recursively, to facilitate clients +discovering how the qualifier part of a name resolves. Use these for everything +but <tt>MethodName</tt>; for <tt>MethodName</tt>, which can appear only +in a method invocation expression, separate the selector identifier from +any preceding qualifier. +<p>(Note: The current internal AST nodes go beyond making the simple/qualified +distinction: they also have simple & qualified type names (classes +<tt>SimpleTypeReference</tt> +and <tt>QualifiedTypeReference</tt>) in additional to simple & qualified +named (classes <tt>SimpleNameReference</tt> and +<tt>QualifiedNameReference</tt>).) +<p><tt>// Name:</tt> +<br><tt>// +SimpleName</tt> +<br><tt>// +QualifiedName</tt> +<br><tt>// SimpleName:</tt> +<br><tt>// +Identifier</tt> +<br><tt>// QualifiedName:</tt> +<br><tt>// +Name <b><u>.</u></b> Identifier</tt> +<br><tt>public interface IName // "marker" interface</tt> +<br><tt> public IBinding resolvedBinding(); // +optional</tt> +<p><tt>public class SimpleName extends ASTNode implements IName, IExpression</tt> +<br><tt> public SimpleName(AST ast);</tt> +<br><tt> public property char[] identifier;</tt> +<p><tt>public class QualifiedName extends ASTNode implements IName, IExpression</tt> +<br><tt> public QualifiedName(AST ast);</tt> +<br><tt> public property IName qualifier;</tt> +<br><tt> public property char[] identifier;</tt> +<h3> +Compilation Units and Major Declarations</h3> +<tt>// CompilationUnit:</tt> +<br><tt>// +[<b><u>package</u></b> Identifier { <b><u>.</u></b> Identifier } <b><u>;</u></b> +]</tt> +<br><tt>// +{ImportDeclaration}</tt> +<br><tt>// +{TypeDeclaration | <b><u>;</u></b>}</tt> +<br><tt>public class CompilationUnit extends ASTNode</tt> +<br><tt> public CompilationUnit(AST ast);</tt> +<br><tt> public property Name packageName; // optional</tt> +<br><tt> public property ImportDeclaration[] imports;</tt> +<br><tt> public property TypeDeclaration[] types;</tt> +<p><tt>// ImportDeclaration:</tt> +<br><tt>// <b><u>import</u></b> +Identifier { <b><u>.</u></b> Identifier } [ <b><u>.</u></b> <b><u>*</u></b> +] +<b><u>;</u></b></tt> +<br><tt>public class ImportDeclaration extends ASTNode</tt> +<br><tt> public ImportDeclaration(AST ast);</tt> +<br><tt> public property Name importName;</tt> +<br><tt> public property boolean onDemand;</tt> +<br><tt> public IBinding resolveBinding();</tt> +<p><tt>// TypeDeclaration:</tt> +<br><tt>// +{Modifier} <b><u>class</u></b> Identifier</tt> +<br><tt>// +[<b><u>extends</u></b> Type]</tt> +<br><tt>// +[<b><u>implements</u></b> Type { <b><u>,</u></b> Type}]</tt> +<br><tt>// +<b><u>{</u></b> {ClassBodyDeclaration | <b><u>;</u></b> } <b><u>}</u></b></tt> +<br><tt>// +{Modifier} <b><u>interface</u></b> Identifier</tt> +<br><tt>// +[<b><u>extends</u></b> Type { <b><u>,</u></b> Type}]</tt> +<br><tt>// +<b><u>{</u></b> {InterfaceBodyDeclaration | <b><u>;</u></b> } <b><u>}</u></b></tt> +<br><tt>// Modifier:</tt> +<br><tt>// <b><u>public</u></b></tt> +<br><tt>// <b><u>protected</u></b></tt> +<br><tt>// <b><u>private</u></b></tt> +<br><tt>// <b><u>static</u></b></tt> +<br><tt>// <b><u>abstract</u></b></tt> +<br><tt>// <b><u>final</u></b></tt> +<br><tt>// <b><u>native</u></b></tt> +<br><tt>// <b><u>synchronized</u></b></tt> +<br><tt>// <b><u>transient</u></b></tt> +<br><tt>// <b><u>volatile</u></b></tt> +<br><tt>// <b><u>strictfp</u></b></tt> +<br><tt>// ClassBodyDeclaration:</tt> +<br><tt>// +MethodDeclaration</tt> +<br><tt>// +ConstructorDeclaration</tt> +<br><tt>// +FieldDeclaration</tt> +<br><tt>// +ClassDeclaration</tt> +<br><tt>// +TypeDeclaration</tt> +<br><tt>// +Initializer</tt> +<br><tt>// InterfaceBodyDeclaration:</tt> +<br><tt>// +MethodDeclaration</tt> +<br><tt>// +FieldDeclaration</tt> +<br><tt>// +TypeDeclaration</tt> +<br><tt>public class TypeDeclaration extends ASTNode implements IStatement, +IMember</tt> +<br><tt> public TypeDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Name superclass; // optional</tt> +<br><tt> public property Name[] superInterfaces;</tt> +<br><tt> public property IMember[] members;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> // convenience methods</tt> +<br><tt> public FieldDeclaration[] getFields; // includes +constants; excludes initializers</tt> +<br><tt> public AbstractMethodDeclaration[] getMethods; +// includes constructors</tt> +<br><tt> public TypeDeclaration[] getTypes;</tt> +<br><tt> public ITypeBinding resolveBinding();</tt> +<p><tt>// MethodDeclaration:</tt> +<br><tt>// +{Modifier} (Type | <b><u>void</u></b>) Identifier <b><u>(</u></b></tt> +<br><tt>// +[FormalParameter { <b><u>,</u></b> FormalParameter}] <b><u>)</u></b> +{<b><u>[</u></b> <b><u>]</u></b>}</tt> +<br><tt>// +[<b><u>throws</u></b> QualifiedIdentifierList] ( MethodBody | <b><u>;</u></b> +)</tt> +<br><tt>// ConstructorDeclaration:</tt> +<br><tt>// +{Modifier} Identifier <b><u>(</u></b> [FormalParameter { <b><u>,</u></b> +FormalParameter}] <b><u>)</u></b></tt> +<br><tt>// +[<b><u>throws</u></b> QualifiedIdentifierList] MethodBody</tt> +<br><tt>// FormalParameter:</tt> +<br><tt>// +[<b><u>final</u></b>] Type Identifier {<b><u>[</u></b> <b><u>]</u></b>}</tt> +<br><tt>public abstract class AbstractMethodDeclaration extends ASTNode +implements IMember</tt> +<br><tt> protected AbstractMethodDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] selector;</tt> +<br><tt> public property FormalParameter[] parameters;</tt> +<br><tt> public property Name[] thrownExceptions;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> public property Block body; // optional</tt> +<br><tt> public IMethodBinding resolveBinding();</tt> +<p><tt>public class MethodDeclaration extends AbstractMethodDeclaration</tt> +<br><tt> public MethodDeclaration(AST ast);</tt> +<br><tt> public property Type returnType; // includes +void</tt> +<p><tt>public class ConstructorDeclaration extends AbstractMethodDeclaration</tt> +<br><tt> public ConstructorDeclaration(AST ast);</tt> +<p><tt>// FieldDeclaration:</tt> +<br><tt>// +{Modifier} Type Identifier {<b><u>[</u></b> <b><u>]</u></b>} [ <b><u>=</u></b> +Expression]</tt> +<br><tt>// +{ <b><u>,</u></b> Identifier {<b><u>[</u></b> <b><u>]</u></b>} [ <b><u>=</u></b> +Expression] }</tt> +<br><tt>public class FieldDeclaration extends ASTNode implements IMember</tt> +<br><tt> public AbstractMethodDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property char[][] javadocComment; // +optional</tt> +<br><tt> public property IExpression initializer; // +optional</tt> +<br><tt> public IFieldBinding resolveBinding();</tt> +<p><tt>// Initializer:</tt> +<br><tt>// +[<b><u>static</u></b>] Block</tt> +<br><tt>public final class Initializer extends ASTNode implements IMember</tt> +<br><tt> public Initializer(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property Block body;</tt> +<p><tt>// LocalVariableDeclaration:</tt> +<br><tt>// +[<b><u>final</u></b>] Type Identifier {<b><u>[]</u></b>} [ <b><u>=</u></b> +Expression ]</tt> +<br><tt>// +{ <b><u>,</u></b> Identifier {<b><u>[]</u></b>} [ <b><u>=</u></b> Expression] +} <b><u>;</u></b></tt> +<br><tt>public class LocalVariableDeclaration extends ASTNode implements +IStatement</tt> +<br><tt> public LocalVariableDeclaration(AST ast);</tt> +<br><tt> public property int modifiers;</tt> +<br><tt> public property char[] name;</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property IExpression initializer; // +optional</tt> +<br><tt> public ILocalVariableBinding resolveBinding();</tt> +<br> +<h4> +Types</h4> +The Type node (= TypeReference) represents a reference to a base type, +a named type, or an array thereof. +<p><tt>// Type:</tt> +<br><tt>// +(BasicType | TypeName ) {<b><u>[]</u></b>}</tt> +<br><tt>// BasicType:</tt> +<br><tt>// <b><u>byte</u></b></tt> +<br><tt>// <b><u>short</u></b></tt> +<br><tt>// <b><u>char</u></b></tt> +<br><tt>// <b><u>int</u></b></tt> +<br><tt>// <b><u>long</u></b></tt> +<br><tt>// <b><u>float</u></b></tt> +<br><tt>// <b><u>double</u></b></tt> +<br><tt>// <b><u>boolean</u></b></tt> +<br><tt>public class Type extends ASTNode implements IExpression</tt> +<br><tt> public Type (AST ast);</tt> +<br><tt> public property int baseType; // either</tt> +<br><tt> public property Name typeName; // or</tt> +<br><tt> public property int dimensions;</tt> +<br><tt> public IBinding resolvedType();</tt> +<h4> +Statements</h4> +There is a different AST node type for each different kind of statement. +Use a "marker" interface (<tt>IStatement</tt>) to bring all constructs +that can appear within a block (nonterminal <tt>BlockStatement</tt>, which +includes local variable and type declarations). +<p><tt>// Block:</tt> +<br><tt>// <b><u>{</u></b> +BlockStatement <b><u>}</u></b></tt> +<br><tt>// BlockStatement :</tt> +<br><tt>// LocalVariableDeclaration</tt> +<br><tt>// TypeDeclaration</tt> +<br><tt>// [Identifier +<b><u>:</u></b> +] Statement</tt> +<br><tt>//Statement:</tt> +<br><tt>// Block</tt> +<br><tt>// <b><u>if +(</u></b>Expression <b><u>)</u></b> Statement [<b><u>else</u></b> Statement]</tt> +<br><tt>// <b><u>for +(</u></b> ForInitOpt <b><u>;</u></b> [Expression] +<b><u>;</u></b> +ForUpdateOpt <b><u>)</u></b> Statement</tt> +<br><tt>// <b><u>while +(</u></b> Expression <b><u>)</u></b> Statement</tt> +<br><tt>// <b><u>do</u></b> +Statement <b><u>while</u></b> <b><u>(</u></b> Expression +<b><u>);</u></b></tt> +<br><tt>// <b><u>try</u></b> +Block [Catches] [ <b><u>finally</u></b> Block ]</tt> +<br><tt>// <b><u>switch +(</u></b> Expression <b><u>)</u></b> <b><u>{</u></b> SwitchBlockStatementGroups +<b><u>}</u></b></tt> +<br><tt>// <b><u>synchronized +(</u></b> Expression <b><u>)</u></b> Block</tt> +<br><tt>// <b><u>return</u></b> +[Expression] <b><u>;</u></b></tt> +<br><tt>// <b><u>throw</u></b> +Expression <b><u>;</u></b></tt> +<br><tt>// <b><u>break</u></b> +[Identifier] <b><u>;</u></b></tt> +<br><tt>// <b><u>continue</u></b> +[Identifier] <b><u>;</u></b></tt> +<br><tt>// <b><u>;</u></b></tt> +<br><tt>// ExpressionStatement</tt> +<br><tt>// Identifier +<b><u>:</u></b> +Statement</tt> +<br><tt>public interface IStatement // "marker" interface</tt> +<p><tt>public class Block extends ASTNode implements IStatement</tt> +<br><tt> public Block(AST ast);</tt> +<br><tt> public property IStatement[] statements;</tt> +<br><tt>public class IfStatement extends ASTNode implements IStatement</tt> +<br><tt> public IfStatement(AST ast);</tt> +<br><tt> public property IExpression test;</tt> +<br><tt> public property IStatement thenPart;</tt> +<br><tt> public property IStatement elsePart; // +optional</tt> +<br><tt>public class WhileStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ForStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class DoStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class TryStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class SwitchStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class SynchronizedStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ReturnStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ThrowStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class BreakStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class ContinueStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class NullStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class LabeledStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<br><tt>public class AssertStatement extends ASTNode implements IStatement</tt> +<br><tt> ...</tt> +<h4> +<font color="#000000">Expression Statements</font></h4> +<font color="#000000">Certain types of expressions can also appear as statements. +The ExpressionStatement node wraps an expression up as a statement. The +source range for the ExpressionStatement includes the trailing semicolon.</font><font color="#000000"></font> +<p><tt><font color="#000000">public class ExpressionStatement extends ASTNode +implements IStatement</font></tt> +<br><tt><font color="#000000"> public ExpressionStatement(AST +ast);</font></tt> +<br><tt><font color="#000000"> public property IExpression +expression;</font></tt> +<h4> +Expressions</h4> +There is a different AST node type for each different kind of expression. +Use a "marker" interface (<tt>IExpression</tt>) to bring all constructs +that can appear as expressions. +<p>(Many details TBD). +<p><tt>// Expression:</tt> +<br><tt>// +Identifier</tt> +<br><tt>// +ArrayAllocationExpression</tt> +<br><tt>// +StringLiteral</tt> +<br><tt>// +FloatingPointLiteral</tt> +<br><tt>// +BooleanLiteral</tt> +<br><tt>// +CharacterLiteral</tt> +<br><tt>// +StringLiteral</tt> +<br><tt>// +NullLiteral</tt> +<br><tt>// +( Type | <b><u>void</u></b> ) <b><u>.</u></b> class</tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>this</u></b></tt> +<br><tt>// <b><u>(</u></b> +Expression <b><u>)</u></b></tt> +<br><tt>// +[ Expression <b><u>.</u></b> ] <b><u>new</u></b> Type <b><u>(</u></b></tt> +<br><tt>// +[ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b> [ ClassBody +]</tt> +<br><tt>// +Expression <b><u>.</u></b> Identifier</tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>super .</u></b> Identifier</tt> +<br><tt>// +MethodName <b>(</b> [ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b></tt> +<br><tt>// +Expression <b><u>.</u></b> Identifier <b>(</b> [ Expression { <b><u>,</u></b> +Expression } ] <b><u>)</u></b></tt> +<br><tt>// +[ ClassName <b><u>.</u></b> ] <b><u>super .</u></b> Identifier <b>(</b></tt> +<br><tt>// +[ Expression { <b><u>,</u></b> Expression } ] <b><u>)</u></b></tt> +<br><tt>// +Expression <b><u>[</u></b> Expression <b><u>]</u></b></tt> +<br><tt>// +Expression InfixOperator Expression</tt> +<br><tt>// +Expression <b><u>instanceof</u></b> Type</tt> +<br><tt>// +Expression PostfixOperator</tt> +<br><tt>// +PrefixOperator Expression</tt> +<br><tt>// <b><u>(</u></b> +Type <b><u>)</u></b> Expression</tt> +<br><tt>// +Expression <b><u>?</u></b> Expression <b><u>:</u></b> Expression</tt> +<br><tt>// +Expression AssignmentOperator Expression</tt> +<br><tt>// ArrayInitializer</tt> +<br><tt>public interface IExpression // "marker" interface</tt> +<br><tt> public IBinding resolvedType(); // optional</tt> +<p><tt>// ArrayAllocationExpression:</tt> +<br><tt>// <b><u>new</u></b> +PrimitiveType <b><u>[</u></b> Expression <b><u>]</u></b> { <b><u>[</u></b> +Expression <b><u>]</u></b> } { <b><u>[</u></b> <b><u>]</u></b> }</tt> +<br><tt>// <b><u>new</u></b> +TypeName <b><u>[</u></b> Expression <b><u>]</u></b> { +<b><u>[</u></b> Expression +<b><u>]</u></b> +} { <b><u>[</u></b> <b><u>]</u></b> }</tt> +<br><tt>// <b><u>new</u></b> +PrimitiveType <b><u>[</u></b> <b><u>]</u></b> { <b><u>[]</u></b> } ArrayInitializer</tt> +<br><tt>// <b><u>new</u></b> +TypeName <b><u>[</u></b> <b><u>]</u></b> { <b><u>[]</u></b> } ArrayInitializer</tt> +<br><tt>public class ArrayAllocationExpression</tt> +<br><tt> extends ASTNode implements IExpression</tt> +<br><tt> public ArrayAllocationExpression(AST ast);</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property Expression[] dimensions; // +optional</tt> +<br><tt> public property Expression arrayInitializer; +// optional</tt> +<p><tt>public class StringLiteral extends ASTNode implements IExpression</tt> +<br><tt> public StringLiteral(AST ast);</tt> +<br><tt> public property String value;</tt> +<p><tt>public class CastExpression extends ASTNode implements IExpression</tt> +<br><tt> public CastExpression(AST ast);</tt> +<br><tt> public property Type type;</tt> +<br><tt> public property IExpression value;</tt> +<p><tt>public class InfixExpression extends ASTNode implements IExpression</tt> +<br><tt> public InfixExpression(AST ast);</tt> +<br><tt> public property int infixOperator;</tt> +<br><tt> public property IExpression leftOperand;</tt> +<br><tt> public property IExpression rightOperand;</tt> +<br><tt> public property IExpression[] extendedOperands; +// L op R op R2 op R3...</tt> +<h3> +Bindings</h3> +The "world of bindings" is an integrated picture of the structure of the +program as seen from the compiler's point of view. The bindings correspond +to named entities (packages, types, fields, methods, local variables). +<p>Clients navigate from AST nodes into the world of bindings to discover +things like: +<ul> +<li> +the entity an identifier resolves to</li> + +<li> +the resolved type of an expression node</li> + +<li> +the resolved binding of a declaration node</li> + +<li> +others?</li> +</ul> +Once in the world of bindings, the client can navigate the web of bindings: +<ul> +<li> +from array type to its component type, and vice versa</li> + +<li> +from field or variable to its declared type</li> + +<li> +from method to its parameter and return types</li> + +<li> +from type to its constructors and its declared method, field, and type +members</li> + +<li> +from constructor, method, or field to its declaring type</li> + +<li> +from nested type to its enclosing type</li> + +<li> +from type to declaring package</li> + +<li> +from type to its supertypes (but, significantly, <i>not</i> to its subtypes)</li> + +<li> +directly to the binding for any base type (int, float, char, etc.)</li> + +<li> +directly to the binding for a handful of well-known types (java.lang.Object, +etc.)</li> +</ul> +Some of the navigations that are not supported (quite intentionally): +<ul> +<li> +from package to its (known) types - very expensive</li> + +<li> +from package to one of its types by name - very expensive</li> + +<li> +from type to its (known) subtypes - very expensive</li> + +<li> +from type or method to the local types it encloses - binding for local +types are only of interest to those with the enclosing type's AST in their +hand</li> + +<li> +from method to the variables declared within it - binding for variables +are only of interest to those with the method's AST in their hand</li> +</ul> +There are no links from the world of bindings back to the world of ASTs. +<p>Other things dealt with in the world of bindings: +<ul> +<li> +synthetic entities stemming from default constructors, abstract method +copy-down from interfaces, and inner class emulation</li> + +<li> +missing bindings for entities that are required (mentioned by name) but +were not found</li> + +<li> +type hierachy circularities</li> + +<li> +internal inconsistencies</li> +</ul> +Other issues: +<ul> +<li> +Compile-time-computed values for constants (public static final fields +with compile-time computable values)</li> +</ul> + +<h4> +Existing Binding classes</h4> +To give an idea of the scope of the existing binding infrastructure, below +is a dump of the type hierarchy of the compiler's binding classes from +package <tt>rg.eclipse.jdt.internal.compiler.lookup</tt>. +<p><tt>public abstract class Binding</tt> +<br><tt> public abstract class TypeBinding</tt> +<br><tt> public final class ArrayBinding</tt> +<br><tt> public final class BaseTypeBinding</tt> +<br><tt> public abstract class +ReferenceBinding</tt> +<br><tt> +public class SourceTypeBinding</tt> +<br><tt> +public class NestedTypeBinding</tt> +<br><tt> +public final class LocalTypeBinding</tt> +<br><tt> +public final class MemberTypeBinding</tt> +<br><tt> +public class ProblemReferenceBinding</tt> +<br><tt> +public class UnresolvedReferenceBinding</tt> +<br><tt> public class PackageBinding</tt> +<br><tt> public class ProblemPackageBinding</tt> +<br><tt> public abstract class VariableBinding</tt> +<br><tt> public class LocalVariableBinding</tt> +<br><tt> +public class SyntheticArgumentBinding</tt> +<br><tt> public class FieldBinding</tt> +<br><tt> +public class SyntheticFieldBinding</tt> +<br><tt> +public class ProblemFieldBinding</tt> +<br><tt> public class MethodBinding</tt> +<br><tt> public class ProblemMethodBinding</tt> +<br><tt> public class SyntheticAccessMethodBinding</tt> +<br><tt> public class ImportBinding</tt> +<br><tt> public class ProblemBinding</tt> +<h4> +Binding API</h4> +The existing binding classes are not immediately suitable for exposing +as a binding API. +<p>However, the Java builder does have an API for the built "image", in +package <tt>org.eclipse.jdt.internal.core.builder</tt>. (This API is a +hold-over from Leapfrog era, and is not exposed in the Eclipse code base). +This API was designed to expose the same kind of integrated picture of +the structure of the program as seen from the compiler's point of view. +This API has a detailed specification that does not expose implementation +details, so the proposal is to use it as the basis for the new binding +API. +<p>Re-purposing this API would entail: +<ul> +<li> +introducing entities for local variables</li> + +<li> +removing protocol for navigations that are not supported (e.g., from package +to its known types)</li> + +<li> +removing unneeded protocol; including states, non-state-specific handles, +deltas, report cards, dependency graph, package references</li> +</ul> +Below is a dump of the relevant interfaces from package <tt>org.eclipse.jdt.internal.core.builder</tt>. +Unnecessary protocol has been omitted. (Note that NotPresentException is +an unchecked exception, and would not be required.) +<p><tt>public interface IHandle</tt> +<br><tt> int K_JAVA_IMAGE = 1;</tt> +<br><tt> int K_JAVA_PACKAGE = 2;</tt> +<br><tt> int K_JAVA_TYPE = 3;</tt> +<br><tt> int K_JAVA_FIELD = 4;</tt> +<br><tt> int K_JAVA_METHOD = 5;</tt> +<br><tt> int K_JAVA_CONSTRUCTOR = 6;</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> int hashCode();</tt> +<br><tt> boolean isFictional() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<br><tt> int kind();</tt> +<p><tt>public interface IMember extends IHandle</tt> +<br><tt> IType getDeclaringClass();</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isBinary() throws NotPresentException;</tt> +<br><tt> boolean isDeprecated() throws NotPresentException;</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<p><tt>public interface IPackage extends IHandle</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getClassHandle(String name);</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isUnnamed();</tt> +<p><tt>public interface IType extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getArrayHandle();</tt> +<br><tt> IType getComponentType();</tt> +<br><tt> IConstructor getConstructorHandle(IType[] parameterTypes);</tt> +<br><tt> IType[] getDeclaredClasses() throws NotPresentException;</tt> +<br><tt> IConstructor[] getDeclaredConstructors() throws +NotPresentException;</tt> +<br><tt> IField[] getDeclaredFields() throws NotPresentException;</tt> +<br><tt> IMethod[] getDeclaredMethods() throws NotPresentException;</tt> +<br><tt> int getDeclaredModifiers() throws NotPresentException;</tt> +<br><tt> String getDeclaredName() throws NotPresentException;</tt> +<br><tt> IType getDeclaringClass() throws NotPresentException;</tt> +<br><tt> IField getFieldHandle(String name);</tt> +<br><tt> IType[] getInterfaces() throws NotPresentException;</tt> +<br><tt> IMethod getMethodHandle(String name, IType[] +parameterTypes);</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> IPackage getPackage();</tt> +<br><tt> String getSimpleName();</tt> +<br><tt> IType getSuperclass() throws NotPresentException;</tt> +<br><tt> boolean isAnonymous() throws NotPresentException;</tt> +<br><tt> boolean isArray();</tt> +<br><tt> boolean isBinary() throws NotPresentException;</tt> +<br><tt> boolean isClass() throws NotPresentException;</tt> +<br><tt> boolean isDeprecated() throws NotPresentException;</tt> +<br><tt> boolean isInnerClass() throws NotPresentException;</tt> +<br><tt> boolean isInterface() throws NotPresentException;</tt> +<br><tt> boolean isLocal() throws NotPresentException;</tt> +<br><tt> boolean isPackageMember() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<br><tt> boolean isPrimitive();</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<br><tt> boolean isTopLevel() throws NotPresentException;</tt> +<p><tt>public interface IMethod extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType[] getExceptionTypes() throws NotPresentException;</tt> +<br><tt> IType[] getParameterTypes();</tt> +<br><tt> IType getReturnType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p><tt>public interface IConstructor extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType[] getExceptionTypes() throws NotPresentException;</tt> +<br><tt> IType[] getParameterTypes();</tt> +<br><tt> boolean isPresent();</tt> +<p><tt>public interface IField extends IMember</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p>In this vein, the interface for local variables would look something +like: +<p><tt>public interface IVariable extends IHandle</tt> +<br><tt> boolean equals(Object obj);</tt> +<br><tt> IType getDeclaringClass();</tt> +<br><tt> int getModifiers() throws NotPresentException;</tt> +<br><tt> String getName();</tt> +<br><tt> boolean isSynthetic() throws NotPresentException;</tt> +<br><tt> IType getType() throws NotPresentException;</tt> +<br><tt> boolean isPresent();</tt> +<p>Also will need to add: +<ul> +<li> +Pseudo-bindings for base types: boolean, int, float, etc.</li> + +<li> +Access to well-known java.lang bindings: Object, String, Throwable, Exception, +RuntimeException, Error, Class.</li> +</ul> + +<h3> +Document History</h3> +18:30 Thursday September 27, 2001 - incorporated first round comments from +PM and DB. +<br><font color="#000000">10:45 Monday October 1, 2001 - incorporated comments +from DB.</font> +<br><font color="#000000">10:45 Tuesday October 2, 2001 - clarify handing +of ExpressionStatement.</font> +<br><font color="#3366FF">14:00 Friday October 26, 2001 - add subtree structural +equality.</font> +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/output folder/output-folder.html b/org.eclipse.jdt.core/notes/r2.0/output folder/output-folder.html new file mode 100644 index 000000000..3cc29d20c --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/output folder/output-folder.html @@ -0,0 +1,1228 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <meta name="Author" content="Build"> + <title>JDT - Java Output Folder</title> +</head> +<body> + +<h2> +Java Output Folder</h2> +Last revised 17:30 Tuesday October 30, 2001 <font color="#3333FF">(recent +changes in blue;</font><font color="#CC0000"> latest in red</font><font color="#3333FF">)</font> +<p>Work item: "Support for dealing with class files generated by external +Java compilers like javac and jikes from an Ant script." +<p>Here's the crux of one problem (from WSAD, via John W.): +<p>In some environments the client has limited flexibility in how they +structure their Java projects. Sources must go here; resource files here; +mixed resource and class files here; etc. +<ul> +<li> +There are situations where the client needs to place additional class files +and resource files in the output directory.</li> + +<li> +There are situations where the client needs to generate class files into +an existing folder filled with their class files and resource files (e.g., +an exploded WAR file).</li> +</ul> +When clients attempts either, they discover that (a) the Java model and +builder ignore any class files in the output folder, and (b) from time +to time these files in the output folder get deleted without warning. +<p>The Java builder was designed under the assumption that it "owns" the +output folder. The work item, therefore, is to change the Java builder +to give clients and users more flexiblility as to where they place their +source, resource, library class, and generated class files. +<h3> +Current Functionality</h3> +Eclipse 1.0 Java builder has the following characteristics (and inconsistencies): +<ul> +<li> +The class files generated by the Java builder go in a single output folder. +Java source files go in one or more source folders. Any other kind of files +can be included in the source folder too; this includes pre-compiled class +files. All these other files will be automatically mirrored to the binary +output directory when a build is done. The mirror is maintained as the +source folder changes; damage made directly to the output folder gets repaired +no later than the next full build.</li> + +<li> +The output folder belongs to the Java builder. It summarily deletes files +from the output folder that it does not think belong there. It is not possible +to get away with adding files directly to the output folder. So you cannot +even mate the extra resource files with the class files manually.</li> + +<li> +When the project source and output folder coincide (perhaps at the project +itself), the builder behaves differently. It grants that source files belong +there, so it never deletes them. It also grants that non-class files belong +there, so it never deletes them either. But it assumes that all class files +are generated, and so it summarily deletes them on every full build, including +class files that were explicitly put there. This is different from the +way things work out when the output folder and the source folder do not +coincide. And it is not what you want if you need to mate other class files +with the generated ones.</li> +</ul> + +<h3> +<font color="#3366FF">WSAD usecase - for the record</font></h3> +<font color="#3366FF">The </font><font color="#CC0000">(proposed)</font><font color="#3366FF"> +WSAD scenario is that they have a src/ folder for source code, a classes/ +folder for pre-existing class files (extracted from a WAR file), and a +bin/ folder for generated classes.</font> +<ul> +<li> +<font color="#3366FF">The typical case is where the source and output folders +are distinct. In this case, the classes/ folder may or may contain class +files.</font></li> + +<li> +<font color="#3366FF">The source code must be compiled against the classes +in classes/ folder.</font></li> + +<li> +<font color="#3366FF">The source and output folders may coincide. The classes/ +folder is always separate from either.</font></li> + +<li> +<font color="#3366FF">In order to be executable, all class files must end +up in the output folder.</font></li> + +<li> +<font color="#3366FF">The client would like a way to delete class files +from the classes/ folder for which there is corresponding source.</font></li> + +<li> +<font color="#3366FF">The client would like to keep resource files in the +output folder on an ongoing basis.</font></li> +</ul> + +<h3> +<font color="#3366FF">Proposal</font></h3> +<font color="#3366FF">The Java builder compiles source files found in the +source folders specified on the build classpath and generates class files +into the output folder. The Java builder also copies "resource" files from +source folders to the output folder (provided that source and output do +not coincide). Once in the output folder, the resource files are available +at runtime because the output folder is always present on the runtime class +path. The proposal is to extend this mechanism.</font><font color="#3366FF"></font> +<p><font color="#3366FF">The following proposal involves:</font> +<ul> +<li> +<font color="#3366FF">Clarifying ownership of files in the output folder.</font></li> + +<li> +<font color="#FF0000">Clarifying semantics of resource file copying from +source folders to output folder.</font></li> + +<li> +<font color="#FF0000">Making resource file copying from source folders +to output folder optional, rather than mandatory.</font></li> + +<li> +<font color="#FF0000">Providing class file copying from library folders +to output folder, also on an optional basis.</font></li> + +<li> +<font color="#FF0000">Providing and promoting useful alternatives to file +copying.</font></li> + +<li> +<font color="#FF0000">Prohibiting cases where expendable copies would end +up mixed with important user data.</font></li> +</ul> + +<h4> +<font color="#3366FF">Output folder ownership</font></h4> +<font color="#3366FF">When the output folder does not coincide with a source +folder, the Java builder owns the output folder and everything in it. The +output folder is taken to contain only files that are "expendable" - either +generated class files or copies of files that live in a source or library +folder.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Users or clients that add, remove, or replace +files in the output folder can expect unpredicatable results. If the user +or client does tamper with files in the output folder, the Java builder +does not attempt to repair the damage. It is the responsibility of the +user or client to clean up their mess (by manually requesting a full build).</font><font color="#3366FF"></font> +<p><font color="#3366FF">When the output folder coincides with a source +folder, the Java builder only owns the class files in the output folder. +Only the class files in the output folder are considered expendable. Users +or clients that add, remove, or replace class files in the output folder +can expect unpredictable results.</font><font color="#3366FF"></font> +<p><font color="#3366FF">(N.B. This is a restatement of the current behavior. +[Verify that damage to output folder is not triggering builds.])</font> +<h4> +<font color="#FF0000">Output folder resource file consolidation</font></h4> +<font color="#FF0000">The Java builder provides resource file consolidation, +for resource files stored in source folders.</font><font color="#FF0000"></font> +<p><font color="#FF0000">When the output folder does not coincide with +a source folder, the Java builder can also be used to consolidate resources +files needed at runtime in the output folder. In some cases, this consolidation +may be preferred over the alternative of including additional runtime classpath +entries for source folders containing resources files.</font><font color="#FF0000"></font> +<p><font color="#FF0000">By flagging a source folder as copied, all non-source, +non-class files become eligible to be copied to the output folder. When +there are multiple entries on the build classpath specifying copying, eligible +files for earlier classpath entries take precedence over ones for later +entries.</font><font color="#FF0000"></font> +<p><font color="#FF0000">When the output folder coincides with a source +folder, the Java builder cannot perform any resource file consolidation +(resource files in the output folder belong to the user, not to the Java +builder). It is considered an error to specify copying from other source +folders.</font><font color="#FF0000"></font> +<p><font color="#FF0000">(N.B. This is different from current behavior +in a couple of regards:</font> +<ul> +<li> +<font color="#FF0000">Resource file copying for source folders is currently +mandatory. It would become optional.</font></li> + +<li> +<font color="#FF0000">Class files are currently copied from source folders. +This would stop.</font></li> +</ul> +<font color="#FF0000">)</font> +<h4> +<font color="#FF0000">Output folder class file consolidation</font></h4> +<font color="#FF0000">The Java builder also provides class file consolidation, +for class files stored in library folders.</font><font color="#FF0000"></font> +<p><font color="#FF0000">The Java builder can also be used to consolidate +class in the output folder, regardless of whether the output folder coincides +with a source folder. In some cases, this consolidation may be preferred +over the alternative of including additional runtime classpath entries +for library folders. Note, however, that this works only when the library +folder contains no important resource files needed at runtime (resource +files are not copied from library folders, because resource files in the +output folder belong to the user rather than to the Java builder).</font><font color="#FF0000"></font> +<p><font color="#FF0000">By flagging a library folder as copied, all class +files become eligible to be copied to the output folder. +Class files generated +in the output folder always take precedence over class files copied from +library folders.</font><font color="#FF0000"></font> +<p><font color="#FF0000">(N.B. This new behavior. Files are not copied +from library folders by the current Java builder.)</font> +<h4> +<font color="#3366FF">Semantics</font></h4> + +<ul> +<li> +<font color="#3366FF">Add a "copy" flag to source and library (and variable) +classpath entries.</font></li> + +<ul> +<li> +<font color="#3366FF">For a source folder, copy flag means that </font><font color="#FF0000">resource +(i.e, non-source, non-class) files </font><font color="#3366FF">in the +source folder are copied to the output folder.</font></li> + +<ul> +<li> +<font color="#3366FF">Primary use is to consolidate </font><font color="#FF0000">resource +files</font><font color="#3366FF"> in output folder so that source folder +does not need to be included on runtime classpath.</font></li> +</ul> + +<li> +<font color="#3366FF">For a library folder (but not a library JAR), copy +flag means that </font><font color="#FF0000">all class files</font><font color="#3366FF"> +in the library folder are copied to the output folder.</font></li> + +<ul> +<li> +<font color="#3366FF">Primary use is to consolidate </font><font color="#FF0000">class +files</font><font color="#3366FF"> in output folder so that library folder +does not need to be included on runtime classpath.</font></li> + +<li> +<font color="#FF0000">N.B. Resource files in the library folder are not +copied.</font></li> +</ul> + +<li> +<font color="#3366FF">The copy flag should be off by default for both types +of folder; i.e., no file copying.</font></li> + +<li> +<font color="#3366FF">Source folder copy flag on describes current behavior.</font></li> + +<li> +<font color="#3366FF">For backward compatibility, existing projects should +have copying on for source folders that might contain resource files.</font></li> + +<li> +<font color="#FF0000">Library folder copy flag off describes current behavior.</font></li> + +<li> +<font color="#FF0000">Error if any source folder copying specified when +a source folder and output folder coincide.</font></li> + +<li> +<font color="#3366FF">API on JavaCore for creating classpath entries with +copy bit set.</font></li> + +<li> +<font color="#3366FF">API on IClasspathEntry for reading copy bit.</font></li> +</ul> + +<li> +<font color="#3366FF">Generated class files in the output folder take precedence +over class files copied from library folders.</font></li> + +<li> +<font color="#3366FF">Files copied from earlier classpath entries take +precedence over ones for later entries.</font></li> + +<li> +<font color="#FF0000">UI provides the user with control over resource file +consolidation.</font></li> + +<ul> +<li> +<font color="#FF0000">For source folder: "Copy resource (non-source, non-class) +files to output folder"</font></li> +</ul> + +<li> +<font color="#FF0000">UI does not provide the user with control over class +file consolidation.</font></li> + +<ul> +<li> +<font color="#FF0000">This feature would be enabled programmatically by +clients that need it.</font></li> +</ul> + +<li> +<font color="#3366FF">Consolidation functionality is built in to Java builder.</font></li> + +<li> +<font color="#3366FF">Pro: Puts us in a position where resource copying +is no longer mandatory.</font></li> + +<li> +<font color="#3366FF">Pro: Gives us an opportunity to improve resource +copying implementation.</font></li> +</ul> +<font color="#FF0000">Summary: Resource files (non-source, non-class) may +be copied from either source folders. Class files may be copied from library +folders, but never override generated files. Source files never get copied.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Output folder invariant:</font> +<ul> +<li> +<font color="#3366FF">For a class file X.class in the output folder O</font></li> + +<ul> +<li> +<font color="#3366FF">if exists a Y.java in a source folder that compiles +to X.class then the X.class in O is the one that results from compiling +Y.java</font></li> + +<li> +<font color="#3366FF">if exists an X.class </font><font color="#FF0000">in +some library folder with copying on </font><font color="#3366FF">then a +copy of the X.class from the earliest such </font><font color="#FF0000">library +folder</font></li> + +<li> +<font color="#3366FF">otherwise no X.class should be present</font></li> +</ul> + +<li> +<font color="#3366FF">For a </font><font color="#FF0000">resource (i.e., +non-source, non-class) </font><font color="#3366FF">file X.other in the +output folder O</font></li> + +<ul> +<li> +<font color="#3366FF">if O is also a source folder then any X.other that +lives there</font></li> + +<li> +<font color="#3366FF">if exists an X.other </font><font color="#FF0000">in +some source folder with copying on</font><font color="#3366FF"> then a +copy of the X.other from the earliest such </font><font color="#FF0000">source +folder</font></li> + +<li> +<font color="#3366FF">otherwise no X.other should be present</font></li> +</ul> + +<li> +<font color="#3366FF">For a source file X.java in the output folder O</font></li> + +<ul> +<li> +<font color="#3366FF">if O is also a source folder then any X.java that +lives there</font></li> + +<li> +<font color="#FF0000">otherwise no X.java should be present</font></li> +</ul> +</ul> +<font color="#3366FF">A full builds must achieve the output folder invariant +from <i>arbitrary</i> initial conditions. When output and source folders +do not coincide, a full build should scrub all existing files from the +output folder, regardless of how they got there. When output and source +folders do coincide, a full build should scrub all existing class files +from the output folder, but leave all other files alone.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Assuming that a user or client is only adding, +removing, or changing files in source or library folders, but not tampering +with any of the files in the output folder that the Java builder owns, +then an incremental build should re-achieve the output folder invariant.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Algorithm:</font><font color="#3366FF"></font> +<p><font color="#FF0000">Full build:</font> +<br><font color="#FF0000"> Scrub all class files from +the output folder.</font> +<br><font color="#FF0000"> if performing resource consolidation +(requires output folder != source folder)</font> +<br><font color="#FF0000"> Scrub +all resource files from the output folder.</font> +<br><font color="#FF0000"> Compile all source files into +class files in the output folder.</font> +<br><font color="#FF0000"> Infill/copy eligible class +files from library folders into the output folder (no overwriting).</font> +<br><font color="#FF0000"> if performing resource consolidation +(requires output folder != source folder)</font> +<br><font color="#FF0000"> Infill/copy +eligible resource files from source folders into the output folder.</font><font color="#FF0000"></font> +<p><font color="#FF0000">Incremental build:</font> +<br><font color="#FF0000"> (phase 1) process changes +to library folders:</font> +<br><font color="#FF0000"> for +add or remove or change file p/x.class in one of the library folders</font> +<br><font color="#FF0000"> +if p/x.class in the output folder was not generated by compiler then</font> +<br><font color="#FF0000"> +scrub p/x.class from the output folder</font> +<br><font color="#FF0000"> +remember to compile source files that depend on p/x</font> +<br><font color="#FF0000"> +remember to infill p/x.class</font> +<br><font color="#FF0000"> (phase 2) process changes +to source folders:</font> +<br><font color="#FF0000"> for +add p/y.java in one of the source folders</font> +<br><font color="#FF0000"> +remember to compile source file at path p/y.java</font> +<br><font color="#FF0000"> for +remove or change p/y.java in one of the source folders</font> +<br><font color="#FF0000"> +scrub any class file p/x.class from the output folder that some p/y.java +compiled into last time</font> +<br><font color="#FF0000"> +remember to infill p/x.class</font> +<br><font color="#FF0000"> +remember to compile source file at path p/y.java</font> +<br><font color="#FF0000"> for +add or remove or change resource p/x.other in one of the source folders</font> +<br><font color="#FF0000"> +if performing resource consolidation (requires output folder != source +folder)</font> +<br><font color="#FF0000"> +scrub p/x.other from the output folder</font> +<br><font color="#FF0000"> +remember to infill p/x.other</font> +<br><font color="#FF0000"> (phase 3) recompile:</font> +<br><font color="#FF0000"> compile +all remembered source files into the output folder (and any dependent source +files)</font> +<br><font color="#FF0000"> (phase 4) infill:</font> +<br><font color="#FF0000"> for +each hole p/x.class to infill</font> +<br><font color="#FF0000"> +copy first-found file p/x.class in a library folder to p/x.class in the +output folder (no overwriting)</font> +<br><font color="#FF0000"> if +performing resource consolidation (requires output folder != source folder)</font> +<br><font color="#FF0000"> +for each hole p/x.other to infill</font> +<br><font color="#FF0000"> +copy first-found file p/x.other in a source folder to p/x.other in the +output folder</font> +<h4> +<font color="#3366FF">How These Changes Solve WSAD's Problem</font></h4> +<font color="#3366FF">WSAD would include their classes/ folder on the build +classpath as a library folder with class file copying turned on. Doing +so means that the pre-compiled class files in the library are available +to build against, and will be used whenever there is no corresponding source +code in a source folder. </font><font color="#FF0000">By turning class +file copying on for that library folder (programatically - there is no +UI), the class files in the library folder are automatically consolidated +with the generated class files.</font><font color="#3366FF"></font> +<p><font color="#FF0000">Resource files can always be kept in the same +folder as the source files. When the source and output folders do not coincide, +the source folder on the classpath could have copying turned on to ensure +that resource files were copied to the output folder. When the source and +output folders do coincide, further resource file consolidation is not +required (or possible) and the source folder on the classpath would have +copying turned off. The resource files that normally live in the source +folder would automatically be included in the output folder (without copying).</font> +<h3> +<font color="#3366FF">Minimizing Class Files</font></h3> +<font color="#3366FF">(This problem is not really an output folder issue.)</font><font color="#3366FF"></font> +<p><font color="#3366FF">WSAD has a special problem. They have class files +in a classes/ folder which they obtain from unzipping a WAR file. They +have a folder of source code; some of the source code may be brand new; +some of the source code may correspond to class files in the classes/ folder. +They need to prune from the classes/ directory those class files for which +corresponding source is available. This allows them to save only those +class files which they actually need.</font><font color="#3366FF"></font> +<p><font color="#3366FF">The heart of this operation is identifying the +class files which could have come from a given source file. A source file +can be lightly parsed to obtain fully qualified names for all top-level +types declared within; e.g., a source file com/example/acme/app/Foo.java +might contain types named com.example.acme.app.Foo and com.example.acme.app.FooHelper. +Such type names map directly to corresponding class file name patterns; +e.g., com.example.acme.app.FooHelper would compile to com/example/acme/app/FooHelper.class +and possibly other class files matching com/example/acme/app/FooHelper$*.class.</font><font color="#3366FF"></font> +<p><font color="#3366FF">This basic operation can be implemented with the +existing JDOM API (or the proposed AST API): simply open the compilation +unit and read off the names from the package declaration and and top-level +type declarations.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Given this basic operation, it is straightforward +to walk any set of source files and use it to prune a given set of class +files. Source files in some folder in the workspace can be monitored with +a resource change listener. It is trivial to delete corresponding class +files incrementally as new source files are added.</font><font color="#3366FF"></font> +<p><font color="#3366FF">Conclusion: New API is not required.</font> +<h2> +Notes Leftover from Earlier Proposals</h2> +The following notes are retained as background material. They include some +of the other approaches we tried, and problems we ran into. +<p>The Java model has 2 primitive kinds of inputs: Java source files, and +Java library class files. The Java builder produces one primary output: +generated Java class files. Each Java project has a build classpath listing +what kinds of inputs it has and where they can be found, and a designated +output folder where generated class files are to be placed. The runtime +classpath is computed from the build classpath by substituting the output +folder in place of the source folders. +<p>Java "resource" files, defined to be files other than Java sources and +class files, are of no particular interest to the Java model for compiling +purposes. However, these resource files are very important to the user, +and to the program when it runs. Resource files are routinely co-located +with library class files. But it is also convenient for the user if resource +files can be either co-located with source code, or segregated in a separate +folder. +<p>Ideally, the Java model should not introduce constraints on where inputs +and outputs are located. This would give clients and users maximum flexibility +with where they locate their files. +<p>The proposal here has 4 separate parts. Taken in conjunction they remove +the current constraints that make it difficult for some clients to place +their files where they need to be. +<ul> +<li> +<a href="#Java Builder Attitude Adjustment">Change the Java builder's attitude +towards the output folder.</a></li> + +<li> +<a href="#Allowing Folders to Play Multiple Roles">Allow folders to play +multiple roles on the same build classpath.</a></li> + +<li> +<a href="#Completely eliminate resource file copying behavior">Completely +eliminate the resource copying behavior of current Java builder.</a></li> + +<li> +<a href="#Minimize the opportunity for obsolete class files to have bad effects">Minimize +the opportunity for obsolete class files to have bad effects.</a></li> +</ul> +[Revised proposal: Rather than write a completely new proposal, I've added +a note like to the end of each subsequent section describing a revised +proposal.] +<h3> +<a NAME="Java Builder Attitude Adjustment"></a>Java Builder Attitude Adjustment</h3> +To appreciate the difficulties inherent with the Java builder sharing its +output folder with other folk, consider the following workspace containing +a Java project. Assume that this project has not been built in quite a +while, and the user has been manually inserting and deleting class files +in the project's output folder. +<p>Java project p1/ +<br> src/com/example/ (source folder on build classpath) +<br> Bar.java +<br> Foo.java +<br> Quux.java +<br> bin/com/example/ (output folder) +<br> Bar.class {SourceFile="Bar.java"} +<br> Foo.class {SourceFile="Foo.java"} +<br> Foo$1.class {SourceFile="Foo.java"} +<br> Internal.class {SourceFile="Foo.java"} +<br> Main.class {SourceFile="Main.java"} +<p>From this arrangement of files (and looking at the SourceFile attributed +embedded in class files), we can infer that: +<ul> +<li> +Bar.class came from compiling a source file named "Bar.java".</li> + +<li> +Foo.class, Foo$1.class, and Internal.class all came from compiling a "Foo.java". +(A single source file will compile to multiple separate class files if +it has nested classes or secondary non-public classes.)</li> + +<li> +There are no existing class files corresponding to "Quux.java".</li> + +<li> +Main.class came from compiling a source file named "Main.java", which the +workspace does't have.</li> +</ul> + +<h4> +Java Builder - Obsolete Class File Deletion</h4> +If the user was to request a full build of this project, how would the +Java builder proceed? Before it compile any source files, it begins by +deleting existing class files that correspond to source files it is about +to recompile. Why? Because obsolete class files left around (a) waste storage +and (b) would be available at runtime where they could cause the program +to run incorrectly. +<p>In this situation, the Java builder deletes the class files corresponding +to Bar.java (i.e., Bar.class), to Foo.java (i.e., Foo.class, Foo$1.class, +and Internal.class), and to Quux.java (none, in this case). The remaining +class files (Main.class) must be retained because it is irreplaceable. +<p>The Java builder takes responsibility for deleting obsolete class files +in order to support automated incremental recompilation of entire folders +of source files. Note that standard Java compilers like javac never ever +delete class files; they simply write (or overwrite) class files to the +output folder for the source files that they are given to compile. Standard +Java compilers do not support incremental recompilation: the user is responsible +for deleting any obsolete class files that they bring about. +<p>If the Java builder is free to assume that all class files in the output +folder are ones that correspond to source files, then it can simply delete +all class files in the output folder at the start of a full build. If it +cannot assume this, the builder is forced to look at class files in the +output folder to determine whether it has source code for them. This is +clearly more expensive that not having to do so. By declaring that it "owns" +the output folder, the current builder is able to makes this simplifying +assumption. Allowing users and clients to place additional class files +in the output folder requires throwing out this assumption. +<p>If the user or client is free to manipulate class files in the output +folder without the Java builder's involvement, then the builder cannot +perform full or incremental builds without looking at and deleting the +obsolete class files from the output folder corresponding to source files +being compiling. +<p>Under the proposed change, the Java builder would need to look at the +class files in the output folder to determine whether it should delete +them. <i>The only files in the output folder that the Java builder would +be entitled to overwrite or delete are class files which the Java builder +would reasonably generate, or did generate, while compiling that project.</i> +<ul> +<li> +The Java builder is entitled to overwrite class files in the output folder +that correspond to current source files. Any class file at such a path +is the Java builder's. Even when the actual contents of the class file +came from elsewhere, the builder is always entitled to delete them or overwrite +them with its contents.</li> + +<li> +The only files in the output folder that the Java model/builder would be +entitled to delete outright are ones that had been generated by the Java +builder when compiling this project but which no longer correspond to a +current source file. This permits the Java builder to clean up obsolete +class files that it knows it generated, or would have generated, on an +earlier build. It does not have the right to delete other class files, +even ones which do not correspond to a current source file. (Otherwise +the Java builder could justify deleting any class file that it does not +have corresponding source for.) Even for a full build, the Java builder +is not allowed to scrub all class files from the output folder (unless +it happens to know for a fact that the only class files in there ones it +generated).</li> + +<li> +The source file is an optional attribute of class files that is not generated +when debug info is suppressed (javac -g:none). Class files in the output +folder without the SourceFile attribute should be treated as if there was +no corresponding source file. This means they never get deleted outright, +although they may still be overwritten as required.</li> + +<li> +Note: changing a project to give it a different output folder should absolve +the Java builder of responsibility for any generated class files in the +former output folder. This means the Java builder does not need to perform +cleanup or track anything outside the current output folder.</li> + +<li> +Note: adding a source entry to the build classpath causes a bunch of new +source files to enter the frame. Some of the existing class files in the +output folder might now map to these source files, possibly in preference +to where they mapped before. Removing a source entry from the build classpath +causes a bunch of source files to leave the picture. Some of the existing +class files in the output folder might now map to other source files, or +not map to any soure file at all. [We need to decide whether obsolete class +files need to be tracked across the additional and/or removal of source +entries from the build classpath.]</li> +</ul> +This change is not a breaking API change. The old spec said that the Java +model/builder owned the output folder, but didn't further specify what +all that entailed. The new spec will modify this position to allow clients +to store files in the output folder; it will promise that these files are +perfectly safe unless they are in the Java builder's direct line of fire. +<h4> +Java Model - Obsolete Class File Deletion</h4> +There is another facet of the obsolete class file problem that the Java +builder is not in a position to help with. +<p>If the source file Foo.java were to be deleted, its three class files +become obsolete and need to be deleted <i>immediately</i>. Why immediately? +Consider what happens if the class files are not deleted immediately. If +the user requests a full build, the Java builder is presented with the +following workspace: +<p>Java project p1/ +<br> src/com/example/ (source folder on build classpath) +<br> Bar.java +<br> Quux.java +<br> bin/com/example/ (output folder) +<br> Bar.class {SourceFile="Bar.java"} +<br> Foo.class {SourceFile="Foo.java"} +<br> Foo$1.class {SourceFile="Foo.java"} +<br> Internal.class {SourceFile="Foo.java"} +<br> Main.class {SourceFile="Main.java"} +<p>Since a full build is requested, the Java builder is not passed a resource +delta tree for the project. This means that the Java builder has no way +of knowing that Foo.java was just deleted. The Java builder has no choice +but to retain the three class files Foo.class, Foo$1.class, and Internal.class, +just as it retains Main.class. This too is a consequence of allowing the +Java builder to share the output folder with the user's class files. +<p>If the obsolete class files are not deleted in response to the deletion +of a source file, these class files will linger around. The Java builder +will be unable to get rid of them. +<p>The proposal is to have the Java model monitor source file deletions +on an ongoing basis and identify and delete any corresponding obsolete +class files in the output folder. This clean up activity must handle the +case of source files that disappear while the Java Core plug-in is not +activated (this entails registering a Core save participant). +<p>Since deleting (including renaming and moving) a source file is a relatively +uncommon thing for a developer to do, the implementation should bet it +does not have to do this very often. When a source file in deleted, its +package name gives us exactly which subfolder of the output folder might +contain corresponding class files that might now be obsolete. In the worst +case, the implementation would need to access all class files in that subfolder +to determine whether any of them have become obsolete. In cases where there +is more than one source folder on the builder classpath, and there is therefore +the possibility of one source file hiding another by the same name, it +is necessary to consult the build classpath to see whether the deleted +source file was exposed or buried. +<h4> +Implementation Tricks</h4> +Some observations and implementation tricks that should help reduce the +space and time impact of doing this. +<ul> +<li> +When one or more source files are deleted from a single source folder, +their position under the source package fragment root gives us the package +name. This package name tells us exactly which subfolder of the output +folder might contain corresponding class files that might now be obsolete. +In the worst case, the implementation would need to access all class files +in that subfolder to determine whether any of them have become obsolete. +This shows that you only need information about a small portion of the +output folder in order to process one or more deletions within a single +source folder.</li> + +<li> +A source file named Foo.java typically compiles to a single class file +named Foo.class. There might be more class files (for nested classes and/or +secondary non-public types); and there might be less (when the source file +contains only non-public types with names other than "Foo"). When recording +the extracted source file name information, only the exceptional cases +need to be represented explicitly. For example, only Foo$1.class (derived +from Foo.java) and Internal.class (derived from Foo.java) are unusual; +Bar.class, Foo.class, and Main.class are all derived from source files +with the expected name. This means you can store the information extracted +from class files much more compactly that a simple class file name to SourceFile +string mapping.</li> + +<li> +There is often only one source folder on the builder classpath. In this +case, all source files in the source folder get compiled; there is no possibility +of one source file "hiding" another by the same name. This observation +can be used to avoid checking for source file hiding.</li> +</ul> + +<h3> +When all else fails</h3> +A special concern is that the user must be able to recover from crashes +or other problems that result in obsolete class files being left behind +in the output folder. It can be very bad when this kind of thing happens +(and it does happen, despite our best efforts), and can undercut the user's +confidence in the Java compiler and IDE. In a large output folder that +contains important user files, the user can't just delete the output folder +and do a full build. The user has no easy way to distinguish class files +with corresponding source from ones without. A simple way to address this +need would be to have a command (somewhere in the UI) that would delete +all class files in the output folder for which source code is available +("Delete Generated Class Files"). This would at least give the user some +help in recovering from these minor disasters. +<p>[Revised proposal: The Java builder remembers the names of the class +files it has generated. On full builds, it cleans out all class files that +it has on record as having generated; all other class files are left in +place. On incremental builds, it selectively cleans out the class files +that it has on record as having generated corresponding to the source files +that it is going to recompile. There is no need to monitor source file +deletions: corresponding generated class files will be deleted on the next +full build (because it nukes them all) or next incremental build (because +it sees the source file deletion in the delta). The Java builder never +looks at class files for their SourceFile attributes. A full build always +deletes generated class files, so there's no need to a special UI action.] +<h3> +<a NAME="Allowing Folders to Play Multiple Roles"></a>Allowing Folders +to Play Multiple Roles</h3> +The proposed change is to consistently allow the same folder to be used +in multiple ways on the same build classpath. +<ul> +<li> +As source folder and as output folder.</li> + +<ul> +<li> +N.B. This is currently supported (e.g., when folder is the project root).</li> + +<li> +Allows generated class files to be co-located with Java source files.</li> + +<li> +Since output folder is automatically included on runtime classpath, this +arrangement would automatically make any class files or resource files +available at runtime.</li> + +<ul> +<li> +However, these class files would not be seen at compile time library folder.</li> + +<li> +Recommendation: when class files or resources are present in a folder, +there should always be a library folder entry on the build classpath for +it.</li> +</ul> +</ul> + +<li> +As source folder and as library folder.</li> + +<ul> +<li> +N.B. This is currently disallowed.</li> + +<li> +Allows library class files to be co-located with Java source files.</li> + +<li> +Allows resource files to be co-located with Java source files.</li> + +<li> +In virtue of being a library entry on the build classpath, the folder is +used at compile time for library class files and is included on the runtime +classpath.</li> +</ul> + +<li> +As library folder and as output folder.</li> + +<ul> +<li> +N.B. This is currently disallowed.</li> + +<li> +Allows library class files to be co-located with generated class files.</li> + +<li> +Allows resource files to be co-located with generated class files.</li> + +<li> +Remove duplicate entry when forming the runtime class path.</li> + +<li> +Note that the generated class files in this library folder are ignored +by the builder because it has source for all these by definition.</li> +</ul> + +<li> +As source folder and as output folder and as library folder.</li> + +<ul> +<li> +This is just a combination of all of above.</li> + +<li> +Allows library class files, generated class files, and resource files to +be co-located with Java source files.</li> + +<li> +Simple "one folder Java development" setup for someone with library class +files and possibly resources.</li> +</ul> +</ul> +This change is not a breaking change; it would simply allow some classpath +configurations that are currently disallowed to be considered legitimate. +The API would not need to change. +<p>[Revised proposal: Many parts of the Java model assume that library +folders are relatively quiet. Allow a library folder to coincide with the +output folder would invalidate this assumption, which would tend to degrade +performance. For instance, the indexer indexes libraries and source folders, +but completely ignores the output folder. If the output folder was also +a library, it would repeatedly extract indexes for class files generated +by the builder. +<p><i>N.B. This means that the original scenario of library class files +in the output folder is cannot be done this way. It will need to be addressed +in some other way (discussed later on).</i> +<p><font color="#3366FF">The identity criteria for package fragment root +handles are based on resources/paths and do not take kind (source vs. binary) +into account. This means that a source folder and a library folder at the +same path map to the same package fragment root handle! Thus allowing a +source folder to coincide with a library folder cannot be supported without +revising Java element identity criteria (which is due for an overhaul, +but that's a different, and bigger, work item).</font> +<br>] +<h3> +<a NAME="Completely eliminate resource file copying behavior"></a>Completely +eliminate resource file copying behavior</h3> +The current Java builder copies "resource" files from source folders to +the output folder (provided that source and output do not coincide). Once +in the output folder, the resource files are available at runtime because +the output folder is always present on the runtime class path. +<p>This copying is problematic: +<ul> +<li> +Copying creates duplicates of resource files.</li> + +<ul> +<li> +Takes up extra disk space.</li> + +<li> +Copying resources takes extra time.</li> + +<li> +Increases risk of user confusion (modify the copy).</li> +</ul> + +<li> +Copying is out of character for Java builder.</li> + +<ul> +<li> +Java builder should compile Java source files to binary class files.</li> +</ul> + +<li> +Copying behavior is quirky.</li> + +<ul> +<li> +Resources are never copied from a source folder that coincides with the +output folder.</li> + +<li> +Resources are copied from a source folder that does not coincide with the +output folder, even if the output folder happens to be another source folder.</li> + +<li> +Modifying the copy and building causes the file to be deleted (!); it is +replaced by a fresh copy on the next full build.</li> + +<li> +When there are several resource files with same name, it is impossible +to reliably control which one ends up in the output folder.</li> + +<li> +When the project source is the project itself, and the output is in a folder +under the project, the builder copies the .classpath file into the output +folder too.</li> +</ul> +</ul> +The proposal is to eliminate this copying behavior. The proper way to handle +this is to include an additional library entry on the build classpath for +any source folders that contain resources. Since library entries are also +included on the runtime classpath, the resource files contained therein +will be available at runtime. +<p>We would beef up the API specification to explain how the build classpath +and the runtime classpath are related, and suggests that one deals with +resource files in source folders using library entries. This would be a +breaking change for clients or users that rely on the current resource +file copying behavior. +<p>The clients that would be most affected are ones that co-locate their +resource files with their source files in a folder separate from their +output folder. This is a fairly large base of customers that would need +to add an additional library entry for their source folder. +<p>It would be simple to write a plug-in that detected and fixed up the +Java projects in the workspace as required. By the same token, the same +mechanism could be built in to the Java UI. If the user introduces a resource +files into a source folder that had none and there is no library entry +for that folder on the build classpath, ask the user whether they intend +this resource file to be available at runtime. +<p>(JW believes that WSAD will be able to roll with this punch.) +<p>[Revised proposal: Retain copying from source to output folder where +necessary. +<ul> +<li> +Source folder different from output folder, no additional source folders: +copy resources from source folder to output folder (current behavior).</li> + +<li> +Source folder different from output folder, additional source folders: +copy resources from all source folders to output folder honoring build +classpath ordering (current behavior).</li> + +<li> +Source folder same as output folder, and no additional source folders: +no copying (current behavior).</li> + +<li> +Source folder same as output folder, and additional source folders: error +(new behavior).</li> +</ul> +This eliminates the screw case where resources get copied from one source +folder into another source folder, possibly overwriting client data.] +<h3> +<a NAME="Minimize the opportunity for obsolete class files to have bad effects"></a>Minimize +the opportunity for obsolete class files to have bad effects</h3> +The Java compiler should minimize the opportunity for obsolete class files +to have bad effects. +<p>Consider the following workspace: +<p>Java project p1/ +<br> src/com/example/ (source folder on build classpath) +<br> C1.java {package com.example; +public class C1 {}} +<br> C2.java {package com.example; +public class C2 extends Secondary {}) +<br> lib/com/example/ (library folder on build classpath) +<br> C1.class {from compiling +an old version of C1.java +<br> that read +package com.example; public class C1 {}; class Secondary {}} +<br> C2.class {from compiling +an old but unchanged version of C2.java} +<br> Secondary.class {from compiling +an old but unchanged version of C2.java} +<br> Quux.class {from compiling +Quux.java} +<p>Assume the source folder precedes the library folder on the build classpath +(sources should always precede libraries). +<p>When the compiler is compiling both C1.java and C2.java, it should not +satisfy the reference to the class com.example.Secondary using the existing +Secondary.class because the SourceFile attributes shows that Secondary.class +is clearly an output from compiling C1.java, not an input. In general, +the compiler should ignore library class files that correspond to source +files which are in the process of being recompiled. (In this case, only +Quux.class is available to satisfy references.) The Java builder does not +do this. +<p>Arguably, the current behavior should be considered a bug. (javac 1.4 +(beta) has this bug too.) Fixing this bug should not be a breaking change. +<p>When the SourceFile attribute is not present in a class file, there +is no choice but to use it. +<p>[Revised proposal: Maintain current behavior.] +<h3> +<font color="#000000">Library Copying Proposal</font></h3> +<font color="#000000">The proposal is to arrange to copy class files from +a certain library folder into the output folder. The library folder would +have to be represented by a library classpath entry so that the compiler +can find any class files it needs to compile source files. Copying the +class files to the output folder would unite them with the class files +generated by the compiler. Since there may be source code in the source +folder corresponding to some of the classes in the library folder, the +builder should only use a class file when source is available.</font> +<p><font color="#000000">Desired semantics:</font> +<p><font color="#000000">S (source folder)</font> +<br><font color="#000000">L (library folder)</font> +<br><font color="#000000">O (output folder)</font> +<p><font color="#000000">Invariant:</font> +<p><font color="#000000">x.class in O =</font> +<br><font color="#000000"> if some y.java in S generates +x.class then</font> +<br><font color="#000000"> x.class +from compiling x.java in S</font> +<br><font color="#000000"> else</font> +<br><font color="#000000"> if +x.class in L then</font> +<br><font color="#000000"> +x.class in L</font> +<br><font color="#000000"> else</font> +<br><font color="#000000"> +none</font> +<br><font color="#000000"> endif</font> +<br><font color="#000000"> endif</font> +<p><font color="#000000">Full builds achieve invariant.</font> +<br><font color="#000000">Incremental builds maintain invariant.</font> +<p><font color="#000000">Full build:</font> +<br><font color="#000000"> Scrub all class files from +O.</font> +<br><font color="#000000"> Compile all source files in +S into class files in O.</font> +<br><font color="#000000"> Infill/copy all class files +from L to O (no overwriting).</font> +<p><font color="#000000">Incremental build:</font> +<br><font color="#000000"> (phase 1) process all changes +to L:</font> +<br><font color="#000000"> for +delete or change x.class in L</font> +<br><font color="#000000"> if +x.class in O was not generated by compiler then scrub x.class from O</font> +<br><font color="#000000"> for +add or change x.class to L</font> +<br><font color="#000000"> +remember to infill x.class</font> +<br><font color="#000000"> (phase 2) process negative +changes to S:</font> +<br><font color="#000000"> for +delete or change y.java from S</font> +<br><font color="#000000"> +scrub any class file x.class from O that y.java compiled into</font> +<br><font color="#000000"> +remember to infill x.class</font> +<br><font color="#000000"> (phase 3) process positive +changes to S:</font> +<br><font color="#000000"> for +add or change y.java from S</font> +<br><font color="#000000"> +compile y.java into O</font> +<br><font color="#000000"> (phase 4) Infill/copy indicated +class files from L to O (no overwriting).</font> +<p><font color="#000000">We will look at ways to implement the above behavior +that do not involve changing the Java builder. This would mean that a customer +(such as WSAD) that requires library copying would be able to add it themselves; +otherwise, we will need to complicate the Java builder (which is complex +enough as it is) and integrate the mechanism into JDT Core.</font> +<h4> +<font color="#000000">Copying pre-builder</font></h4> +<font color="#000000">Could the copying of class files from the library +folder L to the output folder O be accomplished in a separate incremental +project builder that would run <i>before</i> the Java builder?</font> +<p><font color="#000000">Assume the Java builder manages its own class +files in the output folder and knows nothing of the pre-builder. Conversely, +assume that the pre-builder has no access to the insides of the Java builder.</font> +<p><font color="#000000">Pre-copying of class files to the output folder +cannot handle the case where a source file gets deleted and a pre-existing +class file in the library folder should now take its place. The Java builder, +which runs last, deletes the class file; the pre-builder has missed its +chance and does not get an opportunity to fill that hole. When this happens +on a full build, the full build does not achieve the invariant. This is +unacceptable.</font> +<p><font color="#000000">Here's the nasty case:</font> +<br><font color="#000000"> S (source folder): Bar.java +(but recently has Foo.java as well)</font> +<br><font color="#000000"> L (library folder): Foo.class</font> +<p><font color="#000000">On a full build</font> +<br><font color="#000000"> Pre-builder runs first:</font> +<br><font color="#000000"> Scrubs +Foo.class and Bar.class from O.</font> +<br><font color="#000000"> Copies +in Foo.class from L to O.</font> +<br><font color="#000000"> Java Builder runs second:</font> +<br><font color="#000000"> Scrubs +Foo.class from O (generated by Java builder from Foo.java on last build).</font> +<br><font color="#000000"> Compile +Bar.java into Bar.class O (Foo.java is no longer around).</font> +<p><font color="#000000">The output folder should contain a copy of Foo.class +from L since there is no equivalent source file that compiles to Foo.class. +It doesn't.</font> +<h4> +<font color="#000000">Copying post-builder</font></h4> +<font color="#000000">Could the copying of class files from the library +folder to the output folder be accomplished in a separate incremental project +builder that would run <i>after</i> the Java builder?</font> +<p><font color="#000000">Again, assume the Java builder manages its own +class files in the output folder and knows nothing of the post-builder, +and conversely.</font> +<p><font color="#000000">Post-copying of class files to the output folder +(no overwriting) cannot handle the case where library class files are changed +or deleted since the last build, because the post-builder is never in a +position to delete or overwrite class files in the output folder (they +might have been generated by the Java builder). Once lost, the invariant +cannot be reachieved no matter how many full builds you do (you're stuck +with stale or obsolete class files). This is unacceptable.</font> +<h4> +<font color="#000000">Combination of pre- and post-builder</font></h4> +<font color="#000000">Could the copying of class files from the library +folder to the output folder be accomplished by a pair of separate incremental +project builders that run on either side of the Java builder?</font> +<p><font color="#000000">Assume the Java builder manages its own class +files in the output folder and knows nothing of the pre-builder and post-builder, +and the pre- and post-builders have no access to the insides of the Java +builder.</font> +<p><font color="#000000">Full build:</font> +<br><font color="#000000"> Pre-builder runs first:</font> +<br><font color="#000000"> Scrubs +all class files from O.</font> +<br><font color="#000000"> Java Builder runs second:</font> +<br><font color="#000000"> Scrubs +all class files from O generated by Java builder.</font> +<br><font color="#000000"> Compiles +all source files into O.</font> +<br><font color="#000000"> Post-builder runs third:</font> +<br><font color="#000000"> Infill/copy +class files from L to O (no overwriting).</font> +<p><font color="#000000">Incremental build when L changes:</font> +<br><font color="#000000"> Pre-builder runs first:</font> +<br><font color="#000000"> For +delete or change x.class in L</font> +<br><font color="#000000"> +Does nothing (FAILs if no corresponding source file)</font> +<br><font color="#000000"> For +add x.class to L</font> +<br><font color="#000000"> +Infill/copy Foo.class from L to O (no overwriting).</font> +<br><font color="#000000"> Java Builder runs second:</font> +<br><font color="#000000"> Recompiles +classes that depend on affected class files in L.</font> +<br><font color="#000000"> Post-builder runs third:</font> +<br><font color="#000000"> Infill/copy +class files from L to O (no overwriting).</font> +<p><font color="#000000">Incremental build - changes to source folder:</font> +<br><font color="#000000"> Pre-builder runs first:</font> +<br><font color="#000000"> Does +nothing since library did not change.</font> +<br><font color="#000000"> Java Builder runs second:</font> +<br><font color="#000000"> Compiles +source files into O.</font> +<br><font color="#000000"> Post-builder runs third:</font> +<br><font color="#000000"> Infill/copy +class files from L to O (no overwriting).</font> +<p><font color="#000000">An incremental build may fail in the case of a +library class file being changed or deleted, leading to stale or obsolete +class files in the output folder. Fortunately, a full build always achieves +the invariant, and can be used to repair the damage due to changes to the +library.</font> +<p><font color="#000000">So while the combination of pre- and post-builders +is not perfect, it does work in many cases. If the user could do a full +build after making changes to the library folder, they would avoid all +the problems. The solution has the advantage of not requiring anything +special from the Java Core (i.e., WSAD should be able to implement it themselves).</font> +<h3> +<font color="#000000">Resources in Output Folder</font></h3> +<font color="#000000">When the source folder and output folder coincide, +there is no problem keeping resource files in the output folder since they +are not at risk of being overwritten (no with the proposed change to disable +resource copying when the source folder and output folder coincide).</font> +<p><font color="#000000">When the source folder and output folder do not +coincide, keeping resource files in the output folder on a permanent basis +encounters two issues:</font> +<p><font color="#000000">(1) The first issue is that output folder has +no presence in the packages view. Any resources that permanently resided +in the output folder would therefore be invisible during regular Java development. +One would have to switch to the resource navigator view to access them.</font> +<p><font color="#000000">The packages view only shows resource files in +source and library folders. Changing the packages view to show resources +in the output folder is infeasible. Including the output folder on the +classpath as a library folder was discussed at length above and is out +of the question. Including the output folder on the classpath as a source +folder is an option (in fact, it's exactly what you get when your source +and output folders coincide).</font> +<p><font color="#000000">(2) The second issue is that resource files in +the output folder are in harm's way of resources of the same name being +copied from a source folder.</font> +<p><font color="#000000">If resources existing in the output folder are +given precedence over the ones in source folders, then the ones from source +folders would only be copied once and nevermore overwritten. Copies in +the output folder would get stale or obsolete; automatic cleanup would +not be possible.</font> +<p><font color="#000000">On the other hand, if resources existing in source +folders are given precedence over the ones in the output folders, then +one that exists only in the output folders would be permanently lost if +a resource by the same name was ever to be created in a source folder. +It is a dangerous practice to allow the user to store important data in +a place that could be clobbered by an automatic mechanism that usually +operates unseen to the user.</font> +<p><font color="#000000">Conclusion: Keeping resource files in the output +folder on a permanent basis is not well supported at the UI, and should +only be done if the resource files can be considered expendable.</font> +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/pluggable jdks/pluggable-jdks.html b/org.eclipse.jdt.core/notes/r2.0/pluggable jdks/pluggable-jdks.html new file mode 100644 index 000000000..ed1638206 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/pluggable jdks/pluggable-jdks.html @@ -0,0 +1,225 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <meta name="Author" content="Build"> + <title>JDT - Pluggable JDKs</title> +</head> +<body> + +<h2> +Pluggable JDKs</h2> +Last revised 11:45 Friday October 19, 2001 +<p>Work item: pluggable JDKs +<p>Related work item: "Support for dealing with class files generated by +external Java compilers like javac and jikes from an Ant script." +<p>Related issue: remote builds +<p>Other IDEs can claim that when Sun or IBM releases a new JDK, a developer +can just "plug it in" to their IDE. We would like Eclipse to be similarly +flexible, and be able to make a similar claim. +<p>In practice, what does this mean? There are several different aspects. +<ol> +<li> +The ablility to run Java programs with the JDK's JRE.</li> + +<li> +The ablility to compile Java source code against the JDK's class libraries.</li> + +<li> +The ablility to compile Java source code with the JDK's javac compiler.</li> + +<li> +The ablility to debug Java programs with the JDK's JDPA and JVM.</li> + +<li> +The ablility to browse the JDK's class library API javadoc (or perhaps +other release doc).</li> + +<li> +The ablility to run the JDK's utility programs like javap, javadoc, javah.</li> +</ol> +We will take the first four as the most important ones for Eclipse to address, +and look at each in turn. +<h3> +Pluggable JREs</h3> +This is supported in Eclipse 1.0. +<p>The org.eclipse.jdt.launching plug-in provides pluggable support ("vmInstallTypes" +extension point) for describing installed Java2-style Java runtime environments +and launching particular configurations of a Java virtual machine. +<p>JDT defines a workbench preference ("Installed Java Runtime Environments") +so that the user can create, remove, and edit JRE definitions (i.e., instances +of the known VM installed types). The user chooses which one of the JRE +definitions is the default. +<p>JDT also defines a JRE property for each Java project to control which +JRE is used to run programs in that project. By default, each project shares +the workspace-wide default. The user can elect to specify a JRE for that +project, which overrides the workspace-wide default. +<h3> +Pluggable JDK class libraries</h3> +This is supported in Eclipse 1.0. +<p>JDT Core provides a reserved build classpath variable, named "JRE_LIB", +which gets bound to the JAR library (and associated source code ZIP) of +the current workspace-wide default JRE (e.g., "D:\jdk1.4\jre\lib\rt.jar" +with source in "D:\jdk1.4\src.zip"). By default, a classpath entry is included +on the build classpath of a newly created project. This library would ordinarily +supply the compile-time definitions of the standard class libraries used +when browsing and building a Java project. +<p>The client that is not satisfied with this variable is free to remove +the classpath entry from the build classpath and replace it with something +else. The client could declare their own build classpath variable, bind +it to a suitable value, and include that on the build classpath instead +(For instance, VAME/WSDD declares a classpath variable named "IVJ_HOME" +and references various class libraries relative to it; e.g., "IVJ_HOME/lib/jclmax.jar".) +Or they could just hard-wire a library entry for a particular JRE library. +<p>While the basic mechanism is reasonable, it is unfortunate that it is +tied so tightly to the default JRE. It might be more convenient if selecting +a different workspace-wide default JRE definition would prompt the user +to change the JRE_LIB classpath variable as well. +<p>Paralleling the workbench JRE mechanism, we could consider allowing +the user to specify classpath variable bindings at the project level that +override the workspace-wide default. This would allow the user to change +the binding to affect just that project. Something similar can already +be achieved by using distinctly-named classpath variables for each project +(e.g., "P1_JRE_LIB" for project P1's "JRE_LIB"). So it's unclear whether +any interesting new usecases would be supported by this. +<h3> +Pluggable Java compilers</h3> +Java compilers can differ along many axes: +<ul> +<li> +supported Java language level</li> + +<li> +Java bytecode version</li> + +<li> +quality of generated code</li> + +<li> +helpfulness of error messages</li> + +<li> +performance</li> + +<li> +robustness</li> + +<li> +product support for compiler</li> +</ul> +In the simple world of the command line compiler, it's easy to use whatever +Java compiler you choose to use. The command lines are substantially the +same, and the overt compiler behavior of translating .java source files +to .class files is utterly standard. +<p>In additional to the basic compiler functionality, there are usually +a number of IDE features that also need to be "language aware" (to some +extent), including: +<ul> +<li> +source code editing (syntax highlighting)</li> + +<li> +code assist (completion, selection)</li> + +<li> +code reformatter</li> + +<li> +search</li> +</ul> +The language aware features require compiler infrastructure (e.g., a scanner). +<p>The standard Sun Java compiler has no official APIs; the compiler infrastructure +is not available outside the compiler. This means that Java IDEs have no +choice but to reimplement whatever compiler infrastructure they might need. +Without standard Java compiler APIs, no Java IDE can be truely pluggable +in these regards. The best that a Java IDE can do in the circumstances +is to use a pluggable Java compiler for its basic compiler functionality. +<p>In Eclipse 1.0, the IDE's basic Java compiler functionality is provided +by the built-in Eclipse compiler. What would it take to make this part +pluggable? +<p>In Eclipse, the basic Java compiler functionality is provided through +the Java builder. The Java builder is activated when its build method is +called. This happens when (a) an explicit Build commands requested by the +user, (b) the workspace performs an auto build, or (c) some plug-in instigated +a build programmatically. +<p>So the first idea is that the Java builder's build method should invoke +a pluggable Java compiler to do a build. +<h4> +Calling a pluggable javac from within the Java builder</h4> +For a full build, this is clearly doable. The source folders mentioned +on the build classpath can be walked to identify all Java source files. +The corresponding class files in the output folder are deleted, and Java +problem markers are removed. The names of these source files are then passed +to javac as the ones to compile (large file sets perhaps broken up into +reasonable sized batches); the classpath passed is computed from the project's +build classpath; the output folder is passed as the target for the generated +class files. The compiler will generate class files into the target folder +and print text error messages to its output stream. Depending on how "standard" +the format of the output stream was, the Java builder might be able to +analyze the stream of text error messages and convert these into Java problem +markers associated with the offending source files (the complete stream +could also be saved and made available to user through some other mechanism). +Otherwise the net result is close to that of running the Eclipse compiler. +One other difference is that the Java builder would not be able to produce +anything resembling its current internal built state (i.e., no dependency +graph). +<p>For an incremental build, it is impossible to do anything more than +a cursory job without proper dependency information. The Java builder is +passed a resource delta for the project showing which source files have +changed. The delta would also show that the build classpath had changed +(the Java builder could easily remember some classpaths between builds). +<p>How to do an incremental build: +<ul> +<li> +delete a source file => identify and delete the corresponding class files</li> + +<li> +add a source file => identify and delete corresponding class files (just +in case); include source file in list to be recompiled</li> + +<li> +change a source file => identify and delete corresponding class files; +include source file in list to be recompiled</li> +</ul> +The compiler is called to recompile the identified list of source files. +The Java builder might be able to analyze the -verbose output stream to +discover which source files were actually compiled and update their Java +problem markers. +<p>This kind of simple-minded incremental build handles many simple cases +(e.g., changing the body of a method, fixing javadoc comments, reformatting). +The results would usually be less satisfactory when the principal structure +of class is changed because any dependent source files do not get recompiled, +which may lead to incompatible sets of binaries class files. The developer +would need to be educated about when to be asking for a full build. Many +will already be familiar with these rules from using other Java IDEs. With +a Java compiler that does not produce dependency information, it is hard +for an IDE with pluggable Java compilers to do any better. +<p>Autobuild is just an incremental build that is triggered automatically. +Note that the user may find it intolerable to run with autobuild enabled +if the overhead for invoking the pluggable compiler is high (which it is +likely to be if a separate JVM would need to be launched). +<p>The conclusion is that this is feasible, although autobuilding may be +intolerable. As long as the pluggable Java compiler was very javac-like +in terms of command line options and format of generated error messages, +it should be possible to use it to build a Java project. +<h4> +Calling an Ant script instead of the Java builder</h4> +An even more flexible approach would be to allow a Java project to be configured +with a generic Ant-based incremental project builder instead of the standard +Java builder. The Ant-based builder is described in a separate 2.0 Platform +Core feature proposal. +<p>All of the above considerations would still apply; the only real difference +is that everything is implemented in Ant terms. +<h3> +Pluggable JDPA Debuggers</h3> +Eclipse reimplements the JPDA debugger front end; it does not use the JDI +implementation supplied by Sun. Even if it did use Sun JDI, it still would +be work for the Eclipse debugger to capitalize on any new debugger functionality +thereby introduced. So the debugger front end is upgradeable, but not pluggable. +<p>The JDPA debugger back end is logically part of the Java runtime environment, +and ships with the IBM and Sun J2SE JDKs since 1.3. So this part is already +pluggable. +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/release notes/change_summary.txt b/org.eclipse.jdt.core/notes/r2.0/release notes/change_summary.txt new file mode 100644 index 000000000..efdc1bb42 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/release notes/change_summary.txt @@ -0,0 +1,128 @@ +********************************************************** +* BREAKING API CHANGES *********************************** +********************************************************** + None + +********************************************************** +* NON-BREAKING API CHANGES ******************************* +********************************************************** + + * Added API to set both the classpath and the output location at once. + IJavaProject.setRawClasspath( + IClasspathEntry[] newClasspath, + IPath newOutputLocation, + IProgressMonitor monitor) + + * ICodeCompletionRequestor got deprecated, should use ICompletionRequestor + instead to obtain: + (1) - local variable name suggestions + (2) - parameter name hints + (3) - method declaration suggestions. + + ICompletionRequestor is equivalent to ICodeCompletionRequestor except for + the following API changes: + + (1) + Added API for suggest variable name: + void acceptVariableName( + char[] typePackageName, + char[] typeName, + char[] name, + char[] completionName, + int completionStart, + int completionEnd); + + (2) + Added parameterNames to normal method results API: + void acceptMethod( + char[] declaringTypePackageName, + char[] declaringTypeName, + char[] selector, + char[][] parameterPackageNames, + char[][] parameterTypeNames, + char[][] parameterNames,<<<<<<<<<<<<<<<< ADDED + char[] returnTypePackageName, + char[] returnTypeName, + char[] completionName, + int modifiers, + int completionStart, + int completionEnd); + + (3) + Added API for answering method declaration completions: + void acceptMethodDeclaration( + char[] declaringTypePackageName, + char[] declaringTypeName, + char[] selector, + char[][] parameterPackageNames, + char[][] parameterTypeNames, + char[][] parameterNames, + char[] returnTypePackageName, + char[] returnTypeName, + char[] completionName, + int modifiers, + int completionStart, + int completionEnd); + + * SearchEngine.createJavaSearchScope(IResource[]) has been deprecated. + Use SearchEngine.createJavaSearchScope(IJavaElement[]) instead. + The rational is that createJavaSearchScope(IResource[]) was not well + defined for projects, and it could not define a search scope for java + elements that didn't have a corresponding resource (e.g. external jars). + + The specification of createJavaSearchScope(IJavaElement[]) is as follows: + - If an element is an IJavaProject, then the project's source folders, + its jars (external and internal) and its references projects (with their + source folders and jars, recursively) will be included. + - If an element is an IPackageFragmentRoot, then only the package fragments of + this package fragment root will be included. + - If an element is an IPackageFragment, then only the compilation unit and class + files of this package fragment will be included. Subpackages will NOT be + included. + + * Classpath entries (except for source folders) can be tagged as exported upon + creation. When exported, an entry is contributed to dependent projects along + with its output location. + Added APIs: + + Testing status of a given entry + + IClasspathEntry.isExported() + + Creating entries with export flag + + JavaCore.newProjectEntry(IPath, boolean) + + JavaCore.newLibraryEntry(IPath, IPath, IPath, boolean) + + JavaCore.newVariableEntry(IPath, boolean) + + * Search for field read and field write accesses. Two new constants have been added + on IJavaSearchConstants to be used when creating a field reference search pattern: + - READ_ACCESSES: the search results contain *only* read access to a field. + - WRITE_ACCESSES: the search results contain *only* write access to a field. + Note that if REFERENCES is used, then search results contain both read and write + accesss to a field. + + * org.eclipse.jdt.core.search.IJavaSearchResultCollector now clearly states that + the order of the search result is unspecified. + + * Added 2 new APIs on JavaConventions for classpath validation. + - IJavaModelStatus validateClasspath(IJavaProject project, IClasspathEntry[] classpath, IPath outputLocation) + - IJavaModelStatus validateClasspathEntry(IJavaProject project, IClasspathEntry entry, boolean checkSourceAttachment) + + +********************************************************** +* FUNCTIONALITY CHANGES ********************************** +********************************************************** + + * CodeAssist provides variable name suggestions. + * CodeAssist provides argument names in method completions. + * CodeAssist completes on method declarations (just enter selector prefix) + * Project can contribute more than their output folder, using exported classpath entries. + * CodeAssist inserts qualification on field/method/type references in case of ambiguities (optionally these + qualification can be forced to occur). + * CodeAssist optionally performs visibility checks (see JavaCore option: "org.eclipse.jdt.core.codeComplete.visibilityCheck"). + * OpenOnSelection can now locate selected declarations. + * Search can narrow field read/write access. + * Assertions support enabled: by default the compiler is 1.3 compliant, but it can + optionally be turned into source 1.4 mode cf. JavaCore options. + * Evaluation in binaries is functional + * Search for references now finds results in binaries. Indexes in old workspaces are recomputed when restarted + which may result in longer startup times. + * Search in inner-classes now works. Indexes are recomputed automatically on start-up. + +
\ No newline at end of file diff --git a/org.eclipse.jdt.core/notes/r2.0/variable init/ClasspathVariableInitializer.java b/org.eclipse.jdt.core/notes/r2.0/variable init/ClasspathVariableInitializer.java new file mode 100644 index 000000000..94db6e898 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/variable init/ClasspathVariableInitializer.java @@ -0,0 +1,30 @@ +package org.eclipse.jdt.core; + +/** + * Abstract base implementation of all classpath variable initializers. + * Classpath variable initializers are used in conjunction with the + * "org.eclipse.jdt.core.classpathVariableInitializer" extension point. + * <p> + * Clients should subclass this class to implement a specific classpath + * variable initializer. The subclass must have a public 0-argument + * constructor and a concrete implementation of <code>initialize</code>. + * </p> + */ +public abstract class ClasspathVariableInitializer { + + /** + * Creates a new classpath variable initializer. + */ + protected ClasspathVariableInitializer() { + } + + /** + * Binds a value to the workspace classpath variable with the given name, + * or fails silently if this cannot be done. + * + * @param variable the name of the workspace classpath variable + * that requires a binding + * @see JavaCore#setClasspathVariable + */ + protected abstract void initialize(String variable); +} diff --git a/org.eclipse.jdt.core/notes/r2.0/variable init/classpathVariableInitializer.html b/org.eclipse.jdt.core/notes/r2.0/variable init/classpathVariableInitializer.html new file mode 100644 index 000000000..1a5f6c09b --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/variable init/classpathVariableInitializer.html @@ -0,0 +1,47 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>Workbench Extension Point - Classpath Variable Initializer</title> +</head> +<body link="#0000FF" vlink="#800080"> + +<center> +<h1> +Classpath Variable Initializer</h1></center> +<b><i>Identifier: </i></b>org.eclipse.jdt.core.classpathVariableInitializer +<p><b><i>Description: </i></b>This extension point allows a plug-in to +register code for programmatically initializing a particular named classpath +variable. +<p><b><i>Configuration Markup:</i></b> +<p><tt> <!ELEMENT classpathVariableInitializer></tt> +<br><tt> <!ATTLIST classpathVariableInitializer</tt> +<br><tt> variable CDATA #REQUIRED</tt> +<br><tt> class +CDATA #REQUIRED</tt> +<br><tt> ></tt> +<ul> +<li> +<b>variable</b> - the name of the classpath variable</li> + +<li> +<b>class</b> - the class that implements this classpath variable initializer. +The class must implement a public subclass of <tt>org.eclipse.jdt.core.ClasspathVariableInitializer</tt> +with a public 0-argument constructor and an implementation of the <tt>initialize(String)</tt> +method.</li> +</ul> +<b><i>Examples:</i></b> +<br>The following is an example of an IClasspathVariableInitializer for +a classpath variable named "FOO": +<p><tt><extension</tt> +<br><tt> point="org.eclipse.jdt.core.classpathVariableInitializer"></tt> +<br><tt> <classpathVariableInitializer</tt> +<br><tt> variable="FOO"</tt> +<br><tt> class="com.example.CPVInitializer"/></tt> +<br><tt></extension></tt> +<p><b><i>Supplied Implementation:</i></b> +<br>None. +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/variable init/uninit-classpath-vars.html b/org.eclipse.jdt.core/notes/r2.0/variable init/uninit-classpath-vars.html new file mode 100644 index 000000000..6684fb0bd --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/variable init/uninit-classpath-vars.html @@ -0,0 +1,80 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>JDT - Uninitialized Classpath Variables</title> +</head> +<body> + +<h2> +Uninitialized Classpath Variables</h2> +Last revised 11:30 Monday November 19, 2001 +<p>Original work item: "Building with uninitialized class path variables. +You can add a project from the repository that gets built without having +the JavaUI that initializes the JRE_ variables is activated." +<p>The general problem is that a classpath variable can show up in a project's +classpath quite early (for example, when a project is loaded from a repository), +and well before the activation of a plug-in that might willingly initialize +the workspace's binding for that variable. Without a binding for all the +variables mentioned on its build classpath, the project cannot be successfully +built. However, there is currently no mechanism by which these variables +can get initialized. +<p>For variables that the developer (or his team mates) introduces explicitly, +this is not a particular problem. The developer's corrective action is +to explicitly establish a binding for the variable, and then rebuild. +<p>However, there is a problem for variables that are introduced and ordinalrily +initialized by some tool. For these, the developer may not be in a position +to explicitly establish a binding for the variable, and might not even +know which plug-in needs to be activated. +<p>This problem is a symptom of a more widespread problem. For example, +PDE suffers this problem with the "ECLIPSE_HOME" variable. +<h3> +Proposal</h3> +We introduce a JDT Core extension point <tt><a href="classpathVariableInitializer.html">org.eclipse.jdt.core.classpathVariableInitializer</a></tt> +through which plug-ins can supply initializer code for named classpath +variables. +<p>Examples of how this would be used: +<p><tt><extension</tt> +<br><tt> point = "org.eclipse.jdt.core.classpathVariableInitializer"></tt> +<br><tt> <classpathVariableInitializer</tt> +<br><tt> variable="ECLIPSE_HOME"</tt> +<br><tt> class="org.eclipse.pde.internal.core.EclipseHomeInitializer"/></tt> +<br><tt></extension></tt> +<p><tt><extension</tt> +<br><tt> point = "org.eclipse.jdt.core.classpathVariableInitializer"></tt> +<br><tt> <classpathVariableInitializer</tt> +<br><tt> variable="JRE_LIB"</tt> +<br><tt> class="org.eclipse.jdt.internal.ui.CPVInitializer"/></tt> +<br><tt></extension></tt> +<p>The mechanism would work as follows: +<ul> +<li> +It applies automatically each time a classpath containing an unbound classpath +variable is being resolved (e.g., by JavaCore.getResolvedClasspathEntry +or IJavaProject.getResolvedClasspath).</li> + +<li> +If classpath variable is unbound in the workspace, it ask if there is a +registered initializer for that variable.</li> + +<li> +If there is an initializer, it is invoked (it is passed the name of the +classpath variable that needs to be initialized).</li> + +<li> +The initializer returns no result; it simply has the side effect initializing +the variable if it can.</li> + +<li> +After the initializer is invoked, the resolution process proceeds whether +or not the variable is bound or unbound.</li> + +<li> +If there are multiple initializers registered for the same variable, only +the first one is used (this mechanims does not support alternate or hierarchical +classpath variable initializers).</li> +</ul> + +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure-notes.html b/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure-notes.html new file mode 100644 index 000000000..522148903 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure-notes.html @@ -0,0 +1,874 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.5 [en] (Win98; I) [Netscape]"> + <title>JDT - Notes on Workspace structure</title> +</head> +<body> + +<h2> +Notes on Workspace Structure</h2> +Last revised 17:00 Wednesday September 26, 2001 +<p>In large Java development efforts, it is not uncommon for several inter-related +projects to be under development at the same time. For the purposes of +this discussion, assume teams, projects, and components align. All teams +are working in the same repository, sharing a set of projects via a single +stream; each project contains the source code for a single component. +<p>Our example scenario has 4 projects: P1, P2, P3, and P4, with components +C1 through C4, respectively. C1 is the only component that does not depend +on any others; C2 depends on C1; C3 depends on C1 and C2; C4 depends on +C2 (N.B., but not on C1 or C3). +<p>We assume that there is a single CVS repository that contains all 4 +projects in source code form. In addition, we assume that centralized builds +are done periodically and posted to a web server where they can be downloaded +as a unit. These downloads take the form of a zipped directory which includes +a binary jar for each component, along with corresponding source jars to +aid debugging. +<p>We assume that each developer can download builds from the web server +and install them on their local machine. To do their work, they set up +Eclipse workspaces on their local machine and load one or more projects +from the CVS repository. They also upgrade their workspace from time to +time as new builds become available. +<p>These assumptions are a plausible abstraction of what goes on in open +source projects. (It is an open question as to how closely the Eclipse +project with follow this work model.) +<h3> +Developing, Using, and Patching Components</h3> +The following things are the norm for someone actively <b>developing</b> +a component: +<ul> +<li> +source code is available for entire component</li> + +<li> +source code is being browsed and changed</li> + +<li> +source code is being compiled regularly</li> + +<li> +source code changes are shared occasionally with other team members</li> +</ul> +At the other end of the spectrum are components that are passively <b>used</b>: +<ul> +<li> +component is available as pre-compiled binary library</li> + +<li> +source code is available for browsing but not for editing</li> + +<li> +since used component is static, no changes to share with other team members</li> +</ul> +Components that are being <b>patched</b> are one step from being a used +component in the direction of being a component being developed: +<ul> +<li> +component is available as pre-compiled binary library</li> + +<li> +source code is selectively available for editing (patching)</li> + +<li> +patched source code is compiled and used instead of pre-compiled binary</li> + +<li> +patched source code is for local consumption (not shared with other team +members)</li> +</ul> +The general problem can be stated as follows: each developer needs their +own workspace so that they can <i>develop</i> their assigned component. +To to do, they will need to <i>use</i> the components that their component +depends on, and perhaps use some of the other components that depend on +theirs. In order to do their work, a developer may need to <i>patch</i> +a component that they would ordinarily just <i>use</i>; in some cases, +they might even need to actively develop a component that they would ordinarily +just <i>use</i>. How can the developers structure their workspaces so that +they retain sufficient flexibly to switch between using and patching (or +developing) these other components? +<p>The various scenarios presented below are all plausible workspace setups, +each with certain advantages and disadvantages: +<ul> +<li> +<a href="#Component plus libraries">Component plus libraries</a></li> + +<li> +<a href="#Source for everything">Source for everything</a></li> + +<li> +<a href="#Classpath variables to switch between libraries and sources">Classpath +variables to switch between libraries and sources</a></li> + +<li> +<a href="#Unshared Proxy library projects">Unshared proxy library projects</a></li> + +<li> +<a href="#Shared proxy library projects">Shared proxy library projects</a></li> + +<li> +<a href="#Stub projects">Stub projects</a></li> + +<li> +<a href="#Other Scenarios">Assorted Other Scenarios</a></li> +</ul> + +<h3> +<a NAME="Component plus libraries"></a>Component plus libraries</h3> + +<ul> +<li> +Team member's workspace contains just the project for their component.</li> + +<li> +Each team members downloads latest binary build to c:\temp\build\ or some +such and sets classpath variable BUILD = c:\temp\build\</li> + +<li> +Build classpath references individual libraries relative to BUILD classpath +variable.</li> + +<li> +Use of classpath variable allows each developer to choose where to install +the build.</li> + +<li> +Classpath includes libraries for all prerequisite components.</li> + +<li> +Pro: simple workspace setup (load a single shared project from repository +and set single classpath variable).</li> + +<li> +Pro: no restiction on number of jars per project.</li> + +<li> +Pro: easy to upgrade to another binary build (unzip build in place, and +close and reopen project in workspace to force a refresh).</li> + +<li> +Con: unable to view code for a component that is not a prerequisite (requires +loading an additional project).</li> + +<li> +Con: unable to browse code for prerequiste components in proper content +(code completion).</li> + +<li> +Con: switching to developing another component involves loading another +project and changing classpaths on dependent components (bad, because these +are shared).</li> + +<li> +Con: patching only via switching to active development.</li> + +<li> +Note: uses classpath variables whose value is a library jar (works in R1.0).</li> + +<li> +Note: this is roughly the way PDE 1.0 works.</li> +</ul> +project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>| +<br>project P2 (shared) +<br> build classpath = source /P2/C2; BUILD/c1.jar+BUILD/c1src.zip +<br> output folder /P2/bin +<br>| +<br>project P3 (shared) +<br> build classpath = source /P3/C3; BUILD/c1.jar+BUILD/c1src.zip; +BUILD/c2.jar+BUILD/c2src.zip +<br> output folder /P3/bin +<br>| +<br>project P4 (shared) +<br> build classpath = source /P4/C4; BUILD/c2.jar+BUILD/c2src.zip +<br> output folder /P4/bin +<h3> +<a NAME="Source for everything"></a>Source for everything</h3> + +<ul> +<li> +Team member's workspace contains all four projects and works entirely from +source code.</li> + +<li> +Build classpath references individual dependent projects by name.</li> + +<li> +Classpath includes projects for all prerequisite components.</li> + +<li> +Pro: able to browse code for all components.</li> + +<li> +Pro: able to develop any component.</li> + +<li> +Pro: easy to notice downstream impact of any changes made since all components +are present.</li> + +<li> +Pro: easy to keep current (by catching up with stream in repository).</li> + +<li> +Con: Very slow workspace startup and large footprint because entails recompiling +source code for all components.</li> + +<li> +Con: Initial workspace setup involving multiple shared projects (automation +possible).</li> + +<li> +Con: difficulty handling projects with precompiled library jars that must +be exported but do not exist in source (e.g., the Eclipse debugger's JDI +jar)</li> + +<li> +Note: uses required projects (works in R1.0).</li> +</ul> +project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>& +<br>project P2 (shared) +<br> build classpath = source /P2/C2; project P1 +<br> output folder /P2/bin +<br>& +<br>project P3 (shared) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>& +<br>project P4 (shared) +<br> build classpath = source /P4/C4; project P2 +<br> output folder /P4/bin +<h3> +<a NAME="Classpath variables to switch between libraries and sources"></a>Classpath +variables to switch between libraries and sources</h3> + +<ul> +<li> +Team member's workspace contains just the project for their component.</li> + +<li> +Each team members downloads latest binary build to c:\temp\build\ or some +such and sets family of classpath variables, one per component:</li> + +<ul> +<li> +P1_LIB = c:\temp\build\c1.jar</li> + +<li> +P2_LIB = c:\temp\build\c2.jar</li> + +<li> +P3_LIB = c:\temp\build\c3.jar</li> + +<li> +P4_LIB = c:\temp\build\c4.jar</li> +</ul> + +<li> +Build classpath references an individual component via a classpath variable.</li> + +<li> +Use of classpath variable allows each developer to choose where to install +the build.</li> + +<li> +Classpath includes entries for all prerequisite components.</li> + +<li> +Pro: switch to development involves loading an additional project into +workspace and rebinding a single classpath variable to refer to that project. +For example, load project P2 and rebind P2_LIB = /P2.</li> + +<li> +Pro: easy to upgrade to another binary build (unzip build in place, and +close and reopen project in workspace to force a refresh).</li> + +<li> +Con: difficuly attaching debug source to library jar</li> + +<li> +Con: only works if there is exactly one library jar per project.</li> + +<li> +Con: initial workspace setup involving a single shared project and multiple +variable bindings (automation required).</li> + +<li> +Con: classpath variable names must be agreed on across components.</li> + +<li> +Con: unable to view code for a component that is not a prerequisite (requires +loading an additional project).</li> + +<li> +Con: unable to browse code for prerequiste components in proper content +(code completion).</li> + +<li> +Note: uses classpath variables whose value is either a project or a library +(in R1.0, JDT core supports these but JDT UI does not expose).</li> +</ul> +project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>| +<br>project P2 (shared) +<br> build classpath = source /P2/C2; P1_LIB +<br> output folder /P2/bin +<br>| +<br>project P3 (shared) +<br> build classpath = source /P3/C3; P1_LIB; P2_LIB +<br> output folder /P3/bin +<br>| +<br>project P4 (shared) +<br> build classpath = source /P4/C4; P2_LIB +<br> output folder /P4/bin +<h3> +<a NAME="Unshared Proxy library projects"></a>Unshared Proxy library projects</h3> + +<ul> +<li> +Team member's workspace contains a project for each component.</li> + +<li> +Main project is in source and is shared via the repository.</li> + +<li> +All other projects in workspace are proxy library projects (same name as +source project; contains no source code; not shared via repository).</li> + +<li> +Each team members downloads latest binary build to c:\temp\build\ or some +such and sets classpath variable BUILD = c:\temp\build\</li> + +<li> +Build classpath exports libraries relative to BUILD classpath variable.</li> + +<li> +Use of classpath variable allows each developer to choose where to install +the build.</li> + +<li> +Build classpath references individual dependent projects by name.</li> + +<li> +Pro: able to browse code for all components in context.</li> + +<li> +Pro: easy to switch to another binary build.</li> + +<li> +Pro: switch to patching involves adding source and output folders to non-shared +project, populating with selected source files, and building; no other +classpaths need to be changed.</li> + +<li> +Pro: switch to development involves loading an additional shared project +into workspace over top of non-shared proxy library project; no classpaths +need to be changed.</li> + +<li> +Pro: easy to upgrade to another binary build (unzip build in place, and +close and reopen project in workspace to force a refresh).</li> + +<li> +Con: initial workspace setup involving multiple non-shared and shared projects +(automation required; e.g., preconstructed base workspace).</li> + +<li> +Con: project holding component under development is lost in sea of proxy +library projects.</li> + +<li> +Pro: no restriction on number of jars per project.</li> + +<li> +Note: the exported libraries are external to the project.</li> + +<li> +Note: uses library projects (proposed for R2.0).</li> +</ul> +The base workspace for developers on all teams looks like: +<p>project P1 (not shared) +<br> library project +<br> build classpath = export BUILD/c1.jar+BUILD/c1src.zip +<br>& +<br>project P2 (not shared) +<br> library project +<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip; +project P1 +<br>& +<br>project P3 (not shared) +<br> library project +<br> build classpath = export BUILD/c3.jar+BUILD/c3src.zip; +project P1; project P2 +<br>& +<br>project P4 (not shared) +<br> library project +<br> build classpath = export BUILD/c4.jar+BUILD/c3src.zip; +project P2 +<p>None of these projects are shared; however, they have the same names +as the source projects in repository. This means that any of the library +projects can be replaced by loading the corresponding source project from +the repository. (None of the other projects in the workspace need to change.) +<p>project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>| +<br>project P2 (shared) +<br> build classpath = source /P2/C2; project P1 +<br> output folder /P2/bin +<br>| +<br>project P3 (shared) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>| +<br>project P4 (shared) +<br> build classpath = source /P4/C4; project P2 +<br> output folder /P4/bin +<p>Regarding automation, the centralized build could create a simple XML +document describing the collection of proxy library projects for the workspace. +Given this document, a special purpose plug-in could be written that would +create (or modify existing) unshared proxy library projects in the workspace. +<p><tt><projects></tt> +<br><tt> <project name="P1"></tt> +<br><tt> <natures</tt> +<br><tt> +<nature id="org.eclipse.jdt.core.javanature"/></tt> +<br><tt> </natures></tt> +<br><tt> <libraries></tt> +<br><tt> +<classpathentry kind="var" path="BUILD/c1.jar" sourcepath="BUILD/c1src.zip" +export="true"/></tt> +<br><tt> </libraries></tt> +<br><tt> </project></tt> +<br><tt> <project name="P2"></tt> +<br><tt> <natures</tt> +<br><tt> +<nature id="org.eclipse.jdt.core.javanature"/></tt> +<br><tt> </natures></tt> +<br><tt> <libraries></tt> +<br><tt> +<classpathentry kind="var" path="BUILD/c2.jar" sourcepath="BUILD/c2src.zip" +export="true"/></tt> +<br><tt> +<classpathentry kind="project" path="/P1"/></tt> +<br><tt> </libraries></tt> +<br><tt> </project></tt> +<br><tt> <project name="P3"></tt> +<br><tt> <natures</tt> +<br><tt> +<nature id="org.eclipse.jdt.core.javanature"/></tt> +<br><tt> </natures></tt> +<br><tt> <libraries></tt> +<br><tt> +<classpathentry kind="var" path="BUILD/c3.jar" sourcepath="BUILD/c3src.zip" +export="true"/></tt> +<br><tt> +<classpathentry kind="project" path="/P1"/></tt> +<br><tt> +<classpathentry kind="project" path="/P2"/></tt> +<br><tt> </libraries></tt> +<br><tt> </project></tt> +<br><tt> <project name="P4"></tt> +<br><tt> <natures</tt> +<br><tt> +<nature id="org.eclipse.jdt.core.javanature"/></tt> +<br><tt> </natures></tt> +<br><tt> <libraries></tt> +<br><tt> +<classpathentry kind="var" path="BUILD/c4.jar" sourcepath="BUILD/c4src.zip" +export="true"/></tt> +<br><tt> +<classpathentry kind="project" path="/P2"/></tt> +<br><tt> </libraries></tt> +<br><tt> </project></tt> +<br><tt></projects></tt> +<h3> +<a NAME="Shared proxy library projects"></a>Shared proxy library projects</h3> +In the previous approach to using proxy library projects, these projects +were not under VCM. In this variant of it, the proxy library projects are +obtained from a repository as well. We will assume that the proxy library +projects are stored in a different repository from the source projects. +By assuming they're in a separate binary repository, it is easy to use +the same project names for both source and binary forms (doing so in the +same repository would required introducing non-standard binary-only and +source-only branches in the project version histories). +<ul> +<li> +Team member's workspace contains a project for each component.</li> + +<li> +Main project is in source and is shared via the repository.</li> + +<li> +All other projects in workspace are proxy library projects; same name as +source project, but in a different repository.</li> + +<li> +Each proxy library project contains and exports one or more binary jar +libraries (plus attached source zips) equivalent to the source.</li> + +<li> +Build classpath references individual dependent projects by name.</li> + +<li> +Pro: able to browse code for all components.</li> + +<li> +Pro: easy to switch to another binary build.</li> + +<li> +Pro: switch to patching involves adding source and output folders to shared +proxy library project, populating with selected source files, and building; +no other classpaths need to be changed.</li> + +<li> +Pro: switch to development involves loading shared source project into +workspace over top of shared proxy library project with the same name; +no classpaths need to be changed.</li> + +<li> +Pro: easy to upgrade to another binary build (by catching up with binary +stream in repository).</li> + +<li> +Pro: no restriction on number of jars per project.</li> + +<li> +Pro: Proxy library projects are also under VCM.</li> + +<li> +Con: initial workspace setup involving multiple shared projects (automation +possible).</li> + +<li> +Con: Additional effort of building and maintained proxy library projects +in a separate repository (automation possible, perhaps as part of centralized +build process).</li> + +<li> +Con: project holding component under development is lost in sea of proxy +library projects.</li> + +<li> +Note: the exported libraries are internal to the project.</li> + +<li> +Note: uses library projects (proposed for R2.0).</li> +</ul> +The base workspace for developers on all teams looks like: +<p>project P1 (shared via binary repository) +<br> library project +<br> build classpath = export /P1/c1.jar+BUILD/c1src.zip +<br>& +<br>project P2 (shared via binary repository) +<br> library project +<br> build classpath = export /P2/c2.jar+BUILD/c2src.zip; +project P1 +<br>& +<br>project P3 (shared via binary repository) +<br> library project +<br> build classpath = export /P3/c3.jar+BUILD/c3src.zip; +project P1; project P2 +<br>& +<br>project P4 (shared via binary repository) +<br> library project +<br> build classpath = export /P4/c4.jar+BUILD/c4src.zip; +project P2 +<p>The proxy library projects have the same names as the source projects, +but are stored in a separate repository. +<p>project P1 (shared via source repository) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>| +<br>project P2 (shared via source repository) +<br> build classpath = source /P2/C2; project P1 +<br> output folder /P2/bin +<br>| +<br>project P3 (shared via source repository) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>| +<br>project P4 (shared via source repository) +<br> build classpath = source /P4/C4; project P2 +<br> output folder /P4/bin +<h3> +<a NAME="Stub projects"></a>Stub projects</h3> +This is a variation on unshared proxy library projects that makes different +tradeoffs. We call these <i>stub projects</i>. In particular, the ability +to browse a prereqisite component is traded for the ability to have only +a minimal set of extra library projects in the workspace. The only difference +is that stub projects do not require other projects (whereas proxy library +projects did require other projects). This allow a workspace to get by +with a less than complete set of stub projects. Creating a stub project +never forces you to create other stub projects. +<ul> +<li> +Main project is in source and is shared via the repository.</li> + +<li> +That project's required projects are represented in workspace by stub projects +(same name as source project; not shared via repository).</li> + +<li> +Stub projects are library projects (no source code).</li> + +<li> +A stub project only represents a project's outward appearance to other +projects.</li> + +<li> +Stub projects never require any other projects.</li> + +<li> +Each team members downloads latest binary build to c:\temp\build\ or some +such and sets classpath variable BUILD = c:\temp\build\</li> + +<li> +Build classpath exports libraries relative to BUILD classpath variable.</li> + +<li> +Use of classpath variable allows each developer to choose where to install +the build.</li> + +<li> +Build classpath references individual dependent projects by name.</li> + +<li> +Pro: stub projects are needed only for immediate prerequisites of main +project.</li> + +<li> +Con: unable to view code for a component that is not a prerequisite (requires +loading an additional project).</li> + +<li> +Con: unable to browse code for prerequiste components in proper content +(code completion) because stub does not provide sufficient context.</li> + +<li> +Con: patching only via switching to active development.</li> + +<li> +Pro: switch to development involves loading an additional shared project +into workspace, possibly over top of stub project; no classpaths need to +be changed.</li> + +<li> +Pro: easy to upgrade to another binary build (unzip build in place, and +close and reopen project in workspace to force a refresh).</li> + +<li> +Con: initial workspace setup involving multiple shared projects and stub +projects; ongoing need for additional stub projects when new source projects +are loaded (automation required).</li> + +<li> +Pro: no restriction on number of jars per project.</li> + +<li> +Fatal flaw (!): limited able to compile against a stub project due to lack +of sufficient context Note: the exported libraries are external to the +project.</li> + +<li> +Note: uses library projects (proposed for R2.0).</li> +</ul> +The source projects: +<p>project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<br>| +<br>project P2 (shared) +<br> build classpath = source /P2/C2; project P1 +<br> output folder /P2/bin +<br>| +<br>project P3 (shared) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>| +<br>project P4 (shared) +<br> build classpath = source /P4/C4; project P2 +<br> output folder /P4/bin +<p>The corresponding stub projects (note the absence of required projects): +<p>project P1 (not shared) +<br> library project +<br> build classpath = export BUILD/c1.jar+BUILD/c1src.zip +<p>project P2 (not shared) +<br> library project +<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip +<p>project P3 (not shared) +<br> library project +<br> build classpath = export BUILD/c3.jar+BUILD/c3src.zip +<p>project P4 (not shared) +<br> library project +<br> build classpath = export BUILD/c4.jar+BUILD/c3src.zip +<p>For example, a developer working on P3 would need stub projects for +P1 and P2. +<p>project P3 (shared) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>& +<br>project P2 (not shared) +<br> library project +<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip +<br>& +<br>project P1 (not shared) +<br> library project +<br> build classpath = export BUILD/c1.jar+BUILD/c1src.zip +<p>If they then decided to develop P1 as well, they would replace P1 by +the souce project. Their workspace would now look like: +<p>project P3 (shared) +<br> build classpath = source /P3/C3; project P1; project +P2 +<br> output folder /P3/bin +<br>& +<br>project P2 (not shared) +<br> library project +<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip +<br>& +<br>project P1 (shared) +<br> build classpath = source /P1/C1 +<br> output folder /P1/bin +<p>The fatal flaw is clear in the case of a workspace containing P4 and +a stub for P2. +<p>project P4 (shared) +<br> build classpath = source /P4/C4; project P2 +<br> output folder /P4/bin +<br>& +<br>project P2 (not shared) +<br> library project +<br> build classpath = export BUILD/c2.jar+BUILD/c2src.zip +<p>If, for example, a class in C4 subclasses a class in C2 which subclasses +a class in C1, then the compiler will need to get its hands on the class +in C1. Unfortunately, neither the source code nor binary for C1 is anywhere +to be found. +<h3> +<a NAME="Other Scenarios"></a>Assorted Other Scenarios</h3> +A couple of other scenarios can be constructed using a combination of the +techniques employed above: +<ul> +<li> +Unshared proxy library projects with different project names.</li> + +<br>Use a classpath variable per component; bind it initially to the name +of the proxy library project; rebind to name of the source project. +<ul> +<li> +Pro: Allows side-by-side proxy library project and source project.</li> + +<li> +Con: Uses both classpath variables and proxy library projects.</li> +</ul> + +<li> +Shared proxy library projects with different project names.</li> + +<br>Same as preceding except the proxy library projects can be shared. +Use a classpath variable per component; bind it initially to the name of +the proxy library project; rebind to name of the source project. Since +the proxy library project and source projects have different names, there +are no VCM anomalies. +<ul> +<li> +Pro: Allows side-by-side proxy library project and source project.</li> + +<li> +Pro: Proxy library projects are also under VCM.</li> + +<li> +Con: Uses both classpath variables and proxy library projects.</li> +</ul> + +<li> +Shared proxy library projects with same project names and same repository.</li> + +<br>The proxy library projects can use the same name as the source project +and be maintained under VCM in the same repository as the source projects. +Note that this is a non-standard arrangement for a VCM project. The source +and binary versions of a project need to be thought of as two permanently +separate branches in the version history: the source versions contains +source code but no binaries, whereas the binary versions contains binaries +but no source code. The differences are also reflected in the project's +.classpath file. In Eclipse, this can be achieved through the use of a +binaries-only stream separate from the usual stream in which the source +is maintained. In order to avoid generating unwanted outgoing or incoming +changes when switching from binary to source, proceed as follows: unshare +from binary stream; delete from workspace; load and share from source stream. +<ul> +<li> +Pro: Proxy library projects are also under VCM.</li> + +<li> +Pro: One repository holds everything.</li> + +<li> +Con: Project has non-standard dual version history.</li> + +<li> +Con: Separate binaries-only stream means separate branch in version history.</li> +</ul> +</ul> + +<h3> +Discussion</h3> +Of the various workspace setups discussed, the "Component plus libraries" +approach is the most straightforward, but also the weakest. It should work +well in cases where developers work on exactly the component they are assigned. +But it is not recommended in cases where developers assigned to one component +would need to patch or develop another component in the same workspace. +<p>The "Source for everything" approach is best for developers who are +all involved in the joint active development of all of the components. +There is a limit to how large it will scale, since the cost of recompiling +everything from source increases method the number and size of components. +It is not recommended in situations where many of the components are not +under active development; it will be more efficient to use pre-compiled +binary libraries for the static components. +<p>Approaches involving classpath variables do not have much to recommend +them. They share most of the disadvantages (and none of the advantages) +of using proxy library projects. +<p>The "Proxy Library Projects" workspace setup allows each developer to +work on their assigned component while providing ready access to all other +components. The arrangement is flexible in allowing easy switching from +using a component to patching it (or to actively developing it). The two +setups outlined in detail show how the proxy binary projects can be constructed +from a downloadable binary build or obtained from a version-managed repository. +Each has there pluses and minuses. The main drawback of using unshared +proxy library projects is the significant task of setting up a workspace +in the first place. This task would have to be automated by some means; +it would too tedious and error prone to have each developer create a workspace +from scratch. For shared proxy library projects, the tedium is in creating +new versions of the projects in their repository. This task could be automated +as part of the centralized build process. +<p>The "Stub Projects" workspace setup allows each developer to work on +their assigned component while providing placeholders for other components +that it depends on. The arrangement is flexible in allowing easy switching +from using a component to actively developing it by replacing the placeholder. +Unfortunately, this approach is fatally flawed because a compiler might +not have enough information to compile. The other drawbacks are their limited +useful of the stub projects (can't browse the component; can't patch the +component), and the ongoing need to create additional proxy projects to +fill in for the missing prerequisites as real source projects get added +to the workspace. +<h3> +Document History</h3> +15:45 Wednesday September 5, 2001 - first version sent for comments +<br>13:30 Thursday September 6, 2001 - revised for first round of comments +<br>19:40 Wednesday September 12, 2001 - automation for unshared proxy +library projects +<br>10:15 Wednesday September 26, 2001 - added stub projects +<br>17:00 Wednesday September 26, 2001 - documented fatal flaw with stub +projects +<br> +</body> +</html> diff --git a/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure.html b/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure.html new file mode 100644 index 000000000..dd0c806f5 --- /dev/null +++ b/org.eclipse.jdt.core/notes/r2.0/workspace structure/ws-structure.html @@ -0,0 +1,448 @@ +<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> +<html> +<head> + <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> + <meta name="GENERATOR" content="Mozilla/4.75 [en] (Windows NT 5.0; U) [Netscape]"> + <title>JDT - Workspace structure</title> +</head> +<body> + +<h2> +Workspace Structure</h2> +Last revised 11:15 Saturday September 8, 2001 +<p>Original work item: "Make the build path less sensitive with regard +to whether a project is a source or a binary project. This is also related +to patching (i.e., adding a source folder to a binary project)." +<p>Related issue: support for extension directories containing many jars. +<p>There was much discussion. The underlying issue is how to structure +workspaces for significant Java development efforts. The Eclipse Project +is an example of such an effort, so the question is not merely of passing +interest. Since much of what is being developed in the Eclipse Project +are plug-ins, the question also touches on PDE's domain. However, some +aspects of the problem affect all significant Java development efforts +and not just ones for developing plug-ins. So we begin to investigating +those. +<h3> +Problem: Close Collaboration between Adjacent Teams</h3> +The following is an attempt to capture a familiar problem in a general +way that does not presume exactly how developers' workspaces are to be +structured. Consider the case of two components, with PUI dependent on +Core (imagine that Core is the core component, and UI is the corresponding +UI component built atop it). Now imagine that Core and UI are under active +development by two teams, with ownership split along component lines. For +maximum convenience to both teams, the teams would like to be able to do +the following: +<ul> +<li> +Each team would like to be able to work in a workspace that contains the +source code for their component and the binary equivalent (with attached +source) for all other components.</li> + +<li> +Occasionally (e.g., during debugging) each team would like an easy way +to workaround (or perhaps induce) problems in another component.</li> + +<li> +Occasionally each team would like an easy way for members to "join the +other team" and more actively work on another component.</li> +</ul> +Generalize from 2 to N the number of teams and components to get the full +extent of the problem. +<h3> +Problem: Large Numbers of Libraries</h3> +The following Eclipse Corner posting (Jon Skeet <skeet@pobox.com> on +20.8.2001) describes one concern well: +<blockquote><i>Referencing a project's libraries in another project: The +project I'm working on uses a fairly large number of libraries (about 20 +jar files, IIRC). I had hoped I'd be able to create one project which used +these libraries, and make all other projects have that project on their +build-paths to bring in the libraries - that way I wouldn't need to change +*every* project every time I added or removed a library. (It's not a common +operation, but even so...) Unfortunately, I can't get this to work - putting +project A on the classpath of project B only seems to put project A's individual +source/class files on the classpath for project B. I've tried adding as +an external jar and importing the jar file into the project, and had no +joy with either :(</i></blockquote> +Here is the arrangement: +<blockquote>project P1 +<br> build classpath = source /P1/src; library lib1.jar; +library lib2.jar; ...; library lib20.jar +<br> output /P1/bin +<br>project P2 +<br> build classpath = source /P2/src; library lib1.jar; +library lib2.jar; ...; library lib20.jar +<br> output /P2/bin</blockquote> +It is clear that the customer is looking for a way to deal with a set of +libraries that would be required by several projects, and was hoping that +required projects would give him that. +<h3> +Review: Required Projects</h3> +In Eclipse 1.0, the build classpath for a project P can contain an entry +for another project R, called a <b>required project</b>. Required projects +work as follows: +<ul> +<li> +The class files in the binary output directory of R are included, like +a library, on the effective build classpath of P. P's build classpath indicates +the order of this library relative to others. Other library (and project) +entries on the build classpath of R have no bearing on P; they are only +consulted to browse or build R.</li> + +<li> +The names of required projects are recorded in the .classpath file, which +is under VCM. They are therefore shareable with other developers.</li> + +<li> +Projects are presented as top level elements in the standard packages view. +A required project is not presented in the packages view as a child of +the project(s) that requires it.</li> + +<li> +A project's list of required projects (a JDT core notion) is used to computed +the default value for the project's list of project references (a workspace +notion).</li> + +<li> +Missing required projects and libraries, and unbound classpath variables, +generate problems reported against the project resource itself. These kind +of problems are detected by the Java model (rather than the Java builder).</li> +</ul> + +<h3> +Review: Classpath Entries</h3> +In Eclipse 1.0, a classpath is expressed as an array of classpath entries +(<tt>IClasspathEntry</tt>). There are four kinds of classpath entries: +<ul> +<li> +Source folder entry: denotes a project folder containing .java files. The +source folder must be contained inside the project where this classpath +entry is used. Source folder entries are generally used to componentize +the content of a given project (e.g. org.eclipse.jdt.core/Eclipse Java +Compiler/). The action of building a Java project will lead to populating +the project output folder with .class files corresponding to the .java +files which are contained in all the source folders present on this project +classpath. Note that the project root can itself be used as a source folder +(in which case the output folder is also the project itself).</li> + +<li> +Library entry: denotes a binary JAR archive or binary folder. In case of +pointing at a JAR, the entry can also define a source attachment recommendation +(path to source archive, and path of source root inside this archive, e.g. +"c:/jre/src.jar" + "/src").</li> + +<li> +Project entry: denotes a required project. When building, the required +project is built before the dependent project, and contributes its entire +output folder (i.e. all its produced .class files). Thus a project never +directly contributes its source files when building. However, for all source-based +functionalities (code assist, search, ...), a required project directly +contributes its sources (i.e. all of its source folders are exposed) so +that a build action is not mandatory to obtain accurate information; in +particular, when doing intensive code reorganization, code assist and search +will still perform accurately even if auto-build is turned off.</li> + +<li> +Variable entry: indirect reference to either a library or a required project. +A variable entry uses a variable path of the form %variableName%[%pathSuffix%]. +The variable name will be substituted with its actual value, which is a +workspace defined constant. The path suffix is an optional suffix which +can be appended to the variable value. PDE uses a global variable to represent +the Eclipse home directory, and path suffixes to reach the well-known libraries +(e.g. "ECLIPSE_HOME/org.eclipse.jdt.core/jdtcore.jar"). +Note that in a consistent manner with library entries, variable entries +can be provided with a variable source attachment recommendation.</li> +</ul> + +<h3> +Investigation: Make Classpath Variables More Powerful</h3> +Would making classpath variables more powerful address some the problems +facing large scale Java development efforts? +<p>One idea would be to allow variables to be bound to a list of paths +instead of a single path. This would allow, for example, a single classpath +variable "BASE_LIBS" to be bound to the list of the paths of the library +jars lib1.jar; lib2.jar; ...; lib20.jar. Each project referencing this +classpath variable would thereby gain access to all the libraries: +<blockquote>project P1 +<br> build classpath = source /P1/src; BASE_LIBS +<br> output /P1/bin +<br>project P2 +<br> build classpath = source /P2/src; BASE_LIBS +<br> output /P2/bin</blockquote> +Changing the workspace binding of the classpath variable affects all projects +that reference the variable. This makes it easy to change the set of libraries +without having to change the projects individually. +<p>Since classpath variable bindings are local to the workspace, there +is no obvious automatic way by which these classpath variables would get +their bindings. The bindings would have to be configured for each workspace; +loading bindings from a file, or initializing them via a script, are feasible +options.. +<p>The classpath variables currently bind to paths, and classpath entries +can contains paths that begin in a variable. Changing variables to bind +to a list of paths would be a major change, and would likely require reworking +much of existing API. We generally agreed that we would not pursue this +approach since it seems somewhat unlikely to solve much of the problem. +<h3> +Proposal: Explicitly Export Libraries from Required Projects</h3> +The proposal is to extend the required projects mechanism to allow a required +project to contribute more than just its binary output folder. Rather, +a project would be able to indicate that any of its libraries are to be +<b>exported</b>. +Exported libraries also become available to other projects in the workspace +that list this project as a required project. +<p>In the above example, the customer could instead have an arrangement +like: +<blockquote>project P1 +<br> build classpath = source /P1/source; project PLib +<br> output /P1/bin +<br>project P2 +<br> build classpath = source /P2/source; project PLib +<br> output /P2/bin +<br>project PLib +<br> build classpath = source /PLib/source; <b>export</b> +library lib1.jar; ...; <b>export</b> library lib20.jar +<br> output /PLib/bin</blockquote> +What this means is that when P1 (similarly, P2) is built, the libraries +on its build classpath consists of /PLib/bin, lib1.jar, ..., lib20.jar. +<p>The modified semantics of a project P with a required project R are +as follows: +<ul> +<li> +There is an export flag associated with each library explicitly included +on the build classpath of a project. This flag is meaningful for both library +classpath entries (both internal and external), and required projects (it +is not associated with source and output entries.) The export flags are +recorded in the .classpath file, which is under VCM. These flags are therefore +shareable with other developers. (These flags can be added in a way that +does not invalidate existing R1.0 .classpath files.)</li> + +<li> +The class files in the binary output directory of R, along with any libraries +explicitly exported on R's build classpath, are included as libraries on +the build classpath of P. P's build classpath indicates the placement of +this library relative to others; the output folder is always first, with +the exported libraries and projects included in the user-specified order. +Exported required projects are expanded inline. This ordering supports +scenarios where the output folder contains patches to the libraries. Non-exported +library entries and project entries on the build classpath of R have no +bearing on P.</li> + +<li> +The names of required projects are recorded in the .classpath file, which +is under VCM. They are therefore shareable with other developers. (Unchanged.)</li> + +<li> +Projects are presented as top level elements in the standard packages view. +A required project is not presented in the packages view as a child of +the project(s) that requires it; nor are the required project's exported +libraries.</li> + +<li> +A project's list of required projects (a JDT core notion) is used to compute +the default value for the project's list of project references (a workspace +notion). (Unchanged.)</li> + +<li> +Missing required projects and libraries, and unbound classpath variables, +generate problems reported against the project resource itself. These kind +of problems are detected by the Java model (rather than the Java builder). +(Unchanged.)</li> + +<li> +Classpath problems pertaining to libraries exported from a required project +are reported only once, against the project closest to the problem. The +problematic entries are elided from the effective classpath of dependent +projects.</li> +</ul> +Example: +<blockquote>project P +<br> build classpath = source /P/source; project R; library +plib.jar +<br> output /P/bin +<br>project R +<br> build classpath = source /R/source; <b>export</b> +library rlib1.jar; library rlib2.jar; <b>export</b> project Q; <b>export</b> +library rlib3.jar +<br> output /R/bin +<br>project Q +<br> build classpath = source /Q/source +<br> output /Q/bin</blockquote> +Effective build classpath of P: +<blockquote> build classpath = source /P/source; library +/R/bin; library rlib1.jar; library /Q/bin; library rlib3.jar; library plib.jar</blockquote> + +<h4> +Exporting External Libraries</h4> +Relative to a given project, a library is internal iff the path to the +jar or folder lies inside the project's resource tree. Other libraries +are considered external to the project. +<p>Should projects be allowed to export arbitrary libraries, or should +exported libraries always be internal to the project? +<p>Restricting exports to internal libraries ensures that a project that +is to be used by other projects is somewhat self contained. If you load +such a project from a repository, you are guaranteed all the libraries +it exports will be contained therein. +<p>On the other hand, unrestricted exports are more flexible, and allow +a project to export a collection of libraries that are not necessarily +contained within the project's resource tree. (However, it is unclear whether +this additional flexibility would be useful.) +<p>We opted to allow unrestricted imports, but decided to simplify the +error processing by only reporting problems against the project with the +missing library (or required project) explicitly on its classpath. Any +missing entries would simply be omitted from the effective classpath calculation.for +dependent projects. +<h4> +Classpath Variables</h4> +A project's build classpath may include references to classpath variables +that get bound to libraries (or other projects). How do exports and variables +interact? +<p>The export flag could be associated either with the variable reference +or with the variable binding. If the export flag is associated with the +variable reference, it would indicate exporting whatever library the classpath +variable happened to be bound to on a given occasion. If the export flag +is associated with the variable binding itself, the library the classpath +variable happened to be bound to on a given occasion would be exported +conditionally on the flag in the binding. The proposal is to go with the +former (export flag with variable reference) since its semantics are somewhat +simpler and allow the export flag to be shared with other team members +(the bindings of classpath variables are not shareable). +<h4> +Exporting Auxillary Libraries</h4> +Note that explicit exports also address another problem that arises when +a project needs to export a pre-built library in addition to the results +of compiling its source files (the Eclipse debugger's jdi.jar is a fine +instance of this). In Eclipse 1.0, only the class files in the project's +binary output folder are exported to dependent projects. This proposal +allows a project to export any number of additional libraries as required. +<h4> +Exporting Projects</h4> +By exporting other projects, a project can consolidate and concentrate +the outputs from several other projects:.. +<blockquote>project P1 +<br> build classpath = source /P/source; export library +/P1/lib1.jar +<br> output /P1/bin +<br>project P2 +<br> build classpath = source /P2/source; export library /P2/lib2.jar +<br> output /P2/bin +<br>project P1andP2 +<br> build classpath = source /P1andP2/source; export project +P1; export project P2 +<br> output /P1andP2/bin</blockquote> +Effective build classpath of P1andP2 includes everything exported from +both P1 and P2: +<blockquote>build classpath = library /P1andP2/source; library /P1/bin; +library /P1/lib1.jar; library /P2/bin; library /P2/lib2.jar +<br>output /P1andP2/bin</blockquote> + +<h3> +Proposal: Admit Library Projects which Contain No Source Code</h3> +One objection to this whole approach is that the Java project is being +hijacked. In the original design, a Java project is a buildable container +of Java source code. The addition of exported libraries starts to turn +the project into something more general. If we are comfortable with this +general trend, there are ways that the design can embrace it more whole-heartedly. +<p>The above proposal to expand the semantics of required project creates +a new role for a Java project as a container of libraries. This notion +of <b>library project</b> is rounded out by allowing Java projects without +source or output folders. The build classpath of a library project orders +the list of exported libraries and projects (and provides the additional +context required for browsing the project itself). +<p>In our example, the library project could be expressed more directly +without having to postulate source and output folders which are completely +unmotivated in this case: +<p>project PLib +<br> build classpath = <b>export</b> library lib1.jar; +...; <b>export</b> library lib20.jar +<p>A library project is recognizable simply by the absence of source folders +on the build classpath. (We considered adding a new flag to the project +to explicitly mark library projects, but decided this was not necessary +and would be confusing.) +<p>Note that library projects are not "buildable" in any meaningful sense +since they lack source code to compile (and without source code there is +no pressing need for a binary output folder). However, there still needs +to be a mechanism for detecting and reporting errors in the build classpath +for library projects. Except for such checks, the Java incremental project +builder should ignore library projects. +<p>The following kinds of entries on the build classpath of a library project +would serve these purposes: +<ul> +<li> +Source folders - a library project has no source folders, by definition.</li> + +<li> +Output folders - not needed for a library project.</li> + +<li> +Unexported libraries - provides context when browsing this project.</li> + +<li> +Unexported required projects - provides context when browsing this project.</li> +</ul> +However, the API (and UI?) should allow the information in all fields to +be maintained even for library projects, possibly to facilitate switching +a project between a regular and a library project. +<p>The workspace must compute an input delta even for incremental project +builders that ignore the delta they are handed. We considered whether we +should go one step further and remove the Java builder from a library project's +list of incremental project builders. This would make it crystal clear +that there is no Java building going on, and it would reduce workspace +memory footprint because the workspace would not need to remember the shape +of the resource tree at the time of the last build. However, if the builder +was removed for library projects, there would need to be mechanism to add +it if the classpath was changed to include a source project. We decided +that we should simply avoid the additional hassles and leave the Java builder +installed for all Java projects. +<h3> +Background</h3> +From Philippe Mulet 08/31/2001 12:13 PM +<p>I also think this has to be a basic mechanism at the JavaCore level, +which the PDE can then surface to plugin writers. +<br>I would like to see this one addressed asap, so that it is possible +to share projects transparently at least amongst us. +<p>======= +<p>From Erich Gamma on 08/31/2001 12:43 PM +<p>Here is some more input on our favorite topic "make the build class +path less sensitive +<br>with regard to whether a project is a source or binary project". +<p>Our conclusion in SNZ was that this boils down to a PDE project layout +issue (binary +<br>projects etc). However, the EC discussion from below illustrates a +use case for allowing to export +<br>JARs in addition to the output folder from a project, that is independent +of PDE. +<br>The scenario is similar to the WSAD scenario with extension dirs. If +we support to +<br>contribute contained JARs from a project then this problem would be +addressed. +<p>We should therefore reconsider the solution independent of the PDE issue. +<br>If I do that then I come to the conclusion that the proposed mechanism +is valuable +<br>and I suggest to go ahead with specifying it in detail. +<p>Thoughts? +<p>======= +<br>From Jon Skeet <skeet@pobox.com> on Eclipse Corner 20.8.2001 +<p>Referencing a project's libraries in another project: +<br>The project I'm working on uses a fairly large number of libraries +<br>(about 20 jar files, IIRC). I had hoped I'd be able to create one +<br>project which used these libraries, and make all other projects have +<br>that project on their build-paths to bring in the libraries - that +way I +<br>wouldn't need to change *every* project every time I added or removed +a +<br>library. (It's not a common operation, but even so...) Unfortunately, +I +<br>can't get this to work - putting project A on the classpath of project +B +<br>only seems to put project A's individual source/class files on the +<br>classpath for project B. I've tried adding as an external jar and +<br>importing the jar file into the project, and had no joy with either +:( +<h3> +Document History</h3> +[...] +<br>Revised 10:40 Wednesday September 5, 2001 - Library projects not explicitly +marked. +<br>Revised 11:15 Saturday September 8, 2001 - Allow exporting external +libraries and projects. +</body> +</html> |