Build and Test AspectJ

This describes how to build and test AspectJ for developers working on source code for AspectJ. It covers building with Ant or Eclipse and testing with JUnit and the harness used for compiler tests. For information on how the build works and how to debug failed builds, see readme-build-module.html.

Quick start
Requirements
Standard builds
1. Building using Ant
2. Building with Eclipse
3. Running the Ant build scripts from Eclipse
4. Using Eclipse to compile but Ant to assemble
Running build products
1. Running the compiler, browser, or harness from the command-line
2. Running the compiler, browser, or harness from Eclipse
3. Running Ant-built jars from Eclipse
Testing AspectJ
1. Running JUnit tests in Eclipse
2. Running JUnit tests from the command-line without Eclipse
3. Running JUnit tests from Ant without Eclipse
4. Using the test harness to run compiler tests
Releases
1. Release builds
2. Release preconditions and testing
3. Release completion
New modules, Java 5, and Ant-only build problems
Build Problems

Quick start

This is a minimal introduction to building and testing AspectJ.

Command-line users use CVS to check out something like this:

  export CVS_ROOT=":pserver:anonymous@dev.eclipse.org:/home/technology"
  cvs co org.aspectj/modules

If using Eclipse, check out the subdirectories of org.aspectj/modules as Eclipse projects. Skip the aspectj-attic module.

Build an AspectJ distribution:

  cd org.aspectj/modules/build
  ../lib/ant/bin/ant

Install the distribution (e.g., into build/../aspectj-install):

  java -jar ../aj-build/dist/aspectj-DEVELOPMENT.jar

You can skip the GUI by specifying an existing, empty writable target directory using -to {targDir}:

  java -jar ../aj-build/dist/aspectj-DEVELOPMENT.jar -to .

Test it by running the build script in the examples directory:

  cd ../aspectj-install/doc/examples
  ../../lib/ant/bin/ant

This should build and run the spacewar example.

Required sources, libraries, and tools

To build requires only the AspectJ project modules and Java VM's. All necessary libraries and tools are in the lib directory, including Ant.

To get the source, check out from CVS. E.g., from the command line,

   export CVS_ROOT=":pserver:anonymous@dev.eclipse.org:/home/technology"
   cvs co org.aspectj/modules

Eclipse users should check out each relevant subdirectory of org.aspectj/modules as a project (most have .project files).

Not all modules are required. The aspectj-attic module only has old code.

Standard builds

Overview

The build system is designed to work standalone and support Eclipse and Ant. Each module is an Eclipse project, and all required libraries are in lib/. All build dependencies are read from Eclipse .classpath files and managed using an Ant builder task, so no Ant build scripts need to change when dependencies change. However, that means the modules (read: Eclipse projects) can only do what the builder task understands. Currently it expects release source files in src (or java5-src for Java 5 code) and test source files in testsrc (or java5-testsrc), and can handle Java and AspectJ projects. For more information on the build infrastructure and setup, see readme-build-module.html and below.

Building using Ant

You should use the Ant in ../lib/ant. Currently this is a vanilla distribution of Ant 1.6.3, but that might change.

This build directory has a master build.xml script, and the modules have satellite build.xml which support building only that module. The release directory has scripts for doing release builds (in multiple VM's, with testing). These are run by automated build and before any AspectJ distribution. Custom build.xml files are in The ../docs/build.xml, org.aspectj.lib/build-aspectjlib.xml, and ../eclipse.plugin/build.xml. These are invoked by the master build.xml as needed. Other modules have generic build.xml's that just use the master build file to run targets compile and test for that module. (If you do clean, all binaries for all modules are cleaned.)

The master build.xml creates an AspectJ distribution in ../aj-build/dist/aspectj-DEVELOPMENT.jar ; You'll notice module jar results are put in ../aj-build/jars, including {module}.jar and {module}-test.jar (for the release and test classes) and {module}-all.jar, and {module}-test-all.jar (including all antecedant classes and libraries). See build.xml for other targets.

For any build, you should create your own version of local.properties, using sample.local.properties as a template. When using the master build.xml, consider defining "build.config" to pass flags to the Ant builder. E.g., to reuse Eclipse classes and log verbosely,

	ant -Dbuild.config=useEclipseCompiles,verbose

(useEclipseCompiles is incompatible with release builds since it mixes testing and release classes.)

Example builds

For example, to build everything into a release bundle, with verbose logging:

  cd modules/build
  ../lib/ant/bin/ant

To build only the asm module (and any modules it requires) using modules/build:

  cd modules/build
  ../lib/ant/bin/ant -f build.xml any-module -Dmodule.name=asm

To build and test the asm module from that module:

  cd modules/asm
  ../lib/ant/bin/ant test

To build the test harness into ../aj-build/jars/testing-drivers-all.jar:

  cd modules/build
  ../lib/ant/bin/ant build-harness-jar

Building with Eclipse

As mentioned above, the modules are Eclipse projects, so once checked out, they should build as-is. (You will need to be using a version of Eclipse that can handle the .project and .classpath files in the modules/projects.) That will enable you to run the compiler or test harness from within Eclipse (see below), but it will not build the AspectJ release as Ant does. If you are making changes in Eclipse, you should set your default JRE to the minimum supported by the AspectJ tools (currently JDK 1.3) to avoid using later API's. You'll need to set some variables in your Eclipse environment; currently these are:

JAVA_HOME: used to access JAVA_HOME/lib/tools.jar
JRE15_LIB: used to access Java 5 libraries. Some modules/projects use Java 5 API's. These have per-project compiler settings for 5.0 compliance, but cannot be built with the default pre-1.5 VM libraries. Until the minimum VM required by the tools is 1.5, we'll need to define this variable.

Running the Ant build scripts from Eclipse

When running Ant from older versions of Eclipse, be sure to replace the Eclipse Ant libraries with ours. In the Ant configuration, remove all jars specified by Eclipse and add all the libraries in ../lib/ant/lib as well as in ../lib/junit. (Do not add ../lib/build/build.jar, which is added via a taskdef declaration.)

If you find on rebuilding that the build products are not being regenerated, you may need to manually delete them or restart eclipse (the files are not being closed); see readme-build-module.html for more information.

Using Eclipse to compile but Ant to assemble

As mentioned above, assuming Eclipse is compiling the AspectJ modules successfully, you can use Ant to assemble the Eclipse-built .class files into a product by including useEclipseCompiles in the build.config. That reduces the build process to product assembly. (And of course you can run Ant from Eclipse as described above.)

Running the compiler, browser, or harness from the command-line

The build produces jar files in ../aj-build/jars/, some of which have manifests specifying the main class, so they can be run using java -jar {file} {arguments}.

To run the compiler from the command-line, use the ajbrowser jar file:

   java -jar aj-build/jars/ajbrowser-all.jar {compile arguments}

This will run ajbrowser if you provide no arguments or only (unflagged) .lst file arguments. To run the test harness, use the testing-drivers jar file:

   java -jar aj-build/jars/testing-drivers-all.jar tests/ajcTests.xml ...

Running the compiler, browser, or harness from Eclipse

To run things within Eclipse, create a run configuration from the defining module using the main class:

Program Module Main

AspectJ compiler org.aspectj.ajdt.core org.aspectj.tools.ajc.Main

AspectJ browser ajbrowser org.aspectj.tools.ajbrowser.Main

Test harness testing-drivers org.aspectj.testing.drivers.Harness

Program	Module	Main
AspectJ compiler	org.aspectj.ajdt.core	org.aspectj.tools.ajc.Main
AspectJ browser	ajbrowser	org.aspectj.tools.ajbrowser.Main
Test harness	testing-drivers	org.aspectj.testing.drivers.Harness

Running Ant-built jars from Eclipse

You can run build products (built jars) from Eclipse in two ways:

Put them on the classpath of some run configuration
Select the jar files and right-click to "open with default editor" (assuming your system is configured to run .jar files)

You might do this to run the installer or test the browser as built. However, doing so might prevent those jar files from being recreated in the next build. It appears that sometimes these jar files are not close during the Eclipse session, which means they cannot be overwritten in new builds, even those run from a different Ant process. If you find that builds are silently failing, try deleting the build products.

Testing AspectJ

Each module has a tree of JUnit tests in the testsrc directory. These parallel the src directories and contain roll-up suites for each package ({module}/testsrc/{packagePath}/{package}Tests.java) and for the module as a whole ({module}/testsrc/{module}ModuleTests.java).

The AspectJ project also has additional custom tests in the tests module, mainly the compiler tests run by the harness in ajcTests.xml. It is important to run these additional compiler tests (not covered by the JUnit suite) before and after any change to the compiler.

The module run-all-junit-tests refers to all the other modules and the compiler tests adapted through the ../tests/src. So the easiest way to run the JUnit and compiler tests is to run the top-level suite RunTheseBeforeYouCommitTests.

Running JUnit tests in Eclipse

JUnit tests may be run under eclipse by selecting any JUnit source file and creating a run configuration for it, including the roll-up root test RunTheseBeforeYouCommitTests.

Running JUnit tests in Ant from the command line

To run the JUnit tests for any module, just go there and

../lib/ant/bin/ant tests

Running JUnit tests from the command-line without Eclipse or Ant

To run JUnit directly, put JUnit and the {module}-test-all.jar file on the classpath:

	set cp="lib/junit/junit.jar;aj-build/jars/util-test-all.jar"
	java -classpath "$cp" junit.textui.TestRunner UtilModuleTests

The assembled (-all) jar files include all antecedants, except for those "skipped" (like Ant and JUnit). (For more on skipped libraries, see readme-build-module.html).

Using the test harness to run compiler tests

The build.xml build-harness-jar target builds a single jar with the AspectJ binaries and a test harness as the main class. It reads test suite files like ../tests/ajcTests.xml; use the -help flag to see available options. For more information, see ../tests/readme-tests-module.html.

Releases

Release builds

Committers do official release builds to create the distribution released in binary form from the web site. Release builds differ mainly in running from a clean, up-to-date tree and with correct build version values in local.properties, which will update org.aspectj.bridge.Version. Do not run using the build.config value useEclipseCompiles, because this will include testing classes in the release libraries. See Version synchronization below for more details on how the version is updated.

Release preconditions and testing

Normally, we do releases only after fixing all high-priority (P1 and P2) bugs in the bug database ( All open AspectJ bugs with P1 and P2). For bug fixes, associated tests in tests/ajcTestsFailing.xml are fixed and moved to tests/ajcTests.xml.

Before a release, run the release tests as described in ../tests/readme-release-tests.html (deprecated? using: ../build/release-checklist.txt).

Release completion

When the release build is accepted, tag the tree with the release version so others can do diffs or create patches based on the release code. E.g., from the command line:

   cd org.aspectj/
   cvs tag -R -c v1_1_0

Pushing the release out to the web involves manually updating the web with the release files and documentation, verifying the downloads and pages, and sending any release notifications. Save the release installer, test results, and any notes about deferred bugs or tests in org.aspectj/releases/aspectj-{version}/.

New modules, Java 5, and Ant-only build problems

To reiterate (and as described more fully in readme-build-module.html), the build system introspects on module and the .classpath file to determine how to build. Further, it enforces stricter build invariants than the Eclipse projects do; when your code works in Eclipse but fails in the Ant, it's your responsibility to fix it by following the build invariants below.

What to do when adding a new module (i.e., eclipse project):

Setup like the other modules, following the invariants below, esp.
1. same directory level
2. same naming conventions for source folders
3. same use of known classpath variables/entries
Add module to those checked for copyright/license in testsrc/org/aspectj/build/BuildModuleTests.java
Add module to module dependency tree as usual in Eclipse project properties
If the module is to produce a jar visible in {AspectJ}/lib, then add a zero-length file of the correct name to products/tools/dist/lib and an alias to that file from the module name in Builder.properties
To make the JUnit tests visible to the Ant builder, include a top-level Ant suite titled {Module}ModuleTests.java
Test the build in Eclipse/JUnit using testsrc/org/aspectj/internal/build/BuildModuleTest.java

How the build system introspects, and the build invariants that are enforced by the Ant build:

A given module is identified by its directory name.
Most modules are Eclipse Java projects; some are AspectJ projects. Eclipse is not required to build.
The file {module.name}/.classpath is read to determine the source directories and required projects and libraries.
Normal Java source files are in src and testsrc directories.
Java 5 source files are in java5-src and java5-testsrc directories.
Neither testsrc directory is used in production builds.
This only requires what's in the ../lib directory and the usual Java VM's. That means modules can only depend on each other and the libraries in ../lib.
Modules are built into jar files. {module.name}.jar includes only the module classes, and {module.name}-all.jar includes the module classes plus any antecendant modules and any required libraries not skipped (see below).
A file {module.name}/{module.name}.mf.txt, if available, will be used as the jar file manifest, after replacing certain variables in the .mf.txt file.
All required libraries are included in the production jars, except for testing libraries and those "skipped" in Builder.properties (Ant, JUnit, etc.).
Test classes are excluded from the compile process for production builds. These result in the same jar files; there is no way to tell externally whether the test classes were stripped from a jar.
ONLY resources specified as resource.pattern in Builder.properties are included with assembled jars.
No testing-* module is used in production builds, directly or indirectly.
It is an error for any code to depend on any testsrc code, whether in the same or another module.
AspectJ projects are built in Ant using ../lib/aspectj.
Java 5 code is not supported in AspectJ (yet).
Java 5 code is built in Ant only if JAVA_HOME points to Java 5 (or later).
No normal code can depend on any Java 5 code. The build system checks by building everything using the lowest VM supported by the tools.

That means there are a number of legal Eclipse projects and code arrangements which are not valid AspectJ modules. When you add modules, libraries, or dependencies, you must build with Ant to verify these invariants are not broken.

Build problems

Some build problems and fixes encountered in the past:

If the build works in Eclipse, but fails in Ant, do not assume the build scripts are broken. See Invariants enforced only in Ant.
If your compiles fail because Ant cannot find javac, put the JDK bin directory on your PATH and/or define the JAVA_HOME environment variable. Ant requires the path to the javac executable when the BuildModule taskdef runs. I think it either gets it from $JAVA_HOME or if the bin directory is on the PATH.
If using an IBM JDK at version 1.4 or higher within Eclipse, then the default value of JRE_LIB will point to the jar file "core.jar." Unlike the Sun JDK, IBM does not package the full runtime into a single jar ("rt.jar"), but instead has multiple jar files. Since core.jar does not contain the graphics libraries, several of the AspectJ projects will fail to build as checked out of CVS - there are missing dependencies on AWT and swing. The solution is to add graphics.jar to the project classpaths... BUT if this is added as an ordinary (external jar), then the main ant build script will pick up the contents of graphics.jar and include it in the aspectj distribution (obvious clue, the size of the built aspectjtools.jar doubles to about 10MB). Instead, from the Java Build Path properties page of the project, select "Add Library" and add the JDK library. You now have to remove the JRE_LIB entry from the project or Eclipse complains about duplicate jar files in the path.
More generally, if you find extra classes in the assembled jars, most likely you have added unexpected libraries to the build, e.g., when adding libraries for the IBM JRE. To skip those, add them to the "skip.libraries" list in src/org/aspectj/internal/tools/build/Builder.properties. These libraries will always be skipped for any module built this way; there is no way to include a library in one module but not another. For more information on the properties file, see readme-build-module.html.