Detect previously failed downloads by verifying existing file checksums
There was a helpful option hiding in Download Maven plugin, which we use
to download artifacts unavailable on Maven Central, such as the Ant
installer and several source packages: 'checkSignature'. It has the
effect of verifying checksums of existing, i.e. previously downloaded
files too, not only newly downloaded ones. This helps detect interrupted
downloads from previous runs or generally invalid files, whatever the
reason. I was looking for this option before, but did not notice it
because of the name. This is about verifying checksums, not checking
signatures. Anyway, a maintainer just told me about it here:
https://github.com/maven-download-plugin/maven-download-plugin/issues/186
I requested that the option be renamed and described better.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
This is a bugfix release, reverting the essential parts of commit
f70aeb5e, because it causes AspectJ Maven integration tests using
javadoc to fail on JDK 8.
See commit discussion on
https://github.com/eclipse/org.aspectj/commit/f70aeb5e#commitcomment-51417353
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Before, we used 1.9.7.BUILD-SNAPSHOT, which according to Andy Clement
was originally an intent across a group of Spring projects he was
involved in, to ensure that SNAPSHOTS were sorted alphabetically ahead
of MILESTONEs and ahead of RCs.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Create javadoc for all public artifacts, fix dependencies
Sonatype OSSRH repository rules require source and javadoc JARs in
order to create staging repositories for releases to be promoted to
Maven Central. So I added build steps to unzip the source JARs and then
create Javadocs for them.
FIXME: This configuration works with JDK 16, but throws errors on other
JDK versions, e.g. 14. It looks as if the Maven Javadoc plugin does not
do a particularly good job applying the plugin settings in a way making
it work with different JDK javadoc tool versions. I am saying that,
because when using the tool directly on the console, it works with basic
settings and the correct classpath.
In order to enable creating uber JARs via Maven Shade in the future, I
also added missing dependencies. Maven Assembly descriptors just assume
that all the necessary class files and sources already exist where it
copies them from. But several of the dependency modules were not
explicitly listed as such by the uber JAR modules. I fixed that.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
There is a strange effect in Maven builds: Depending on which profiles
are active when building the project - even seemingly unrelated ones
like 'create-docs' or 'clean-libs' - the execution order of plugins in
the 'process-resources' phase can vary. Specifically, Build Helper vs.
Enforcer in module 'lib', which both were in the same phase, can
sometimes be executed in lexical order, which I expected, or the other
way around, which makes the build fail if the existence of the marker
file is checked by Enforcer before Build Helper even had a chance to
create it. Probably, this is because Build Helper is defined inside a
profile and Enforcer outside of any.
Therefore, the safest way to ensure correct ordering of the two is to
place Enforcer in a later phase, in this case 'compile'.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
I tripped over not finding aspectjtools in my IntelliJ Maven view many
times, because it was listed as "AspectJ Compiler". So I renamed it to
"AspectJ Tools (Compiler)". Now it resembles more the artifact ID and
still retains the information that it is the artifact containing AJC.
For the 'lib' module I removed the 'name' tag again, because it is not
one of the main artifacts we publish. Now the POMs are more like Andy
might have intended them to be, using a human-readable 'name' only for
the main artifacts.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
In the previous GitHub build, there were warnings in the log because of
failed downloads. Actually, the default is to fail the build, but that
did not happen. Let us try a more recent version, maybe it fixes an old
bug, even though in the diff between the versions I did not see anything
obvious here.
Anyway, I created an issue ticket:
https://github.com/maven-download-plugin/maven-download-plugin/issues/185
BTW, our build only failed later during the Maven Enforcer sanity check,
because several files from the check list were missing after a seemingly
successful provisioning. Actually, I am glad I added this "redundant"
double-checking step to the build, otherwise the build would not have
failed in the 'lib' module but much later in a hard to detect spot.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Upon special request by Andy Clement, I included 'lib' as a child module
in the parent POM again, making several modules which refer to
downloaded library files dependent the 'lib' module. I am not sure I
caught all of them, but I hope so.
Now after cloning the project and configuring the token for reading from
GitHub Packages (sorry!), you can just run a Maven build for the main
project and no longer need to fail the first build, read the Maven
Enforcer message and run 'cd lib && mvn compile' as a first step. This
convenience comes at the price of a more complex POM and two new
profiles:
- Profile 'provision-libs' is auto-activated by the absence of a
marker file, kicking off the library provisioning process and
creating same marker file at the end, if successful. Therefore,
during subsequent builds libraries will not be re-provisioned,
because the marker file exists and Maven skips all download and
(un)zip steps, which saves build time and bandwidth. Otherwise
offline builds would not work either.
- Profile 'clean-libs' needs to be activated manually, because by
default 'mvn clean' will not erase provisioned libraries. In most
cases, even after a clean a developer does not want to re-provision
all libraries if they have not changed (e.g. new JDT Core build).
But if you do wish too erase the libraries and the marker file, just
call 'cd lib && mvn -P clean-libs clean'.
Please note: The Maven Enforcer build step, which additionally checks
for existence of other files, still exists and was moved from the parent
POM to 'libs'. No matter if provisioning was just done or skipped
because the main marker file exists, a quick heuristic check for that
list of files is done during each build, failing the build with a
comprehensive message if an inconsistency was found. The error message
says which files are missing and tells the user:
"There is an inconsistency in module subdirectory 'lib'. Please run
'mvn --projects lib -P clean-libs clean compile'. This should take
care of cleaning and freshly downloading all necessary libraries to
that directory, where some tests expect them to be."
This should cover the topic.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
The new string AjcTestCase.CLASSPATH_ASM_RENAMED dynamically determines
the 'asm-renamed' location from the classpath, system property
'java.class.path'.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Delete all remaining docbook contents, download them instead
Actually, only fop:fop:0.20.5 and batik:batik-1.5-fop:0.20-5 are really
used in addition to lib/saxon/saxon.jar (saxon:saxon:6.5.3). So the rest
does not need to be replaced and can just be wiped.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Delete docbook XSL + DTD directories from libs, download instead
It was kind of difficult to identify and find the vintage versions used
in AspectJ in download archives, but finally I managed to. Docs
generation looks good visually, tests to be run on GitHub CI.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Switch from 'libx' to 'lib', delete all obsolete binaries
Because 'cd lib && mvn compile' can now download and (un)zip many
previously SCM-committed third-party dependencies, the following 'lib'
subdirectories have been deleted:
- ant
- asm
- commons
- jarjar
- junit
- regexp
- saxon
This one is new (but not stored in SCM):
- jdtcore-aj
For each of them, there now is a .gitignore entry, so as to prevent
developers from accidentally committing the downloaded binaries again.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
In two places, the documentation now contains this text:
"Since AspectJ 1.9.7, the obsolete Oracle/BEA JRockit agent is no longer
part of AspectJ. JRockit JDK never supported Java versions higher than
1.6. Several JRockit JVM features are now part of HotSpot and tools like
Mission Control available for OpenJDK and Oracle JDK."
The decision to drop JRockit support was made during a discussion
between Alexander Kriegisch and Andy Clement:
Andy Clement wrote on 26 Mar 2021:
> Yes I think so.
>
>
> Alexander Kriegisch wrote on 26 Mar 2021:
>
>> https://en.wikipedia.org/wiki/JRockit
>>
>> Can we get rid of that? AspectJ requires Java 8, JRockit never
>> supported more than Java 6.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Download correct JDiff binaries and sources to 'libx'
This enables us to replace the original file from SCM. There is even an
improvement, like in other packages before: We create separate binary
and source archives, copying files from the compound download archive.
This way the library should be easy to use in an IDE.
TODO: This still does not get rid of the system path. Maybe it is better
to upload source and binary JARs to GitHub Packages.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Before, the Maven Clean configuration overrode the one from the parent
POM. Now it leaves it intact, adding a separate module-specific
execution to delete the downloads and libraries.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Module 'asm-renamed' now deploys to GitHub Packages
This means that instead of a system-scoped dependency we now have a
regular one.
The 'libx' module also downloads binary and source JARs redundantly into
the libraries directory in order to be found there by other scripts and
tests.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Replace JDT Core system dependency by deployed one
Get rid of system paths. Instead, rely on JDT Core Shadows to deploy
both binary and source JARs to GitHub Packages. The former module
directory was deleted completely. Instead, the JARs are redundantly
copied into 'libs/jdtcore-aj' in order to be found there by tests and
other Ant scripts.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Build libs without additional profiles, add Enforcer rule
Because 'libx' no longer is a submodule of the AspectJ parent POM, it
will not be built automatically each time an AspectJ build runs.
Therefore, is is no longer necessary to shield the zip/unzip steps from
repetitive execution by profiles with auto-activation based on the
(non-)existence of files. An AspectJ developer knows when to build the
module, she does it manually anyway.
A new Enforcer rule makes sure to warn the developer if some files it
expects to exist in the libs folder are not actually present.
Now we also have a Maven Clean rule which wipes away all downloaded and
(un-)zipped files.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Replace system-scoped dependency on commons by granular dependencies
There are only two direct dependencies used in AspectJ code:
- Commons Digester (module 'testing')
- Commons Logging (module 'org.aspectj.matcher')
I declared those two and experimentally removed all the other
system-scoped dependencies, as it should be. Let's see if the build
works with transitive dependencies.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Recreate lib/commons from Apache Commons downloads
Project archeology, binary and source code comparisons of contents in
lib/commons/commons.jar and lib/commons/commons-src.zip yielded the
following results:
- All binaries are available on Maven Central in 4 different legacy
Apache Commons dependencies:
* commons-beanutils:commons-beanutils:1.4
* commons-collections:commons-collections:2.0
* commons-digester:commons-digester:1.3
* commons-logging:commons-logging:1.0.1
- Those Maven Central binaries are not accompanied by source JARs,
i.e. in order to recreate lib/commons/commons-src.zip we have to
download source archives from the corresponding Git tags. All
projects are available on GitHub, so it is possible to download them
using Maven Download Plugin.
- Both the compound binaries and compound sources archives currently
checked in in AspectJ can be recreated using TrueZIP Maven Plugin.
This is rather tedious and involves additional Maven profiles in
order not to generate the compound archives during every build, but
fully implemented now.
Unfortunately, all of the above does not make the system-scoped
dependency on commons.jar obsolete. In order to achieve that, we either
have to publish the compound files on Maven Central or GitHub Packages,
or we find out which AspectJ modules use classes from which of the 4
individual Apache Commons packages and replace the compound system
dependency by the relevant single dependencies. Probably I am going to
try that in a next step.
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Recreate lib/ant from Apache source/binary downloads
- Download Ant 1.6.3 binaries and sources ZIPs from Apache releases
download server
- Verify expected SHA-1 checksums
- Unpack binary distribution
- Repack main sources into source package as it is checked in now
- Redundantly add JUnit JAR in order to 100% replicate existing
directory layout
- Move downloads from 'validate' phase to 'generate-resources'
- Unpack/repack phase is 'process-resources'
- Make sure that download, unpack, repack only occur if necessary
instead of overwriting existing artifacts during each build
Signed-off-by: Alexander Kriegisch <Alexander@Kriegisch.name>
Alexander Kriegisch
Andy Clement