FACT: The compiler is a complex beast.

Sometimes its hard to test that widget you've stuck five levels deep down in the 
compiler infrastructure, sometimes you want to ship some experimental function
that is only enabled in certain environments.

The solution is that we design the code for testability and configurability.

Q. What does this mean?

It means that throughout the complex are public fields or methods that are
exposed to enable some components to have their behaviour changed, from a
testcase or hosting environment (AJDT).  Andy has put a few of these in
over the last few months, and Adrian has just put one in - I don't think
we can get around the need for these things.

Q. How do I identify them?

I keep forgetting where I've put them, so from now on they are all marked
with a task tag of SECRETAPI.  When we have a little time to rest and
reflect on what we've been doing we should look at whether they can be
consolidated into an uber-control API module that centralises control of
all these secret buttons and widgets.

Q. Give me an example?

1. The structure model produced post build is a complex series of data
structures.  Incremental compilation is complex too.  Incremental
repairs to the structure model to ensure it stays up to date when
incrementally compiling are particularly complex so there is a flag
to turn on incremental repairs.

2. Testing what happens when processing incremental state is difficult
as there is no nice API to the state object that you can call at the
appropriate point in your test.  The StateListener instance that
can be set on the State object enables you to get callbacks when
certain things happen - for example if a change is detected on a 
classpath entry the compiler is looking at.  Doing it via an optional
listener means we don't have to record endless amounts of
information whilst state processing that would only be consumed
by the test infrastructure.

Q. Can I add them wherever I like?

NO! They are a last resort if you can't test something cleanly
*or* its so experimental you don't want to destabilise existing consumers
of the compiler.  For example the state management for incremental
compilation is currently configurable (defaults to off) as it has an
impact on memory usage and probably on other IDEs - when we've resolved
these issues we can look to making ON the default.  For any non-test
related secret APIs you should think about when they ought to be made
default !