#summary The new design for compilation units as of GWT 1.5
#labels Phase-Design

= How Compilation Units are Managed in the GWT Infrastructure =
== Overview == 
In GWT, both running hosted mode and compiling for web mode share infrastructure for building {{{TypeOracle}}} and managing Generators.  Up to now we have not had a good name or well-defined boundaries for this viscera, but hopefully this design doc will shed some light on things related to the {{{CompilationUnit}}}.  All doc is circa GWT 1.5 RC1, when a big refactoring effort over all this infrastructure occurred, code named "Flaming Sword of Death".

_*GRAVEYARD* is not yet implemented, all generated units are killed on refresh.  Associated sections are italicized._

== Goals ==
  * Read Java source files off the classpath based on module source inclusions
  * Compile included Java source using JDT
  * Perform custom error checking for GWT-specific constructs
  * Log any compilation errors
  * Build and maintain a {{{TypeOracle}}}, which reflects the full body of compilation units
  * Assimilate new compilation units from Generators
  * Produce Java class files for use in hosted mode
  * Efficiently refresh from disk (important for hosted mode refreshes)
  * Provide compiled class files for hosted mode execution without additional compilation

== Glossary of Moving Parts ==
==={{{CompilationState}}}===
  Represents the total set of {{{CompilationUnits}}} for a module; a member of a GWT module.  The {{{CompilationState}}} is reponsible for managing the set of {{{CompilationUnits}}} and ensuring orderly state transitions.  The {{{CompilationState}}} also manages the module's {{{TypeOracle}}}.

==={{{CompilationUnit}}}===
  The central piece of the whole puzzle.  A {{{CompilationUnit}}} represents a single Java source file, but includes all accumulated state and output as compilation and validation proceeds.  Subparts include:
    * the parsed JDT AST for the {{{CompilationUnit}}}
    * a set of {{{CompiledClasses}}}, which contain class-specific artifacts

==={{{CompilationUnit.State}}}===
  Any given {{{CompilationUnit}}} can be in exactly one of several discreet states.  Here are the possible states and associated invariants:
  * *FRESH*: All internal state is cleared; the unit's source has not yet been compiled by JDT.
  * *COMPILED*: In this intermediate state, the unit's source has been compiled by JDT.  The unit will contain a set of {{{CompiledClasses}}}.
  * *ERROR*: In this final state, the unit was compiled, but contained one or more errors.  Those errors are cached inside the unit, but all other internal state is cleared.
  * *CHECKED*: In this final state, the unit has been compiled and is error free.  Additionally, all other units this unit depends on (transitively) are also error free.  The unit contains a set of checked {{{CompiledClasses}}}.  The unit and each contained {{{CompiledClass}}} releases all references to the JDT AST.  Each class contains a reference to a valid JRealClassType, which has been added to the module's {{{TypeOracle}}}, as well as byte code, JSNI methods, and all other final state.
  * _*GRAVEYARD*: A *CHECKED* generated unit enters this state at the start of a refresh.  If a generator generates the same unit with identical source, the unit is immediately promoted to *CHECKED*, bypassing costly compilation, validation, and {{{TypeOracle}}} building._

==={{{CompiledClass}}}===
  Represents a single compiled Java class; a member of a {{{CompilationUnit}}}.  Subparts include:
  * the parsed JDT AST for the class
  * a reference to a JRealClassType in the module's {{{TypeOracle}}}
  * the actual byte code of the class file
  * a set of JSNI methods associated with the class

==={{{TypeOracleMediator}}}===
  Helper class to handle the details of reflecting the JDT AST nodes provided by the {{{CompilationState}}} as a {{{TypeOracle}}}.

==={{{JdtCompiler}}}===
  Helper class to handle the details of transforming Java source files into both JDT AST nodes and compiled Java byte code.

==={{{JavaSourceOracle}}}===
  An abstraction that provides a single unified view of all source files available to the current module.  Implemented on top of a {{{ResourceOracle}}}.

==={{{ResourceOracle}}}===
  A classpath abstraction that provides a unified view of all resources available on the Java classpath given a set of included packages and optional filters.  This is used for both source files and public files, and the set of included packages is determined by the module's source and public declarations.

== Class Connection Diagram ==

http://google-web-toolkit.googlecode.com/svn/wiki/TypeOracleCompilationDesign.png

== Algorithm Overview for Various Use Cases ==
===#1: {{{ModuleDef}}}.getTypeOracle() called the first time===
  # {{{CompilationState}}} compiles
  # {{{JdtCompiler}}} compiles all *FRESH* and *ERROR* units; those units become *COMPILED*
  # All valid type names for classes with corresponding source code are recorded for validation
  # Any units with JDT errors transition to *ERROR* state; all errors are logged
  # Custom validation is performed over *COMPILED* units, recording new errors
    # JSO restrictions checked using JDT AST
    # Primitive long access from JSNI methods checked using JDT AST
    # References to binary types checked using set of valid source types and JDT AST
    # Any units with custom errors transition to *ERROR* state; all errors are logged
  # Any units (transitively) depending on *ERROR* units are invalidated and become *FRESH*
    # Each *COMPILED* unit lazily computes its direct references to other units
    # All units depending on *FRESH* or *ERROR* units are transitively invalidated and become *FRESH*
    # Each unit that is invalidated generates an error log message
  # JSNI methods are collected from each unit that remains *COMPILED*
  # {{{TypeOracleMediator}}} builds a JRealClassType for each class that belongs to a *COMPILED* unit, and places that JRealClassType into {{{TypeOracle}}}
  # All *COMPILED* units become *CHECKED*
    # Unneeded internal state (such as references to JDT's AST) are now cleared

===#2: {{{ModuleDef}}} wants to refresh===
  # {{{ResourceOracle}}} for source path refreshes against file system
  # {{{JavaSourceOracle}}} refreshes against {{{ResourceOracle}}} if {{{ResourceOracle}}} state changed
  # {{{CompilationState}}} refreshes against {{{JavaSourceOracle}}}
    # _All *GRAVEYARD* units are removed_
    # _All generated units become *GRAVEYARD*_
      # _Any contained JRealClassTypes are removed from {{{TypeOracle}}}_
    # Any source files whose corresponding file was changed in {{{JavaSourceOracle}}} are invalidated and become *FRESH*
    # All units whose corresponding file was removed from {{{JavaSourceOracle}}} are removed
    # Any new source files in {{{JavaSourceOracle}}} have new {{{CompilationUnits}}} created
    # All units depending on *FRESH* units are transitively invalidated and become *FRESH*
  # The sequence of events in use case #1 occurs

===#3: Newly generated units need to be assimilated===
  # A set *FRESH* {{{CompilationUnits}}} contained generated source are added to the {{{CompilationState}}}
  # _For each new unit, if a *GRAVEYARD* unit with the same type name already exists, the source of the new unit is compared to the source of the existing unit._
    # _If the new unit's source is identical to the old unit's source, the old unit becomes *CHECKED* and the new unit is discarded_
      # _All JRealClassTypes in the contained {{{CompiledClasses}}} are re-added to {{{TypeOracle}}}_
    # _If the new unit's source differs from the old unit's source, the old unit is discarded and the new unit is added_
  # If any *FRESH* units remain, the sequence of events in use case #1 occurs

===#4: A {{{CompilationUnit}}} becomes invalid===
  # The reference to the JDT cud is removed
  # The set of cached references to other units is cleared
  # For each {{{CompiledClass}}}
    # The contained JRealClassType is removed from {{{TypeOracle}}}
    # All internal state is cleared
  # The set of {{{CompiledClasses}}} is cleared