Key Differences between JavaCC 21 and Legacy JavaCC

From the end user's point of view, the most important difference is that JavaCC 21 has undergone quite a bit of re-design to make it much more usable “out of the box” than the legacy JavaCC. One of the most basic (and obvious) things that JavaCC 21 provides is the INCLUDE statement. With legacy JavaCC, the only way to reuse commonly used constructs across different grammars was via the classic copy-paste antipattern.

It is quite clear that building an AST Abstract Syntax Tree is the most typical use case for such a tool. However, in legacy JavaCC, generating a parser that builds an AST is actually a rather baroque build process. You write a grammar with special “tree-buildling annotations” that you process with the JJTree tool, which is really a pre-processor that in turn generates a JavaCC grammar. Then you run JavaCC on that to generate your Java source code.

With JavaCC 21, the JJTree pre-processor functionality is merged into the JavaCC tool and the generated parser simply builds an AST by default. (N.B. You can still generate a parser that does not automatically build an AST, but you need to specify that via TREE_BUILDING_ENABLED=false in the settings. There is no need to specify any special annotations. The tool generates the various classes that represent the nodes in the AST following some common-sense conventions.

• The tree building functionality has been enhanced considerably compared to what was available in JJTree. In particular, the generated Token class now implements the Node interface. So, optionally, Tokens (both regular tokens and special tokens) may be added to the generated parse tree. See Tree Building Enhancements for more info

• There has been an effort to clean up the set of configuration options. In general, the philosophy of JavaCC 21 is to make configuration options largely unnecessary, at least for typical usage, since the defaults are set sensibly and, in the absence of configuration settings, the tool simply infers naming via conventions. See JavaCC 21 Conventions for more information.

• JavaCC 21 introduces a new statement called INJECT that allows you to "inject" code into the files that the tool generates. This can help you to avoid the error-prone anti-pattern of generating code and editing it afterwards. See Code Injection for more information.

• FreeCC introduced a new INCLUDE statement which allows you to break your grammar into more than one physical file. See The Include Statement for more information.

• As of the latest 0.9.4 release (2019-12-28) FreeCC supports Java up to Java 8, including Lambda expressions. Since the Java.freecc is embedded in FreeCC.freecc using FreeCC's INCLUDE mechanism, that Java grammar is usable on its own. Note that this grammar successfully parses all the Java source code in the JDK 1.8, as well as all the source code in JRuby, Jython, and FreeMarker. So, if anybody needs a Java code parser for use in their own projects, this is quite usable!