Here is a list of settings that exist in JavaCC 21 but are not (and never were) present in the legacy JavaCC tool:
BASE_SRC_DIR This supersedes the older OUTPUT_DIRECTORY setting. Files are generated relative to the BASE_SRC_DIR, i.e. taking into account the package naming. If this is unset, BASE_SRC_DIR is assumed to be the directory where the grammar is.
With this setting turned on (it is off by default) the generated parser ensures that the input file ends with a newline character. (It tacks one on if it is not present.) This is a nitpicking detail but it is surprisingly difficult to write certain grammars (ones that are very line-oriented) if you cannot be sure that every line (including the last one!) ends with a newline.
This turns on the experimental support for building a fault tolerant
parser. It is off by default.
Since we believe that the normal usage of the tool is simply to build a tree, it makes little sense to have any qualms about reading in the entire input into memory. So this is the default. This option allows you to turn on the legacy behavior of only maintaining a (fairly small) buffer in memory of the input file. See The Gigabyte is the new Megabyte
for more information on the reasoning behind all this. Note that the experimental fault-tolerant parsing features only work with HUGE_FILE_SUPPORT off. Also, having TREE_BUILDING_ENABLED set to true (which is the default) means that HUGE_FILE_SUPPORT is automatically turned off.
If you turn on this setting, the tool generates code that is more compatible with legacy JavaCC. One example is that JavaCC 21 removes publicly visible fields like Token.kind and Token.image and replaces them with getter/setter methods. If you have LEGACY_API
set, it leaves these fields as publicly visible. Also, it generates static final int constants for your various token types, as a convenience to keep older code working. (JavaCC 21 uses type-safe enums in these cases.) Note, however, that the existence of this setting is not guaranteed to keep all older code working. It simply makes it less work to get legacy JavaCC code working with JavaCC 21. Projects that migrate to JavaCC 21 should, as soon as they reasonably can, refactor their code so that the LEGACY_API
setting can be turned off. In other words, it is just meant to provide a temporary stopgap, not any sort of permanent solution for people migrating their projects.
PRESERVE_LINE_ENDINGS This is true by default (though this could change in the future based on user feedback. If you turn this setting off, all Windows/DOS style line endings (\r\n) are converted to UNIX/MacOS style (\n) internally when the file is read in. Note, by the way, that one advantage of this and the TABS_TO_SPACES option is that if you convert tabs to spaces and line endings to \n then your grammar's lexical specification can be a bit simpler. And your own code that runs over Tokens and Nodes. Your code can just assume that any line endings are a simple \n and and your horizontal whitespace is just spaces, not a mix of tabs and spaces, independently of what platform the generated code is running on.
SMART_NODE_CREATION This is the default behavior, so you would have to explicitly turn it off. It means that if no JJTree-style tree-building annotation is used, then a new Node will be created if there are more than one Nodes on the stack. So, a production like
A (B)* will create a new Node if there are one or more B's after the A. If there is only an
A then the production will just leave it on the stack. It is our belief that this is the behavior that most people would want most of the time.
SPECIAL_TOKENS_ARE_NODES This sets whether to add so-called “special tokens” to the AST. By default, it is set to false. (Note that this option and TOKENS_ARE_NODES are meaningless if TREE_BUILDING_ENABLED is set to false.)
TABS_TO_SPACES This is an integer (typically from 1 to 8, in practice) that defines how many spaces a tab stop is. This is off by default, but if you use this setting, all TAB characters (\t) are converted to spaces when the file is read in. Note that, if you do not have this turned on, all reported error locations simply treat a tab character as one horizontal offset. If you want JavaCC to report errors as if a TAB stop is 4 spaces, say, you need to set TAB_SPACES=4 in your settings.
TOKENS_ARE_NODES This sets whether we add Tokens as terminal nodes to the AST. By default, it is true.
See deprecated settings for a list of legacy JavaCC options that no longer exist in JavaCC 21.