User Tools

Site Tools


new_settings_in_javacc_21

Here is a list of settings that exist in JavaCC 21 but are not (and never were) present in the legacy JavaCC tool:

  • BASE_SRC_DIR This supersedes the older OUTPUT_DIRECTORY setting. Files are generated relative to the BASE_SRC_DIR, i.e. taking into account the package naming. If this is unset, BASE_SRC_DIR is assumed to be the directory where the grammar is.
  • DEACTIVATE_TOKENS This setting allows you to indicate that certain token types are de-activated by default when you instantiate the parser. Something like: DEACTIVATE_TOKENS=OPEN_PAREN,CLOSE_PAREN; would mean that those two tokens are not active by default when you instantiate the parser. Of course, you could use ACTIVE_TOKENS(….) to activate them when needed in the parse.
  • ENSURE_FINAL_EOL With this setting turned on (it is off by default) the generated parser ensures that the input file ends with a newline character. (It tacks one on if it is not present.) This is a nitpicking detail but it is surprisingly difficult to write certain grammars (ones that are very line-oriented) if you cannot be sure that every line (including the last one!) ends with a newline.
  • EXTRA_TOKENS This setting allows you to indicate some additional token types that are not defined with regular expressions in the lexical grammar. This can be useful particularly in token hook routines.
  • FAULT_TOLERANT This turns on the experimental support for building a fault tolerant parser. It is off by default.
  • LEGACY_API If you turn on this setting, the tool generates code that is more compatible with legacy JavaCC. One example is that JavaCC 21 removes publicly visible fields like Token.kind and Token.image and replaces them with getter/setter methods. If you have LEGACY_API set, it leaves these fields as publicly visible. Also, it generates static final int constants for your various token types, as a convenience to keep older code working. (JavaCC 21 uses type-safe enums in these cases.) Note, however, that the existence of this setting is not guaranteed to keep all older code working. It simply makes it less work to get legacy JavaCC code working with JavaCC 21. Projects that migrate to JavaCC 21 should, as soon as they reasonably can, refactor their code so that the LEGACY_API setting can be turned off. In other words, it is just meant to provide a temporary stopgap, not any sort of permanent solution for people migrating their projects.
  • PRESERVE_LINE_ENDINGS This is true by default (though this could change in the future based on user feedback. If you turn this setting off, all Windows/DOS style line endings (\r\n) are converted to UNIX/MacOS style (\n) internally when the file is read in. Note, by the way, that one advantage of this and the TABS_TO_SPACES option is that if you convert tabs to spaces and line endings to \n then your grammar's lexical specification can be a bit simpler. And your own code that runs over Tokens and Nodes. Your code can just assume that any line endings are a simple \n and and your horizontal whitespace is just spaces, not a mix of tabs and spaces, independently of what platform the generated code is running on.
  • SMART_NODE_CREATION This is the default behavior, so you would have to explicitly turn it off. It means that if no JJTree-style tree-building annotation is used, then a new Node will be created if there are more than one Nodes on the stack. So, a production like A (B)* will create a new Node if there are one or more B's after the A. If there is only an A then the production will just leave it on the stack. It is our belief that this is the behavior that most people would want most of the time.
  • SPECIAL_TOKENS_ARE_NODES This sets whether to add so-called “special tokens” to the AST. By default, it is set to false. (Note that this option and TOKENS_ARE_NODES are meaningless if TREE_BUILDING_ENABLED is set to false.)
  • TABS_TO_SPACES This is an integer (typically from 1 to 8, in practice) that defines how many spaces a tab stop is. This is off by default, but if you use this setting, all TAB characters (\t) are converted to spaces when the file is read in. Note that, if you do not have this turned on, all reported error locations simply treat a tab character as one horizontal offset. If you want JavaCC to report errors as if a TAB stop is 4 spaces, say, you need to set TAB_SPACES=4 in your settings.
  • TOKENS_ARE_NODES This sets whether we add Tokens as terminal nodes to the AST. By default, it is true.

See deprecated settings for a list of legacy JavaCC options that no longer exist in JavaCC 21.

new_settings_in_javacc_21.txt · Last modified: 2021/10/19 19:54 by revusky