This is an old revision of the document!

The INCLUDE Statement

JavaCC 21's INCLUDE statement allows you to break up your grammar file into multiple physical files. This feature is not present in legacy JavaCC.

The motivation behind INCLUDE should be fairly obvious. By allowing you to reuse a base grammar or generally useful fragment in various files, you can avoid the copy-paste-modify antipattern that would have been necessary when using legacy JavaCC. Also, allowing you to organize a large grammar into multiple physical files can be a big win in terms of maintainability.

Still, as they say, the devil is in the details, and there are some various wrinkles that need to be covered here.

The DEFAULT_LEXICAL_STATE setting

In legacy JavaCC, if you defined a token production without specifying a lexical state, any lexical definitions corresponded to a lexical state called "DEFAULT". Now, obviously, this is a problem once you have an INCLUDE disposition, because the including and included grammar are liable to have a "DEFAULT" lexical state and, typically, we don't want the definitions to clobber one another.

Thus, JavaCC 21 introduces a setting called DEFAULT_LEXICAL_STATE. That means that any lexical specifications where the lexical state is unspecified are in that state. Thus, a JSON grammar would likely have something like:

options {
   DEFAULT_LEXICAL_STATE="JSON";
}

at the top. In that case, any grammar for a language that wants to handle embedded JSON data would presumably define a different "default" lexical state, and when it wants to embedded JSON data, would have to explicitly switch to the JSON lexical state.

At the moment, DEFAULT_LEXICAL_STATE is the only setting you can put in an INCLUDEd grammar that has any effect. All of the other options are simply ignored, since they are presumably set in the including grammar. In legacy JavaCC, if you defined a token production without specifying a lexical state, it those patterns are matched in a lexical state called "DEFAULT" -- by default, obviously. This is a problem in terms of its interaction with the INCLUDE directive, since both grammars are liable to have a "DEFAULT" lexical state. The solution is that the default lexical state (the one you are using if none is explicitly specified) should be different in the including grammar from the included one.

Wrinkles with Code Injection

JavaCC still supports the legacy JavaCC constructs of PARSER_BEGIN...PARSER_END and TOKEN_MGR_DECLS. (For how much longer, I am not making any promises...). However, those constructs are ignored within an INCLUDEd grammar.

If you want to inject code into the generated parser or lexer class, from within an included grammar, you need to write something like:

INJECT(PARSER_CLASS) : 
{
   ...
}
{
   ...
}

or:

INJECT(LEXER_CLASS) : 
{
   ...
}
{
   ...
}

JavaCC 21 will replace the PARSER_CLASS and LEXER_CLASS holders with the appropriate names. So, if you have a Foo language in which you want to embed JSON expressions, so you include a JSON grammar, if that JSON grammar is to include some code within the generated parser, it cannot be:

INJECT(JSONParser) :
{
   ...
}
{
   ...
}

because the parser class we are generating is not JSONParser, it is FOOParser! However, the person writing a JSON grammar who wants it to be included does not know the name of the class. So, he needs to use the alias PARSER_CLASS or possibly LEXER_CLASS for the injected code to be included.

So, do not be surprised when the code within PARSER_BEGIN...PARSER_END is ignored if it is within an INCLUDEd grammar. You need to write INJECT(PARSER_CLASS) to achieve the desired result.

In fact, the aliases PARSER_CLASS, LEXER_CLASS, CONSTANTS_CLASS, PARSER_PACKAGE, and LEXER_PACKAGE can be used in code injections and java actions to make an included grammar (or grammar fragment) more generally useful.

To see a concrete example of INCLUDE in use, you can take a look at https://github.com/javacc21/javacc21/tree/master/src/grammars. Specifically, you can see that the JavaCC.javacc grammar INCLUDEs the Java.javacc. Another point is that this Java.javacc grammar file is, on its own quite generally usable. And useful!

INCLUDE with Java Source files

 INCLUDE("SomeJavaCode.java")
 

  INJECT : {
     (contents of the SomeJavaCode.java file here)
  }