The SideKick plugin provides a dockable window in which other plugins can display buffer structure. There are a number of such plugins, supporting languages such as Inform, Java, Javascript, EcmaScript, Perl, XML, HTML, CSS, and so on. SideKick makes browsing code for these languages easier, and has the ability to provide code completion (although this is not implemented in all of the sidekicks). Javacc is an excellent tool to parse language files and create the SideKick tree.
Some useful resources:
Javacc home: https://javacc.dev.java.net/
This site contains the latest releases of javacc, links to documentation, and has a repository of grammars.
Javacc FAQ: http://www.engr.mun.ca/~theo/JavaCC-FAQ/
This is an excellent resource.
I'll assume you have javacc installed and understand how it works (here is the briefest of overviews: javacc converts a .jj file into .java files, which are then compiled with javac. jjtree converts a .jjt file into a .jj file, then javacc converts the .jj file into .java files, which are then compiled with javac). See the build file for the XML plugin for an example of how to use the javacc and jjtree Ant tasks. I'll use the CSS sidekick as a working example of how javacc can be used to create a parser for SideKick. The CSS sidekick is part of the XML plugin. In the examples below, I'll be referencing the CSS2Parser.jj file, revision 6774. That file can be found here. This link will open a new browser window so you can follow the code while reading this document.
I started with a "clean" .jj file. A clean file is one that will parse appropriate input, but doesn't actually do anything. We want to add our own code to create a tree for SideKick. I want the parser to create tree nodes for specific items. In SideKick, the root node represents the file being parsed. All other nodes are based on the language, so a java file would show class and method nodes, a css file would show selectors, javascript would show functions, and so on. Generally, a good .jj file, that is, one that fully supports the language to be parsed, will have a lot more in it than is strictly necessary for creating the tree structure. While that level of detail isn't necessary for the tree, it is nice for code completion, so it's good to have.
The overall strategy is to use the javacc-generated parser to read the contents of the current jEdit buffer and create a set of tree nodes that can be displayed in a javax.swing.JTree. The SideKick plugin itself handles the tree, we just have to make the nodes.
To represent the nodes, I created a CSSNode class. This class implements sidekick.util.SideKickElement
:
package sidekick.util;
import javax.swing.text.Position;
public interface SideKickElement {
public void setStartLocation( Location loc );
public Location getStartLocation();
public void setEndLocation( Location loc );
public Location getEndLocation();
public void setStartPosition( Position s );
public Position getStartPosition();
public void setEndPosition( Position s );
public Position getEndPosition();
}
A sidekick.util.Location
represents an offset in the file based on line and column. Javacc tokens provide this information. A javax.swing.text.Position
represents an offset in the file based on character count from the start of the file. SideKick uses Position
s to locate items in the jEdit editor pane. Clicking on an item in the SideKick tree moves the cursor to the corresponding element in the editor, and moving the cursor in the editor moves the tree selection to the corresponding tree node. sidekick.util.ElementUtil
provides methods to convert the Location
s provided by javacc to the Position
s needed by jEdit and SideKick.
The CSSNode itself is fairly simple, it has a "name" attribute that is used for display in the SideKick tree, it has methods to add child nodes, and concrete methods for the SideKickElement methods.
To actually create a CSSNode, the .jj file is modified. The main entry point is the styleSheet
production at line 317. The top level node for our tree is created, then the styleSheetRuleList
production is called. This production returns a list of child nodes, which are added to our top level node as children. In this case, we don't really care about the top level node, it's just a holder for the children. We'll throw it away later when the actual Swing tree nodes are created.
styleSheetRuleList
calls other productions to create individual nodes. These nodes are assembled into a list and will become the first nodes below the root/file node. Below is an example of a production that actually creates a node (line 444). First is the original production from my "clean" .jj file, followed by my edits:
Original This will find things like @import url(imptest1.css);
in a css file:
Editted:
void importRule() :
{
}
{
<IMPORT_SYM> ( <S> )*
( <STRING>
| <URI> ) ( <S> )*
( mediaList() )?
<SEMICOLON>
}
CSSNode importRule() :
{
Token start = null;
CSSNode middle = null;
Token uri = null;
Token end = null;
CSSNode node = null;
}
{
try {
start=<IMPORT_SYM> ( <S> )*
( uri=<STRING>
| uri=<URI> ) ( <S> )*
( middle=mediaList() )?
end=<SEMICOLON>
}
catch(ParseException pe) {
addException(pe);
return null;
}
{
if (notNull(start, end)) {
String name = start.image + (uri != null ? " " + uri.image : "") + (middle != null ? " " + middle.getName() : "");
node = new CSSNode(name);
node.setStartLocation(getStartLocation(start));
node.setEndLocation(getEndLocation(end));
}
return node;
}
}
A Token
provides the starting and ending line and column. The Token.java
file is generated automatically by javacc. Passing the Token
to getStartLocation
creates a Location
for the start of the Token
, getEndLocation
is similar. These two methods must be added to the parser, but they are pretty simple:
public Location getStartLocation(Token t) {
if (t == null)
return new Location(0 + lineOffset, 0);
return new Location(t.beginLine + lineOffset, t.beginColumn);
}
Note the reference to lineOffset
. By default, lineOffset is 0, but can be set to some other value. I added the setLineOffset method to the parser to set the value. This is useful for when the css code is embedded in an html page in a <style> block. The html parser can call the css parser to provide part of the tree for the style, and can set the line number for the start of the style block so the offsets will be correct. Not all sidekicks will need to provide this functionality.
Once the call to styleSheet
returns, there is a CSSNode that contains other CSSNodes in a tree-like structure. All that is left is to convert those nodes into javax.swing.tree.DefaultMutableTreeNode
s. There are a few other things to point out in the .jj file, though:
importRule
above. The addException
, getExceptionLocation
, and getParseErrors
methods have been added to the parser so that parse errors can be displayed in the ErrorList plugin. These are pretty much boiler-plate code, I've reused these methods in several parsers. ParseException.java
is also a file generated by javacc.setTabSize
method has been added so that the javacc parser can know the tab size in use by the jEdit buffer being parsed. Javacc uses a default tab size of 8, so setting the tab size correctly for the buffer is essential so that Location
s and Position
s are accurate.PARSER_BEGIN
definition. This is from the JavaCC documentation:
The name that follows "PARSER_BEGIN" and "PARSER_END" must be the same and this identifies the name of the generated parser. For example, if name is "MyParser", then the following files are generated: MyParser.java: The generate parser. MyParserTokenManager.java: The generated token manager (or scanner/lexical analyzer). MyParserConstants.java: A bunch of useful constants. Other files such as "Token.java", "ParseException.java", etc. are also generated. However, these files contain boilerplate code and are the same for any grammar and may be reused across grammars.
There is some "magic" in the parser created by javacc. While you can declare constructors in the .jj file itself, javacc will automatically create constructors that accept a java.io.InputStream and a java.io.Reader. Most likely, it is one of these constructors that you'll want to use. You may want a main
method so that you can test the parser from the command line.
The next file to look at is sidekick.css.CSS2SideKickParser.java
. This class extends sidekick.SideKickParser
and will be called by SideKick itself when it determines a buffer should be parsed as CSS. This is the file that actually produces the Swing tree nodes for the SideKick tree and will call the javacc-generated parser. The main entry point for SideKick is the public SideKickParsedData parse(Buffer buffer, DefaultErrorSource errorSource)
method, which uses a lineOffset
of 0. This will cause the entire buffer to be parsed and will return a sidekick.SideKickParsedData
which will contain a DefaultMutableTreeNode that SideKick will show in its tree.
Stepping through the parse
method:
addTreeNodes
method.Location
s are changed to the Position
s needed by SideKick and jEdit.(line 135)
SimpleNode
, which is also a generated file. I edited SimpleNode.java
so that it implements SideKickElement
. To get the Location
s set into the SimpleNode
s, I did edit the .jjt file. From the JJTree documentation:
If the NODE_SCOPE_HOOK option is set to true, JJTree generates calls to two user-defined parser methods on the entry and exit of every node scope. The methods must have the following signatures: void jjtreeOpenNodeScope(Node n) void jjtreeCloseNodeScope(Node n)
I set the NODE_SCOPE_HOOK
value to true, then filled in the method bodies of jjtreeOpenNodeScope
and jjtreeCloseNodeScope so that the start and end locations are populated. Notice too that I added in the setLineOffset
, setTabSize
, and getParseErrors
(which needs to be finished) methods into the .jjt file like I did for the .jj example above.
The majority of the various AST files are unchanged from when they were generated by javacc. I changed the toString
methods in ASTFunctionDeclaration.java
, ASTIdentifier.java
, and ASTVariableStatement.java
so that the display in the SideKick tree is nicer.
EcmaScriptSideKickParser.java
is almost identical to CSS2SideKickParser. It follows the same steps to create the DefaultMutableTreeNodes for SideKick.
Dale Anson, Sep 2006