LPG


This page contains various sorts of useful information about LPG (formerly known as JikesPG), the LALR parser generator included with SAFARI:

LPG Quickstart for SAFARI Users
Running LPG Manually
LPG Documentation
Troubleshooting

(You may still find references to JikesPG in the SAFARI code, documentation, and interface. Those should all eventually be converted to LPG.)

LPG Quickstart for SAFARI Users

The SAFARI installation includes a version of LPG for generating a lexer, parser and related components in support of the instantiation of a SAFARI IDE for a new programming language.  The lexer and parser are generated from "grammar" files that specify the lexer and parser grammars for the language.  SAFARI uses LPG with three grammar files:  a "KeywordLexer.gi" file (".gi" stands for "grammar include"), which specifies the keywords of the language, a "Lexer.gi" file, which specifies the general lexical grammar (given the keyword lexer), and a "Parser.g" file (".g" stands for "grammar), which specifies the parser garmmar (given the general lexer).

When the New Programming Language wizard is run on a SAFARI project, it installs a LPG builder in the project, creates a parser package in the source folder of the project, and copies in three default grammar files.  The builder then generates Java classes for the lexer, parser, and related components, using templates provided with LPG and SAFARI.

A brief overview of the sections of the LPG grammar files that SAFARI users may encounter (not all sections will occur in all files):

  • The "Options" section includes options for LPG.
  • The "$Define" section is used to define macro symbols that are used by LPG itself.  The macros may be defined to include other symbols or source texts that will be substituted for subsequent occurrences of the symbols.  If a symbol is defined more than once, an earlier definition is overridden by a later one.
  • The "$Globals" section is intended for material that should appear at the head of a generated file, such as Java "import" directives.
  • The"$Headers" and "$Trailers" sections include members that are to be incorporated (respectively) at the beginning or end of the generated parser class  (more specifically, before or after the expansion of the action blocks).
  • An "$Include" section causes material to be interpolated into the input file at the point at which the $Include occurs.
  • The "$Export" section defines the token types recognized by the lexer or parser (in effect, the output elements)
  • The "$Terminals" section defines the terminal symbols recognized by the lexer or parser (in effect, the input elements)
  • The "$Start" section gives the root symbol for the processing of grammar rules.  In the parser grammar it is typically something like "program"
  • The "$Rules" section defines the grammatical rules that map the terminals in to the exported tokens
What do these mean for the SAFARI user?
  • The $Export, $Terminals, and $Rules sections contain the representation of the grammar.  These will be the focus of user customizations.  The rules may include arbitrary Java code (default delimiters "/." and "./").  The inclusion of such code may entail additions of supporting material to the $Globals and $Headers sections.
  • LPG is confgurable with many different options.  The generated grammar-file skeletons are given a basic set of options that are sufficient for generating a SAFARI-ready parser and these options should not generally need to be edited.  (Once you have attained some mastery of LPG by other means, then you may wish to edit the generated options if it suits your purposes.)
  • SAFARI makes some use of the $Globals section to add import statements for interfaces that the generated classes will implement.  SAFARI does not by default make any use of the $Headers or $Trailers sections.  However, if special-processing code is included in the grammar rules, then the $Globals, $Headers, and $Trailers sections can be used to add  imports, fields, methods, and so on in support of that code.
  • SAFARI uses the $Define section to define macros that are used elsewhere in the grammar files to more completely parameterizes the templates.  It is not generally necessary for the SAFARI user to add definitions (although exerienced LPG users may wish to do so).
  • SAFARI makes some use of $Include sections to incorporate portions of grammar specifications that have been put into separate files.  There is not any necessary reason for SAFARI users to customize the $Include sections, unless this is needed to support special-purpose language processing.
The LPG grammar templates used by SAFARI, along with the Java classes generated from these, usually give some examples and brief instructions about how the files should be completed for a new language.

Running LPG Manually


You can run LPG manually, in the form of “lpg”, as an Eclipse external tool.  To run lpg as an external tool you may first need to create a launch configuration for it.  To do this select

"Run" -> "External tools" -> "External tools ..."

and click on "New."

In the "Main" tab, give a suitable name to the configuration (such as "lpg") and enter appropriate parameters, such as:

  • Location:  ${workspace_loc:/lpg/bin/lpg-win32_x86.exe}
  • Working directory:   ${container_loc}
  • Arguments:    -include-directory=./;${workspace_loc}/lpg/templates ${resource_name}

The location assumes that lpg is stored in a project called "lpg".  If t;his location isn't appropriate, change this parameter to point to your lpg exectable file.  The working directory is set to be the container of the grammar file selected for processing.  The arguments specify an "include-directory" parameter.  One of the included directories is the "templates" directory in the lpg project, which contains templates that are used for generated grammar files and parser classes.

Note:  The grammar file templates contain path specifications that are defined relative to the "templates" directory, so this parameter should not be changed unless dependent templates are changed along with it.

Also note:  These parameters are themselves parameterized by the current workspace location.  If you have installed lpg under one workspace (say, your development workspace) and then want to invoke it from another workspace (say, a runtime workbench), then the workspace location in the second workspace will be incorrect.  Your parameters may have to be modified to account for this.

On the "Refresh" tab, if the working directory is "${container_loc}", then it is only necessary to select the button for "The folder containing the selected resource."  There are no environment variables needed.  On the "Common" tab it may be helpful to display the launch configuration in the "External tools" favorites menu.

With this launch configuration, to actually run lpg, select the grammar file to be processed then and select "lpg" from the "Run" menu or "External tools" favorites menu.  lpg can be run on both ".g" and ".gi" files and can be used to separately generate a keyword lexer, general lexer, and parser.


LPG Documentation


There is documentation for LPG on the LPG website at sourceforge.net (http://sourceforge.net/projects/lpg/)


Troubleshooting:


Still in the works!