Class Grammar

java.lang.Object
org.antlr.tool.Grammar

public class Grammar extends Object
Represents a grammar in memory.
  • Field Details

    • SYNPRED_RULE_PREFIX

      public static final String SYNPRED_RULE_PREFIX
      See Also:
    • GRAMMAR_FILE_EXTENSION

      public static final String GRAMMAR_FILE_EXTENSION
      See Also:
    • LEXER_GRAMMAR_FILE_EXTENSION

      public static final String LEXER_GRAMMAR_FILE_EXTENSION
      used for generating lexer temp files
      See Also:
    • INITIAL_DECISION_LIST_SIZE

      public static final int INITIAL_DECISION_LIST_SIZE
      See Also:
    • INVALID_RULE_INDEX

      public static final int INVALID_RULE_INDEX
      See Also:
    • RULE_LABEL

      public static final int RULE_LABEL
      See Also:
    • TOKEN_LABEL

      public static final int TOKEN_LABEL
      See Also:
    • RULE_LIST_LABEL

      public static final int RULE_LIST_LABEL
      See Also:
    • TOKEN_LIST_LABEL

      public static final int TOKEN_LIST_LABEL
      See Also:
    • CHAR_LABEL

      public static final int CHAR_LABEL
      See Also:
    • WILDCARD_TREE_LABEL

      public static final int WILDCARD_TREE_LABEL
      See Also:
    • WILDCARD_TREE_LIST_LABEL

      public static final int WILDCARD_TREE_LIST_LABEL
      See Also:
    • LabelTypeToString

      public static String[] LabelTypeToString
    • ARTIFICIAL_TOKENS_RULENAME

      public static final String ARTIFICIAL_TOKENS_RULENAME
      See Also:
    • FRAGMENT_RULE_MODIFIER

      public static final String FRAGMENT_RULE_MODIFIER
      See Also:
    • SYNPREDGATE_ACTION_NAME

      public static final String SYNPREDGATE_ACTION_NAME
      See Also:
    • ANTLRLiteralEscapedCharValue

      public static int[] ANTLRLiteralEscapedCharValue
      When converting ANTLR char and string literals, here is the value set of escape chars.
    • ANTLRLiteralCharValueEscape

      public static String[] ANTLRLiteralCharValueEscape
      Given a char, we need to be able to show as an ANTLR literal.
    • LEXER

      public static final int LEXER
      See Also:
    • PARSER

      public static final int PARSER
      See Also:
    • TREE_PARSER

      public static final int TREE_PARSER
      See Also:
    • COMBINED

      public static final int COMBINED
      See Also:
    • grammarTypeToString

      public static final String[] grammarTypeToString
    • grammarTypeToFileNameSuffix

      public static final String[] grammarTypeToFileNameSuffix
    • validDelegations

      public static MultiMap<Integer,Integer> validDelegations
      Set of valid imports. E.g., can only import a tree parser into another tree parser. Maps delegate to set of delegator grammar types. validDelegations.get(LEXER) gives list of the kinds of delegators that can import lexers.
    • tokenBuffer

      public org.antlr.runtime.CommonTokenStream tokenBuffer
      This is the buffer of *all* tokens found in the grammar file including whitespace tokens etc... I use this to extract lexer rules from combined grammars.
    • IGNORE_STRING_IN_GRAMMAR_FILE_NAME

      public static final String IGNORE_STRING_IN_GRAMMAR_FILE_NAME
      See Also:
    • AUTO_GENERATED_TOKEN_NAME_PREFIX

      public static final String AUTO_GENERATED_TOKEN_NAME_PREFIX
      See Also:
    • name

      public String name
      What name did the user provide for this grammar?
    • type

      public int type
      What type of grammar is this: lexer, parser, tree walker
    • options

      protected Map<String,Object> options
      A list of options specified at the grammar level such as language=Java. The value can be an AST for complicated values such as character sets. There may be code generator specific options in here. I do no interpretation of the key/value pairs...they are simply available for who wants them.
    • legalLexerOptions

      public static final Set<String> legalLexerOptions
    • legalParserOptions

      public static final Set<String> legalParserOptions
    • legalTreeParserOptions

      public static final Set<String> legalTreeParserOptions
    • doNotCopyOptionsToLexer

      public static final Set<String> doNotCopyOptionsToLexer
    • defaultOptions

      public static final Map<String,String> defaultOptions
    • legalBlockOptions

      public static final Set<String> legalBlockOptions
    • defaultBlockOptions

      public static final Map<String,String> defaultBlockOptions
      What are the default options for a subrule?
    • defaultLexerBlockOptions

      public static final Map<String,String> defaultLexerBlockOptions
    • legalTokenOptions

      public static final Set<String> legalTokenOptions
      Legal options for terminal refs like ID<node=MyVarNode>
    • defaultTokenOption

      public static final String defaultTokenOption
      See Also:
    • global_k

      protected int global_k
      Is there a global fixed lookahead set for this grammar? If 0, nothing specified. -1 implies we have not looked at the options table yet to set k.
    • actions

      private Map<String,Map<String,Object>> actions
      Map a scope to a map of name:action pairs. Map<String, Map<String,GrammarAST>> The code generator will use this to fill holes in the output files. I track the AST node for the action in case I need the line number for errors.
    • nfa

      public NFA nfa
      The NFA that represents the grammar with edges labelled with tokens or epsilon. It is more suitable to analysis than an AST representation.
    • factory

      protected NFAFactory factory
    • composite

      public CompositeGrammar composite
      If this grammar is part of a larger composite grammar via delegate statement, then this points at the composite. The composite holds a global list of rules, token types, decision numbers, etc...
    • compositeTreeNode

      public CompositeGrammarTree compositeTreeNode
      A pointer back into grammar tree. Needed so we can add delegates.
    • label

      public String label
      If this is a delegate of another grammar, this is the label used as an instance var by that grammar to point at this grammar. null if no label was specified in the delegate statement.
    • charVocabulary

      protected IntSet charVocabulary
      TODO: hook this to the charVocabulary option
    • lineColumnToLookaheadDFAMap

      Map<String,DFA> lineColumnToLookaheadDFAMap
      For ANTLRWorks, we want to be able to map a line:col to a specific decision DFA so it can display DFA.
    • tool

      public Tool tool
    • ruleRefs

      protected Set<GrammarAST> ruleRefs
      The unique set of all rule references in any rule; set of tree node objects so two refs to same rule can exist but at different line/position.
    • scopedRuleRefs

      protected Set<GrammarAST> scopedRuleRefs
    • tokenIDRefs

      protected Set<org.antlr.runtime.Token> tokenIDRefs
      The unique set of all token ID references in any rule
    • decisionCount

      protected int decisionCount
      Be able to assign a number to every decision in grammar; decisions in 1..n
    • leftRecursiveRules

      protected Set<Rule> leftRecursiveRules
      A list of all rules that are in any left-recursive cycle. There could be multiple cycles, but this is a flat list of all problematic rules. This is stuff we couldn't refactor to precedence rule.
    • externalAnalysisAbort

      protected boolean externalAnalysisAbort
      An external tool requests that DFA analysis abort prematurely. Stops at DFA granularity, which are limited to a DFA size and time computation as failsafe.
    • numNonLLStar

      public int numNonLLStar
    • nameToSynpredASTMap

      protected LinkedHashMap<String,GrammarAST> nameToSynpredASTMap
      When we read in a grammar, we track the list of syntactic predicates and build faux rules for them later. See my blog entry Dec 2, 2005: http://www.antlr.org/blog/antlr3/lookahead.tml This maps the name (we make up) for a pred to the AST grammar fragment.
    • precRuleInitCodeBlocks

      public List<String> precRuleInitCodeBlocks
      Each left-recursive precedence rule must define precedence array for binary operators like: static int[] e_prec = new int[tokenNames.length]; static { e_prec[75] = 1; } Track and we push into parser later; this is computed early when we look for prec rules.
    • atLeastOneRuleMemoizes

      public boolean atLeastOneRuleMemoizes
      At least one rule has memoize=true
    • atLeastOneBacktrackOption

      public boolean atLeastOneBacktrackOption
      At least one backtrack=true in rule or decision or grammar.
    • implicitLexer

      public boolean implicitLexer
      Was this created from a COMBINED grammar?
    • nameToRuleMap

      protected LinkedHashMap<String,Rule> nameToRuleMap
      Map a rule to it's Rule object
    • overriddenRules

      public Set<String> overriddenRules
      If this rule is a delegate, some rules might be overridden; don't want to gen code for them.
    • delegatedRuleReferences

      protected Set<Rule> delegatedRuleReferences
      The list of all rules referenced in this grammar, not defined here, and defined in a delegate grammar. Not all of these will be generated in the recognizer for this file; only those that are affected by rule definitions in this grammar. I am not sure the Java target will need this but I'm leaving in case other targets need it. see NameSpaceChecker.lookForReferencesToUndefinedSymbols()
    • lexerRuleNamesInCombined

      public List<String> lexerRuleNamesInCombined
      The ANTLRParser tracks lexer rules when reading combined grammars so we can build the Tokens rule.
    • scopes

      protected Map<String,AttributeScope> scopes
      Track the scopes defined outside of rules and the scopes associated with all rules (even if empty).
    • grammarTree

      protected GrammarAST grammarTree
      An AST that records entire input grammar with all rules. A simple grammar with one rule, "grammar t; a : A | B ;", looks like: ( grammar t ( rule a ( BLOCK ( ALT A ) ( ALT B ) ) <end-of-rule> ) )
    • indexToDecision

      protected Vector<Grammar.Decision> indexToDecision
      Each subrule/rule is a decision point and we must track them so we can go back later and build DFA predictors for them. This includes all the rules, subrules, optional blocks, ()+, ()* etc...
    • generator

      protected CodeGenerator generator
      If non-null, this is the code generator we will use to generate recognizers in the target language.
    • nameSpaceChecker

      public NameSpaceChecker nameSpaceChecker
    • ll1Analyzer

      public LL1Analyzer ll1Analyzer
    • lexerGrammarTemplate

      protected String lexerGrammarTemplate
      For merged lexer/parsers, we must construct a separate lexer spec. This is the template for lexer; put the literals first then the regular rules. We don't need to specify a token vocab import as I make the new grammar import from the old all in memory; don't want to force it to read from the disk. Lexer grammar will have same name as original grammar but will be in different filename. Foo.g with combined grammar will have FooParser.java generated and Foo__.g with again Foo inside. It will however generate FooLexer.java as it's a lexer grammar. A bit odd, but autogenerated. Can tweak later if we want.
    • lexerGrammarST

      protected org.stringtemplate.v4.ST lexerGrammarST
    • fileName

      protected String fileName
      What file name holds this grammar?
    • DFACreationWallClockTimeInMS

      public long DFACreationWallClockTimeInMS
      How long in ms did it take to build DFAs for this grammar? If this grammar is a combined grammar, it only records time for the parser grammar component. This only records the time to do the LL(*) work; NFA→DFA conversion.
    • numberOfSemanticPredicates

      public int numberOfSemanticPredicates
    • numberOfManualLookaheadOptions

      public int numberOfManualLookaheadOptions
    • setOfNondeterministicDecisionNumbers

      public Set<Integer> setOfNondeterministicDecisionNumbers
    • setOfNondeterministicDecisionNumbersResolvedWithPredicates

      public Set<Integer> setOfNondeterministicDecisionNumbersResolvedWithPredicates
    • blocksWithSynPreds

      public Set<GrammarAST> blocksWithSynPreds
      Track decisions with syn preds specified for reporting. This is the a set of BLOCK type AST nodes.
    • decisionsWhoseDFAsUsesSynPreds

      public Set<DFA> decisionsWhoseDFAsUsesSynPreds
      Track decisions that actually use the syn preds in the DFA. Computed during NFA to DFA conversion.
    • synPredNamesUsedInDFA

      public Set<String> synPredNamesUsedInDFA
      Track names of preds so we can avoid generating preds that aren't used Computed during NFA to DFA conversion. Just walk accept states and look for synpreds because that is the only state target whose incident edges can have synpreds. Same is try for decisionsWhoseDFAsUsesSynPreds.
    • blocksWithSemPreds

      public Set<GrammarAST> blocksWithSemPreds
      Track decisions with syn preds specified for reporting. This is the a set of BLOCK type AST nodes.
    • decisionsWhoseDFAsUsesSemPreds

      public Set<DFA> decisionsWhoseDFAsUsesSemPreds
      Track decisions that actually use the syn preds in the DFA.
    • allDecisionDFACreated

      protected boolean allDecisionDFACreated
    • builtFromString

      protected boolean builtFromString
      We need a way to detect when a lexer grammar is autogenerated from another grammar or we are just sending in a string representing a grammar. We don't want to generate a .tokens file, for example, in such cases.
    • sanity

      Factored out the sanity checking code; delegate to it.
    • target

      Target target
      Useful for asking questions about target during analysis
  • Constructor Details

    • Grammar

      public Grammar(Tool tool, String fileName, CompositeGrammar composite)
      Create a grammar from file name.
    • Grammar

      public Grammar()
      Useful for when you are sure that you are not part of a composite already. Used in Interp/RandomPhrase and testing.
    • Grammar

      public Grammar(Tool tool)
    • Grammar

      public Grammar(String grammarString) throws org.antlr.runtime.RecognitionException
      Used for testing; only useful on noncomposite grammars.
      Throws:
      org.antlr.runtime.RecognitionException
    • Grammar

      public Grammar(Tool tool, String grammarString) throws org.antlr.runtime.RecognitionException
      Used for testing and Interp/RandomPhrase. Only useful on noncomposite grammars.
      Throws:
      org.antlr.runtime.RecognitionException
  • Method Details

    • setFileName

      public void setFileName(String fileName)
    • getFileName

      public String getFileName()
    • setName

      public void setName(String name)
    • setGrammarContent

      public void setGrammarContent(String grammarString) throws org.antlr.runtime.RecognitionException
      Throws:
      org.antlr.runtime.RecognitionException
    • parseAndBuildAST

      public void parseAndBuildAST() throws IOException
      Throws:
      IOException
    • parseAndBuildAST

      public void parseAndBuildAST(Reader r)
    • dealWithTreeFilterMode

      protected void dealWithTreeFilterMode()
    • translateLeftRecursiveRule

      public void translateLeftRecursiveRule(GrammarAST ruleAST)
    • defineGrammarSymbols

      public void defineGrammarSymbols()
    • checkNameSpaceAndActions

      public void checkNameSpaceAndActions()
      ANALYZE ACTIONS, LOOKING FOR LABEL AND ATTR REFS, sanity check
    • validImport

      public boolean validImport(Grammar delegate)
      Many imports are illegal such as lexer into a tree grammar
    • getLexerGrammar

      public String getLexerGrammar()
      If the grammar is a combined grammar, return the text of the implicit lexer grammar.
    • getImplicitlyGeneratedLexerFileName

      public String getImplicitlyGeneratedLexerFileName()
    • getRecognizerName

      public String getRecognizerName()
      Get the name of the generated recognizer; may or may not be same as grammar name. Recognizer is TParser and TLexer from T if combined, else just use T regardless of grammar type.
    • addArtificialMatchTokensRule

      public GrammarAST addArtificialMatchTokensRule(GrammarAST grammarAST, List<String> ruleNames, List<String> delegateNames, boolean filterMode)
      Parse a rule we add artificially that is a list of the other lexer rules like this: "Tokens : ID | INT | SEMI ;" nextToken() will invoke this to set the current token. Add char literals before the rule references. If in filter mode, we want every alt to backtrack and we need to do k=1 to force the "first token def wins" rule. Otherwise, the longest-match rule comes into play with LL(*). The ANTLRParser antlr.g file now invokes this when parsing a lexer grammar, which I think is proper even though it peeks at the info that later phases will (re)compute. It gets a list of lexer rules and builds a string representing the rule; then it creates a parser and adds the resulting tree to the grammar's tree.
    • parseArtificialRule

      public GrammarAST parseArtificialRule(String ruleText)
    • addRule

      public void addRule(GrammarAST grammarTree, GrammarAST t)
    • getArtificialRulesForSyntacticPredicates

      protected List<? extends GrammarAST> getArtificialRulesForSyntacticPredicates(LinkedHashMap<String,GrammarAST> nameToSynpredASTMap)
      for any syntactic predicates, we need to define rules for them; they will get defined automatically like any other rule. :)
    • addRulesForSyntacticPredicates

      public void addRulesForSyntacticPredicates()
    • createRuleStartAndStopNFAStates

      public void createRuleStartAndStopNFAStates()
      Define all the rule begin/end NFAStates to solve forward reference issues. Critical for composite grammars too. This is normally called on all root/delegates manually and then buildNFA() is called afterwards because the NFA construction needs to see rule start/stop states from potentially every grammar. Has to be have these created a priori. Testing routines will often just call buildNFA(), which forces a call to this method if not done already. Works ONLY for single noncomposite grammars.
    • buildNFA

      public void buildNFA()
    • createLookaheadDFAs

      public void createLookaheadDFAs()
      For each decision in this grammar, compute a single DFA using the NFA states associated with the decision. The DFA construction determines whether or not the alternatives in the decision are separable using a regular lookahead language. Store the lookahead DFAs in the AST created from the user's grammar so the code generator or whoever can easily access it. This is a separate method because you might want to create a Grammar without doing the expensive analysis.
    • createLookaheadDFAs

      public void createLookaheadDFAs(boolean wackTempStructures)
    • createLL_1_LookaheadDFA

      public DFA createLL_1_LookaheadDFA(int decision)
    • updateLineColumnToLookaheadDFAMap

      private void updateLineColumnToLookaheadDFAMap(DFA lookaheadDFA)
    • makeEdgeSetsDisjoint

      protected List<IntervalSet> makeEdgeSetsDisjoint(List<IntervalSet> edges)
    • createLookaheadDFA

      public DFA createLookaheadDFA(int decision, boolean wackTempStructures)
    • externallyAbortNFAToDFAConversion

      public void externallyAbortNFAToDFAConversion()
      Terminate DFA creation (grammar analysis).
    • NFAToDFAConversionExternallyAborted

      public boolean NFAToDFAConversionExternallyAborted()
    • getNewTokenType

      public int getNewTokenType()
      Return a new unique integer in the token type space
    • defineToken

      public void defineToken(String text, int tokenType)
      Define a token at a particular token type value. Blast an old value with a new one. This is called normal grammar processsing and during import vocab operations to set tokens with specific values.
    • defineRule

      public void defineRule(org.antlr.runtime.Token ruleToken, String modifier, Map<String,Object> options, GrammarAST tree, GrammarAST argActionAST, int numAlts)
      Define a new rule. A new rule index is created by incrementing ruleIndex.
    • defineSyntacticPredicate

      public String defineSyntacticPredicate(GrammarAST blockAST, String currentRuleName)
      Define a new predicate and get back its name for use in building a semantic predicate reference to the syn pred.
    • getSyntacticPredicates

      public LinkedHashMap<String,GrammarAST> getSyntacticPredicates()
    • getSyntacticPredicate

      public GrammarAST getSyntacticPredicate(String name)
    • synPredUsedInDFA

      public void synPredUsedInDFA(DFA dfa, SemanticContext semCtx)
    • defineNamedAction

      public void defineNamedAction(GrammarAST ampersandAST, String scope, GrammarAST nameAST, GrammarAST actionAST)
      Given @scope::name {action} define it for this grammar. Later, the code generator will ask for the actions table. For composite grammars, make sure header action propogates down to all delegates.
    • setSynPredGateIfNotAlready

      public void setSynPredGateIfNotAlready(org.stringtemplate.v4.ST gateST)
    • getActions

      public Map<String,Map<String,Object>> getActions()
    • getDefaultActionScope

      public String getDefaultActionScope(int grammarType)
      Given a grammar type, what should be the default action scope? If I say @members in a COMBINED grammar, for example, the default scope should be "parser".
    • defineLexerRuleFoundInParser

      public void defineLexerRuleFoundInParser(org.antlr.runtime.Token ruleToken, GrammarAST ruleAST)
    • defineLexerRuleForAliasedStringLiteral

      public void defineLexerRuleForAliasedStringLiteral(String tokenID, String literal, int tokenType)
      If someone does PLUS='+' in the parser, must make sure we get "PLUS : '+' ;" in lexer not "T73 : '+';"
    • defineLexerRuleForStringLiteral

      public void defineLexerRuleForStringLiteral(String literal, int tokenType)
    • getLocallyDefinedRule

      public Rule getLocallyDefinedRule(String ruleName)
    • getRule

      public Rule getRule(String ruleName)
    • getRule

      public Rule getRule(String scopeName, String ruleName)
    • getRuleIndex

      public int getRuleIndex(String scopeName, String ruleName)
    • getRuleIndex

      public int getRuleIndex(String ruleName)
    • getRuleName

      public String getRuleName(int ruleIndex)
    • generateMethodForRule

      public boolean generateMethodForRule(String ruleName)
      Should codegen.g gen rule for ruleName? If synpred, only gen if used in a DFA. If regular rule, only gen if not overridden in delegator Always gen Tokens rule though.
    • defineGlobalScope

      public AttributeScope defineGlobalScope(String name, org.antlr.runtime.Token scopeAction)
    • createReturnScope

      public AttributeScope createReturnScope(String ruleName, org.antlr.runtime.Token retAction)
    • createRuleScope

      public AttributeScope createRuleScope(String ruleName, org.antlr.runtime.Token scopeAction)
    • createParameterScope

      public AttributeScope createParameterScope(String ruleName, org.antlr.runtime.Token argAction)
    • getGlobalScope

      public AttributeScope getGlobalScope(String name)
      Get a global scope
    • getGlobalScopes

      public Map<String,AttributeScope> getGlobalScopes()
    • defineLabel

      protected void defineLabel(Rule r, org.antlr.runtime.Token label, GrammarAST element, int type)
      Define a label defined in a rule r; check the validity then ask the Rule object to actually define it.
    • defineTokenRefLabel

      public void defineTokenRefLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
    • defineWildcardTreeLabel

      public void defineWildcardTreeLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
    • defineWildcardTreeListLabel

      public void defineWildcardTreeListLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST tokenRef)
    • defineRuleRefLabel

      public void defineRuleRefLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST ruleRef)
    • defineTokenListLabel

      public void defineTokenListLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST element)
    • defineRuleListLabel

      public void defineRuleListLabel(String ruleName, org.antlr.runtime.Token label, GrammarAST element)
    • getLabels

      public Set<String> getLabels(Set<GrammarAST> rewriteElements, int labelType)
      Given a set of all rewrite elements on right of ->, filter for label types such as Grammar.TOKEN_LABEL, Grammar.TOKEN_LIST_LABEL, ... Return a displayable token type name computed from the GrammarAST.
    • examineAllExecutableActions

      protected void examineAllExecutableActions()
      Before generating code, we examine all actions that can have $x.y and $y stuff in them because some code generation depends on Rule.referencedPredefinedRuleAttributes. I need to remove unused rule labels for example.
    • checkAllRulesForUselessLabels

      public void checkAllRulesForUselessLabels()
      Remove all labels on rule refs whose target rules have no return value. Do this for all rules in grammar.
    • removeUselessLabels

      protected void removeUselessLabels(Map<String,Grammar.LabelElementPair> ruleToElementLabelPairMap)
      A label on a rule is useless if the rule has no return value, no tree or template output, and it is not referenced in an action.
    • altReferencesRule

      public void altReferencesRule(String enclosingRuleName, GrammarAST refScopeAST, GrammarAST refAST, int outerAltNum)
      Track a rule reference within an outermost alt of a rule. Used at the moment to decide if $ruleref refers to a unique rule ref in the alt. Rewrite rules force tracking of all rule AST results. This data is also used to verify that all rules have been defined.
    • altReferencesTokenID

      public void altReferencesTokenID(String ruleName, GrammarAST refAST, int outerAltNum)
      Track a token reference within an outermost alt of a rule. Used to decide if $tokenref refers to a unique token ref in the alt. Does not track literals! Rewrite rules force tracking of all tokens.
    • referenceRuleLabelPredefinedAttribute

      public void referenceRuleLabelPredefinedAttribute(String ruleName)
      To yield smaller, more readable code, track which rules have their predefined attributes accessed. If the rule has no user-defined return values, then don't generate the return value scope classes etc... Make the rule have void return value. Don't track for lexer rules.
    • checkAllRulesForLeftRecursion

      public List<? extends Collection<? extends Rule>> checkAllRulesForLeftRecursion()
    • getLeftRecursiveRules

      public Set<Rule> getLeftRecursiveRules()
      Return a list of left-recursive rules; no analysis can be done successfully on these. Useful to skip these rules then and also for ANTLRWorks to highlight them.
    • checkRuleReference

      public void checkRuleReference(GrammarAST scopeAST, GrammarAST refAST, GrammarAST argsAST, String currentRuleName)
    • isEmptyRule

      public boolean isEmptyRule(GrammarAST block)
      Rules like "a : ;" and "a : {...} ;" should not generate try/catch blocks for RecognitionException. To detect this it's probably ok to just look for any reference to an atom that can match some input. W/o that, the rule is unlikey to have any else.
    • findFirstTypeOutsideRewrite

      protected GrammarAST findFirstTypeOutsideRewrite(GrammarAST block, BitSet types)
    • isAtomTokenType

      public boolean isAtomTokenType(int ttype)
    • getTokenType

      public int getTokenType(String tokenName)
    • getTokenIDs

      public Set<String> getTokenIDs()
      Get the list of tokens that are IDs like BLOCK and LPAREN
    • getTokenTypesWithoutID

      public Collection<Integer> getTokenTypesWithoutID()
      Return an ordered integer list of token types that have no corresponding token ID like INT or KEYWORD_BEGIN; for stuff like 'begin'.
    • getTokenDisplayNames

      public Set<String> getTokenDisplayNames()
      Get a list of all token IDs and literals that have an associated token type.
    • getCharValueFromGrammarCharLiteral

      public static int getCharValueFromGrammarCharLiteral(String literal)
      Given a literal like (the 3 char sequence with single quotes) 'a', return the int value of 'a'. Convert escape sequences here also. ANTLR's antlr.g parser does not convert escape sequences. 11/26/2005: I changed literals to always be '...' even for strings. This routine still works though.
    • getUnescapedStringFromGrammarStringLiteral

      public static StringBuffer getUnescapedStringFromGrammarStringLiteral(String literal)
      ANTLR does not convert escape sequences during the parse phase because it could not know how to print String/char literals back out when printing grammars etc... Someone in China might use the real unicode char in a literal as it will display on their screen; when printing back out, I could not know whether to display or use a unicode escape. This routine converts a string literal with possible escape sequences into a pure string of 16-bit char values. Escapes and unicode specs are converted to pure chars. return in a buffer; people may want to walk/manipulate further. The NFA construction routine must know the actual char values.
    • importTokenVocabulary

      public int importTokenVocabulary(Grammar importFromGr)
      Pull your token definitions from an existing grammar in memory. You must use Grammar() ctor then this method then setGrammarContent() to make this work. This was useful primarily for testing and interpreting grammars until I added import grammar functionality. When you import a grammar you implicitly import its vocabulary as well and keep the same token type values. Returns the max token type found.
    • importGrammar

      public void importGrammar(GrammarAST grammarNameAST, String label)
      Import the rules/tokens of a delegate grammar. All delegate grammars are read during the ctor of first Grammar created. Do not create NFA here because NFA construction needs to hook up with overridden rules in delegation root grammar.
    • addDelegateGrammar

      protected void addDelegateGrammar(Grammar delegateGrammar)
      add new delegate to composite tree
    • importTokenVocabulary

      public int importTokenVocabulary(GrammarAST tokenVocabOptionAST, String vocabName)
      Load a vocab file <vocabName>.tokens and return max token type found.
    • getTokenDisplayName

      public String getTokenDisplayName(int ttype)
      Given a token type, get a meaningful name for it such as the ID or string literal. If this is a lexer and the ttype is in the char vocabulary, compute an ANTLR-valid (possibly escaped) char literal.
    • getStringLiterals

      public Set<String> getStringLiterals()
      Get the list of ANTLR String literals
    • getGrammarTypeString

      public String getGrammarTypeString()
    • getGrammarMaxLookahead

      public int getGrammarMaxLookahead()
    • setOption

      public String setOption(String key, Object value, org.antlr.runtime.Token optionsStartToken)
      Save the option key/value pair and process it; return the key or null if invalid option.
    • legalOption

      public boolean legalOption(String key)
    • setOptions

      public void setOptions(Map<String,Object> options, org.antlr.runtime.Token optionsStartToken)
    • getOption

      public Object getOption(String key)
    • getLocallyDefinedOption

      public Object getLocallyDefinedOption(String key)
    • getBlockOption

      public Object getBlockOption(GrammarAST blockAST, String key)
    • getUserMaxLookahead

      public int getUserMaxLookahead(int decision)
    • getAutoBacktrackMode

      public boolean getAutoBacktrackMode(int decision)
    • optionIsValid

      public boolean optionIsValid(String key, Object value)
    • buildAST

      public boolean buildAST()
    • rewriteMode

      public boolean rewriteMode()
    • isBuiltFromString

      public boolean isBuiltFromString()
    • buildTemplate

      public boolean buildTemplate()
    • getRules

      public Collection<Rule> getRules()
    • getDelegatedRules

      public Set<? extends Rule> getDelegatedRules()
      Get the set of Rules that need to have manual delegations like "void rule() { importedGrammar.rule(); }" If this grammar is master, get list of all rule definitions from all delegate grammars. Only master has complete interface from combined grammars...we will generated delegates as helper objects. Composite grammars that are not the root/master do not have complete interfaces. It is not my intention that people use subcomposites. Only the outermost grammar should be used from outside code. The other grammar components are specifically generated to work only with the master/root. delegatedRules = imported - overridden
    • getAllImportedRules

      public Set<? extends Rule> getAllImportedRules()
      Get set of all rules imported from all delegate grammars even if indirectly delegated.
    • getDelegates

      public List<Grammar> getDelegates()
      Get list of all delegates from all grammars directly or indirectly imported into this grammar.
    • getHasDelegates

      public boolean getHasDelegates()
    • getDelegateNames

      public List<String> getDelegateNames()
    • getDirectDelegates

      public List<Grammar> getDirectDelegates()
    • getIndirectDelegates

      public List<Grammar> getIndirectDelegates()
      Get delegates below direct delegates
    • getDelegators

      public List<Grammar> getDelegators()
      Get list of all delegators. This amounts to the grammars on the path to the root of the delegation tree.
    • getDelegator

      public Grammar getDelegator()
      Who's my direct parent grammar?
    • getDelegatedRuleReferences

      public Set<Rule> getDelegatedRuleReferences()
    • getGrammarIsRoot

      public boolean getGrammarIsRoot()
    • setRuleAST

      public void setRuleAST(String ruleName, GrammarAST t)
    • getRuleStartState

      public NFAState getRuleStartState(String ruleName)
    • getRuleStartState

      public NFAState getRuleStartState(String scopeName, String ruleName)
    • getRuleModifier

      public String getRuleModifier(String ruleName)
    • getRuleStopState

      public NFAState getRuleStopState(String ruleName)
    • assignDecisionNumber

      public int assignDecisionNumber(NFAState state)
    • getDecision

      protected Grammar.Decision getDecision(int decision)
    • getDecisions

      public List<Grammar.Decision> getDecisions()
    • createDecision

      protected Grammar.Decision createDecision(int decision)
    • getDecisionNFAStartStateList

      public List<NFAState> getDecisionNFAStartStateList()
    • getDecisionNFAStartState

      public NFAState getDecisionNFAStartState(int decision)
    • getLookaheadDFA

      public DFA getLookaheadDFA(int decision)
    • getDecisionBlockAST

      public GrammarAST getDecisionBlockAST(int decision)
    • getLookaheadDFAColumnsForLineInFile

      public List<Integer> getLookaheadDFAColumnsForLineInFile(int line)
      returns a list of column numbers for all decisions on a particular line so ANTLRWorks choose the decision depending on the location of the cursor (otherwise, ANTLRWorks has to give the *exact* location which is not easy from the user point of view). This is not particularly fast as it walks entire line:col→DFA map looking for a prefix of "line:".
    • getLookaheadDFAFromPositionInFile

      public DFA getLookaheadDFAFromPositionInFile(int line, int col)
      Useful for ANTLRWorks to map position in file to the DFA for display
    • getLineColumnToLookaheadDFAMap

      public Map<String,DFA> getLineColumnToLookaheadDFAMap()
    • getNumberOfDecisions

      public int getNumberOfDecisions()
    • getNumberOfCyclicDecisions

      public int getNumberOfCyclicDecisions()
    • setLookaheadDFA

      public void setLookaheadDFA(int decision, DFA lookaheadDFA)
      Set the lookahead DFA for a particular decision. This means that the appropriate AST node must updated to have the new lookahead DFA. This method could be used to properly set the DFAs without using the createLookaheadDFAs() method. You could do this Grammar g = new Grammar("..."); g.setLookahead(1, dfa1); g.setLookahead(2, dfa2); ...
    • setDecisionNFA

      public void setDecisionNFA(int decision, NFAState state)
    • setDecisionBlockAST

      public void setDecisionBlockAST(int decision, GrammarAST blockAST)
    • allDecisionDFAHaveBeenCreated

      public boolean allDecisionDFAHaveBeenCreated()
    • getMaxTokenType

      public int getMaxTokenType()
      How many token types have been allocated so far?
    • getMaxCharValue

      public int getMaxCharValue()
      What is the max char value possible for this grammar's target? Use unicode max if no target defined.
    • getTokenTypes

      public IntSet getTokenTypes()
      Return a set of all possible token or char types for this grammar
    • getAllCharValues

      public IntSet getAllCharValues()
      If there is a char vocabulary, use it; else return min to max char as defined by the target. If no target, use max unicode char value.
    • getANTLRCharLiteralForChar

      public static String getANTLRCharLiteralForChar(int c)
      Return a string representing the escaped char for code c. E.g., If c has value 0x100, you will get "Ā". ASCII gets the usual char (non-hex) representation. Control characters are spit out as unicode. While this is specially set up for returning Java strings, it can be used by any language target that has the same syntax. :) 11/26/2005: I changed this to use double quotes, consistent with antlr.g 12/09/2005: I changed so everything is single quotes
    • complement

      public IntSet complement(IntSet set)
      For lexer grammars, return everything in unicode not in set. For parser and tree grammars, return everything in token space from MIN_TOKEN_TYPE to last valid token type or char value.
    • complement

      public IntSet complement(int atom)
    • isValidSet

      public boolean isValidSet(TreeToNFAConverter nfabuilder, GrammarAST t)
      Given set tree like ( SET A B ), check that A and B are both valid sets themselves, else we must tree like a BLOCK
    • getSetFromRule

      public IntSet getSetFromRule(TreeToNFAConverter nfabuilder, String ruleName) throws org.antlr.runtime.RecognitionException
      Get the set equivalent (if any) of the indicated rule from this grammar. Mostly used in the lexer to do ~T for some fragment rule T. If the rule AST has a SET use that. If the rule is a single char convert it to a set and return. If rule is not a simple set (w/o actions) then return null. Rules have AST form: ^( RULE ID modifier ARG RET SCOPE block EOR )
      Throws:
      org.antlr.runtime.RecognitionException
    • getNumberOfAltsForDecisionNFA

      public int getNumberOfAltsForDecisionNFA(NFAState decisionState)
      Decisions are linked together with transition(1). Count how many there are. This is here rather than in NFAState because a grammar decides how NFAs are put together to form a decision.
    • getNFAStateForAltOfDecision

      public NFAState getNFAStateForAltOfDecision(NFAState decisionState, int alt)
      Get the ith alternative (1..n) from a decision; return null when an invalid alt is requested. I must count in to find the right alternative number. For (A|B), you get NFA structure (roughly): o->o-A->o | o->o-B->o This routine returns the leftmost state for each alt. So alt=1, returns the upperleft most state in this structure.
    • FIRST

      public LookaheadSet FIRST(NFAState s)
    • LOOK

      public LookaheadSet LOOK(NFAState s)
    • setCodeGenerator

      public void setCodeGenerator(CodeGenerator generator)
    • getCodeGenerator

      public CodeGenerator getCodeGenerator()
    • getGrammarTree

      public GrammarAST getGrammarTree()
    • setGrammarTree

      public void setGrammarTree(GrammarAST value)
    • getTool

      public Tool getTool()
    • setTool

      public void setTool(Tool tool)
    • computeTokenNameFromLiteral

      public String computeTokenNameFromLiteral(int tokenType, String literal)
      given a token type and the text of the literal, come up with a decent token type label. For now it's just T<type>. Actually, if there is an aliased name from tokens like PLUS='+', use it.
    • toString

      public String toString()
      Overrides:
      toString in class Object
    • grammarTreeToString

      public String grammarTreeToString(GrammarAST t)
    • grammarTreeToString

      public String grammarTreeToString(GrammarAST t, boolean showActions)
    • printGrammar

      public void printGrammar(PrintStream output)