Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require superClass for Lexer in split mode for JavaScript target #1966

Merged
merged 5 commits into from
Sep 17, 2017

Conversation

venkatperi
Copy link
Contributor

  • In split mode grammars, the generated lexer.js wasn't requiring the superclass.

  • Lexer ids such as token/mode/channel names were set on the generated lexer and not its prototype making them class/static variables. For grammar actions to be universal across Java and JavaScript, these names must be accessible on the lexer instance (via [this.]SOME_MODE in Java, or this.SOME_MODE in JavaScript since JS doesn't scope to the local class without this.).

  • Finally, Utils.escapeWhiteSpace was replacing only the first instance of \n etc. Switched to a regex with global replace.

…s available on Lexer prototype for use in lexer actions (this.SOME_MODE) like in Java actions, Utils.escapeWhiteSpace uses regex for global replace (was replacing only first occurnce of \n etc).
if (escapeSpaces) {
s = s.replace(" ", "\u00B7");
s = s.replace(/ /g, "\u00B7");
}
return s;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok with these

@@ -160,6 +160,7 @@ Object.defineProperty(<parser.name>.prototype, "atn", {
});

<parser.name>.EOF = antlr4.Token.EOF;
<parser.name>.prototype.EOF = antlr4.Token.EOF;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry but this is superfluous

@@ -802,6 +803,9 @@ var antlr4 = require('antlr4/index');
>>

Lexer(lexer, atn, actionFuncs, sempredFuncs, superClass) ::= <<
<if(superClass)>
var <superClass> = require('./<superClass>').<superClass>;
<endif>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@@ -819,14 +823,18 @@ function <lexer.name>(input) {
<lexer.name>.prototype.constructor = <lexer.name>;

<lexer.name>.EOF = antlr4.Token.EOF;
<lexer.name>.prototype.EOF = antlr4.Token.EOF;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

superfluous

<lexer.tokens:{k | <lexer.name>.<k> = <lexer.tokens.(k)>;}; separator="\n", wrap, anchor>
<lexer.tokens:{k | <lexer.name>.prototype.<k> = <lexer.tokens.(k)>;}; separator="\n", wrap, anchor>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

superfluous


<if(lexer.channels)>
<lexer.channels:{c| <lexer.name>.<c> = <lexer.channels.(c)>;}; separator="\n">
<lexer.channels:{c| <lexer.name>.prototype.<c> = <lexer.channels.(c)>;}; separator="\n">

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

superfluous


<endif>
<if(rest(lexer.modes))>
<rest(lexer.modes):{m| <lexer.name>.<m> = <i>;}; separator="\n">
<rest(lexer.modes):{m| <lexer.name>.prototype.<m> = <i>;}; separator="\n">

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

superfluous

@ericvergnaud
Copy link
Contributor

Hi,
thanks for this.
Tokens are already members of the lexer and parser type, so should not be members of each instance. In java they are static, so should not be accessed through parser/lexer instances.
ANTLR does not support 'universal' grammar actions. Instead, you should use listener or visitor.
Other proposals make sense.
Also, if not already done, you need to sign the license agreement.

@ericvergnaud
Copy link
Contributor

Hi
I can't bleed this until you remove the prototype level stuff

@ericvergnaud
Copy link
Contributor

I meant bless not bleed

@venkatperi
Copy link
Contributor Author

So the downside of ids on the class vs instance is that superclasses will need to reach into the subclass (usually the generated Lexer) to access definitions. e.g. if I have a token:

tokens { TOKEN }

From my lexer's super class, I need to do this:

SuperLexer.prototype.someAction = function() {
  console.log(Object.getPrototypeOf(this).TOKEN);
}

I guess its the same with Java currrently:

//superclass member
  public void someAction() {
     int type = GeneratedLexer.TOKEN;
  }

Example:
https://github.com/venkatperi/asciidoc-antlr/blob/master/src/main/js/AbstractAsciidocLexer.js
https://github.com/venkatperi/asciidoc-antlr/blob/master/src/main/java/AbstractAsciidocLexer.java

@venkatperi
Copy link
Contributor Author

Reversed proto changes.

@ericvergnaud
Copy link
Contributor

@parrt blessed

@parrt parrt added this to the 4.7.1 milestone Sep 17, 2017
@parrt parrt merged commit a7b0afb into antlr:master Sep 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants