add TokenStreamRewriter to Python2 runtime

[lygav]

Hello, i've added the TokenStreamRewriter to Python2 runtime.
If all goes well i hope to expand on this contribution.

Please advise about how to integrate into the auto-generated test suit if this is necessary.

P.S: Closes ISSUE #1115

[parrt]

@ericvergnaud this would be a nice addition. can you bless it?

[ericvergnaud]

blessed

[ericvergnaud]

blessed

[ericvergnaud]

should this not be is instance(op, ...

[lygav]

Seems you are right.
I'll fix that, and will add tests and will review the code again.

[ericvergnaud]

@lygav please add some tests, following https://github.com/antlr/antlr4/blob/master/doc/adding-tests.md

[lygav]

@ericvergnaud Thank you for taking time to review the PR.
I'll add the needed tests.

[lygav]

@ericvergnaud Hello, although TokenStreamRewriter belongs to the 'runtime' it looks like TokenStreamRewriter has tests only as part of the "tool" testsuite, and only for Java.
The only other implementation is in C# 'runtime' and i just cannot see where is the test for it?
How do you propose to create test for it then:

• create python test in the Pyhon2 runtime dir (will not run with the build?)
• try to create TokenStreamDescriptor (or similar name..)

[ericvergnaud]

Hi,
create TokenStreamRewriterDescriptor is the way to go
Eric

[lygav]

Thanks Eric, i am going to try to figure this out.

[parrt]

@lygav I'll move my java tests to runtime.

[parrt]

Actually, it looks like a major pain to make this general. So far we test ANTLR itself and not runtime lib stuff that is accessed only with code. I'm not sure it's worth the effort for me at this time to make a general mechanism. It would be easier to have runtime tests access just library stuff in the target language. Better than not having tests I guess. @ericvergnaud open to suggestions.

[ericvergnaud]

Ok so let's leave as is and create independent tests in the respective runtimes.
We'll see on the long term whether there is a pattern that is worth sharing across runtimes.

[lygav]

Hello @ericvergnaud and @parrt , please look into the pull request with all the tests
replicated after the Java runtime, and fixed errors across the runtime.
Plus further enhancements to make Python2 runtime to be of the same consistent feel with Java (the original) as much as possible.
Also fixed another issue from some time ago (#550), turned out to be simple, i think can be easily replicated into Java.

Thank you