Feature/gomod4110

.gitignore

@@ -104,3 +104,23 @@ runtime/PHP
 
 # Swift binaries
 .build/
+
+# Cpp generated build files
+runtime/Cpp/CMakeCache.txt
+runtime/Cpp/CMakeFiles/
+runtime/Cpp/CPackConfig.cmake
+runtime/Cpp/CPackSourceConfig.cmake
+runtime/Cpp/CTestTestfile.cmake
+runtime/Cpp/Makefile
+runtime/Cpp/_deps/
+runtime/Cpp/cmake_install.cmake
+runtime/Cpp/runtime/CMakeFiles/
+runtime/Cpp/runtime/CTestTestfile.cmake
+runtime/Cpp/runtime/Makefile
+runtime/Cpp/runtime/antlr4_tests
+runtime/Cpp/runtime/antlr4_tests\[1]_include.cmake
+runtime/Cpp/runtime/antlr4_tests\[1]_tests.cmake
+runtime/Cpp/runtime/cmake_install.cmake
+runtime/Cpp/runtime/libantlr4-runtime.4.10.1.dylib
+runtime/Cpp/runtime/libantlr4-runtime.a
+runtime/Cpp/runtime/libantlr4-runtime.dylib

doc/go-target.md

@@ -8,50 +8,162 @@
 
 #### 2. Get the Go ANTLR runtime
 
-Each target language for ANTLR has a runtime package for running parser generated by ANTLR4. The runtime provides a common set of tools for using your parser.
+Each target language for ANTLR has a runtime package for running parser generated by ANTLR4. 
+The runtime provides a common set of tools for using your parser. Note that if you have existing projects and have 
+yet to replace the `v1.x.x` modules with the `v4` modules, then you can skip ahead to the section *Upgrading to v4
+from earlier versions*
 
-Get the runtime and install it on your GOPATH:
+The Go runtime uses modules and has a version path of `/v4` to stay in sync with the runtime versions of all the other runtimes.
+Setup is the same as any other module based project:
 
 ```bash
-go get github.com/antlr/antlr4/runtime/Go/antlr
+$ cd mymodproject
+$ go mod init mymodproject
 ```
 
-#### 3. Set the release tag (optional)
+After which, you can use go get, to get the latest release version of the ANTLR v4 runtime using:
 
-`go get` has no native way to specify a branch or commit. So, when you run it, you'll download the latest commits. This may or may not be your preference.
+```bash
+go get github.com/antlr/antlr4/runtime/Go/antlr/v4
+```
 
-You'll need to use git to set the release. For example, to set the release tag for release 4.9.3:
+If your project is already using the v4 runtime, then you can upgrade to the latest release using the usual:
 
 ```bash
-cd $GOPATH/src/github.com/antlr/antlr4 # enter the antlr4 source directory
-git checkout tags/4.9.3 # the go runtime was added in release 4.9.3
+go get -u github.com/antlr/antlr4/runtime/Go/antlr/v4
+```
+If you have not yet upgraded existing projects to the `/v4` module path, consult the section *Upgrading to v4
+from earlier versions*
+
+The ANTLR runtime has only one external dependency, and that is part of the go system itself:
+
+```
+golang.org/x/exp
+```
+
+A complete list of releases can be found on [the release page](https://github.com/antlr/antlr4/releases). The Go 
+runtime will be tagged using standard Go tags, so release 4.11.0 will be tagged with `v4.11.0` and go get will pick
+that up from the ANTLR repo.
+
+#### 3. Configuring `go generate` in your project
+
+In order to promote the use of repeatable builds, it is often useful to add the latest tool jar to your project's
+repo and configure a `generate.sh` and `generate.go` file. You can of course globally alias the java command required to run the
+tool. Your own CI and dev environment will guide you.
+
+Here is how you can configure `go generate` for your project, assuming that you follow the general recommendation to
+place the ANTLR grammar files in their own package in your project structure. Here is a general template as a starting point:
+
+```
+	.
+	├── myproject
+	├── parser
+	│     ├── mygrammar.g4
+	│     ├── antlr-4.11.0-complete.jar
+	│     ├── error_listeners.go
+	│     ├── generate.go
+	│     ├── generate.sh
+	├── go.mod
+	├── go.sum
+	├── main.go
+	└── main_test.go
+```
+
+Make sure that the package statement in your grammar file(s) reflects the go package they exist in.
+The `generate.go` file then looks like this:
+
+```golang
+	package parser
+
+	//go:generate ./generate.sh
+```
+
+And the `generate.sh` file will look similar to this:
+
+```shell
+
+	#!/bin/sh
+
+	alias antlr4='java -Xmx500M -cp "./antlr4-4.11.0-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
+	antlr4 -Dlanguage=Go -no-visitor -package parser *.g4
+```
+
+From the command line at the root of your package “myproject” you can then simply issue the command:
+
+```shell
+	go generate ./...
 ```
 
-A complete list of releases can be found on [the release page](https://github.com/antlr/antlr4/releases).
+If you have not yet run a `go get`, you can now run `go mod tidy` and update your 
 
-#### 4. Generate your parser
+#### 4. Generate your parser manually
 
 You use the ANTLR4 "tool" to generate a parser. These will reference the ANTLR runtime, installed above.
 
-Suppose you're using a UNIX system and have set up an alias for the ANTLR4 tool as described in [the getting started guide](getting-started.md). To generate your go parser, you'll need to invoke:
+Suppose you're using a UNIX system and have set up an alias for the ANTLR4 tool as described in
+[the getting started guide](getting-started.md). 
+
+To generate your go parser, you'll need to invoke:
 
 ```bash
 antlr4 -Dlanguage=Go MyGrammar.g4
 ```
 
 For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
 
+### Upgrading to `/v4` from the default path
+
+*NB: While switch to new module path would normally imply that the public interface for the runtime has changed, this is
+not actually the case - you will not need to change your existing code to upgrade. The main point of the path change is so
+that git tagging works with the ANTLR Go runtime.*
+
+Prior to release v4.11.0 the Go runtime shipped with a module but the module had no version path. This meant that
+the tags in the ANTLR repo did not work, as any tag above `v1` must refer to a matching module path. 
+So the command `go get github.com/antlr/antlr4/runtime/Go/antlr` would just bring in
+whatever was the `HEAD` of the master branch. While this *kind of* worked, it is obviously subject to problems and does
+not fit properly with the idiomatic ways of Go.
+
+As of v4.11.0 the module path for the Go runtime is properly in sync with the repo tags. However, this means you need to
+perform a few simple actions in order to upgrade to the `/v4` path. 
+
+ - Firstly, make sure that you are using an ANTLR tool jar with a version number of 4.11.0 or greater.
+ - Next you replace any mention of the old (default) path to ANTLR in your go source files. Don't worry that this will
+modify your generated files as...
+ - Now regenerate your grammar files either manually or using `go generate ./...` (see above)
+
+A quick way to replace original module path references is to use this script from your module's base directory:
+
+```shell
+find . -type f \
+    -name '*.go' \
+    -exec sed -i -e 's,github.com/antlr/antlr4/runtime/Go/antlr,github.com/antlr/antlr4/runtime/Go/antlr/v4,g' {} \;
+```
+After performing the steps above, issuing:
+
+```shell
+go mod tidy
+```
+Should fix up your `go.mod` file to reference only the `v4` version of the ANTLR Go runtime:
+
+```shell
+require github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.11.0-xxxxxx-xxxxxxxxx
+```
+
+From this point on, your go mod commands will work correctly with the ANTLR repo and upgrades and downgrades will work
+as you expect. As will branch version such as @dev
+
 ### Referencing the Go ANTLR runtime
 
 You can reference the go ANTLR runtime package like this:
 
-```go
-import "github.com/antlr/antlr4/runtime/Go/antlr"
+```golang
+import "github.com/antlr/antlr4/runtime/Go/antlr/v4"
 ```
 
 ### Complete example
 
-Suppose you're using the JSON grammar from https://github.com/antlr/grammars-v4/tree/master/json.
+Suppose you're using the JSON grammar from https://github.com/antlr/grammars-v4/tree/master/json placed in the parser
+directory and have initialized your `go mod` file.
 
 Then, invoke `antlr4 -Dlanguage=Go JSON.g4`. The result of this is a collection of .go files in the `parser` directory including:
 ```
@@ -61,15 +173,17 @@ json_lexer.go
 json_listener.go
 ```
 
-Another common option to the ANTLR tool is `-visitor`, which generates a parse tree visitor, but we won't be doing that here. For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
+Another common option to the ANTLR tool is `-visitor`, which generates a parse tree visitor, but we won't be doing that here. 
+For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
 
-We'll write a small main func to call the generated parser/lexer (assuming they are separate). This one writes out the encountered `ParseTreeContext`'s. Suppose the gen'ed parser code is in the `parser` directory relative to this code:
+We'll write a small main func to call the generated parser/lexer (assuming they are separate). This one writes out the 
+encountered `ParseTreeContext`'s. Assuming the generated parser code is in the `parser` directory relative to this code:
 
 ```golang
 package main
 
 import (
-	"github.com/antlr/antlr4/runtime/Go/antlr"
+	"github.com/antlr/antlr4/runtime/Go/antlr/v4"
 	"./parser"
 	"os"
 	"fmt"

runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers/Test.go.stg

@@ -1,7 +1,7 @@
 package main
 import (
 	"test/parser"
-	"github.com/antlr/antlr4/runtime/Go/antlr"
+	"github.com/antlr/antlr4/runtime/Go/antlr/v4"
 	"fmt"
 	"os"
 )

runtime-testsuite/test/org/antlr/v4/test/runtime/go/GoRunner.java

@@ -64,7 +64,7 @@ public String[] getExtraRunArgs() {
 		return new String[]{"run"};
 	}
 
-	private static final String GoRuntimeImportPath = "github.com/antlr/antlr4/runtime/Go/antlr";
+	private static final String GoRuntimeImportPath = "github.com/antlr/antlr4/runtime/Go/antlr/v4";
 
 	private final static Map<String, String> environment;
 
@@ -138,4 +138,4 @@ protected CompiledState compile(RunOptions runOptions, GeneratedState generatedS
 	public Map<String, String> getExecEnvironment() {
 		return environment;
 	}
-}
+}

runtime/Go/antlr/v4/LICENSE

@@ -0,0 +1,26 @@
+Copyright 2021 The ANTLR Project
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+    1. Redistributions of source code must retain the above copyright notice,
+    this list of conditions and the following disclaimer.
+
+    2. Redistributions in binary form must reproduce the above copyright notice,
+    this list of conditions and the following disclaimer in the documentation
+    and/or other materials provided with the distribution.
+
+    3. Neither the name of the copyright holder nor the names of its
+    contributors may be used to endorse or promote products derived from this
+    software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

runtime/Go/antlr/v4/antlrdoc.go

@@ -0,0 +1,68 @@
+/*
+Package antlr implements the Go version of the ANTLR 4 runtime.
+
+# The ANTLR Tool
+
+ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing,
+or translating structured text or binary files. It's widely used to build languages, tools, and frameworks.
+From a grammar, ANTLR generates a parser that can build parse trees and also generates a listener interface
+(or visitor) that makes it easy to respond to the recognition of phrases of interest.
+
+# Code Generation
+
+ANTLR supports the generation of code in a number of [target languages], and the generated code is supported by a
+runtime library, written specifically to support the generated code in the target language. This library is the
+runtime for the Go target.
+
+To generate code for the go target, it is generally recommended to place the source grammar files in a package of
+their own, and use the `.sh` script method of generating code, using the go generate directive. In that same directory
+it is usual, though not required, to place the antlr tool that should be used to generate the code. That does mean
+that the antlr tool JAR file will be checked in to your source code control though, so you are free to use any other
+way of specifying the version of the ANTLR tool to use, such as aliasing in `.zshrc` or equivalent, or a profile in
+your IDE, or configuration in your CI system.
+
+Here is a general template for an ANTLR based recognizer in Go:
+
+	.
+	├── myproject
+	├── parser
+	│     ├── mygrammar.g4
+	│     ├── antlr-4.11.0-complete.jar
+	│     ├── error_listeners.go
+	│     ├── generate.go
+	│     ├── generate.sh
+	├── go.mod
+	├── go.sum
+	├── main.go
+	└── main_test.go
+
+Make sure that the package statement in your grammar file(s) reflects the go package they exist in.
+The generate.go file then looks like this:
+
+	package parser
+
+	//go:generate ./generate.sh
+
+And the generate.sh file will look similar to this:
+
+	#!/bin/sh
+
+	alias antlr4='java -Xmx500M -cp "./antlr4-4.11.0-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
+	antlr4 -Dlanguage=Go -no-visitor -package parser *.g4
+
+depending on whether you want visitors or listeners or any other ANTLR options.
+
+From the command line at the root of your package “myproject” you can then simply issue the command:
+
+	go generate ./...
+
+# Copyright Notice
+
+Copyright (c) 2012-2022 The ANTLR Project. All rights reserved.
+
+Use of this file is governed by the BSD 3-clause license, which can be found in the [LICENSE.txt] file in the project root.
+
+[target languages]: https://github.com/antlr/antlr4/tree/master/runtime
+[LICENSE.txt]: https://github.com/antlr/antlr4/blob/master/LICENSE.txt
+*/
+package antlr