all: switch to Go 1.12

Differences in code formatting between Go versions cause constant problems for us (golang/go#25161). Currently we support 1.9 and 1.10. Switch to newer 1.11 and 1.12. Fixes google#1013
dvyukov · Mar 14, 2019 · eb2251e · eb2251e
1 parent 1e82084
commit eb2251e
Show file tree

Hide file tree

Showing 14 changed files with 54 additions and 108 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -2,8 +2,8 @@ language: go
 dist: xenial
 
 go:
-  - "1.9"
-  - "1.10"
+  - "1.11"
+  - "1.12"
 
 before_install:
   - echo $PATH

diff --git a/Makefile b/Makefile
@@ -308,9 +308,8 @@ presubmit:
 presubmit_parallel: test test_race arch check_links
 
 test:
-ifeq ("$(TRAVIS)$(shell go version | grep 1.9)", "true")
-	# Collect coverage report for codecov.io when running on travis (uploaded in .travis.yml).
-	# Note: Go 1.9 does not support -coverprofile when testing multiple packages.
+ifeq ("$(TRAVIS)$(shell go version | grep 1.11)", "true")
+	# Collect coverage report for codecov.io when testing Go 1.12 on travis (uploaded in .travis.yml).
 	env CGO_ENABLED=1 $(GO) test -short -coverprofile=coverage.txt ./...
 else
 	# Executor tests use cgo.

diff --git a/docs/contributing.md b/docs/contributing.md
@@ -46,6 +46,33 @@ Unassigned issues from the [bug tracker](https://github.com/google/syzkaller/iss
 If you want to work on something non-trivial, please briefly describe it on the [syzkaller@googlegroups.com](https://groups.google.com/forum/#!forum/syzkaller) mailing list first,
 so that there is agreement on high level approach and no duplication of work between contributors.
 
+## Go
+
+`syzkaller` is written in [Go](https://golang.org), and a `Go 1.11` or `Go 1.12`
+toolchain is required for build. The toolchain can be installed with:
+
+```
+go get golang.org/dl/go1.12
+go1.12 download
+# Default download path is here.
+~/sdk/go1.12/bin/go version
+export GOROOT=$HOME/sdk/go1.12
+export PATH=$HOME/sdk/go1.12/bin:$PATH
+```
+
+Then get and build `syzkaller`:
+
+``` bash
+go get -u -d github.com/google/syzkaller/...
+cd $HOME?/go/src/github.com/google/syzkaller/
+make
+```
+
+Note: older versions of Go toolchain formatted code in a slightly
+[different way](https://github.com/golang/go/issues/25161).
+So if you are seeing unrelated code formatting diffs after running `make generate`
+or `make format`, you may be using Go 1.10 or older. In such case update to Go 1.11+.
+
 ## How to create a pull request
 
 - First, you need an own git fork of syzkaller repository. Nagivate to [github.com/google/syzkaller](https://github.com/google/syzkaller) and press `Fork` button in the top-right corner of the page. This will create `https://github.com/YOUR_GITHUB_USERNAME/syzkaller` repository.
@@ -64,20 +91,6 @@ This adds git origin `my-origin` with your repository and checks out new branch
 - Change/add files as necessary.
 - Commit changes locally. For this you need to run `git add` for all changed files, e.g. `git add sys/linux/sys.txt`. You can run `git status` to see what files were changed/created. When all files are added (`git status` shows no files in `Changes not staged for commit` section and no relevant files in `Untracked files` section), run `git commit` and enter commit description in your editor.
 - Run tests locally (`make install_prerequisites` followed by `make presubmit`).
-- *Important* If you've run `go fmt` and you're seeing the presubmit fail on
-  `check_diff`, then you may need to use an older version of go to format your
-  code. (Version 1.11 in particular introduced a breaking change, see
-  [here](https://github.com/golang/go/issues/25161) and
-  [here](https://github.com/golang/go/issues/26228) for details). A
-  simple way to do this is:
-
-```
-go get golang.org/dl/go1.10
-go1.10 download
-# Default download path is here.
-~/sdk/go1.10/bin/go fmt [target files]
-```
-
 - Push the commit to your fork on github with `git push my-origin my-branch`.
 - Nagivate to [github.com/google/syzkaller](https://github.com/google/syzkaller) and you should see green `Compare & pull request` button, press it. Then press `Create pull request`. Now your pull request should show up on [pull requests page](https://github.com/google/syzkaller/pulls).
 - If you don't see `Create pull request` button for any reason, you can create pull request manually. For that nagivate to [pull requests page](https://github.com/google/syzkaller/pulls), press `New pull request`, then `compare across forks` and choose `google/syzkaller`/`master` as base and `YOUR_GITHUB_USERNAME/syzkaller`/`my-branch` as compare and press `Create pull request`.

diff --git a/docs/executing_syzkaller_programs.md b/docs/executing_syzkaller_programs.md
@@ -4,10 +4,10 @@ This page describes how to execute existing syzkaller programs for the purpose
 of bug reproduction. This way you can replay a single program or a whole
 execution log with several programs.
 
-1. Setup Go toolchain (if you don't yet have it, you need version 1.9 or higher):
-Download latest Go distribution from (https://golang.org/dl/). Unpack it to `$HOME/go1.9`.
+1. Setup Go toolchain (if you don't yet have it, you need version 1.11 or higher):
+Download latest Go distribution from (https://golang.org/dl/). Unpack it to `$HOME/go1.11`.
 ``` bash
-$ export GOROOT=$HOME/go1.9
+$ export GOROOT=$HOME/go1.11
 $ export GOPATH=$HOME/gopath
 ```
 

diff --git a/docs/fuchsia/README.md b/docs/fuchsia/README.md
@@ -77,7 +77,7 @@ Then run FIDL compiler backend `fidlgen` with syzkaller generator, which compile
 
 To run `syz-ci` locally for Fuchsia, you need:
 
-- Go 1.10 toolchain (in `/go1.10` dir in the example below)
+- Go 1.12 toolchain (in `/go1.12` dir in the example below)
 - bootstrapped Fuchsia checkout (in `/bootstrap/fuchsia` dir in the example below)
 - bootstrap `syz-ci` binary (in the current dir, build with `make ci`)
 - `syz-ci` config similar to the one below (in `ci.cfg` file in the current dir)
@@ -87,7 +87,7 @@ To run `syz-ci` locally for Fuchsia, you need:
 	"name": "testci",
 	"http": ":50000",
 	"manager_port_start": 50001,
-	"goroot": "/go1.10",
+	"goroot": "/go1.12",
 	"syzkaller_repo": "https://github.com/google/syzkaller.git",
 	"managers": [
 		{

diff --git a/docs/linux/setup.md b/docs/linux/setup.md
@@ -66,7 +66,7 @@ For some details on fuzzing the kernel on an Android device check out [this page
 
 ### Syzkaller
 
-`syzkaller` is written in [Go](https://golang.org), a `Go 1.9+` toolchain is required for build.
+`syzkaller` is written in [Go](https://golang.org), a `Go 1.11+` toolchain is required for build.
 
 Go distribution can be downloaded from https://golang.org/dl/.
 Unpack Go into a directory, say, `$HOME/goroot`.

diff --git a/docs/linux/setup_linux-host_android-device_arm64-kernel.md b/docs/linux/setup_linux-host_android-device_arm64-kernel.md
@@ -1,11 +1,8 @@
 # Setup: Linux or Mac OS host, Android device, arm64 kernel
 
 Prerequisites:
- - go1.8+ toolchain (can be downloaded from [here](https://golang.org/dl/))
  - Android Serial Cable or [Suzy-Q](https://chromium.googlesource.com/chromiumos/platform/ec/+/master/docs/case_closed_debugging.md) device to capture console output is preferable but optional. syzkaller can work with normal USB cable as well, but that can be somewhat unreliable and turn lots of crashes into "lost connection to test machine" crashes with no additional info.
 
- - Build syzkaller
-
 In case you have old Android `/dev/ion` driver:
 
 ```sh

diff --git a/docs/linux/setup_linux-host_isolated.md b/docs/linux/setup_linux-host_isolated.md
@@ -53,28 +53,8 @@ Host *
 Before fuzzing, connect to the machine and keep the connection open so all scp
 and ssh usage will reuse it.
 
-## Go
-
-Install Go 1.8.1:
-``` bash
-wget https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz
-tar -xf go1.8.1.linux-amd64.tar.gz
-mv go goroot
-export GOROOT=`pwd`/goroot
-export PATH=$PATH:$GOROOT/bin
-mkdir gopath
-export GOPATH=`pwd`/gopath
-```
-
 ## Syzkaller
 
-Get and build syzkaller:
-``` bash
-go get -u -d github.com/google/syzkaller/...
-cd gopath/src/github.com/google/syzkaller/
-make
-```
-
 Use the following config:
 ```
 {

diff --git a/docs/linux/setup_ubuntu-host_android-device_arm32-kernel.md b/docs/linux/setup_ubuntu-host_android-device_arm32-kernel.md
@@ -15,37 +15,14 @@ These were tested on an NXP Pico-Pi-IMX7D following the instructions [here](http
 
 If feasible, recompile and reinstall the Linux kernel with any debugging options available on your board.
 
-## Install Go
-
-Install Go as follows:
-``` bash
-wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz
-tar -xf go1.9.2.linux-amd64.tar.gz
-export PATH=`pwd`/go/bin:$PATH
-mkdir gopath
-export GOPATH=`pwd`/gopath
-```
-
-## Build syzkaller code
-
-### Initialize a working directory and set up environment variables
-
-Create a working directory. Also make sure GOROOT, GOPATH are defined and exported as instructed earlier.
-
-``` bash
-go get -u -d github.com/google/syzkaller/...
-cd gopath/src/github.com/google/syzkaller/
-mkdir workdir 
-```
-
-### Build syzkaller executables
+## Build syzkaller executables
 
 Run make. 
 ```
 make TARGETOS=linux TARGETARCH=arm
 ```
 
-### Create a manager configuration file
+## Create a manager configuration file
 
 Create a manager config myboard.cfg, replacing the environment
 variables `$GOPATH`, `$KERNEL` (path to kernel build dir for the ARM32 board), and `$DEVICES` (the device ID for your board as reported by adb devices) with their actual values. Change any other flags as needed for your ARM board.

diff --git a/docs/linux/setup_ubuntu-host_qemu-vm_x86-64-kernel.md b/docs/linux/setup_ubuntu-host_qemu-vm_x86-64-kernel.md
@@ -196,29 +196,8 @@ To kill the running QEMU instance:
 kill $(cat vm.pid)
 ```
 
-## Go
-
-Install Go 1.8.1:
-``` bash
-wget https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz
-tar -xf go1.8.1.linux-amd64.tar.gz
-mv go goroot
-export GOROOT=`pwd`/goroot
-export PATH=$GOROOT/bin:$PATH
-mkdir gopath
-export GOPATH=`pwd`/gopath
-```
-
 ## syzkaller
 
-Get and build syzkaller:
-``` bash
-go get -u -d github.com/google/syzkaller/...
-cd gopath/src/github.com/google/syzkaller/
-mkdir workdir
-make
-```
-
 Create a manager config like the following, replacing the environment
 variables `$GOPATH`, `$KERNEL` and `$IMAGE` with their actual values.
 ```
@@ -243,6 +222,7 @@ variables `$GOPATH`, `$KERNEL` and `$IMAGE` with their actual values.
 
 Run syzkaller manager:
 ``` bash
+mkdir workdir
 ./bin/syz-manager -config=my.cfg
 ```
 

diff --git a/pkg/build/akaros.go b/pkg/build/akaros.go
@@ -86,8 +86,8 @@ bash
 		return fmt.Errorf("failed to write image file: %v", err)
 	}
 	for src, dst := range map[string]string{
-		".config": "kernel.config",
-		"key":     "key",
+		".config":                    "kernel.config",
+		"key":                        "key",
 		"obj/kern/akaros-kernel":     "kernel",
 		"obj/kern/akaros-kernel-64b": "obj/akaros-kernel-64b",
 	} {

diff --git a/pkg/build/fuchsia.go b/pkg/build/fuchsia.go
@@ -31,7 +31,7 @@ func (fu fuchsia) build(targetArch, vmType, kernelDir, outputDir, compiler, user
 	}
 	for src, dst := range map[string]string{
 		"out/" + arch + "/obj/build/images/fvm.blk": "image",
-		".ssh/pkey":                                 "key",
+		".ssh/pkey": "key",
 		"out/build-zircon/kernel-" + arch + "-gcc/obj/kernel/zircon.elf": "obj/zircon.elf",
 		"out/build-zircon/multiboot.bin":                                 "kernel",
 		"out/" + arch + "/fuchsia.zbi":                                   "initrd",

diff --git a/pkg/vcs/vcs_test.go b/pkg/vcs/vcs_test.go
@@ -33,12 +33,12 @@ func TestCheckRepoAddress(t *testing.T) {
 		"http://host.xz:123/path/to/repo.git/":                                  true,
 		"https://chromium.googlesource.com/chromiumos/third_party/kernel":       true,
 		"https://fuchsia.googlesource.com":                                      true,
-		"":           false,
-		"foobar":     false,
-		"linux-next": false,
-		"foo://kernel.ubuntu.com/ubuntu/ubuntu-zesty.git":    false,
-		"git://kernel/ubuntu.git":                            false,
-		"gitgit://kernel.ubuntu.com/ubuntu/ubuntu-zesty.git": false,
+		"":                                                                      false,
+		"foobar":                                                                false,
+		"linux-next":                                                            false,
+		"foo://kernel.ubuntu.com/ubuntu/ubuntu-zesty.git":                       false,
+		"git://kernel/ubuntu.git":                                               false,
+		"gitgit://kernel.ubuntu.com/ubuntu/ubuntu-zesty.git":                    false,
 	})
 }
 

diff --git a/prog/prog_test.go b/prog/prog_test.go
@@ -219,11 +219,11 @@ func TestEscapingPaths(t *testing.T) {
 		"file/../../file":        true,
 		"../file":                true,
 		"./file/../../file/file": true,
-		"":          false,
-		".":         false,
-		"file":      false,
-		"./file":    false,
-		"./file/..": false,
+		"":                       false,
+		".":                      false,
+		"file":                   false,
+		"./file":                 false,
+		"./file/..":              false,
 	}
 	for path, want := range paths {
 		got := escapingFilename(path)