Upgrading Go from v1.17 to v1.18

Our experience migrating our code base to access generics
Alec Fong
·
·
June 10, 2022
5 min
read

Why we upgraded

At Brev.dev, We recently upgraded from v1.17 to v1.18 to access generics, which allows us to build type-generic libraries. While there is a lot of controversy in the go community over this functionality, our personal experience has been very positive with generics in other languages so we decided to be early adopters.

Our first use of generics has been for a shared collections library, importing functions from a typical functional collections library in a language like Haskell, Scala, or similar functional libraries in dynamic languages such as lo-dash, underscore and rambda for JS, and raskell for Ruby.

While 1.18 itself works flawlessly so far, the tooling around it, specifically with respect to golangci-lint, led to a memory leak and later panic crashes which the below will provide means of recognition and methods for handling until all of the linters you use are compatible with 1.18.

How we upgraded

Installing or upgrading to golang 1.18 is straightforward - all we need to do is update the binary and edit the version number in the go.mod file.

## on os x curl https://go.dev/dl/go1.18.1.darwin-amd64.pkg . open go1.18.1.darwin-amd64.pkg # and then follow the on-screen instructions # you may need to add $HOME/go/bin to your path as well if it isn't already rm go1.18.1.darwin-amd64.pkg

Once the go binary is upgraded you’ll be able to compile and build go 1.18 code. Don’t forget to update your IDE extensions, plugins, and other dev tools. Specifically, if using the vscode Go extension, make sure to open the Command Palette (Cmd+Shift+P) and run Go: Install/Update Tools .

For golangci-lint, make sure the version in your go.mod is 1.45 or later. Finally, if you are not the only person on this project, you'll need to communicate to the rest of the team(s) how to upgrade. If that doesn't sound appealing, or you don't want to manually install these things yourself, consider using a developer environment management tool like Docker, Nix or Brev.

Issues we encountered

There are still a few kinks in the tooling that need working out. Additionally, one must be careful when upgrading - we found failing to fully upgrade tools such as the linter will result in memory leaks. We have a separate go.mod for dev tools such as golangci-lint, goreleaser, and gofumpt. In the case of golangci-lint, we observed an inexplicable memory leak when not upgrading golangci-lint from v1.42 to v1.45. If you want to reproduce this memory leak:

git clone https://github.com/brevdev/brev-cli git checkout lintMemoryLeak

Running make lint will reproduce the issue which looks like a process using gigabytes of memory until the process is killed our consumes all available memory. After upgrading golangci-lint to v1.45, which is the first version compatible with 1.18, our linting tools no longer leaked memory, but still crashed on generic code.

➜ brev-cli git:(lintCrash) ✗ make lint Executing target: lint golangci-lint run --timeout 5m ERRO [runner] Panic: buildssa: package "importpkg" (isInitialPkg: true, needAnalyzeSource: true): T: goroutine 7142 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:24 +0x65 github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyzeSafe.func1() /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_action.go:101 +0x155 panic({0x142bb60, 0xc00380e7b0}) /usr/local/go/src/runtime/panic.go:838 +0x207 golang.org/x/tools/go/ssa.(*Program).needMethods(0xc00399bc70, {0x16e37d0?, 0xc00380e7b0?}, 0x0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:237 +0x5b1 golang.org/x/tools/go/ssa.(*Program).needMethods(0xc00399bc70, {0x16e37a8?, 0xc0037ecbe8?}, 0x0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:233 +0x708 golang.org/x/tools/go/ssa.(*Program).needMethods(0xc00399bc70, {0x16e3708?, 0xc0003e4780?}, 0x0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:209 +0x448 golang.org/x/tools/go/ssa.(*Program).needMethods(0xc00399bc70, {0x16e37a8?, 0xc0037ecc18?}, 0x0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:233 +0x708 golang.org/x/tools/go/ssa.(*Program).needMethods(0xc00399bc70, {0x16e3708?, 0xc0003e4700?}, 0x0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:209 +0x448 golang.org/x/tools/go/ssa.(*Program).needMethodsOf(0xc00399bc70, {0x16e3708?, 0xc0003e4700?}) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/methods.go:145 +0x70 golang.org/x/tools/go/ssa.(*Package).build(0xc0039bfda0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/builder.go:2284 +0x111 sync.(*Once).doSlow(0xc00399bc70?, 0xc0038080a0?) /usr/local/go/src/sync/once.go:68 +0xc2 sync.(*Once).Do(...) /usr/local/go/src/sync/once.go:59 golang.org/x/tools/go/ssa.(*Package).Build(...) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/ssa/builder.go:2272 golang.org/x/tools/go/analysis/passes/buildssa.run(0xc00399bba0) /home/brev/go/pkg/mod/golang.org/x/tools@v0.1.10/go/analysis/passes/buildssa/buildssa.go:72 +0x2ee github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyze(0xc0035ccef0) /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_action.go:187 +0x9c4 github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyzeSafe.func2() /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_action.go:105 +0x1d github.com/golangci/golangci-lint/pkg/timeutils.(*Stopwatch).TrackStage(0xc0011de6e0, {0x14974bf, 0x8}, 0xc002c99f48) /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/timeutils/stopwatch.go:111 +0x4a github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*action).analyzeSafe(0xc00023cf60?) /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_action.go:104 +0x85 github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze.func2(0xc0035ccef0) /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_loadingpackage.go:80 +0xb4 created by github.com/golangci/golangci-lint/pkg/golinters/goanalysis.(*loadingPackage).analyze /home/brev/go/pkg/mod/github.com/golangci/golangci-lint@v1.45.0/pkg/golinters/goanalysis/runner_loadingpackage.go:75 +0x1eb WARN [runner] Can't run linter goanalysis_metalinter: goanalysis_metalinter: buildssa: package "importpkg" (isInitialPkg: true, needAnalyzeSource: true): T ERRO Running error: 1 error occurred: * can't run linter goanalysis_metalinter: goanalysis_metalinter: buildssa: package "importpkg" (isInitialPkg: true, needAnalyzeSource: true): T make: *** [Makefile:50: lint] Error 3

This crash occurs because not all linters currently support go’s new generic syntax, and we were using several of them configured in the .golangci.yml config file. Note that all non-generic code works fine it is specifically the generic syntax which breaks many of the linters. We found the following linters did not support generic syntax.

- gosimple - unused - bodyclose - noctx - rowserrcheck - sqlclosecheck - stylecheck - tparallel - unparam

If you would like to reproduce this issue, git clone https://github.com/brevdev/brev-cli ,  git checkout lintCrash, and make lint. To see the linters that, when removed, prevent the crash, git checkout lintCrashFixed, and make lint. For more information on golang 1.18 linting-related issues see this Github issue.

A (Temporary) Fix

Until these issues are resolved, there were two approaches we considered taking: comment out the broken linters and don’t use them on any of our code, or isolate  the generic code and disable linting for it. We decided to pull out all generic code to separate packages, and disable the linter for those files. Disabling disabling linting for a specific file can be done with the //go:build !codeanalysis comment at the top of the file seen here. Then on each import of the generic package, we also disable the “typecheck” linter (the compiler will still fail if this code is not typed correctly).

import "github.com/brevdev/brev-cli/pkg/collections" //nolint:typecheck // uses generic code

While not an ideal solution, we believe that the ecosystem will iron itself out quickly, and we eagerly await updates to golangci-lint, which we very much enjoy using.

Final Thoughts

Although our initial onboarding experience with the Go 1.18 ecosystem has not been the smoothest our experience with generics has been overwhelmingly positive. If your project could benefit from generics then the above experience report will hopefully ease the transition for you and your team.