Getting Go onto Internet of Things
How to compile Go code for embedded Linux targets using Yocto
Go is primarily known as the language for creating high performance cloud applications. What developers may overlook is that Go is also great for developing Internet of Thing devices. Its ability to efficiently handle requests, ease of development, reasonable memory footprint and ability to interface well with C, make it a great alternative to Java, C++ or Python for IoT projects.
The biggest challenge for most Go developers when targeting an IoT device is not writing the code, but compiling it. There are several techniques that can be used to cross compile, all of which will require different flags to be passed to the Go compiler. The limited disk space of the embedded target might also incentivize developers to opt for non-traditional binary architectures, such as enabling linked libraries, which can complicate build flags even more. For these reasons, Go embedded compilation can be especially frustrating for developers coming from cloud platforms where compilations ‘just worked’.
In this post I’ll explain how to simplify Go compilation for IoT, embedded boards by using the Yocto Build system. I will first go over why I choose Yocto, as well as how it works. I will not go into how to set up the Yocto development environment for a board because this is normally provided by manufactures or the open source community. Instead, I will focus on how to compile Go code via the creation of Yocto input file type: bitbake recipes. This will be done by showing a very simple recipe that will compile the Official Go Project Example and then examining the output Yocto generates from it. Next, I will go over some common parameters that are customized in a Go recipe to tailor a build. Then I will go over some more advanced Go recipes types that may need to be used under special circumstances. Finally I will briefly discus my experience with using Yocto to compile Go.
Why Cross Compile with Yocto?
There are many build systems that aim to help automate cross compiling, but none that cater to both the novice and the professional as much as Yocto. It is very popular with system-on-a-chip/module/board manufacturers and also has support for hobby embedded boards (BeagleBone board and Raspberry Pi). Yocto works by breaking the process of building a Linux image into components and allows for adaptable compilation instructions to be given for each components via ‘bitbake recipes’ (the primary tool used by Yocto is called bitbake). The recipes take information regarding the target and compile instructions that use them, to generate an image (OS+applications) or a deployment package (apt-get,dnf,yum,etc.). To simplify the creation of recipes, there are recipe libraries, bitbake classes and include files, that greatly simplify compilation for specific languages. Recent version of Yocto, Rocko onward, have extended Go support via such libraries. The result is that a 10 line bitbake recipe can allow a Go application to be built on the hundreds of boards supported by Yocto.
The Basic, Default Recipe
The following is a working bitbake recipe for the Go Example Project. This section will explain how it was created as well as some of the basic compile mechanisms that it controls. This recipe does require an additional recipe for the golang/x libraries which is given and explained in the Advanced Recipe section. Once both files are in the Yocto search path then Go Example Project can be built.
Creation of Basic Go Recipe
//github.com-golang-example.bb DESCRIPTION = "The Go Example Project"SRC_URI = "git://github.com/golang/example.git;protocol=https;"
SRCREV = "${AUTOREV}"
LICENSE = "Apache-2.0"
LIC_FILES_CHKSUM = "file://${WORKDIR}/${PN}-${PV}/src/${GO_IMPORT}/LICENSE;md5=3b83ef96387f14655fc854ddc3c6bd57"DEPENDS += "golang.org-x"inherit go
GO_IMPORT = "github.com/golang/example"
Go bitbake recipe files are named according to certain guidlines. The Yocto/Go community came up with the idea to make the filename mirror the Go import string as much as possible by replacing ‘ /’ with ‘ -’ to simplify dependence management. There is some division regarding if the filenames should keep the top-level domain in the Go import path, so this or may not be in other developer’s recipe filenames. For the Go Example Project, the Go import path would be github.com/golang/example so the recipe filename should be github.com-golang-example.bb .
The simple recipe contains several fields that are not unique to Go recipes that pertain to the description, source location and license. The description field is meant to be a quick blurb explaining what the recipe is for. The SRC_URI and SRCREV fields are meant to tell Yocto where to find the source. In this case the SRC_URI field tells yocto to download the source from github using git via https and the SRCREV field has the special values of ${AUTOREV}to tell Yocto to grab from the repositories head. These fields can also be tailored to select specific branches or commits of the code for tighter version control. One default behavior that is different in Go recipes is that the code is not downloaded to ${WORKDIR}/${PN}-${PV}/src/, but instead to ${WORKDIR}/${PN}-${PV}/src/${GO_IMPORT} because the GOPATH is being created at ${WORKDIR}/${PN}-${PV}. The license fields, LICENSE and LIC_FILES_CHKSUM, convey the license file type, location and checksum. In this example, we tell Yocto that the license file is in the cloned git repository and provide its MD5 checksum. If your project does not have a standard license or a license at all, then replaced the LICENSE field value with CLOSED or find a comparable licence in ${COMMON_LICENSE_DIR} . These fields are present on the majority of Yocto recipes, so if there is any trouble then general tutorials on Yocto should provide answers.
Dependencies also need to be conveyed in the recipe. This is done by adding other recipe names to the DEPENDS field. We cannot use the go get command so we have to replace this functionality with other Yocto recipes. While this can be troublesome, recipes can be reused by other packages allowing for easier reuse. For our example, the Go Example Project requires the golang-x libraries. The recipe for these tightly coupled libraries is more advanced and I will go over it in the Advanced Recipes-Mulipackage Recipe section. For the time being we just need to know that the recipe name for this is golang.org-x.bb and add it as a dependency by inserting DEPENDS += "golang-x" into the recipe.
The final step is to add the Go recipe libraries. The include go statement inherits the go.bbclass which adds Go compile functions and modify existing behaviors behind the scenes. This is why the recipe downloads the source code to a special directory using the GO_IMPORT field value. The statement also adds other special parameters to our recipe that we will explore in later sections.
Building and Result
Assuming you have activated your Yocto environment, building this project is very simple. First, make sure that github.com-golang-example.bband golang.org-x.bb are in a directory that Yocto can find them in(putting them next to other usablebb files will work if you’re having trouble). Next, if the project has been built before you will likely need to remove old artifacts with the command bitbake -c cleanall github.com-golang-example. Then run the command bitbake github.com-golang-example to begin the primary build. This will automatically build the golang-x libraries as well as all the other tools needed for Go compilation.
During the compilation, Yocto is will try to compile the code and then package it for deployment. The compiler flags are abstracted by the recipes, as well as what binaries are put into what packages. Normally the package type is an RPM, but it can be other formats. Upon a successful standard build you should be able to find the RPMs containing the name of the recipe github.com-golang-example in the <YOCTO-DIR>/tmp/deploy/rpm directory. The RPM whose name begins with github.com-golang-example-1.0-r0 contains the primary application and can be installed onto your target using dnf or rpm.
Error:
Problem: conflicting requests
- nothing provides go-runtime >= 1.9.4 needed by github.com-golang-example-1.0-r0.cortexa7hf_neonWhen you try to install this RPM you will probably get the error above. The reason for this is that the default bitbake compiler configuration for Go applications generate binaries in the shared mode and links them to the go-runtime library. You can read more about the shared build-mode here. This is likely done to reduce the binary sizes for embedded targets so many Go apps can all share the runtime. The go-runtime-1.9.4 RPM should be in the same folder you found the main application RPM and can be installed without any other dependency. After the go-runtime package is installed you should be able to install the main RPM without error and all the generated binaries will be placed in /usr/bin .
The Other Generated RPMs
In addition to the RPM containing the compiled Go applications, there are four other generated RPMs generated by default. I will briefly go over the purpose of each of these RPMs as well as what they contain for our example project.
pTest RPM— github.com-golang-example-ptest-1.0-r0
Go bitbake recipes by default create an rpm filled with compiled Go tests written associated with a package. If you install this rpm you can run all the tests by going into the corresponding directory in /usr/lib and running run-ptest. Below is an example of the ptest being run:
:~# /usr/lib/github.com-golang-example/ptest/run-ptest
=== RUN TestIsTagged
--- PASS: TestIsTagged (0.02s)
=== RUN TestIntegration
--- PASS: TestIntegration (0.04s)
PASS
=== RUN TestReverse
--- PASS: TestReverse (0.00s)
PASS
PASS: github.com-golang-exampledev RPM — github.com-golang-example-dev-1.0-r0
The static dev RPM contains all the source files used for development and will install them into a /usr/lib/go. This can be used as another way to control source. In the case of the Go Example Project, the static dev RPM would install:
/usr/lib/go/src/github.com/golang/example/LICENSE
/usr/lib/go/src/github.com/golang/example/README.md
/usr/lib/go/src/github.com/golang/example/appengine-hello
/usr/lib/go/src/github.com/golang/example/appengine-hello/README.md
/usr/lib/go/src/github.com/golang/example/appengine-hello/app.go
/usr/lib/go/src/github.com/golang/example/appengine-hello/app.yaml
/usr/lib/go/src/github.com/golang/example/appengine-hello/static
...
...
...
/usr/lib/go/src/github.com/golang/example/stringutil/reverse_test.go
/usr/lib/go/src/github.com/golang/example/template
/usr/lib/go/src/github.com/golang/example/template/image.tmpl
/usr/lib/go/src/github.com/golang/example/template/index.tmpl
/usr/lib/go/src/github.com/golang/example/template/main.godbg RPM — github.com-golang-example-dbg-1.0-r0
The DBG package contains debug symbols for generated applications and tests. The debug symbols can be used by gdb to actively debug Go code. For the Go Example Project the dbg RPM would install:
/usr/bin/.debug/defsuses
/usr/bin/.debug/doc
/usr/bin/.debug/gotypes
/usr/bin/.debug/hello
/usr/bin/.debug/hugeparam
/usr/bin/.debug/implements
/usr/bin/.debug/lookup
/usr/bin/.debug/nilfunc
/usr/bin/.debug/outyet
/usr/bin/.debug/pkginfo
/usr/bin/.debug/skeleton
/usr/bin/.debug/template
/usr/bin/.debug/typeandvalue
/usr/lib/github.com-golang-example/ptest/github.com/golang/example/outyet/.debug/outyet.test
/usr/lib/github.com-golang-example/ptest/github.com/golang/example/stringutil/.debug/stringutil.test
/usr/src/debug/github.com-golang-example/1.0-r0/build/src/github.com/golang/example/outyet/main.go
/usr/src/debug/github.com-golang-example/1.0-r0/build/src/github.com/golang/example/outyet/main_test.go
/usr/src/debug/github.com-golang-example/1.0-r0/build/src/github.com/golang/example/stringutil/reverse.go
/usr/src/debug/github.com-golang-example/1.0-r0/build/src/github.com/golang/example/stringutil/reverse_test.gostaticdev RPM— github.com-golang-example-staticdev-1.0-r0
The staticdev RPM contains the compiled Go libraries used in the package. These can be used to speed up compilations. For the Go Example Project, this RPM would install:
/usr/lib/go/pkg/linux_arm_dynlink/github.com/golang/example/appengine-hello.a
/usr/lib/go/pkg/linux_arm_dynlink/github.com/golang/example/stringutil.aModifying the Basic Recipe
Generate a Single, Non-linked Executable
Go bitbake recipes by default produce applications which link to the go runtime library. This can complicate deployment and dependency management. For these and other reasons, you may want to have Yocto build Go application as the default binary type of a completely contained application. To do this just add GO_LINKSHARED = ""to the recipe. The result is that the main RPM will have the same name, but will not require the go-runtime rpm be installed.
Scoping Builds
Go bitbake recipes by default try to build all possible products in a project. This is done by issuing the Go compiler the ... argument at the end of the Go import string. This can be troublesome when you only want to compile a library and not the associated commands. To prevent this you can scope what is built by altering the GO_INSTALL field in the recipe. For example, to scope the build to just the package in the top level directory, you would enter GO_INSTALL = "${GO_IMPORT}” into the recipe.
Disabling Tests
Sometimes tests require additional packages or are not ready to be run in the target environment and developers would prefer that they are not built. To tell Yocto to not build tests, you can add PTEST_ENABLED="0"to the recipe. This will prevent the ptest rpm from being generated.
Advanced Recipes
Multi-package Recipes — Golang.org-X libraries
There are some Go packages that are so coupled with other packages that it is best to group them together into one Yocto recipe. The most common example of this is the golang-x libraries. They are required by many Go projects and also depend on each other. The following is multi-package bitbake recipe that will import all the golang-x recipes in order to ensure dependencies are met.
//golang.org-x.bb
DESCRIPTION = "Go X libraries"SRC_URI = "\
git://github.com/golang/net.git;protocol=https;name=net;destsuffix=${PN}-${PV}/src/golang.org/x/net \
git://github.com/golang/text.git;protocol=https;name=text;destsuffix=${PN}-${PV}/src/golang.org/x/text \
git://github.com/golang/tools.git;protocol=https;name=tools;destsuffix=${PN}-${PV}/src/golang.org/x/tools \
git://github.com/golang/crypto.git;protocol=https;name=crypto;destsuffix=${PN}-${PV}/src/golang.org/x/crypto \
git://github.com/golang/sys.git;protocol=https;name=sys;destsuffix=${PN}-${PV}/src/golang.org/x/sys \
"
SRCREV_text = "${AUTOREV}"
SRCREV_net = "${AUTOREV}"
SRCREV_crypto = "${AUTOREV}"
SRCREV_tools = "${AUTOREV}"
SRCREV_sys = "${AUTOREV}"LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"inherit go
GO_IMPORT = "golang.org/x"
This recipe is different from the basic Go recipe in a few ways. First all the SRC_URI values given have two extra parameters: name and destsuffix. The name parameter is used to identify the source as part of a subpackage. The destsuffix parameter is to tell Yocto where to put the source manually. This is required because each of these packages have their own GO_IMPORT value so we cannot rely on the default behavior. The next difference is that each subpackage has its own SRCREV field. Right now they are all set to pull from the head of their corresponding repository, but this can be used to version control them individually. The final difference is that the LICENSE referenced is not in a repository but built into Yocto. This is not not unique to multi-package recipes, and is only required in the case because the libraries have an atypical license (the Go License).
Cgo Recipes — github.com-mattn-go-sqlite3
When cross-compiling, you must specify a C cross-compiler for cgo to use. — cgo GoDoc
Cgo complicates Yocto recipes in two ways. The first way is that when cross compiling, the Go compiler must explicitly be told that cgo is being used, unlike traditional Go compilation that just figures this out.To do this in Yocto, CGO_ENABLED = "1"must be added to the recipe. The second way is that C dependencies have to be accounted for. This includes the library and may include linker flags depending on cgo directives in the code. Libraries are normally handed by adding their recipe as a dependency via DEPENDS. Linker flag are handled by LDFLAGS and values are given just like gcc arguments ( IE: LDFALGS = "-lpthread” for the pthread library). There are many other flags being set by Yocto under the hood and overall Yocto simplifies cgo cross compilation greatly.
//github.com-mattn-go-sqlite3.bb
DESCRIPTION = "The mattn SQLite3 Go Library"
GO_IMPORT = "github.com/mattn/go-sqlite3"inherit goDEPENDS="sqlite3"SRC_URI = "git://github.com/mattn/go-sqlite3.git;protocol=https;"
SRCREV = "${AUTOREV}"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${S}/src/${GO_IMPORT}/LICENSE;md5=2b7590a6661bc1940f50329c495898c6"CGO_ENABLED = "1"
Discussion
Yocto has really started to hit its stride with Go development. Before the Rocko version, there was several different versions of Go recipe libraries being used, all of which would require different options be set in the Go recipes that used them. This meant it was hard to create recipes for dependencies that everyone could use. With the Rocko version and later, Yocto has standardized the Go recipe libraries and greatly extended their capabilities. Overall it has made the process of compiling Go very easy.
The one shortfall of Yocto and Go right now is dependency management. There is not a central repository containing Go package recipes, though this should be easier to create now with the Go standard recipe libraries. Ideally once Go starts to standardize version control more with vgo , Yocto recipe’s can be generated programmatically. In the mean time, recipe writing is not that difficult so even if a project requires 10 packages, you can normally write them all within an hour. While a little annoying, the amount of functionality Yocto provides greatly outweighs this.
