Packaging in Golang

This article assumes a basic familiarity with the Go language. Check out Golang if you’re not familiar! Go is one of those languages that I love to use but haven’t gotten a chance to do a major project in yet. So I thought I’d write a quick article about how packaging works within the Go ecosystem, as it is a bit different than what most people are familiar with coming from Python, Ruby, etc.

Golang provides excellent documentation for its packaging system and my article follows it’s concepts pretty closely.

Workspaces

All go code is developed, maintained, and executed inside of workspaces. A workspace is a folder that contains the following subfolders:

# I use a folder called `projects` to store all my code
mkdir /Users/rlbaker/projects/go;# `bin` contains all command-line executables will be available in the bin folder
mkdir bin# `pkg` contains package objects which will be covered later
mkdir pkg# `src` is where your .go source files will be kept. 
# this folder will be organized into one subdirectory per package
mkdir src

Go gives us a lot of flexibility in the choice to use one or multiple workspaces. The usage of one workspace is much more manageable thanks to godep, a dependancy management tool that is quite simple to use. It produces a Godeps/Godeps.json that is similar to the package.json of Node.js or Ruby Bundler Gemfiles. Godeps will be covered further down.

Setting the GOPATH

$GOPATH is an environment variable that allows the Go runtime to find the current go environment. It needs to be set to do any sort of development. It also helps to export the $GOPATH/bin folder to your system path so that any generated executables are also easily accessible.

# Inside ~/.bashrc or ~/.zshrc
export GOPATH=/Users/rlbaker/projects/go;
export PATH=$PATH:$GOPATH/bin;# Make sure to reload your shell or source your config file to put these environment variables into effect

To change your go workspace, all you must do is modify the GOPATH to point to a different directory.

Package Naming Conventions

The packages in the go standard library are quite succinct. The packaging documentation provides the examples fmt and net/http. Packages you develop yourself will need more descriptive names as not to collide with packages provided from both the standard library and other developers. The common naming scheme used for third-party programs and libraries is:

<remote>/<user>/<package_name># Example
github.com/rlbaker/my_first_go_package
bitbucket.org/rlbaker/my_bitbucket_package

You can then create a directory for your user and project inside of your go workspace.

mkdir -p $GOPATH/src/github.com/rlbaker/my_first_go_package

This directory can then either have the source of a pre-existing project cloned into it or be a fresh start for a new package.

It really is that simple. At the end of this, I’ll walk through creating a golang library from start to finish to provide an example of all of these concepts.

Executables

If you are developing an executable application, it is important to remember that all executables must use the following at the top of the file in order to work correctly.

package main

This allows go to discover the “main” function for your program.

Go provides two methods of compiling your executable. The first is to perform a go build which will build your executable inside of the source directory. The other option is to perform a go install which will place the executable inside $GOPATH/bin. If you added that to your system path as above, this will let you access it from anywhere on your system.

Libraries

A library’s name should usually match its its directory name except under specific circumstances or larger projects. For example, if you’re working on github.com/rlbaker/redis/client.go, the following would be a good package name.

# Example first line of .go file
package redis

When working with libraries, you will need to perform go install to make the library accessible to other packages. This compiles your library and places it in a system-architecture specific folder inside the pkg folder we created when setting up our workspace.

Below is the output of tree after running go install inside of an example library $GOPATH/src/github.com/rlbaker/redis:

[ryans-mbp] ~/projects/go/pkg
→ tree
.
└── darwin_amd64
    └── github.com
        └── rlbaker
            └── redis.a

Namespacing

An important concept to understand in golang is how names are exported from packages. Some languages have keywords such as public/private/protected to specify package visibility but golang takes a unique approach of using capital letters to signify public functions. As you look through the go standard library documentation, every function has a leading capital letter. This is how go determines which functions will be accessible outside the package.

package sandboximport (
        "fmt"
)func hello() {
        fmt.Printf("Can't see me\n")
}func World() {
        fmt.Printf("But you can see me\n")
}func Greetings() {
        hello()
}

In the above example, the World function would be able to be called from outside the package but the hello function would not be accessible. However, you would be able to call hello from inside another function in the sandbox package.

Godep

Godep is a tool that was created of somewhat necessity. When working with multiple projects on the same system, headaches are often caused by different projects needing the same package, but slightly different versions. All programming languages solve this problem in a slightly different way. There is the heavy-handed Python approach of virtual environments and the comparatively lightweight Ruby Bundler approach.

Godep is closer to Bundler than a Virtual Environment. It requires the packages to be under some form of version control because Golang does not assign version numbers to packages. So instead, Godep looks at the version control revision number and uses that instead.

I won’t spend a lot of time going over Godep here, but in short it allows us to make sure that the package has the correct dependancies no matter what the current state of the workspace looks like. It is best to start at the Godep repo to learn more about it.

Tying it all together

To bring this all together, we’re going to create a go library and use it in a separate executable. We’ll start from scratch by creating our workspace.

# Create our go workspace
mkdir -p ~/projects/go
mkdir -p ~/projects/go/src
mkdir -p ~/projects/go/pkg
mkdir -p ~/projects/go/binexport GOPATH=/Users/rlbaker/projects/go
export PATH=$PATH:$GOPATH/bin

Next we’ll create the package for our executable which we’ll call demo and initialize a git repository.

# Create a new package for our executable
cd ~/projects/go/src
mkdir -p github.com/rlbaker/demo
cd github.com/rlbaker/demotouch demo.gogit init
git add demo.go
git commit -m "Initial commit"

Opening up demo.go, we’ll add the following. This is temporary until we’ve created our library and can import it.

package mainimport (
        "fmt"
)func main() {
        fmt.Println("Hello, world!")
}

We’ll then commit the changes

# Commit changes to demo
git add demo.go
git commit -m "Stub code for main"

and begin setting up a library package.

# Create a new package for the library
cd ~/projects/go/src
mkdir -p github.com/rlbaker/calc
cd github.com/rlbaker/calctouch calc.gogit init
git add calc.go
git commit -m "Initial commit"

Open calc.go and add the following

package calcfunc Add(a int, b int) int {
        return a + b
}

Commit these changes

git add calc.go
git commit -m "Initial library implementation"

and build our library!

# Builds the library and places it inside the pkg directory
go install

Switch back to our executable package and lets get working with it. Open up demo.go and modify it so it looks similar to below

package mainimport (
        "fmt"
        "github.com/rlbaker/calc"
)func main() {
    fmt.Printf("2 + 2 = %v\n", calc.Add(2, 2))
}

Commit our changes and build the executable!

git add demo.go
git commit -m "Utilize the library package"go build
./demo
=> 2 + 2 = 4

Now that we’ve successfully build a library and executable, it might be handy to record this dependency in the executable repo.

cd ~/projects/go/src/github.com/rlbaker/demo
godep save

This creates a Godeps directory inside the package which can be committed to the git repo. If anyone else wished to install your executable, they could now easily install the correct dependency version. Check out Godeps/Godeps.json if you’re interested in seeing how it resolves dependancies based on git revisions and package names.

That should just about do it! While I mainly wrote this to help teach myself the intricacies of go packages, I hope that it is useful to others as well.