Seven Story Rabbit Hole

Sometimes awesome things happen in deep rabbit holes. Or not.

   images

A Successful Git Branching Model With Enterprise Support

This further extends A Slight Tweak on a Successful Git Branching Model with the addition of the concept of support branches.

diagram

Release Branches

  • When completed the release branch would be merged into both the master and stable branches, the commit on the stable branch would be tagged with a release tag (eg, 1.0.0).

  • The release branch would be discarded after being merged back into master and stable.

  • Release branches would be named “release/xxx”, where xxx is the target release tag for that release. Eg, “release/1.0.0”.

  • Release branches should only have bugfixes related to the release being committed to them. All other changes should be on feature branches or the master branch, isolated from the release process.

  • Release branches would help avoid making developers having to “double-commit” bugfixes related to a release to both the release branch and the master branch — because the release branch will be merged into master at the time of release, they only need to commit the fix to the release branch.

  • Release branches should be periodically merged back into the master branch if they run longer than normal (eg, if it was expected to last 3 weeks and ended up lasting 8 weeks), rather than waiting until the time of release. This will reduce the chance of having major merge conflicts trying to merge back into master.

  • When a release is ready to be tagged, if the release branch does not easily merge into master, it is up to the dev lead on that team to handle the merge (not the build engineer). In this case, the build engineer should not be blocked, because the merge into stable will be a fast-forward merge, and so the release can proceed despite not having been merged into master yet.

Support Branches

  • Support branches would be created “on demand” when requested by customers who are stuck on legacy releases and are not able to move forward to current releases, but need security and other bug fixes.

  • Support branches should be avoided if possible, by encouraging customers to move to the current release, because they create extra work for the entire team.

  • Support branches would follow a similar naming scheme and would be named “support/xxx”, where xxx is the release tag from which they were branched off of. Eg, “support/1.0.1”.

  • Support branches are essentially dead-end branches, since their changes would unlikely need to be merged back into master (or stable) as the support branch contains “ancient code” and most likely those fixes would already have been integrated into the codebase.

  • If a customer is on the current release, then there is no need to create a support branch for their required fix. Instead, a hotfix branch should be used and a new release tag should be created.

Hotfix Branches

  • Hotfix branches would branch off of the stable branch, and be used for minor post-release bugfixes.

  • Hotfix branches would be named “hotfix/xxx”, where xxx might typically be an issue id. Once their changes have been merged into master and stable, they should be deleted.

  • Hotfix branches are expected to undergo less QA compared to release branches, and therefore are expected to contain minimum changes to fix showstopper bugs in a release. The changes should not include refactoring or any other risky changes.

  • If it’s being branched off the master branch, it’s not a hotfix branch. Hotfixes are only branched off the stable branch.

  • Hotfix branches should verified on the CI server using the automated QA suite before being considered complete.

  • After being accepted by QA, hotfix branches are merged back into master and stable, and the latest commit on stable is tagged with a release tag. (eg, 1.0.1)

  • Similar to release branches, if hotfixes do not easily merge back into master, the build engineer would assign the dev lead the responsibility for completing the merge, but this should not block the release. However since hotfix branches are so short-lived, this is very unlikely to happen.

Stable Branch

  • The stable branch would represent the “released” mainline development.

  • The latest commit on stable should always correspond to the latest release tag.

  • All release tags should be made against commits on stable, except for those on legacy support branches.

  • Developers who wanted to subscribe to the latest released code would follow the stable branch.

Master Branch

  • The master branch would represent the “as-yet-unreleased” mainline development.

Feature Branches

  • All non-trivial changes should be done on feature branches and undergo code review before being merged into the master branch.

A Slight Tweak on a Successful Git Branching Model

A great “best practices” for juggling git branches in a release cycle is A successful git branching model. It is also accompanied with a tool called git flow that makes it easy to implement in practice.

It does however have one major issue: many people use a different naming scheme.

Instead of:

  • Master – the latest “stable” branch
  • Development – bleeding edge development branch

a slightly more common naming pattern is:

  • Master – bleeding edge development branch
  • Stable – the latest “stable” branch

To that end, I’ve tweaked the original diagram to be.

diagram

Every branch solves a problem

The natural reaction to most people seeing this diagram is dude, that’s way too many branches, this is way more complicated than it needs to be. Actually, I’d argue that it’s minimally complex to solve the problems that these branches are designed to solve. In other words, each type of branch justifies it’s existence by the problem it’s designed to solve.

From the perspective of a library developer (in my case, Couchbase Lite for Android), here are the problems each branch is intended to solve.

Permanent and External Facing Branches

These branches are permanent, in the sense they have a continuous lifetime. They are also external, and consumers of the library would typically depend on one or both of these branches

Master

  • A home for the latest “feature complete” / reviewed code.
  • Anyone internal or external who wants to stay up to date with latest project developments needs this branch.

Stable

  • A home for the code that corresponds to the latest tagged release.
  • This branch would be the basis for post-release bugfixes, eg an external developer who finds a bug in the latest release would send a pull request against this branch.
  • Developers depending on this code directly in their app would likely point to this branch on their own stable branch.

Ephemeral and Internal Only Branches

These branches are emphemeral in nature, and are thrown away once they are no longer useful. Developers consuming the library would typically ignore these

FeatureX

  • A place for in progress features to live without de-stabilizing the master branch. Can be many of these.

Hotfix

  • An in progress fix would go here, where that fix is destined to be merged back into latest stable but not part of a new release that is branched off of master.
  • While this hotfix is in progress, since these commits are not part of a tagged release, they cannot go on stable (yet), otherwise it would be a violation of the stable branch contract that says that stable reflects the latest tagged release.
  • They could go directly on a “local stable” branch, which is only on the developers workstation, but that prevents sharing the branch or running CI unit tests against it, so it’s not a good solution.
  • NOTE: when hotfix merged, new release tagged simultaneously with a merge to stable, so the stable branch contract stays satisfied.

Release

  • During QA release process, need a place to QA and apply bugfixes, while isolated from destabilizing changes such as merging feature branches.
  • Release branch allows feature branch merging to happen concurrently on master branch, which is especially crucial if release gets delayed.

Playing With Go and OpenGL

Get this spinning gopher working with Go and OpenGL on OSX Mavericks via gogl

Steps

1
2
3
4
5
6
7
8
$ brew install glfw2
$ go get -u -v github.com/tleyden/gogl
$ go get -u -v github.com/go-gl/glfw  
$ cd $GOPATH/src/github.com/tleyden/gogl
$ make bindings
$ go get -u -v github.com/chsc/gogl
$ cd $GOPATH/src/github.com/tleyden/gogl/examples/gopher
$ go run gopher.go

NOTE: if pull request #37 has already been merged into gogl, then replace github.com/tleyden/gogl with https://github.com/chsc/gogl in steps above.

Why Use Go When You Can Just Use C++?

I recently saw this on a mailing list:

What are the advantages of Go against C++ or other languages?

Go has sane concurrency.

In C++ the main tool to deal with concurrency is pthreads, but making threaded programming correct is extremely difficult and error prone. Trying to make it performant by minimizing locking makes it even more challenging.

Go, OTOH, has a concept of goroutines and channels for concurrency. The idea has its roots in CSP (communicating sequential processing), and is not unlike Erlang’s style of having processes that communicating by message passing.

In Go, instead of having threads communicating by sharing memory (and locking), goroutines share memory by communicating over channels. Eg, concurrent goroutines communicate over channels, and each goroutine’s internal state is private to that goroutine. Also, concurrency and their constructs like channels are built right into the Go language, which affords many advantages over languages that have had concurrency slapped on as an afterthought.

Other Strengths

  • No unsafe pointer arithmetic.
  • Array bound checking
  • Write once run anywhere
  • Closures (eg, lambdas)
  • Functions are first class objects
  • Multiple return values
  • Does not have IFDEF’s, so no IFDEF hell and unmaintainable code.
  • Compiles blazingly fast
  • Gofmt – All code is uniformly formatted, making codebases much easier to read. (a la Python)
  • Garbage collection

Weaknesses

  • Lack of generics
  • Not quite as fast as C/C++ (partly due to GC overhead)
  • Integration with existing native code is a bit limited (you can’t build libraries in Go or link Go code into a C/C++ executable)
  • IDE support is limited compared to C/C++/Obj-C/Java
  • Doesn’t work with regular debugging tools because it doesn’t use normal calling conventions.

A Stubbed Out Gradle Multi-project Hierarchy Using Git Submodules

I’m about to split up the couchbase-lite android library into two parts: a pure-java and an android specific library. The android library will depend on the pure-java library.

Before going down this project refactoring rabbit hole for the real project, I decided to stub out a complete project hierarchy to validate that it was going to actually work. (I’m calling this project hierarchy “stubworld” for lack of a sillier name)

There are five projects in total, here are the two top-level projects

https://github.com/tleyden/stubworld-app-android (an Android app)

https://github.com/tleyden/stubworld-app-cmdline (a pure java Command Line application)

The projects are all self contained gradle projects and can be imported in either Android Studio or IntelliJ CE 13, and the dependencies on lower level projects are done via git submodules (and even git sub-submodules, gulp). In either top level project you can easily debug and edit the code in the lower level projects, since they use source level dependencies.

The biggest sticking point I ran into was that initially I tried to include a dependency via:

1
compile project(':libraries:stubworld-corelib-java')

which totally broke when trying to embed this project into another parent project. The fix was to change it to use a relative path by removing the leading colon:

1
compile project('libraries:stubworld-corelib-java')

I’m posting this in case it’s useful to anyone who needs to do something similar, or if any Gradle / Android Studio Jedi masters have any feedback like “why did you do X, when you just could have done Y?”

Objective C Coding Interview Challenge: URLPuller

Programming Challenge

The goal is to create an Objective-C class which takes an array of NSURL’s, downloads their contents asynchronously, and writes the contents of each URL to a unique file.

The class should be named URLPuller and should have the following instance methods:

  • - (void) downloadUrlsAsync: (NSArray*)urls
  • - (void) waitUntilAllDownloadsFinish
  • - (NSString*) downloadedPathForURL: (NSURL*)url

Here are the detailed descriptions of each method:

downloadUrlsAsync

- (void) downloadUrlsAsync: (NSArray*)urls

The urls array will contain an array of NSURL’s. The method should return immediately, eg, it’s an asynchronous API call that will run in the background.

The downloaded files can be stored in any directory, where the filename is {sha1 hash of url}.downloaded

waitUntilAllDownloadsFinish

- (void) waitUntilAllDownloadsFinish

Block until all of the URLs have been downloaded.

If this is called while there is no download in progress, it should return immediately. Otherwise, it should block until all the downloads have finished.

downloadedPathForURL

- (NSString*) downloadedPathForURL: (NSURL*)url

Return the path where the given url was downloaded to.

If the given url has not been downloaded yet, or was never requested to be downloaded, then it should return nil.

Objectives

Demonstrate that:

  • You can write a program that satisfies the API methods and described behavior.
  • You can write code that is well factored, and easy to understand.
  • You can clearly communicate questions (if any arise) regarding this specification.

Deliverable

  • Zipped Xcode project directory or a link to github repo that contains the Xcode project. It can be either an iOS or MacOS project.

Rules

  • You can use any available resource you can find on the Internet — eg, Stack Overflow, API docs, etc.
  • You may not ask questions related to the problem on Stack Overflow, IRC, etc. (eg, the Internet is “read-only”)
  • You cannot use any 3rd party libraries or products, you can only use classes and API’s in the standard iOS/Cocoa Objective-C runtime. If in doubt, just ask.
  • You are welcome to go above and beyond the call of duty in any way that you see fit, however with the constraint that you must provide the above methods with signatures exactly matching with the specification.

How to Downgrade Android Studio

I just tried to update to Android Studio 0.4, however was stung by Issue 61573: Gradle duplicate file exception when package apk file with jar dependencies that contains resources.

The good news is that it’s very easy to downgrade! Thank Google, because there is nothing more annoying than not being able to back out of an upgrade gone bad.

Here’s how I did it on OSX Mavericks:

1
2
3
$ mv /Applications/Android\ Studio.app /tmp/
$ mv ~/Downloads/Android\ Studio.app /Applications/
$ cp -R /tmp/Android\ Studio.app/sdk /Applications/Android\ Studio.app/

The last step is necessary because the dowloaded 0.3.7 Android Studio will not have an SDK directory. (Note: at this point you can also use mv instead of cp if you don’t want to use the extra disk space)

Understanding Chan Chan’s in Go

A channel describes a transport of sorts. You can send a thing down that transport. When using a chan chan, the thing you want to send down the transport is another transport to send things back.

They are useful when you want to get a response to something, and you don’t want to setup two channels (it’s generally considered bad practice to have data moving bidirectionally on a single channel)

Visual time lapse walkthrough

Keep in mind that Goroutine C is the “real consumer” even though it will be the one which writes to the request channel.

The request channel starts out empty.

Screenshot

Goroutine C passes a “response channel” to go routine D via the request channel

Screenshot

Goroutine C starts reading from the (still empty) response channel.

Screenshot

Goroutine D writes a string to the response channel

Screenshot

Goroutine C now is able to read a value from response channel, and get’s the “wassup!” message

Screenshot

And now we are back to where we started

Screenshot

Here is some code that uses chan chan’s

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
package main

import "fmt"
import "time"

func main() {

     // make the request chan chan that both go-routines will be given
     requestChan := make(chan chan string)

     // start the goroutines
     go goroutineC(requestChan)
     go goroutineD(requestChan)

     // sleep for a second to let the goroutines complete
     time.Sleep(time.Second)

}

func goroutineC(requestChan chan chan string) {

     // make a new response chan
     responseChan := make(chan string)

     // send the responseChan to goRoutineD
     requestChan <- responseChan

     // read the response
     response := <-responseChan

     fmt.Printf("Response: %v\n", response)

}

func goroutineD(requestChan chan chan string) {

     // read the responseChan from the requestChan
     responseChan := <-requestChan

     // send a value down the responseChan
     responseChan <- "wassup!"

}

This code can be run on Go playground

For more information

See Google I/O 2013 – Advanced Go Concurrency Patterns around 17:00. This contains another explanation of chan chan’s.

Android Studio + Genymotion Emulator

The Android emulator is dog slow. The x86 intel emulators only work with certain versions of OSX, otherwise they crash miserably.

Genymotion is apparently pretty good, so I’m giving that a shot.

Here’s how to do it.

Download and Install VirtualBox

Download VirtualBox + follow instructions to install.

Signup with Genymotion + Download installer

  • Go to genymotion.com and signup

  • Click the “Get Genymotion” button and download the appropriate version for your OS.

Run the Genymotion application

It will install Genymotion and Genmotion shell — run the Genymotion application (not the shell).

When you run it, you will see:

Screenshot

Click add new virtual device. It will ask you to login, then show you a list:

Screenshot

Pick the virtual device you want and it will download it.

Try starting a virtual device

Here’s what happened on my system:

  • Hit Play to run virtual device
  • It asked me to configure the Android SDK location, I said “No”
  • Then it gave me an error that VirtualBox was not installed
  • I started VirtualBox manually
  • I retried running the virtual device, this time it worked and I saw

Screenshot

Configure Android Studio plugin

Go to Preferences / Plugins and click “Browse Repositories”, then search for Genymotion. Right click and choose “Download and Install”.

Screenshot

Restart Android Studio and you should see a new icon in your IDE.

Screenshot

Click the icon and it will bring up Preferences, and choose the path to the Genymotion application.

Screenshot

Now if you click the Genymotion icon again, you will see the list of devices available:

Screenshot

Start an emulator

Click the Genymotion icon within Android studio, select a virtual device, and hit start.

It will ask you to choose the location of the SDK:

Screenshot

On Mac OSX, I was not able to brows to the path since the application was greyed out in the chooser dialog, so I went onto the shell and found it (/Applications/Android Studio.app/sdk) and just copied and pasted it:

Screenshot

Select a virtual device, and hit start (again). It should come up now.

Deploy an application/test to that emulator

In Android Studio, hit the “Play” or “Debug” button, and you should see the dialog that asks you to choose an emulator:

Screenshot

and one of the emulators will be the Genymotion emulator. After you choose that emulator and hit “OK”, it will run your application in the Genymotion emulator (this is showing the Grocery Sync demo app for Couchbase Lite):

Screenshot

A Simple Data Race in Go and How to Fix It

Suppose you want to add an http handler which can show some things that in memory.

Simple, right? Actually, if you aren’t careful, doing this can lead to data races, even though only one goroutine is writing the data at any given time.

Here’s the golang code (also on github) that reproduces the issue:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
type Foo struct {
  content string
}

type FooSlice []*Foo

func updateFooSlice(fooSlice FooSlice) {
  for {
      foo := &Foo{content: "new"}
      (*fooSlice)[0] = foo
      time.Sleep(time.Second)
  }
}

func installHttpHandler(fooSlice FooSlice) {
  handler := func(w http.ResponseWriter, r *http.Request) {
      for _, foo := range fooSlice {
          if foo != nil {
              fmt.Fprintf(w, "foo: %v ", (*foo).content)
          }
      }

  }
  http.HandleFunc("/", handler)
}

func main() {
  foo1 := &Foo{content: "hey"}
  foo2 := &Foo{content: "yo"}
  fooSlice := FooSlice{foo1, foo2}

  installHttpHandler(fooSlice)

  go updateFooSlice(fooSlice)

  http.ListenAndServe(":8080", nil)
}

if you run this code with go run main.go and then open http://localhost:8080 in your browser, it will work as expected.

But not so fast! Lurking in this code are data races, and if you run this with race detection enabled using go run -race main.go and then access it with the browser, it will panic with:

1
2
3
4
5
6
7
8
9
10
==================
WARNING: DATA RACE
Read by goroutine 6:
  main.func·001()
      /../concurrentaccess/main.go:45 +0x9e
      ...
Previous write by goroutine 4:
  main.updateFooSlice()
      /../concurrentaccess/main.go:35 +0x98
      ...

because there are two goroutines accessing the same slice without any protection — the main goroutine running the http server, and the goroutine running updateFooSlice.

Fix #1 – use sync.mutex to lock the slice

This isn’t necessarily the best way to fix this, but it’s the simplest to understand and explain.

Here are the changes to the code (also on github):

  • Import the sync package
  • Create a sync.Mutex object in the package-global namespace
  • Before updating the slice, lock the mutex, and after updating it, unlock it.
  • Before the http handler access the slice, it locks the mutex, and after it’s done, it unlocks it.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
type Foo struct {
  content string
}

type FooSlice []*Foo

var mutex sync.Mutex

func updateFooSlice(fooSlice FooSlice) {
  for {
      mutex.Lock()
      foo := &Foo{content: "new"}
      fooSlice[0] = foo
      fooSlice[1] = nil
      mutex.Unlock()
      time.Sleep(time.Second)
  }
}

func installHttpHandler(fooSlice FooSlice) {
  handler := func(w http.ResponseWriter, r *http.Request) {
      mutex.Lock()
      defer mutex.Unlock()
      for _, foo := range fooSlice {
          if foo != nil {
              fmt.Fprintf(w, "foo: %v ", (*foo).content)
          }
      }
  }
  http.HandleFunc("/", handler)
}

func main() {

  foo1 := &Foo{content: "hey"}
  foo2 := &Foo{content: "yo"}

  fooSlice := FooSlice{foo1, foo2}

  installHttpHandler(fooSlice)

  go updateFooSlice(fooSlice)

  http.ListenAndServe(":8080", nil)

}

If you now re-run the code with the -race flag and access http://localhost:8080, it won’t panic.

Digression – chan chan

Before we can talk about Fix #2, we need to take a digression to talk about chan chan’s — channels which contain other channels as the morsels of data that pass through the channel tubes (metachannels, if you will).

tl;dr A channel describes a transport of sorts. You can send a thing down that transport. When using a chan chan, the thing you want to send down the transport is another transport to send things back.

If it’s still a little fuzzy for you, here’s the long description of chan chan’s with diagrams

Fix #2 – Use channels instead of sync.Mutex

In this version of the fix, the goroutine running the http handler (the main goroutine in this case), makes a response chan and passes it into a request chan chan that both goroutines can access. It then reads the response from the response chan, which will contain a copy of the FooSlice.

1
2
3
4
5
6
7
8
9
10
11
12
13
func installHttpHandler(fooSlice FooSlice) {
        handler := func(w http.ResponseWriter, r *http.Request) {
                response := make(chan FooSlice)
                request <- response
                fooSliceCopy := <-response
                for _, foo := range fooSliceCopy {
                        if foo != nil {
                                fmt.Fprintf(w, "foo: %v ", (*foo).content)
                        }
                }
        }
        http.HandleFunc("/", handler)
}

The other goroutine updates the FooSlice and is also checking the request chan chan for new messages. If it gets a new request message, it makes a copy of the FooSlice and sends it to the response chan.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
func updateFooSlice(fooSlice FooSlice) {
        t := time.Tick(time.Second)
        for {
                select {
                case <-t:
                        foo := &Foo{content: "new"}
                        fooSlice[0] = foo
                        fooSlice[1] = nil
                case ch := <-request:
                        fooSliceCopy := make(FooSlice, len(fooSlice))
                        copy(fooSliceCopy, fooSlice)
                        ch <- fooSliceCopy
                }
        }
}

Again, if you now re-run this code with the -race flag and access http://localhost:8080, it won’t panic.

The full code sample is available on gihub