Function closures are really powerful.
Essentially you can think of them like stateful functions, in the sense that they encapsulate state. The state that they happen to capture (or “close over” — hence the name “closure”) is everything that’s in scope when they are defined.
First some very basic higher order functions.
Higher order functions
Functions that take other functions and call them are called higher order functions. Here’s a trivial example:
1 2 3 4 5 6 7 8 9 10 11 12
main() function, we define a function called
mySender and pass it to the
sendLoop() takes a confusing looking argument called
sender func() — the parameter name is
sender, and the parameter type is
func(), which is a function that takes no arguments and returns no values.
To make this slightly less confusing, we can define a named
SenderFunc function type and use that:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
sendLoop() has been updated to take
SenderFunc as an argument, which is easier to read than taking a
func() as an argument (which looks a bit like a function call!) If the
SenderFunc type took more parameters and/or returned more values, having this in a defined type would be crucial for readability.
Adding a return value
Let’s make it slightly more realistic — let’s say that the
sendLoop() might need to retry calling the
SenderFunc passed to it a few times until it actually works. So the
SenderFunc definition will need to be updated so that it returns a boolean that indicates whether a retry is necessary.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
One thing to note here is the clean separation of concerns — all
sendLoop() knows is that it gets a
SenderFunc which it should call and it will return a boolean indicator of whether or not it worked or not. It knows absolutely nothing about the inner workings of the
SenderFunc, nor does it care.
A stateful sender — the wrong way
You have a new requirement that you need to only retry the
SenderFunc 10 times, and then you should give up.
Your first inclination might be to take this approach:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
This will work, but it makes the
sendLoop() less generally useful. What happens when your co-worker hears about this nifty
sendLoop() you wrote, and wants to use it with their own
SenderFunc but wants it to retry 100 times? (side note: your
SenderFunc implementation simply prints to the console, whereas theirs might write to a Slack channel, yet the
sendLoop() will still work!)
To make it more generic, you could take this approach:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Which will work — but there’s a catch. Now that you’ve changed the method function signature of
sendLoop() to take a second argument, all of the code that consumes
sendLoop() will now be broken. If this were an exported function, it would be even worse problem.
Luckily there is a much better way.
A stateful sender — the right way using function closures
Rather than making
sendLoop() do the retry-related accounting and passing it parameters for that accounting, you can make the
SenderFunc handle this and encapsulate the state via a function closure. In this case, the state is the number of retries that have been attempted, which will start at 0 and then increase on every call to the
SenderFunc keep internal state? It can “close over” any values that are in scope, which become associated with the function instance (I’m calling it an “instance” because it has state, as we shall see) and will be bound to the function instance as long as the function instance is around.
Here’s what the final code looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
counter state variable is bound to the
mySender function instance, which is able to update
counter on every failed send attempt since the function “closes over” the
counter variable that is in scope when the function instance is created. This is the heart of the idea of a function closure.
sendLoop() doesn’t know anything about the internals of the
SenderFunc in terms of how it tracks whether or not it should retry or not, it just treats it as a black box. Different
SenderFunc implementations could use vastly different rules and/or states for deciding whether the
sendLoop() should retry a failed send.
If you wanted to make it even more flexible, you could update the
SenderFunc to return a
time.Duration in addition to a
bool to indicate retry, which would allow you to implement “backoff retry” strategies and so forth.
What about thread/goroutine safety?
If you’re passing the same function instances that have internal state (aka function closures) to multiple goroutines that are calling it, you’re going to end up causing data races. There’s nothing special about function closures that protect you from this.
The simplest way to deal with is to make a new function instance for each goroutine you are sending the function instance to, which is probably what you want. In theory though, you could also wrap the state update in a mutex, which is probably not what you want since that will cause goroutines to block eachother trying to grab the mutex.