Three Patterns for Concurrent Functions in Go
Support for channels in Go gives us the opportunity to compose code using pure functions — functions that take state and return state without side effects.
From experience 3 signatures for these pure functions emerge:
Simple Functions
These functions don’t use channels as parameters, rather take and return standard data structures (structs, slices, maps etc.) and optionally an error, blocking until results are available.
Concurrency and channels stay outside the function leaving us with several advantages: simplicity of code, ease of testing, focus on logic and control flow without the distraction of concurrency.
The problem of course is that this will only work for cases where we can afford to block and fit the full results in memory.
On the other hand, if we need to be concurrently using the results as they’re being produced or need to stream the results in a pipeline to restrict memory use, we need functions that use channels.
Functions that take channels as input parameters
Since channels are first class citizens in Go, they can be passed in as parameters from the calling code. The function has straightforward control flow as it can block until done. Along the way as results become available it publishes them into the channel making them available to the caller.
The function should not close the channel. This should be the responsibility of the caller since it’s the one that created the channel.
Instead, the function should feel free to return on the first error to indicate it’s done or of course if it’s indeed fully done.
The drawback of this approach is one of semantics. The function would seem to suggest it takes in the channel as input and returns just an error as output while of course the channel itself is for the output. The ability to specify direction on the channel does go some way in clarifying this point.
What we have gained of course is that the caller can start using the results concurrently to the function execution allowing for parallel processing and limiting memory.
Functions that return channels as parameters
To fix the semantics issue we can indeed construct a function that creates and returns a channel to communicate it’s output. However, this means the function can no longer be a blocking function and should perform it’s work in a background goroutine.
In addition to the messy control flow the other drawback is the need to return errors also as a channel even if it will only ever communicate one error since the function cannot block.
So, while we fixed the semantics of this function we impacted readability within the function and this trade-off should decide which approach to take.
Conclusion
Channels in Go open up the possibility for writing pure functions that can run concurrently. I use it coupled with Structured Concurrency to make readability easy by making the control flow obvious.
Separating out concerns within functions is fundamental to writing sustainable code and core to refactoring. In doing so, most functions will likely end up being simple functions.
Higher functions that combine multiple simple functions and work loops or I/O will need to use channels at which point consider the 2 patterns above and pick the one that wins the trade-off between control flow simplicity and signature semantics.
