More on Go Channels, Parallelism and Concurrency

As a follow up of Basic Parallel Computing in Go, I wanted to build a more complex example where we use more advanced techniques in Go concurrent programming.

Let’s first create a function ProduceInts that generates random data into a channel for some time t . After time t passes, it closes the channel to indicate that no more data will be generated. Notice that ProduceInts does not block.

Now, we can create two functions that process the data (ints) being produced.

In the following snippet we have added two new functions, CountInts and Odds. The first one just creates a map to keep track of how many time a particular number has been generated. The second function check if the generated number is odd, and print it if the check passes.

Notice that both new function are reading from a channel, they don’t block, they run independently of each other, and many instances of themselves could potentially be running at the same time (important for scalability).

Now, we can use the Splitter we created in Basic Parallel Computing in Go to route the values from the original channel (stream) into two different channels so that Odds and CountInts can read from.

In the router package we have added the implementation of Splitter along with some functionality that will allow us to compose the channel and control how the data flows.

Finally, we need to pipe everything together.

In the main package, we basically start generating random ints, but only for certain amount of time (20 seconds).

Then, we use the router package to pipe the output of ProduceInts the Splitter which in turns adds two outputs which are going to be in the input for CountInts and Odds.

Finally we start the Splitter by doing router.Run(). Since don’t want the program to finish before all items has been processed, we wait until the Splitter is not running any longer.

As we can see, synchronizing streams and pipelining is quite easy in Go, we only need to think about channels as queues where we can write and read from independent process at any moment in time.

Happy Coding.