Simple bidirectional messaging in Node.js Worker Threads
Talk to your workers, and they will talk back.
What are Worker Threads?
Node.js Worker Threads allow concurrent execution of JavaScript code. If you are familiar with the Unix thread model, you know that for every process group there is a parent process and several child processes. A similar model is used for Worker Threads as will be shown.
Why would you use them?
If you have several CPU intensive tasks that need to be done, and those tasks operate independently of each other, then threads allow tasks to run in parallel, saving time. (You don’t need worker threads for I/O processing — this is something that Node.js non-blocking I/O model handles capably already.)
For example, I have been working (slowly) on a GraphQL wrapper for the Universal Chess Interface (a.k.a., UCI). One of the things I want to do is pool several chess engines and have them run independently.
Creating a worker thread
There’s really nothing to it! All that is required is a Node version > 10.5.0. The 10.x versions of Node require the --experimental-worker flag to be passed on the command line, but I am running version 11.11.0 and did not have to do that.
Here’s the example given at the start of the documentation for Worker Threads:
const {
Worker, isMainThread, parentPort, workerData
} = require('worker_threads');if (isMainThread) {
module.exports = function parseJSAsync(script) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, {
workerData: script
});
worker.on('message', resolve);
worker.on('error', reject);
worker.on('exit', (code) => {
if (code !== 0)
reject(new Error(`Worker stopped with exit code ${code}`));
});
});
};
} else {
const { parse } = require('some-js-parsing-library');
const script = workerData;
parentPort.postMessage(parse(script));
}
Not terribly exciting and a little cryptic. The stuff imported from ‘worker_threads’ is:
- Worker: a class
- isMainThread: true if this is the master process, false otherwise
- parentPort: the communication port between parent and worker
- workerData: a clone of the data passed to the thread’s Worker constructor.
Here, the main thread exports a method that takes some JavaScript and hands it off to a worker thread to process. We know we’re the main thread if isMainThread is true. Otherwise, we are the worker thread (else{} block), and we parse the script and returned the result of that back to the parent thread using the Worker method postMessage().
If that’s still confusing, here’s a link to another example that is perhaps easier to understand.
Talking to a worker thread
So far, the only data the worker receives is the workerData sent form the constructor arguments. The main thread gets a message from the worker in the above example, and that message is some parsed JavaScript.
That’s pretty boring, especially since the worker, as soon as it gets its workerData:
- does its thing,
- sends the result back to the main thread,
- quits.
For my planned chess engine workers, there’s going to be a lot of interaction (via UCI) that will be ongoing for as long as there are games to set up, moves to be made, and chess analysis to be done. I will write about the details of that in the future, but the title of this post says “Simple”, so I’ll illustrate bidirectional interaction with a basic worker. First, the worker code:
So all this mostly does is echo the messages it’s given, with the exception of “exit”, which I’ll get to in a minute.
The main thread that creates the worker is similar to the examples noted earlier, but no Promise is involved:
The main thing to note is that several messages are being sent to the worker, besides the one that is sent via the Worker constructor (i.e., workerData). So what happens when I run this?
Let’s map these message lines with lines of code:
{isMainThread: true} comes from line 19 of index.js
{ incoming: { start: “let’s begin”, isMainThread: false } } : line 14 of service.js (the worker thread).
{ incoming: { going: 'once' } }
{ incoming: { going: 'twice' } }
{ incoming: { going: 'three times' } }These all come from line 10 of service.js
{ incoming: ‘sold!’ } is line 7 of service.js
Worker stopped with exit code 0 brings us back to index.js, line 9.
Exiting the thread
I added the ability to exit the worker thread from the main thread. When the main thread posts an “exit” message, the worker closes the port. This triggers an exit event back in the main thread, and the exit status is noted (line 9).
What happens if you try to post another message after the worker has exited? Line 14 of index.js tries to do just that. Note that the message does not appear in the console output.
Done!
There’s a lot more to worker threads than this, but that about wraps it up for “simple”. Full code can be found here.
