Inter process communication between Node.js and Python
IPC with Named Pipes
Recently, I came across a use-case which involved running a PyTorch model for inference in real-time, along with a Node.js server which handles I/O. After an initial failed attempt of using torch.js, we decided to go with Tensorflow.js (tfjs). However, the layers model is a synchronous and a blocking process which forces the use of worker threads (this route is detailed in a separate story). This approach has its own difficulties: (1) a tfjs specific worker thread needs to be running separately. (2) the use of GPU via WebGL backend is not straightforward.
PyTorch is convenient for AI development, with the ease of using Python. Also, PyTorch provides a nice C++ frontend which is especially attractive for low-latency systems. For the above use-case, we decided to continue using PyTorch. While Node.js handles input and output, Python/PyTorch will handle the AI model inference. The two systems will communicate using named pipes.
Named Pipe
‘Pipe’ is a fundamental feature of Linux that allows separate processes to communicate and pass data to each other. A named pipe (uses the file system) is a file that can be accessed by two unrelated processes; one is a reader and and the other, writer. More details can be found here. Named pipes are sometimes referred to as FIFOs: first-in first-out, since the byte order (going in and coming out) is preserved.
Simplifying the problem statement…
The goal is to use Named Pipes for communication.
- A Node.js process will write some data to a pipe (A)
- A Python process reads from pipe (A) and manipulates the data
- Above python process then writes the data into a pipe (B)
- Node.js reads from pipe (B)
Python: read, process and write
First up, let’s create a simple Python process that reads from a named pipe A, processes the data and then writes it to named pipe B (for simplicity, the process_msg() function here returns the read data). The script first creates the named pipe A using os.mkfifo() command.
Node.js: write and read
Next, let’s write a simple Node.js program that writes into the pipe. There are a few packages which do this: named-pipe, fifo-js. However, node offers functions which makes it convenient to set up IPC with pipes. The script below writes data every 1 second into Pipe A (send data), and reads from Pipe B (receive processed data from Python). For simplicity, the data here is the current time as string.
Note that while createReadStream and createWriteStream appear to have a synchronous interface, they are async operations: they don’t return a promise or accept a callback to communicate when an operation is done. More details can be found in this stackoverflow answer.
Next, fire up two terminals, one for Python and another for Node.js. The pipeline adds roughly 1ms latency.
Conclusion
Named Pipes offer a convenient way of communication between processes. I presented a simple way to pass data from a Node.js thread to a Python process which can do data manipulation. For slower tasks, multiple threads can be used in Python to read and write separately.
Thanks for reading!
