Handling User Input in Rich Browser Apps
The right architecture for handling mouse and keyboard events
Simple Application
The browser (JavaScript) is very helpful about handling user input. For example:
var doneButton = document.getElementById(“done”)
doneButton.onclick = function () { sendDataToServer() }
This is a simple and straightforward code that works perfectly well for the following context.
- The user spends a minute filling out a form with his data.
- The user clicks the “done” button.
- The user waits two seconds for the page to be updated, which he considers very acceptable.
Let’s assume that the client doesn’t check the data; the client just sends anything (even a blank form) to the server. Therefore, you only need to write client-side code to handle one event: the click on the button.
Of course, every time the user types a letter writing his name in a field is also an event. But you don’t have to write code to handle this event because the browser does it automatically.
Now, what can the user do to break the client?
- Close the page?
- Click the “done” button while typing his name?
- Send a blank form?
- Write letters where should be only numbers?
None of them would break the client, not even a combination of them. And the user would know that he is to blame for any failure. The client is fine. The application appears to be reliable!
Complex Application
Let’s consider a browser based drawing/painting tool as our complex application. There are key differences from that simple, linear, application.
First, the painting tool must be absolutely responsive. At the same time the user drags the mouse on the canvas, the canvas must show the result. The code must be really efficient.
Second, the application must be absolutely reliable. It can never crash or let the user lose his work even when he is to blame (for closing the browser tab). Filling administrative data on a form is a 2 minute job and needs no inspiration. But lose that unique, precious artwork is a dramatic event! Which the artist is not prone to let happen again (bye bye app).
Third, basically everything happens on the canvas element and you have to handle every minimal input. The browser is not going to be as helpful as when you create a text box.
These 3 key differences are enough for the purpose of this article.
The Golden Rule
Considering the text above we come to a golden rule that will guide us when writing code for the complex application:
The application must fully process any input before receiving the next input.
Or else the application is not going to be responsive. And not reliable at all, when severely overloaded.
Small Convention
In this article canvas refers the HTMLCanvasElement. And layer refers the virtual painting surface that is drawn on the canvas. Their sizes can be completely different. And the layer can be positioned in any part of the canvas.
The Pixel Info
One of the basic features of a rich drawing/painting tool is give feedback about the layer pixel under the mouse. The user can inspect the color and position of any pixel just by placing the mouse cursor over it.
Some remarks in the code above tell the cost to process the function. “Smart” means that the function has an expensive part which runs if necessary, otherwise it is cheap to process.
So far, so good! We have a clean and simple code. And it looks efficient.
It Is Not That Easy
The code above has important problems. Sorry.
- Suppose the user wants to rotate the layer 90°. He just types “R” in the keyboard. And the layer pixel under the mouse becomes another one (different coordinates, color may be the same by chance). But, as no mouse move event is raised, function updatePixelInfo is not going to be called! And our application shows wrong values about the pixel under the mouse. It is not reliable!
- Suppose the user moves the mouse fast over the canvas. Passing, let’s say, the mouse over 600 pixels during 30 ms. This means that you have (*) less than 0.05 ms to fully process each input (don’t forget the Golden Rule). We can not prevent the user to make this action. And our code is not efficient enough for this. As consequence the application will accumulate events to be processed becoming unresponsive (at least not fully responsive as it must be). Fortunately, the hardware, operating system and/or the browser skips many of these inputs. So we will receive MUCH less than 600 mouse move events to handle. Still the application probably will break the Golden Rule.
(*) We always make theoretical calculations about how much time is available for a procedure. In practice the available time is a half, very often a third, of the theoretical time because the operating system and the browser have other jobs to run.
3. Suppose the user repeats the fast mouse movement described in ‘2’, but this time he is not just moving the mouse. He is dragging the mouse expecting to paint a (continued) line. Now, besides giving feedback about the pixel under the mouse, we stress the application with a much more intensive/expensive task, painting the layer:
- changing layer pixels (expensive)
- memorizing the layer for undo/redo (very expensive)
- updating the canvas with the changed layer (very expensive)
NO WAY it is going to work. Remember that part “Fortunately, the hardware… skip so many inputs.”? Skipping the mouse move (drag) inputs when the user wants to draw a line will produce a line with *HUGE* gaps.
Note: As there is no mouse drag event in browser canvas, we have to refactor our mouse move event handler (if mouse button pressed then…).
An Architecture Born to Crash
Notice how our application fails while doing any of 3 very basic actions:
- giving pixel feedback after changing the layer by keyboard input
- giving pixel feedback while moving the mouse very fast
- drawing a line fast (need not to be very fast)
Imagine a more sophisticated use like drawing an ellipse, which requires dealing with an invisible over layer, adjusting its shape and size before drawing on the standard layer…
The Overpower of the Event Handlers
You have no control of the application
Each time an event is raised, his handler starts firing functions (some very expensive) without thinking twice!
It is not the user, the browser and/or the operating system that raises “EXCESSIVE” events to be handled. It is our event handlers that have a bad design.
We need to find a way to classify the events:
- which events we shall completely ignore
- which events we shall fully process
- which events we shall partially process
Consider that case of giving pixel feedback during a very fast and long mouse movement. We only need (and should) show the info about the CURRENT pixel under the mouse. For this purpose, every old, non processed mouse event becomes obsolete by definition! We must ignore them; NOT fully process them.
Consider that case of drawing a line by (very) fast movement of the mouse. We need to create a list of points that represents the mouse path. And then use interpolation to fill the gaps in the path. For sure, we are NOT going to fire the very expensive full process of painting 600 times in 20 ms, or something like that.
The Chaining of Functions
You have no control of the code
Do you remember the nice function updatePixelInfo? It seems very good when we read the code but is a tragedy when running the application. One of its problems is that it does not work if the layer pixel under the mouse changes without moving the mouse (by using keyboard to rotate the layer).
A linear architecture, with chained functions (A calls B, B calls C, C calls D…) is doomed to failure in this environment: an event machine.
A linear architecture (in this environment) makes the code UNMAINTAINABLE.
The Root of All Evil
(in a complex application)
The root of all EVIL is an event handler that CALLS a function.
By “function” I mean other important segment of the code. I am not talking about small functions that work like subroutines of the event handler (snippets that help to organize the code).
In a complex application, an event handler should only SIGNAL to other function(s), NOT CALL them.
The Solution: Event-Driven Architecture
The event-driven architecture is the key to solve all the problems of efficiency and turn the code EXTREMELY maintainable!
Instead of chained functions we have loosely coupled functions!
Again, I mean “function” in the broad sense; a software component with specific role.
One function does not call another. One function doesn’t even know about the existence of other functions! It just offer its production to the system and that’s all.
This is the general idea. Some exceptions must exist, of course.
Implementing the Concept
First we define the modules as described below. By “module” I mean any file written with related functions (some public, some private) that performs an exclusive/independent ROLE.
- “main.js” is the manager of all other modules
- “mouse.js” handles all mouse events
- “keyboard.js” handles all keyboard events
- tens of modules with specific tasks; like creating the interface, updating the interface, painting, undo/redo, save/load, rotate, etc..
The module “main.js” is the pivot and commands all other modules. Basically this is the only module allowed to call functions in other modules.
All other modules are allowed only to read global variables in any other module.
Maybe what we need right now is a little pseudocode. So be it!
Warning: this pseudocode is extremely simplified. It misses a lot of essential features and it is only purpose is illustrate the concept.
Module main.js
This is the brain of our application. Here decisions are made. setTask is the function modules “mouse.js” and “keyboard.js” call after handling input events.
And setTask may accept or reject the task. But even when it accepts the task, setTask NEVER runs the task!!! It just memorizes the task, to run in the next execution of runMainLoop.
Come rain or shine, here is the end of line for the (processed) input event.
The user/browser/system may raise thousands of input events per second. No problem. Our application only runs 1 task per loop (16.666 ms). Of course we need a clever code to avoid, for example, that a drawn line have any gap.
Module mouse.js
The first duty of module “mouse.js” is to release information about the mouse position to whoever wants it (by updating its public global variables).
This module processes the canvas coordinates into useful layer coordinates, which are the one other modules want to consume.
Its second duty is, when proper, call setTask in the module “main.js”:
“Hey, the user wants this task (drawing) to be executed. Now it is up to you. Good luck. I am done here!”.
Note: I am considering that handling input events is not a duty. It is the internal mechanism. It is how the module fulfills its duties to the “community”.
Module keyboard.js
This is a straightforward module. We clearly can see its role as interface. After it calls setTask the program hasn’t a clue about the key that was pressed or if any event was raised. It only knows about the task to be executed (if accepted).
There is no data to be processed/released for later consumption.
Like “mouse.js”, it does not check if task is null before call setTask because this decision is not up to it. Taking decisions is not its role!
Module interface.js
Remember the problem we had to update the pixel info when the layer was rotated by keyboard event (because updatePixelInfo was bound to a mouse event)? SOLVED!
We can call updateMousePositionLabel at any time. No need to inform the coordinates of the mouse. Module “mouse.js” makes them available all the time.
Sparks of Life
There are 3 sources of “sparks” that make our application alive (run).
Mouse events, keyboard events, and the timer (requestAnimationFrame).
Our code was designed to use and harmonize these sparks. If you write a decent code you may be assured that everything will run fine. These sparks were not invention of our code.They are created by the browser. We are just processing them.
Finally
I hope I have been helpful!
