Clone Wars: Increase Performance 3X by Switching to HarperDB
One of my favorite coding pastimes is watching the Clone Wars, which is where developers create open-source reproductions of popular software we all know and love. It gives us a look at how some neat features could be implemented, how designs are developed, and what is possible with the latest versions of frameworks from time to time.
I found a project I could actually use day-to-day, however, it used Firestore, which I don’t really use too often. My platform of choice is HarperDB, so I thought it might be cool to modify this project to use HarperDB so that I could use it on my local home lab server.
I also thought it would be a neat experiment to see how easily HarperDB can be implemented into an existing project and the steps I took to accomplish it. Let’s jump into it!
Todoist
The project I am starting with is a straightforward reproduction of Todoist, a popular note-taking application loved by many. Karl Hadwen created this popular clone a while back, and it seemed like the perfect starting spot for my project since it already had many of the building blocks I needed. Since this project uses React, hooks, and the context system, it should be straightforward to jump in and edit it further for any of your needs too!
Modifications
One of the first modifications I did was to create separate clients for HarperDB and Firebase to use. This way, I could reference the functions required for either client without having to embed the functions into the components as they were previously. A step further would have been to abstract this and have a generic client that you use across the application and have it connect the dots on which client to use depending on your configuration.
Since all of our logic lives inside our Custom Functions, our HarperDB client is much simpler and straightforward compared to the Firebase functions. It also gives us a chance to have all of our business logic live in one location that can be easily edited versus having to build and deploy a user interface every time we make a change.
With the components modified, I made a few last changes to let the project work on M1 Macs, as I had some trouble with the older packages:
# Inside of the React project's directory
$ yarn remove node-sass
$ yarn add sass node-fetch@2Custom Functions
One of my favorite features of HarperDB, the Custom Functions, allows us to build an API alongside our Database. This lowers the overall latency and gives us a streamlined development experience. We also have the flexibility to add authentication to our API if we had something more important to keep secure, or if it was a public project.
For the Custom Functions, I’ve broken up the Project and Task routes into their own files for better organization and used helpers extensively to lower code reuse and make development quicker. Also, since HarperDB supports SQL and No-SQL operations, I’ve used both where it made more sense, it was quicker, and/or safer.
The schema and attributes are automatically created, so no setup process is required. The only bit of manual setup I did was set up a list of the tables and attributes (columns) that I wanted to be preconfigured for my queries.
Overall, it was an incredibly easy-going process, and being able to edit the Custom Functions on HarperDB Studio makes it much easier if you transfer the project to a Cloud Instance and no longer have the local edit option.
User Interface
Since the Custom Functions can host our static user interface too, I went ahead and bundled the user interface and uploaded it to the static folder inside of the Custom Function’s directory. This way, we are hosting our entire project from our one Instance which makes things much easier during development and even later when and if we have to make changes.
To get it ready, we can run these commands in our terminal:
# Ensure you are inside of the React project code
$ cd todoist
# Set your Endpoint URL / project name
$ nano src/harperdb.js
# Bundle the user interface
$ yarn build
# Copy the bundled code to our Custom Function's directory
$ cp -r build ~/hdb/custom_functions/todoist/staticTesting
Since our HarperDB Instance is hosting everything, once we have everything uploaded and we’ve reloaded the Custom Functions, you should be able to navigate to the project in your browser, for example:
https://10.128.2.150:9926/todoist/staticWith the project open, we can now add projects or tasks as we see fit:
We can also quick-add tasks at the top, or even archive the tasks as we complete them. Since all of our logic is contained with the Custom Functions, we can also make some quick changes if we wanted to a query without having to interrupt our flow and go back to the React application, re-bundle, and re-upload.
Performance Gains
While this wasn’t my primary goal with this project, another huge benefit to HarperDB I’ve found is the performance. To test this, I set the application up with both clients configured and loaded 500 sample tasks into my Firestore database, my local HarperDB Instance, and a HarperDB Cloud Instance.
After running some test requests and plotting the results, I was able to get performance gains of around 3x by switching the underlying database away from Firestore. Both of the HarperDB Instances I tested were only on the free tier as well — which was awesome to see! If you wanted to see the raw data for yourself, you can take a look here.
Often I’ve found that performance improvements (or in our case here, a decrease in latency specifically) can be matched with a similar drop in infrastructure costs. Instead of having to allocate a larger instance, we can use a smaller Instance and still deliver the same, if not better performance when compared to other solutions. If we used even better hardware and set up HarperDB’s replication system, we could see even greater performance.
Conclusion
With these changes complete, I can now host this project in my local home lab alongside my numerous other HarperDB projects very easily. It was also nice to use the power of HarperDB’s Custom Functions to quickly write an API layer to interface with the project. And since the project is self-contained, I can easily deploy and edit the project with a few clicks.
It’s also great that I am going to get a huge performance boost by switching but that is just the icing on the cake!
