How to Choose Your Stack Rationally
How can you tell which language will serve your solution best? I work as Cloud Engineer in a polyglot project and I feel at home working with multitude of technologies. But it sometimes makes me think — is there a consistent method of picking right tool for the job? Problem is multidimensional and takes looking from many different perspectives.
I cannot emphasize enough that the decisions based on word of mouth introduce a lot of bias. That can cost you a lot of time — in the first place and in the worst case — even a job. After all, nobody knows your specific use case better than YOU.
Making data-driven decisions grounded in hard facts can drastically improve your outcomes. And so, doing some research before jumping to conclusion is a must. Throughout the article I will show you various aspects to consider. Also I want to share with you my personal thoughts, complemented with what is on the Internet.
Dimensions
When analyzing languages (and stacks in general) there are many perpendicular aspects to consider. Some of these might be
- Requirements volatility — it goes without saying that every project is in motion. Perspectives change… Customers change their minds… New technologies give advantage to other companies, and so on…
- Road-map — what is required for POC? What is required for MVP?
- Language’s and ecosystem’s health metrics— the statistics taken from StackOverflow and GitHub can tell you a lot
- Adaptability— is it one-time project or meant to stay forever? Will it be set in stone or should it evolve with time?
- Impact— not every service is equal. Some are critical to functioning of the whole company others are perks.
- Scale — how large will the desired code-base be? Some languages are better for maintenance than others.
- Performance — different languages carry different performance weights
- Domain— domain limits the pool of languages to choose from
- Talent — the more niche language the more cost of attracting right people. Sometimes even it will be hard to find any.
- Risk— how much are you willing to put at stake. Do you finish it at all costs?
- Targeted devices — does the language has support for Windows | Linux | Mac Os X? Is it for embedded or x86?
- Integration with other tools — how well does it integrate with other languages in the company, databases, crypto, etc.
- Cost — how much money does it take to operate in the long run
Where to get data from
Wouldn’t it be nice to back your decisions with actual facts? Statistic dashboards can save you a lot of time figuring things out. Good start are these:
It is great for getting historical time series for various qualities of a repository. Like:
- volumes of Pull Requests over time and their size classifications
- time of response to Issues over time
- time series of cumulative Issues Opened vs Issues Closed
It provides various static metrics for GitHub repository, most notably:
- number of stars
- number of contributors
- daily issue/pr/commit rate
The most interesting is tech section. You learn most loved languages and most popular ones in percents of total.
'java' tag wiki
Stack Overflow | The World's Largest Online Community for Developers
stackoverflow.com
Here you can inspect particular tag for data.
- total vs unanswered posts number
- related tags for coincident technologies
These resources combined can give a general understanding where the ecosystem is heading. Then after scanning several candidates next step is to drill down and read further materials like posts or PRs and Issues.
Plan ahead and conceptualize
Gone are the days of Waterfall Method. Modern world forced us to adopt agile methods for software development. It helps you better to adapt to the unknowns. But still, when it comes to technologies some future-thinking is still necessary.
Come up with road map for at least two years. Then think about components that will help to achieve what is needed. Pick one:
- Check metrics of the framework
- Scan top posts on StackOverflow for tag
- Read documentation
- Predict pain points
Then repeat…
There are lots of things to consider. Are they self-contained in one ecosystem? Are they replaceable for something else? How well the language integrate with database of your choice? Does it have cryptography? Do I need bindings to C? To Assembly? Other languages in general? How about writing tests — is it easy?
We don’t live in perfect world and sometimes not all criteria are met:
- effectiveness — the tool only does it’s job half-way or covers only subset of functionality
- coherence — framework doesn’t fit good withing existing code and you have to make effort to integrate it
- efficiency — it is slow or consumes too much memory
- sustainability — it requires a lot of time to train the team, the knowledge has to be refreshed / updated
You should discuss these with stakeholders and agree on the tradeoffs per framework. It is your role to understand the options on the table and present them to non technical staff in several variants.
For the languages I recommend paying notice to is the support quality, first. When bug found — how fast will the maintainers respond? I would also search StackOverflow for potential pitfalls.
Secondly, worth considering is all the functionality contained within language’s ecosystem. Some languages go hand in hand with some use-cases and it can narrow down your choices a lot. From my own perspective I can tell that most problems are on the boundaries between technologies, so it is important for them to integrate well.
Other key points to consider:
- if major sponsor backs up the technology
- if it’s key pieces are stable
- how much training does it require
- is it supported by current tools
- if it is secure
- if it is maintainable
- how hard to introduce the technology
Proof of Concept then MVP
Being practical always beats theoretic concepts. Creating model is the only way you learn if the stack is well suited. When you have working POC you can jump-start the project from it. This is the creative part, after then further developments are much more predictable.
I advise developing many versions concurrently, eliminating some along the way while experimenting with different frameworks. I wouldn’t bet everything on one stack I would choose 3–4 of them. What should be the time-span is up to you, but best not to exceed length of two weeks. The longer you take the more tired or hooked up you are towards particular technology. And the outcome should be cold and calculated estimate.
Developing POC is equal to evaluating feasibility of the solution. This is time where the budget is decided. You also assess risks that can sabotage the product in the long run. Remember that you should be working at rapid pace.
After the Proof of Concept stabilizes, you start working on MVP (Minimum Viable Product). The difference is that the latter should be a product ready to be shown to the stakeholders / customers. It’s main goal is to elicit feedback and make corrections if necessary. MVP should take at most couple of months. In the end you will have UX ready to show to business.
Remember, it is only a core subset of features. A mistake is to try to implement too much. Solution is presented to the consumers so the Observability solutions have to be in place. Sometimes routing of traffic will be necessary for the sake of A/B testing.
For dynamic environment favor adaptability
Some businesses require constant shifting and aligning to new trends. These tend to be projects on the edge of research and engineering.
Ideal language should:
- provide way of abstracting things (Open/Closed Principle)
- be multi-purpose with large ecosystem of libraries
- make rapid development easy
- have good support from the maintainers
New languages represent most of these features— but not newest ones.
- Python — it is simple and easy to understand. It has vast collection of libraries and strong community
- JavaScript — it is good for web-development. It is flexible, with good support from the community, allows for really fast development
- Go — good for building scalable systems, because of it’s simplicity. It is also very performant and designed for system programming
- Kotlin — it has strong ecosystem because of running on JVM. It’s very versatile and statically typed. It is good for mobile development.
Stable tools for critical components
We should treat language and framework hypes with a grain of salt. The pace of software evolution is pretty fast these days. Survival of a particular tool is a matter of a couple of years and after this time I would consider it to be future-safe. Having been here for a while these languages proven to be “worthy” and gained momentum. What is more important they have major features stable.
Most of the software has layered architecture with core in the middle. It doesn’t change much throughout life of an application (if ever changes at all). So rapid development is not what we are after. We rather would like a language that is predictable and will be around in the future.
Ask HN: Why Did Pascal Fail? | Hacker News
Why Lisp Failed (2010) | Hacker News
What is the best way to go? To stay in the middle. So find a programming language in its golden age, which has been for a while. A C++, Java, and Python are all versatile and excellent choices. They also show active community support, are general purpose and are adaptable to new trends.
Standardize on what you got
Some companies set up internal standards to foster communication and effectiveness of teams. Standardization can incur the same languages, processes and tools. Most notable is Netflix:
Netflix OSS and Spring Boot — Coming Full Circle | by Netflix Technology Blog
which standardized on Java and then Spring.
The question is which is a better polyglot stack or standardized? The answer is as always — it depends.
The cost is proportional to the conceptual distance between the ecosystems. One thing is rewriting some services from Java to Kotlin. A completely different thing is rewriting a native assembly solution to micro-services in Java.
Then also there is the concept of synergy — combined power of teams that is greater that the sum of them. Excessive communication and debugging problems on the boundaries doesn’t help synergy. And this is what you ask for when having many stacks. That’s why standardization is better on the global scale.
My personal opinion is this: you should have a good reason to introduce a new language to the stack. And this can be:
- easier to attract talent
- does the job better
- old solution went commercial / is dead
Choosing boring languages
Boring languages are not so popular and marketable, most of the times they do the heavy-lifting. Their maintainability, makes them perfectly suited for large systems and long running projects.
One year ago I learned programming in Go. Having tried lots of different languages already, I constantly had a feeling that it is deficient in features. But I must admit I have never been coding so effortlessly and having fun in my whole life.
Ease of use trumps conciseness. The expressiveness is good but the less features the less cognitive load on the mind. Someone said that systems programming languages just have to be dead simple. This explains the C’s popularity didn’t fade away.
Boring languages are usually the most scalable. Each syntax feature has a cost associated with it. First every developer has to learn it exists. And even then it makes chances of bugs creeping in. Boring also means stability and low number of quirks, which is a good thing.
There are two kinds of unknowns — know unknown and unknown unknown. When you choose boring language it doesn’t take too much time to fill in the gaps. But imagine that you saw Scala for the first time — scary right? As the presentation in the link tells us there is much larger dichotomy between two kinds of unknowns in the feature-pack languages.
Stay with old solution when you can
Developers are in constant temptation for rewriting. But sometimes there’s just no value in doing so, apart from good marketing. If you decide to rewrite the app you put all the knowledge you attained for years to the trash. And the old one has to still be maintained.
There are times that the code becomes so messy that rewriting becomes the only option. You still shouldn’t do that. Instead every effort should be made to learn what it does and refactor it (it’s how some companies got out of business).
The changing of your stack is a cost. Maybe even a considerable undertaking. Because the company you are working for is not a charity, it should be accountable. So you should have a good reason to do so, and benefits should outweigh the cons.
New shiny tools are mostly immature and burdened with a high risk of something going wrong. If there are latent bugs / vulnerabilities in these tools be prepared to bump up its versions months after months.
Manage risk
Everything we do is burdened with some degree of risk, that’s a fact. But we shouldn’t restrain from it, we should be proactively managing it. The same case is for software development.
You may be familiar to innovation tokens. You spend most of the time fixing bugs and maintaining software. You reduce technical debt and in the end your reward is some innovation tokens. You can use them for whatever you like. May it be new framework, rewrite of an service or whatever.
When planning for using a new stack, I would use something similar. Let’s call it frogs. First you sit down with stakeholders and agree how much frogs are you willing to swallow. Then you agree on how much risk is per one frog.
Then when it comes to picking technology you think of how risky they are in terms of tokens. Take into consideration language itself + all the ecosystem dependencies and their interactions.
Then in the final step you throw away those stacks that surpass the maximum risk you were willing to spend in the first place. I’m not swallowing excess number of frogs, no thank you!
Conclusion
This is it. This article presented my own thoughts sometimes overlapping with what is on the web. Everyone’s perspective is different and you may not agree with some statements here. What I wanted to achieve is to encourage you to do your own research.
Thank you for reading.
