How to Build a Scalable Software Architecture Part 1: Monolith vs. Microservices

In March, Amazon published a blog post stating they’d reduced costs for a Prime Video monitoring service by 90% after restructuring their microservices architecture into a monolithic one. For many, this article was confirmation of a growing bias in the industry towards microservices - even when they don’t make sense.

Knowing when to use a monolith vs. when to use microservices can play a pivotal role in shaping business outcomes and customer experiences. These architectural decisions are critical for maximising the performance, reliability, cost-efficiency, and scalability of your applications.

In this blog, we will discuss:

• Why the monolith vs. microservices debate matters
• How to determine when to use a monolith and when to use microservices
• What logical modularity is and how you can use it to develop a scalable architecture
• How we implemented both monoliths and microservices into the software architecture for our developer platform at Northflank.

Why the Monolith vs. Microservices Debate Matters: Amazon Prime Video Case Study

To understand the importance of sound architectural decisions, we can look at the Prime Video monitoring service blog post. In their article, the Amazon team discussed how they reduced their cloud spend by 90%, and improved their system’s scalability by consolidating several microservices into a monolith. Now it’s worth noting that multiple factors contributed to that cost reduction, including migration from expensive AWS Lambda and Step Functions to Elastic Container Service (ECS) on EC2 instances. However, the main architectural improvement was the elimination of excessive data shuffling. Initially, Amazon’s system decoded videos into frames and analysed them in several distinct services using S3 as a buffer for frame data. By consolidating their services into a single process, intermediate data could be kept in memory, eliminating the need for S3.

The team at Amazon Prime Video fell into a common pitfall with their initial microservice-based implementation. Their services needed to exchange large volumes of data, which wouldn’t have been the case if they implemented their system with a centralised monolithic architecture. The 90% difference in their resulting cloud spend speaks to the consequences of implementing the wrong architectural paradigm.

What can we learn from this? When choosing between a monolith or microservices architecture, there are many angles to consider, including the cost and implementation of auxiliary tools and resources. It’s also important to prevent industry bias from clouding decision making for your specific use case.

How to Determine When to Use a Monolith and When to Use Microservices

As with most aspects of software engineering, there is no one-size-fits-all solution. It’s important to approach your decisions thoughtfully, and with a comprehensive understanding of the tradeoffs involved.

First, gather a thorough list of your system requirements for:

• Resource fluctuations
• Fault tolerance
• Data consistency
• Scalability

You also need to consider the resources and expertise of your engineering organisation, and your capacity for automated build and release processes. From there, use the below as guiding principles to assist in the decision-making process.

Monolithic architecture

Monolithic architectures are generally cheaper to build and maintain. If your project’s scope is relatively limited or needs to be built quickly, first consider a monolithic implementation. Monoliths are also best for use cases where a high degree of data consistency is required.

Microservices architecture

Only start considering microservices when one of the following is true:

• Your system has components with highly-variant resource requirements
• Your system will be worked on by multiple teams with different domains of expertise
• Your system has high fault-tolerance requirements
• Your application can tolerate weaker data consistency guarantees

Under the right circumstances, a microservices architecture can facilitate higher agility and reliability.

If you choose to adopt a microservices architecture, it’s important to adhere to loose coupling between services. Also, be prepared for unexpected performance bottlenecks and weaker consistency guarantees.

The other downside to microservices is the high cost to set up, maintain, and manage them. As a minimum, teams considering a microservices architecture should plan to implement CI/CD. Luckily, there are a number of tools and platforms that make microservices architecture accessible to teams that might otherwise lack the resources required to manage a complex deployment from scratch.

Northflank is a good example, as it provides the simplicity, pricing, and performance of container services like AWS ECS, but it can accommodate microservices just as easily as monoliths. It also provides built-in CI/CD, so managing complex deployments remains simple.

When evaluating cloud platforms and other auxiliary tooling, make sure they can scale with your team and the complexity of your application both at a functional level, and a financial one. Many solutions lock you in with specific features, don’t allow for high levels of configurability, or become exorbitantly expensive as you scale.

Regardless of your chosen architecture and tooling, it is crucial to bear in mind that good coding practices win the game. One fundamental objective is to strive for loose coupling between application modules, and strong cohesion within them. If a monolithic implementation aligns with this principle, it provides the flexibility to separate some of its modules into microservices more seamlessly as you scale. It’s possible to start out with a monolithic service and gradually transition to a microservice architecture when the need for it arises. It’s also possible to employ different architectural patterns for different parts of your application. In other words, monoliths and microservices are not mutually exclusive, and can be hybridised.

How to Leverage Logical Modularity to Build a Scalable Software Architecture

There is a spectrum between purely monolithic and fully distributed architectures. In building Northflank, we’ve found that the best architecture implementations don’t demand 100% commitment to microservices or monoliths. However, it is essential to strive for logical modularity.

Logical modularity refers to the degree to which an application’s modules are independent of one another. High logical modularity implies that it’s easier to replace the implementation of a module without affecting other parts of the application. It also means that functionality can be changed more easily while modifying a minimum of modules.

Neglecting to design with logical modularity in mind can result in a system that becomes complicated and eventually untenable. This phenomenon is often dubbed as a “big ball of mud”. Perhaps an even worse result is the distributed monolith, which combines the drawbacks of both microservices and monoliths.

When striving for logical modularity, follow proven software design patterns and adhere to the following guidelines:

• Application modules should be as independent of each other as possible. In particular, there should be no dependencies to a module’s specific implementation details (low coupling)
• The logic within a module should belong together and be responsible for exactly one particular function of the application (high cohesion)

By focusing on the above best practices, we have been able to build a scalable system that gets the best out of both monolithic and microservice architectures.

How We Built a Scalable Software Architecture at Northflank

At Northflank, we built a platform that empowers developers to build, deploy, and manage cloud-native applications from a unified interface. To do this, we needed a system that could coordinate thousands of user deployments at a time. Thus, fine-grained scalability and fault-isolation were top concerns.

To meet these requirements, we chose to build several monolithic services that are supplemented by smaller, more specialised “satellite” services. Our core components like our API and platform services are monolithic. We also went with a monolithic approach for more complex subsystems, like the controllers for "Bring Your Own Cloud" clusters or our addon system. By focusing on monoliths for our core services, we were able to develop and deploy them faster and more easily. In separating the monolithic components, we’ve enabled them to scale independently. For example, our add-on controller component can be scaled separately from our user-facing platform.

In addition to our monoliths are our satellite services. They cover clear-cut concerns like workload monitoring, log forwarding, event dispatching, and so on. These specialised services enjoy all the benefits of a microservices architecture - they can be shared across our monoliths and easily added or removed.

Our platform consists of about 40 services of varying complexity and size. We believe that the modularity of our architecture enables our team to work more efficiently on independent parts of our platform in parallel. It also allows performance-critical components of the platform to be (re-)implemented with low-level languages more readily.

So far, this approach has proven itself quite effective.

Takeaways

• The choice between monolithic and microservices architectures isn't binary, and hybridising both paradigms can be a viable option.
• Whether you choose to go with a monolith, microservices, or a combination of the two, make sure to apply good coding practices. Always aim for logical modularity, which will empower your team to manoeuvre effectively, no matter how business requirements change.
• Be sure to consider the impact of auxiliary tooling when making architectural decisions, so you can effectively manage your expenses and human resources.

For more content like this follow us on LinkedIn and Twitter.