Hi! My name is Dima, and I'm an architect at Kuper. Today I want to talk about the Strangler design pattern, which we used to move business logic out of a monolithic system into a separate service.
First we turn to the original source, and then move on to the practical points we ran into in the process.
History of the Strangler pattern
Martin Fowler first coined the term after he saw a particular kind of fig that settles in the upper branches of trees. Gradually the plant descends to the ground, takes root, and, wrapping around the tree, uses it as a temporary support and a source of nutrients. Eventually the tree turns into rotten pulp, and by that point the fig stands on its own.
Mapping the metaphor onto software development, Fowler proposes migrating critical systems along an evolutionary path: gradually build the new system on the base of the old one, letting it grow slowly over time until the old one is eliminated ("strangled", in the author's terminology).
In this article Fowler calls such an approach the Strangler Fig Application, illustrating it with a photograph of a fig. To get ahead of myself: to me it looks more like a treehouse, so here is my own photo:
This approach lowers the risks:
- through incremental migration to the new system in small steps, each of which is easy to reverse. So much so that the migration can even be put on pause;
- through the ability to roll back to the baseline configuration (the monolith) whatever happens.
For simplicity, we will call this approach the Strangler pattern.
The Strangler pattern for decomposing legacy systems
Sam Newman describes using the approach to decompose monoliths in detail in Monolith to Microservices, proposing three steps:
- Identify the functionality we want to move.
- Implement it in a new service.
- Switch over to the new implementation.
By identifying the functionality the author means studying the logic to see whether it can be localized, plus a preliminary refactoring pass. The decision of how to carry out the extraction is left to the readers: rewrite everything from scratch in the new service, or refactor and port the code. After that, user traffic is moved to the new service.
It makes sense to use the pattern when:
- the legacy monolithic system is a black box;
- there is no way to change it (or we do not want to — a long release cycle, for example).
The book has a funny story on this topic: at one point the author was working on decomposing a large legacy system, an old boxed product installed at an organization. Reverse-engineering a compiled program is a thankless job, so to get some grip on the business logic he contacted the author of the solution and asked for the source code. He was ready to give any non-disclosure guarantees, but the code's author refused.
A few years later Newman met the system's author at a conference and asked why he had never sent the sources, since there had been no risk. The author answered awkwardly that he had simply lost the system's source code.
This is an extreme and degenerate case. In reality we mostly deal with code that can be viewed and changed, even if it is tangled enough.
Starting conditions
The goal of this article is to share our experience with the Strangler pattern. But first, a bit more context to explain why we chose this path.
-
A large Rails monolith. One of its parts, a very heavily loaded one, was causing failures due to a shortage of database resources. Heavy queries were knocking out the DB replicas, threatening the operation of the whole system.
-
The functionality itself is a set of related endpoints for search and filtering, exposed to the user based on their geolocation and the locations of the delivery stores. This simplified the task, because the first item in Sam Newman's plan was effectively already done.
-
By that point we had already run Event Storming and identified a bounded context and its boundaries. The scope of the business logic to be extracted was clear.
-
This concerned part of the business logic on the critical user path (without choosing a store you cannot put an item in the cart or place an order). So the priority was a focus on stability. There was no option to simply switch traffic to the new service and hope everything would be fine. As it became clear later: it did not go well right away.
For the Strangler pattern, the conditions were ideal:
- we do not want to make changes to the monolith (a long release cycle) and focus right away on moving the business logic out of it into a new service;
- the endpoints can be replaced wholesale, since the business logic in them is effectively localized (it turned out not to be);
- we keep the focus on stability — this is exactly where the Strangler pattern shines!
Theoretical migration plan
An iterative, incremental and safe (in short, a very good 🙂) plan for migrating the business logic from the monolith into a separate service was drawn up:
- Start with proxying traffic through the new service. Then, "parasitizing" on that traffic, we can grow into it (proxying mode).
- In parallel, run the business logic in the new service and gradually move it to its new home (mirroring mode).
- Replace the old implementation with the new one (replacement mode).
- And in the finale we will probably arrive at a scheme where the response is enriched in the legacy system (API composition mode).
Let's go through the steps in more detail.
Proxying traffic
From the API gateway we send user traffic into the new service. It does nothing, it only proxies requests to the monolithic system. This step allows us to start studying the business logic and how the monolithic endpoints are used.
Mirroring the business logic
This is a mode in which requests are executed both in the service and in the monolith, in parallel.
At this stage we move the business logic into the service. To understand how identically the old and new implementations behave, we compare the responses from the monolith and the service, calculating a metric of their match (the accuracy metric). The business logic is moved out step by step here, which eventually gives high values of the accuracy metric.
Important: the client receives the response from the monolith, so mirroring does not affect users.
Replacing the functionality
This is the target mode of migration to the new endpoint in a separate service: requests are processed only in the service. At this stage it is assumed that the monolith will be dropped from the scheme as no longer needed (in theory).
The replacement process itself is smooth and gradual. Using a set of feature flags we can flexibly manage the share of traffic between the mirroring and replacement modes. On the one hand, this allows us to track the accuracy metric on real traffic; on the other, to adapt the new service to the load profile in the process.
API composition (optional)
API composition is a scheme in which the business logic fires in the new service, while the final response for the client is partially enriched in the legacy system.
This step may not be necessary if the service's new API is localized and does not return data outside of its own domain. In our case it was not: the necessary data was insufficient, and the response had to be partially enriched in the monolith. To do that we had to add instrumentation to it.
The choice came down to two options:
- compose the API and keep compatibility;
- implement a new API that is more idiomatic from an architectural point of view and move the clients over to it.
Because the mobile app could not be updated at that point, we had to choose the first.
Migration plan: a symbiosis of patterns and approaches
The new plan diverged from the original Strangler pattern in many details, although the main idea was preserved. It turned out to be a situational mix of various methods and approaches, with a number of features:
- API gateway pattern: traffic switching assumes that the system has an API gateway or another kind of reverse proxy to manage traffic.
- Authentication: we need to be able to authenticate traffic, because the business logic depends on it.
- Parallel execution pattern: the next idea was borrowed from Newman's book: the new and old functionality work in parallel. And we have the opportunity to compare the results of both outcomes. You have probably come across the scientist library, which is designed for safe code refactoring. It is roughly the same thing, only for refactoring a monolith into a service.
- A few more approaches to managing and migrating data from chapter four of Sam Newman's book: the service's business logic is based on data owned by another service or the monolith.
Practice
We start executing the plan.
Proxying, the first problems
Earlier I called the proxying mode parasitic: nothing useful has been done yet, and the coupling of the system is already growing. Because of the new intermediary, request latency grows from network calls. Turning on a persistent client for Ruby, which reuses an established TCP connection, helped improve the situation. But until the moment we completely abandon the legacy system this is unavoidable, so we treat the situation as normal.
Proxying is very sensitive to the capacity of the web server (Puma for Ruby), especially under load, when the monolith starts to respond slower and requests pile up in the proxy. The problem showed itself sharply on load tests. We had to fine-tune autoscaling, allocating more capacity, and reduce the response waiting timeout.
Already during proxying the advantages of the evolutionary approach started to show: we had the ability to move traffic (even for the proxy mode) to the new system in iterations, minimizing the negative impact on clients.
The multifaceted business logic of the monolith
The full scope of the legacy system's business logic cannot be grasped. Sooner or later you will find code behind three feature flags and a pile of nested conditions that fires unexpectedly.
In the service we wanted from the very beginning to do everything cleanly, without carrying over redundant business logic. We researched the code and identified which incoming parameters the business logic is based on, and introduced a whitelist of HTTP headers that we pass through during proxying.
At night we successfully switched traffic to the new service, and in the morning we learned that old mobile clients had lost part of the functionality because of HTTP header filtering that the multifaceted business logic relied on.
We switched back, fixed it, turned it on again.
Stocking up on rounds
The method of carefully staring at the monolith's code let us down. And we do not want to acquire new knowledge about the legacy system through incidents and impact on users.
A functional regression, preferably an automatic one, could have protected us from the problem. Ideally it would use real request profiles with real attributes (query parameters, HTTP headers) that affect the business logic.
Having analyzed the logs, we collected historical data on user requests and called them "rounds". With them we "fired" at production "like users" and ran a regression, comparing the results of identical requests to the service and the monolith.
Significant parameters can also be separated from insignificant ones with the help of rounds: we run several iterations of the regression with a reduced set of rounds. If the responses do not match, then the behavior on the reduced set differs, and the discarded parameter was an important one.
A statistically significant sample of rounds and a simple script for comparing results closed the problem of functionality regression, and automatically at that.
We successfully passed the first step of the migration plan using the Strangler pattern and moved on to the most interesting part — mirroring.
Mirroring
Mirroring is the stage of moving the business logic: the logic runs in the service in parallel with the monolith, which remains the primary one.
But it is impossible to move what you do not know: we had an idea of the basic scenarios, but we did not understand the full scope and multifaceted nature of the business logic.
At this step we had to:
- work out the logic of the original endpoints;
- move it to the new service;
- validate that nothing broke.
Do not solve all problems at once
In the new service you want to solve all problems at once: refactor the code, fix the flaws in the business logic and patch the vulnerabilities found, change the implementation stack. After all, just moving legacy code from one service to another is ugly and boring!
In the second chapter of the book Newman warns that such an approach is destructive and suggests following a simple rule without fail. The goal of a microservice migration must be exactly one. It is fatal for a project not only to have no goals, but to have many of them.
It happens that several side goals are added to the main one, for example:
- Switching to a different stack;
- Fixing errors and vulnerabilities;
- Improving performance;
- Fixing bugs;
- Any other changes/improvements.
In our case the goal was to stabilize the problematic part of the monolith by moving the load to the new service.
It seems obvious: why not switch from Ruby to Golang right away? But in order to move business logic, you need to understand it. Changing the stack, in my opinion, adds extra cognitive load and takes away part of the mental resources that you will definitely need to untangle the legacy code.
I wrote more about this in my Telegram channel.
In the process we realized that if you do not frame it strictly as a business-logic migration and try to do everything at once, the project will never end (remember the HTTP header whitelist).
Business logic: expectation vs reality
When you have run EventStorming, described business processes, isolated bounded contexts — everything is smooth and beautiful: independent services describing localized business processes, connected to each other asynchronously…
In practice a large part of this exists neither as separate services nor as localized modules in the monolith. And the business logic being extracted is based on projections of data whose source is a bounded context that is no longer autonomous. It only exists in the monolith now! In short, everything is coupled to everything.
Data dependence
The first attempts to pull part of the functionality out of the monolith are blocked by the absence in the new service of the data needed for the business logic to work. The problem is not the absence of the data itself (it is in the monolith), but rather that there are no formalized sources (neither Kafka topics nor implemented endpoints) to obtain it. Let's call this phenomenon missing-data-sources. I wrote in more detail about approaches to the problem in another post.
The problem is especially acute when the first services move out of the monolithic application, when all the necessary data sources are in the monolith. With the gradual move to services, new Kafka topics appear that partially cover the needs.
In the process of working with missing data sources we identified three main approaches to solving the problem (a detailed post in the Telegram channel). I will list them in increasing order of soundness.
A synchronous call into the monolithic system
The most obvious thing is to go to the monolithic system synchronously and request the data you are interested in. Bad, because it is synchronous and not idiomatic. Good, because it is fast and cheap.
The asynchronous method
The two other options are to move to an asynchronous method of data delivery, for example through a Kafka topic, and, depending on the effort available, to work out its contract. This will be either a temporary transitional topic (which you will also have to move away from later) or a properly worked out and agreed with the potential data owners topic with a full idiomatic contract.
In case of a loosely worked out contract, it may turn out that the topic itself will have a "monolithic character": for example, it may contain information from several subject domains, and later it will need to be decomposed.
In the second case, greater involvement of the owners of the part of the monolith that serves as the data source is needed. But an idiomatic contract can be agreed upon that will be preserved when the service moves out (though that is not certain).
Hand-cranked synchronization
All three options are my interpretation of the approaches that Sam Newman described in the fourth chapter of the book (the list presents their names).
The fourth option: if the data is static or changes rarely, it does not have to be synchronized at all, just copied. In this case it is better to agree in advance with the data owners on joint actions in case of a change in the source (for example, the owners update the copies too). Such is "Kafka on a hand crank", with the corresponding "hand-crank guarantees".
There is no single correct approach to data synchronization. We based our decision on the capacity we had to make it as asynchronous as possible, and on whether the potential data owners could get involved in agreeing the Kafka topic contract.
As a result, to move the business logic we needed to run several mini-projects to launch the corresponding data sources, in some cases on the base of the monolith, in others on the base of services. In some places we had to copy the data, in others move ownership of it into the service.
This preparatory stage took up most of the time.
Degradation of business logic
This is my favorite option for migrating business logic — through its removal.
As we sorted out the nuances of the historical business logic, we ran into ancient and dubious artifacts.
For example, once we came across some dubious business logic. The complexity of formulating the task unequivocally indicated its narrow applicability. And the investigation showed that the feature had no customer. The author of the task in JIRA and the implementer no longer work at the company.
It pays off to get rid of such business logic in advance, so as not to move it into the new service. The best code extracted from a monolith is the code that is cut out.
To confirm or refute the hypothesis, we had to ship metrics counting the number of firings of this part of the code. For two weeks — not a single firing.
The result: the feature is not needed and does not work. So it does not need to be moved! So there is no dependence on additional data!
Lucky? But note that in order to cut out the code, we had to write code that measures the number of firings. As they say: "Painful? At least it's free."
Product blockers
Later on, we began to call product blockers the cases where complex interconnections in the business logic hinder a simple port into a service.
A product blocker is an element of business logic whose full preservation during migration is problematic and requires additional effort. In one case the solution may be building the required data sources; in another, temporarily or completely dropping part of the functionality.
The question of what to do with a product blocker is open to discussion. On the one hand it runs into technical limitations (how long to preserve the functionality), on the other — into product ones (impact on the product's business metrics, conversion, GMV, etc). Each case requires study and agreement.
Product technical debt
Continuing to work on moving the business logic, we ran into another class of problems — feature flags, A/B tests and other elements of the monolith's codebase that increase the variability of the code.
And during the move we wanted to freeze the business logic as much as possible, so as not to do extra work and not to encounter a difference in behavior between the new and old implementations due to desynchronization of feature flag values or A/B test settings.
We classified such moments as product technical debt: we agreed with the owners to stop using the flags, to cut the code out, or, as a last resort, a manual process of synchronizing feature flags in two places. The principle is the same: the less code we move and the more deterministic it is, the fewer problems there will be in the process.
Accuracy metric
How do you make sure that the business logic has been moved into the service in full and nothing has been lost?
A way is needed to validate the quality and completeness of the migration. The accuracy metric took that role. During mirroring we compare the response from the monolithic system with the result of the moved code.
The accuracy metric is the percentage of matching results: how much the business logic in the service matches the business logic in the legacy system. This depends on how completely and accurately it was moved, as well as on the freshness of the data we stream into the service.
At the start, when there is no business logic in the service at all, the value of the accuracy metric is zero.
As the business logic is moved, the value increases. A big jump happens when the main functionality is moved. We pick the low-hanging fruit — we move the main functionality at a fast pace.
And then it gets harder, since the increase in the accuracy metric happens through analysis of mismatches of specific results and searching for edge cases that we did not account for.
Analyzing edge cases requires greater immersion into the specifics of the business logic. The time spent searching for problems increases: you need to investigate, identify nuances and historical backgrounds, involve experts.
There comes a moment when further increase of accuracy becomes unfeasible. In this case it is worth stopping, having agreed in advance with the product on what mismatch you are ready for.
We identified three reasons that affect the accuracy metric:
- incompleteness of the transferred business logic (some edge case in the code was not accounted for / not extracted);
- irrelevance of data (not all events come through topics, which is why computations in the service are performed on lagging data);
- the effect of asynchrony (a special case of the second).
I will tell you more about the last reason.
The effect of asynchrony (eventual consistency)
If in the monolith any changes to data in the DB happen instantaneously (in one transaction), then when moving to an asynchronous architecture you cannot count on that — data arrives with a delay. For example, if a consumer accumulates a lag and there are delays in processing and updating data in the service, then computations are performed on lagging data. At critical moments of large lags there is a clear correlation between the accuracy metrics dropping and consumer lag growing. This effect is called Eventual Consistency. It implies that data in distributed systems is eventually consistent, but at a moment in time they can differ. This is the price of ensuring high availability.
What to do with this problem?
- Do not call it a problem — it is an effect of the transition to independent services. We must accept this effect and learn to work with it, including in product terms.
- Optimize data ingestion, reducing the delay rate. In a talk about the inbox-outbox pattern (video, slides) I described what we did for this.
However much we might want it, we will not reach 100% accuracy of responses. Therefore we agreed with the product on a value of 98-99% as acceptable and finished moving the business logic into the service.
Gradual rollout
Even in mirroring mode we cannot afford to turn it on for 100% of requests at once, because we are not sure what volume of resources the extracted business logic will require in the separate service.
The right approach is to enable mirroring gradually, keep assessing the situation, adjust the system to the load as needed, and tune autoscaling.
By the way, remember the rounds for automatic regression? We loaded them into K6 for load testing that is close to reality.
Replacing the business logic
Everything is ready: the business logic is moved and works in parallel with the primary one, the accuracy satisfies us and has been agreed with the product. Ahead is the most important regression — on real users.
And the smooth switching to replacement mode immediately brings us to new, unaccounted-for edge cases. They are rare, and the accuracy metric (within the agreed error margin) does not catch them. At the same time these edge cases can be significant in terms of business value. That is exactly what happened.
We rolled back to the previous step several times to figure things out. As they say, play with an axe and you'll cut yourself.
In retrospect I noted several important points:
- The ability to roll back to the previous step came to the rescue more than once, helping to reduce risks and investigate the problem calmly (without bringing it to an incident);
- The number of rollbacks in the process exceeded all expectations. In this respect rollback became a working tool rather than an emergency situation;
- The numeric basis of the accuracy metric formula is traffic from users, therefore it detects common cases well. But rare does not mean unimportant.
With that, the replacement stage was complete.
Enrichment
Although the business logic was successfully moved out, we could not handle part of the data the endpoints returned: the endpoints being replaced returned data conceptually belonging to other contexts (and several at that). We did not want to pull this data into the service, and in order to preserve API compatibility we moved to an "enrichment" scheme.
The business logic is executed autonomously, and to render a complete response the monolith is involved. In the future such technical debt can move to BFF (backend-for-frontend) or be eliminated through migration of the client to the new API. But that is not quick.
The scheme is tightly coupled and not elegant, but temporary and necessary. Therefore it is worth accepting it as a normal stage of the migration process. Especially since Fowler himself describes the use of the Strangler pattern as long-term: the approach is flexible and assumes pauses in the evolution or the process of strangling the old system, sometimes even for an indefinite period.
At the enrichment (API composition) stage we finished, the goal was achieved — the load was moved from the monolith to the service!
Retrospective
The Strangler pattern is a working option that allowed us to focus on reliability. More than once we rolled changes back a step and always kept in mind the possibility of temporarily returning to the original legacy system or rolling back to the previous step. This added confidence and allowed us to move safely. Subsequently we resorted to using it several times in other projects, and in order not to write the same code several times we implemented a library for Ruby — the sbmt-strangler gem.
The Strangler pattern is a reliable, mature and deliberate (old-school) evolutionary approach, which the experience of its use confirms. But the migration process requires more time to implement and twice as many compute resources. Because for a while, two versions of the system run at once.
The Strangler pattern certainly will not save you from diving into the thickets of legacy business logic. It is all just a technical method of migration, which will not help solve the problems of high coupling and entanglement of business logic in the legacy system. You will have to deal with this in any case.
Changes to the legacy system will have to be made anyway. Despite the long release cycle, changes in the monolith before the extraction can be justified, because they save weeks of work (for example, degradation of business logic).