Why we Rewrite (Even When we Shouldn't)

As we saw in the previous chapter, a software rewrite can be a dangerous undertaking - there are a number of very real costs and risks that can sink even the most well-intentioned effort. And yet we do rewrite! Often. Despite the warnings and obvious dangers, we come together, business and technology, and collectively declare: "damn the torpedoes, we're rewriting this old pile of code, and this time we'll succeed!"

The question I'd like to explore in this chapter is "why". Given all the costs and risks, why do we so often choose to rewrite a working legacy system rather than refactor it in place? Leaning on insights from psychology, sociology, and other fields, I'd like to expose some subtle and not-so subtle forces that serve to nudge each of us toward the path of the full-scale rewrite. Though none of these individually are the ultimate reason we choose “big modernization”, together I think they form a powerful undercurrent that float us in that direction.

I want to be clear though. I'm absolutely not arguing that a rewrite is always the wrong choice - there are many good and justified reasons to rewrite, which we'll see in the next chapter. And further I'm not trying to cast aspersions on developers or other stakeholders for making "bad" decisions. We do all have blind spots and biases, however. When we're aware of them, we can work to counteract them - which in this case might mean sparing ourselves and our organizations from sinking countless hours and dollars into what might be a doomed rewrite. With those caveats out of the way, let's take a look at some of these forces at play...

The Enjoyment of Creation

Developers enjoy the act of creation. We are builders by nature. Though we understand that the job often requires us to bravely venture into the dark recesses of others' code to slay some lurking defect or tweak some abstruse calculation, this isn't what attracts us to programming. It's just a necessary evil. When given the chance, we want to build our own systems. We want to craft a solution to a problem in the way that makes sense to us, in the style that appeals to us. At the risk of sentimentalizing, programming is our art. In the same way a painter would prefer to start from a clean canvas than paint within the portrait of another, all things being equal, we too want to give life to our own creation.

In fact, there is evidence that this fondness we have for the things we've made ourselves is not just a part of the psyche of programmers but is built into all of us. Dan Ariely, in his book Payoff: The Hidden Logic That Shapes Our Motivations, gives an interesting anecdote that underscores this human tendency:

Consider cake mixes. Back in the 1940s, when most women worked at home, a company called P. Duff and Sons introduced boxed cake mixes. Housewives had only to add water, stir up the batter in a bowl, pour it into a cake pan, bake it for half an hour, and voilà! They had a tasty dessert. But surprisingly, these mixes didn’t sell well. The reason had nothing to do with the taste. It had to do with the complexity of the process — but not in the way we usually think about complexity.

Duff discovered that the housewives felt these cakes did not feel like the housewives’ own creations; there was simply too little effort involved to confer a sense of creation and meaningful ownership. So the company took the eggs and milk powder out of the mix. This time, when the housewives added fresh eggs, oil, and real milk, they felt like they’d participated in the making and were much happier with the end product.

Ariely goes on to describe a number of scientific studies that show this exact same effect: we are in fact hard-wired to enjoy creating things, and further when we do create, we overly value that which we've built to the creations of others (independent of how good what we've produced actually is!).

I think this directly speaks to how we as developers can feel about maintaining legacy systems. Sure, we’re capable of rotely following some prescribed pattern to add a new column to the report or button to the screen, but that doesn't give us that feeling of attachment or meaning to our work. What we'd like is to crack our own eggs and mix in our own milk and oil. In other words, our brains may naturally tend toward wanting to rewrite over refactor.

The Drudgery of Maintenance

The flip side of seeking out enjoyment from the act of creation is avoiding frustration from the drudgery of maintenance. And there is plenty of frustration built into legacy systems. Over time, software tends toward decay. Business requirements shift, forcing developers to remodel what used to be the kitchen into the front porch. Critical deadlines force us to cut corners, and ship code with TODOs and copy-and-paste. And the comings-and-goings of old and new developers to the system create what, over time, look like sedimentary layers in the architecture - "you can see from the style that this code comes from the pre-IPO period, before the time of dependency injection".

Working within legacy code like this can be frustrating to say the least. To fix even the simplest bug might require us to trace through convoluted execution paths as they wind in and out of different software ghettos, where weeds have grown tall and broken windows have gone un-fixed. Keeping the logic (or illogic!) of the system in our heads can be a herculean task. And while we dutifully accept our fate and fix those defects and add those small enhancements, all the while we think about how we wish we could do it differently. We think about how cathartic it would be to put a torch to this bastion of complexity and technical debt and start over from scratch.

To the end user and perhaps even to the business, however, it may be a different story. Sure the legacy system might not be as visually pleasing or it might require a few more clicks or seconds than might be preferred, but it likely still works. The system is functional, relatively reliable, and the business processes around it, though not optimal, are at least well understood. But to the developer, especially one who was not the original creator (see above!), the system can represent not comfort but frustration and constraint. And the more time we work within it, the more we want to escape and rewrite it.

Self-Interest and Fashion

Beyond just seeking enjoyment or avoiding frustration, we as developers are also well aware of what tangibly benefits us and our careers, and often it's not maintaining legacy systems. A 2019 Stack Overflow developer survey shows a direct correlation between the newness of a language and the amount of money a developer makes. Whereas the average age of top 5 languages by salary is just 15 years old, the average age of the bottom 5 languages is 39 years old.

Top 5 Languages by Average Salary

Bottom 5 Languages by Average Salary

Now of course many other factors play into these numbers, but all things being equal, it's pretty clear that if we don't keep pace with newer technologies, the amount of money we can expect to make will be less. This is probably not surprising for any developer who has recently been on the job market. Recruiters, resume bots, and even fellow developers are hunting through our experiences for the latest buzz words: microservices, Kubernetes, progressive web apps, or whatever is the technology du jour. The newer and more cutting edge, the better. The logic, presumably, is that a developer who is up on the latest skills must be more motivated and invested in their craft than someone who has only been doing the yeoman's work of maintaining a legacy system, (the irony being that working within the legacy system might have been the right thing from the perspective of the prior employer!).

Now some of this does make sense. The job of programming does entail a high degree of exploration and experimentation, and so a passion for learning is indeed important. However, there's a tendency, I think, for our industry to sometimes favor fashion over substance. When we pick a language, tool, or framework, of course we want to choose the one that is right for the job, but we also consider what the choice says about us. Are we the kind of person that uses a proven but stodgy language like Java, or are we "forward-thinking" and "innovative" and pick Go or Kotlin? In a sense, the technologies we use are like the clothes we wear - they speak to who we are, what we stand for, our worth. It can be technology as fashion.

Now of course the analogy is far from perfect, but consider some parallels from this description of the fashion cycle by sociologist Herbert Blumer:

The elite class seeks to set itself apart by observable marks or insignia, such as distinctive forms of dress. However, members of the immediately subjacent classes adopt these insignia as a means of satisfying their striving to identify with a superior status. They, in turn, are copied by members of classes beneath them.

In this way, the distinguishing insignia of the elite class filter down through the class pyramid. In this process, however, the elite class loses these marks of separate identity. It is led, accordingly, to devise new distinguishing insignia which, again, are copied by the classes below, thus repeating the cycle.

In a nutshell, fashion, whether in clothing, music, or programming, is about class differentiation. The elites innovate. The masses catch up so they can be associated with the elites. As soon as they do, however, the elites are no longer "elite", and so they must break away and innovate again. The masses eventually follow, and the cycle continues, ad inifinitum.

When it comes to the question of rewriting or refactoring, I think this can be another subtle force that pushes us as developers toward the choice of full-on modernization. To keep an existing system alive is akin to wearing your clothes from a decade ago. Sure, they may still keep you warm and dry, but they are old and worn and probably don't present to the world the image you want show. And for those who aren't prone to the frivolousness of fashion, we have to at least know that it does tangibly affect our careers. Keep pace or make less.

Faulty Intuition and Mental Short-cuts

Putting motivations and self-interest aside, there is another powerful force which can compel us to believe that rewriting is the obvious and best decision for our legacy systems even when it's not: our own intuition. In his amazing book on decision making and cognitive bias, Thinking Fast and Slow, Daniel Kahneman explains that while it’s easy for us to form opinions on all types of questions (and to be confident in these opinions to boot!) oftentimes they’re based on simple mental short-cuts, and not solid reasoning. For decisions where the stakes are small this might be ok, but for the question of whether to refactor or rewrite an entire system, where hundreds if not thousands of hours of effort could be at stake, this isn’t going to cut it. We need to be sure.

One of these subtle mental short-cuts we use is called substitution. When confronted with a question that would require deep effort and analysis (which is hard work!), we often just swap in an easier proxy question that we can answer more quickly. For example, to know whether it's worth it to rewrite vs. refactor, we should understand what the cost of a rewrite would be, how much value it would bring to the organization, when ROI would be realized, and so on. But this is hard! So instead, we substitute in questions that are easier to answer like "is the existing system buggy?" or "is the UI out-of-date?". You bet it is! And while it seems like we addressed the original (hard) question, we actually didn’t – we just swapped it for something simpler.

Over the years, I've seen this type of justification over and over. Rather than pause to try to answer the real but difficult question - whether the net value of a rewrite is greater than the net value of a refactor - we often fall back to easier substitute questions. We say things like “we’re going to rewrite because the legacy system is hard to work with" or "because no one understands it". Yes, these answers are relevant, but they're just not sufficient. Maybe it's true that no one understands the code, but how much would it cost for a person to gain that understanding? And is that cost greater than or less than rewriting the entire system from scratch (which would probably require understanding it all anyway!). Ditto for bugs or tech debt. What are the costs and returns?

Now of course we live in a world that moves fast - we don't have infinite time to analyze, make business cases, and pontificate. But for a question as crucial as rewrite or refactor, we should try to slow down a little and make sure we're at least answering the real question, and not an easier substitute one.

Throw-Away Culture

This last force that can push us toward a rewrite is not rooted within us, but rather in the society around us. We live in a throw-away culture. Our food and beverages come in one-time use containers. The products we buy are garbed in excessive plastic and paper packaging. And our appliances and devices are designed for obsolescence, so they can't be repaired even if we wanted to. In other words, we've been conditioned to throw things away without a second thought - whether its dented, cracked, malfunctioning, or just slightly old or worn, we toss it out.

And so, I think it's at least possible that this same "throw-away first" attitude affects the way we think about our software as well. When a system ages, our inclination is not to pull out the duct tape and WD40 to give it a few more years of life, it's to throw it to the curb and go out and get (or in this case build!) the newer model. In fact, it seems that the very word "legacy" can induce groans of sympathy from our colleagues. "I'm sorry you have to work on that", they say, "when are you going to rewrite?"

Now to be fair, our industry is one of constant improvement and innovation. We are continually learning better and cleaner ways of doing things and then incorporating them into our tools, frameworks, and languages. No one would ever purposely use Java 7 when they could use streams and lambdas from Java 8, or even a more concise or functional language like Kotlin or Scala. Additionally, new platforms and devices are constantly being released, which our existing systems may not port to. So I'm not saying that scrapping the old for the new is necessarily a frivolous choice, as we'll see in the next chapter.

I do think it's helpful though, like with the other forces, to be aware of this potential bias. In the same way that there are very real costs to our society for our thoughtless waste, there can be costs to the organization when we too quickly abandon a legacy system that may be reliable, albeit a little old or complex. In the following chapters, I'd like to argue for a different three "R"s of legacy modernization: "repair", "reuse", and "recycle". Instead of seeing our existing systems as things we need to distance ourselves from, I think we should appreciate and leverage the components and elements that are working, and only seek to rewrite that which we need to.

Wrapping Up

At this point, we've seen the many dangers and risks that await us on our journey to rewrite, and we've seen the subtle forces that nudge us in that direction despite it all. In the next chapter we'll look at when a full-scale rewrite does make sense - the reasons, constraints, and drivers that justifiably push us to abandon our existing systems and start afresh. If done for the right reasons, a rewrite can be a real opportunity to improve things we care about - user experience, maintainability, performance, and the ability to add new features and address new use cases.