Why Do Software Rewrites ALWAYS Take Longer Than Expected?

Transcript is auto-generated and may contain errors.

Hey folks, we're going to go to Experience Dev subreddit for a topic today. This one is about rewrites. Um, it's not I don't know if it's even a question that they have, but I figured it'd be a interesting topic at least. And uh you know the framing is maybe not a surprise for most people, but the person said, "Hey, notice that like I think they said after like 8 years of being in the industry that um every single rewrite that they've seen happen for for software systems always takes like significantly longer than teams plan for." And for some reason, like people always act surprised. And um again, I don't know if there was actually a question or if it was just a topic to bring up, but I thought it would be good to talk through cuz uh uh yeah, spoiler alert, that's also been my experience.

So, friendly reminder for folks, if you want questions answered, leave them below in the comments. Software engineering career related. Otherwise, go to codemute.com and you can submit stuff anonymously if you don't want to leave a comment because it's public. Okay. So, um yeah, I think from my experience this is very much the same where um for whatever motivation comes up for a rewrite of you know an application, a service, whatever it is, the um the amount of effort to do so is always like greatly underestimated. Um, and it's like I don't even know like proportionally or like with I don't have data to back it up I guess to like give you numbers, but uh like I don't know sometimes like at least twice as long. Like I would I would almost say with a high degree of confidence that if someone said, "Hey, it's going to take us x amount of time to rewrite this." I would at least double it.

If they said it's going to take a year, I would say it's going to take you two years. like just that's it like at least um and that's just based on my own lived experience with various rewrites. And so I I think there's a handful of different things to talk about here because um you know I I don't like making videos or talking about things uh to make it sound like there's only one way or this is always how it is or it's never this way, that kind of stuff. I like making sure we can look at different angles even though of course I have my own bias, my own perspective on things. I think it's important we challenge that kind of stuff when we're exploring these topics. So my goal is not to, you know, in this video to say like, hey, a rewrite is the devil.

It's bad. Never do it. Never consider it. Um, it's just that I think how a lot of the times I've seen rewrites approached, it's like this is going to be a silver bullet solution for all the problems we have and again grossly underestimating um the amount of time and complexity to go into it. Okay. So, maybe to start like we can we can kind of think through what some of the different motivators are. Okay. And I personally have not been through rewrites where uh it's not totally true. I was going to say where it's like an external factor or external to the the engineering team. So at least for me it has been um more common where the engineering team is the one to say like we want to do the rewrite we are going to rewrite versus um an external factor to the team where it's like a the business is saying like you need to rewrite.

So the I feel like one of the reasons we we hit this scenario a lot where engineering teams say it is um and I I read maybe a comment or two in this thread that I um like that seemed to resonate and someone was kind of getting this point across where when we have these like really large systems or complex systems uh it's not it's not the case that like one person knows uh all of the things going on, right? So as systems grow and scale and uh they become older, people on teams and stuff change, the kind of information about the system is not number one, it's not like one person knows it and then by that definition uh there it means that everyone on the team doesn't know all of the things either.

And so when that knowledge the depth of knowledge of the system becomes uh so dispersed I suppose or so I don't know what the right word is right now um in inconsistent there's gaps like you know there's people you have some people on the team that know some parts and no one on the team knows everything and certainly everyone doesn't know everything when you have that kind of disperate knowledge Um, it seems like it's easier to get behind the idea of like, well, if we if we just start from something we all agree on, then certainly certainly we can make this better. We know there's a problem here. We know there's this big god class that is, you know, super legacy and brittle and no one seems to be able to touch it. Um, you know, like we we can solve this. like we can we can be the ones that make this better and then we'll also be fixing this problem of like this legacy thing that no one really understands.

And the it's interesting cuz like I think the you know the the motivation is fair. I like it's hard I feel like it's kind of hard to argue with that on the surface. Like it seems um logical I suppose, right? And given u the severity or complexity of some of the issues like uh structurally in a codebase, I could certainly see why people would be motivated to do that. I have literally been parts of conversations where like I certainly feel motivated to be like, man, if we just like didn't have this thing here, we could do this so much better, right? Like we're paying this crazy tax to have this in our codebase. And so like I I can rationalize that. Um, but I think what ends up happening is that if that is a a motivating factor where um we need to be able to factor in the business part of it and I'm not saying that always goes missing.

I just think that there's a that's where one of the really big discrepancies comes in. Okay. So to be to to say like we want to go rewrite something. Um the way that I've mostly seen this done and why I feel like it often falls apart is that rewrite means absolute and it means everything. And if you're in a situation where you're rewriting everything, um, that often means that you need to start from scratch with something else, right? So you have this green field from an engineering perspective to go kind of do do all the right things, right? You can get it right this time. And um when you have a green field from the engineering side, what's kind of I think uh the big danger or one of the big dangers from the business side is when you have systems or products that have

been around for a long time, there are a lot of I mean I can't say this ex for everything, but I feel like there's a lot of features and functionality that exist in the product. And if you have lots of users that are paying your bills, if what you're making is not identical to them, right? Like if it is different in any way, and spoiler alert, it's going to be different there. There's a there's a tax to that. There's an overhead to that. Um, I think when you have systems or products that are uh that are old and you're you're starting it from scratch, there's going to be, you know, there might be hidden menus that you never thought about and there's like some users that absolutely use those menus or buttons or workflows or whatever. Um, and there's all this knowledge and history that went into getting those flows together.

And they might not be the best thing. They might not be perfect, but there are people that use them. Okay? So, when you don't have complete feature parity, and I'm not saying that you have to, but when you don't, the expectation of how to use your software changes. Okay? And I think that what happens a lot of the time is that there's so much focus from the engineering perspective having this green field to go make things perfect that that the par is like it's completely dropped. And again I don't mean to say that you must have 100% par. I mean the focus on that discussion is is just not being emphasized enough. Right? If if product is involved and they're saying like we need to have 100% par like does product even know all of the things that they need to have parity on like

probably not cuz I don't think like when you have these types of systems I think the same thing applies to people in product they don't know all the things they don't know that there's a pocket of users that use the software some certain way just the same way that all the engineers don't know every single branch you know in the in the logic in the codebase. So we end up having these two clean slates like not only from a code and architecture perspective but from like a feature parody perspective. And so I think that when engineers are are talking about this kind of stuff, they they focus on how are we going to make this system like we're going to solve all the problems technically from the architecture perspective, the infrastructure perspective. So much attention goes there.

there's not enough conversation around par or what needs to happen in terms of usability or if you're going to have completely different feature sets uh or workflows because you know maybe some of the workflows that are in the product are are also right like people use them but like they use them a certain way and you can't change them because of all this architecture like hey we're going to make this better going forward like if if you have to do that do you have a migration strategy to communicate to users all this stuff. Um, and I feel like that's a huge miss uh almost all the time. Beyond that, uh, we're dividing resources, right? So, if you have a legacy system that's running that users are paying for and you start to rewrite, you have to split resources somehow. Okay? So, you could take everyone that is supporting this legacy product that people are literally paying for.

you could move them all to the new thing. Okay? If you want to get that new thing done as fast as possible, you move everyone over to it. Now, while that's happening, you have literally paying users that may have issues that may be hoping for more feature updates because you've been on some release cadence that they've come to know and love, um, and that's not getting attention. So then you might say okay well we'll keep some people back right we'll keep some capacity back okay well now now you're you have this uh this overhead of managing two groups to focus on different different road maps different sets of priority and you're you're not prioritizing the rewrite as much so it's going to take longer um you could and then like so where do you where do you draw the balance right I think the the point is that uh there's a sliding scale here there's a a spectrum of of how you split resources.

That means people, that means time, right? So, I don't like I'm not saying there's one right answer to this. I'm just saying that this is one huge uh factor in terms of complexity and how this stuff's all going to pan out. And I think that also gets grossly uh underestimated for just how much overhead that is, right? And a lot of the times like I feel like just based on interest and focus like engineers often will not be uh like hyperfocused on on that aspect. you know, product managers might be because they're saying, "Okay, well, and depending on the stance they take with the rewrite, if they're highly motivated for it, like they are going to have to factor in, you know, not not shipping functionality or how they're going to balance that, but you know, it it's a ton of overhead on that front.

the the kind of the the scariest trap that I see with this kind of stuff is that like say you go you go all in on a rewrite and like I said most of the time when I'm hearing people talk about rewrites it's literally doing it all from scratch. And so when that happens um you know there there's going to be excitement in the beginning because everything's new. there's going to be like these phases that people go through. Um it might take you know there might be tons of momentum right in the beginning just because like things are being stood up and then there'll be a bit of a lull and then you'll hit this uh trajectory where like you know because things are coming together there's now people focused on the same things together you'll start to get more and more momentum so things move fast um which is great but the I think what starts to happen is that as the time is elapsing.

You you realize like, hey, I know we said we're going to build the architecture perfect this time, but as the complexity is being added in with the feature sets and more things are coming together, oh, like, oh, like we we didn't factor this part in. So, like our architecture has to change a little bit, right? Got we got to adjust a little bit for it. And this keeps happening. And it keeps happening, right? So now a little bit more unexpected, a little bit more unexpected. Now your pristine architecture is starting to get some, you know, some cracks in it, but that's okay. Like you know, you might have tried to accommodate this kind of thing in the design up front. There's some flexibility here and there. These are just a couple of exceptions, but hey, like we're starting to have a couple of cracks in it.

So that's slowing things down a little bit on the engineering side. And then what will happen is that you know as maybe people in product are looking at what's being produced and they're going oh wait like we don't we didn't account for this feature or like there's a gap here in terms of functionality like we didn't get that perfect like what do we like we need to address this. So, so now, okay, there's one more thing to add back in and another and another and this continues to move the timeline out. And I would say that like I don't know, like any rewrite I've been in, I feel like there's always this point or multiple points where people are questioning, should we keep going?

And I I just I'm saying this out loud because I feel like the I feel like this is the the biggest sort of scenario to be in is like we've we tried to make this conscious decision to go all in on something and now we're like, well, what the hell are we going to do here because this isn't panning out to be, you know, perfect sunshine and rainbows and like no It's software development. It's never going to be perfect sunshine and rainbows. Like to this person's point in the Reddit post, why are we always surprised when you try to do a full rewrite of something that's taken years to build, right? Like software systems that have taken 3, 5, 10 years to build. We're just going to we're going to build it from scratch. Like, are you me that you think it's going to be perfect?

that you that your estimate we can't even estimate feature deliveries in like the order of days or weeks or story points like and you're gonna you're gonna rewrite this thing from scratch and you got a timeline that you feel confident in like It's just it's it's completely crazy to think that we're going to get that right. Doesn't mean that it can't be successful. It's not what I'm saying. But it's crazy to think that we're going to get it right or that there's not going to be surprises. There's going to be surprises the entire time. You're building software. Um, we can minimize those things. We can, you know, we can uh we can pivot. We can adapt. Like there's I'm not saying it's impossible. I'm just saying like to to not be surprised is like to me that's crazy dog. So, how are we doing for time here?

This is It says another 35 minutes to work. This is I got some real shitty drives this year so far. Um I'm going to miss my meeting which is uh ridiculous. Starts in 15 minutes. I'm going to miss literally the whole thing. So um I want to spend some time talking about like alternatives, right? And it doesn't mean that we can't talk about things being rewritten. doesn't mean that you can't have that conversation or there aren't situations where a full rewrite makes sense. Um I think that one of the ways to to approach this is like is thinking about software systems as like um especially when you have paying users. If you don't have any paying users like doesn't do whatever you want man do what you want. uh when you have paying users, your software is kind of like a living system. It's like a living being.

And so what we really don't want to do at any point in time because the software, assuming the software is the thing that's paying the bills for your company, um, what we don't want to do at any time is kind of like cut off the life support to it. Because I feel like as soon as you do that, now we're in this weird spot, especially when you have been neglecting it. You've cut off the life support. This thing's dying. And all of your eggs are in this other basket. And you're going, "Uh-oh, like this thing's delayed. Uh-oh. This is way more complex. Uh-oh. Should we pull the rip cord?" Like, if you don't give up the life support for the existing product, then I feel like you can position yourself to approach things in uh safer ways, right?

What I my my sort of strategy or perspective on this is like when we do largecale efforts like rewrites, the the bigger the scope, the more surprises we should expect, the less uh accurate we should expect that we're going to be. Like with anything where we're trying to predict over a large time or scope, there's just more uncertainty that that gets dropped into it. So if we want to derisk these things, if the goal is to be able to be successful and we want to derisk and still make forward progress, um Martin Fowler and some people will know who this person is. Um he had coined this term called the the strangler fig and he has a blog article and stuff on it. This isn't a new thing, by the way. I'm not I'm not trying to talk about this like it's some new science or like I invented this.

No, like Martin Fowler, I don't know how long ago, a long time ago now, um coined this term and he has a blog about it where he talks about where the the naming comes from cuz it talks about uh some types of trees that he saw kind of uh slowly taking over, strangling out the other uh vegetation around them. And the idea here when it comes to rewrites is that like instead of taking your entire product or your entire service, whatever it is, and going we're starting from zero, we're going to go build the new thing and like I said, like cut off life support for this other product. Um, the idea is that you do it in pieces. You find ways to segment how you can start rewriting things. You start building up the new pieces and then you slowly or depending on the rate of your development, you start taking code paths and pointing them to the new parts.

So you start building up new things the new way and you start pointing the old paths to these new things and you start replacing parts of your system this way. Right. So the the idea being that it instead of it being a this is how my interpretation instead of it being a full-on rewrite of the entire product where you do an all or nothing thing you can still go on to approach trying to fully rewrite something but you do it in a way where you're like incrementally rewriting chunks of the software. If you have um you know whether it's microservices or you have a a modular monolith or whatever like that might mean that you say okay well um let's take a modular monolith for as an example just take something where there's uh different components in the system you can you can sort of

uh imagine that however you'd like I'm envisioning something like a modular monolith where we just have a you know big big service or big application and there are different types of modules for different types of functionality in this system. And there's probably patterns that are common across many of these, right? Because they fit into this big architecture. Maybe, you know, some of them have different commonalities versus others, whatever. There's going to be some common parts. And there's also, of course, when you have these legacy systems, there's going to be parts that are just like that are not common, and they're sort of the noisy bad part. So, we got these things going on. I would say that like if you're like, "Okay, well, we want to rewrite. We want to get rid of these bad patterns." Like, okay, if you if you were to start rewriting it, like what is one of the first things like the first areas that you'd want to go address?

And then essentially, can you can you rewrite a piece of it? Can you demonstrate that like this is the pattern we want to use is the architectural approach. We're going to go address this module or these sets of modules with this new pattern with this new architecture with this new approach and then you effectively have like almost like a shim layer that you introduce in front of it. So you can you can start having code paths call into the new stuff and then over time you kill off the old code paths. Now how this is done in practice will you know vary tremendously depending on what components we're talking about in a system right for example if you have a set of microservices that all work together to to do something maybe a single micros service is the um the granularity of rewrite that you can go address.

So you stand up a new process or new service or whatever like the the new thing. You start standing that up and then you start sending some of the traffic to it proportionally, right? Get it tested, get it working. Okay, cool. Once you're confident, you cut over all the traffic to the new thing and you cut out the old one, right? You didn't you didn't rearchitect and rewrite your entire system from scratch. You picked a micros service. In that that example, the cut over is like traffic that you're sending. Okay? And you might, like I said, you might have to introduce some type of shim layer depending on um I don't know like the how you're communicating between your your microservices. um if it's not microservices and it's uh modules in an application, it's likely not going to be, you know, HTTP calls or some other type of um I don't know, messaging like that that's over the wire.

Maybe it is, but say it's not. Okay, it's function calls. Okay, so like same concept, it's just a different, you know, transport that you're using. So how can you start rewriting a module, cut over things to it? So, why I like this? I mean, there's many reasons, but I I have I'll pitch you on a course now. Um, so I do I have a course that's on dome train that talks about refactoring and rewriting. It's about refactoring in general, but I have a a bit of a section on rewriting. And as I'm talking through this, you might say, "Well, what's the difference between what you're talking about versus refactoring?" People get really pedantic about the the word refactor. Like it's not supposed to change any observable behavior at all. You should be able to change things in a way that has no effect on behavior. And I'm like in practice like if you want to say that then I think that you know 99% of the refactoring work is then by definition not refactoring.

So instead of trying to be super pedantic about it, I just like to to talk about um you know refactoring being closer to like that is a goal of refactoring is to not change uh behavior, but like if there's minimal behavior changes, I'm I'm still like not, you know, excluding that from the refactoring bucket. Anyway, um the strangler fig approach to me starts to to move things from a uh the end of the spectrum that's a full rewrite like where it's all at once, you know, all or nothing to the moves it towards the other end of the spectrum which is like let me refactor something and it changes no behavior, no observed behavior, right? That's at the complete opposite end.

And so like you have this spectrum to operate in now where you can effectively incrementally start changing parts of the system and maybe not have observed behavior changes or maybe because of how you're rewriting the behavior change that you're expecting is a lot more easily observed and gives you feedback that you're moving in the right direction before you try to invest years into rewriting the whole damn thing. Okay. So like it affords you this opportunity to do things incrementally and uh I think at least from my lived experiences in software engineering that um when we have opportunities to do things incrementally. Yes, there is often an overhead associated with doing that, right? because you might have to plan for things a certain way because there's intermediate extra steps that you have to take. So there there may be some extra overhead versus you know going allin but I often find that that extra overhead for doing things incrementally affords you flexibility.

Like with anything, if we can minimize the amount of overhead and still maximize the flexibility we get from it, I think that's still aligned with, you know, my my uh best interest in in software development. I don't uh I don't like you know especially from a business perspective I don't like putting so much risk on something especially if we can mitigate it by doing it incrementally. Um, another like totally, you know, um, tangential example to this. It's a t it's a tangent uh, example, but I think it parallels the concept. People might people might not love this, but like waterfall versus agile, right? Like ah, and okay, well, what what even is agile based on how people do it? Um the the idea just being like having um like tighter feedback loops. All right. It doesn't mean no planning. It doesn't mean like waterfall doesn't work.

Like I'm not saying that. But uh in that comparison, you know, the built into the the approach for agile at least is to have these feedback loops to make sure that we can we can keep a steering in the right direction. And I I really like that about something like the strangler fig approach or approaching a rewrite incrementally is that we have more opportunity for feedback. Yes, there could be more overhead as a result of that, but instead of going so offc course or completely misunderstanding our our estimates of things, we do it in smaller chunks, we get feedback, we adjust. Okay. So, um sort of as maybe like a a meta point or like a recommendation if I may is that um you know trust that when people are talking about rewrites and estimating the the scope of them and the amount of effort and the timelines, trust that that is going to be way off.

It's going to be completely underestimated. I have never experienced something being remotely close to on track. I've never heard of from other experiences from talking to people. I've never heard of a rewrite being remotely close to on track. Okay. So, assume assume whatever you say with as much confidence as you have, double it. Just double it. I it's it's going to be off if you're doing a full rewrite like I was kind of describing earlier. So, um what I would encourage you to do though is figure out like if you have this idea for a rewrite, there's this alignment that we should be moving in this direction. Please do find ways to step back from an all or nothing approach and see how you can incrementally do it. Okay, I'm going to give you a couple of examples from from my professional experience. I'm not saying that I was responsible in these situations or um trying to over or underell my involvement.

Um but uh I'll give you a couple from a service perspective. Oh boy. Let me in here. Nice. Okay. Um, one is going to be for a desktop product and so we had this uh database format we were using um that uh we needed the database to be portable. Okay. Requirements for the software we used a SQLite database. SQLite is a very very capable database. Um we had this schema in our database that was really starting to be limiting for us. Uh the amount of data that we were putting into the database had like grown orders of magnitude. And again, it's not that SQLite's not capable of doing it, but our schema just made it so that um working with that data was an absolute nightmare for some of the features in the direction we were going. So, um we we did I think a couple things right and a lot of things wrong, which is okay.

But one of the things we wanted to do was basically almost like this strangler fig approach but for our data layer. And so that way we could start reading in um you know this this new uh this new format. Now I think where we went wrong was that we were also just rewriting the product that particular product from scratch and like so I I think what would have been really helpful was instead to incrementally move parts of that product over to like the new new design. Um, so we started doing this data layer where we could share things and then it kind of just like didn't and then we were all in on this thing and so we we literally lived this problem of like well you know users actually do use this other thing this particular way and like we didn't think about that um and now we got to we got to try to find a way to wedge that in.

It was it was a nightmare. It be it was successful in the end, but holy Uh I would do that completely different if I had to do it again. I would absolutely try to build out modules and and move things over to the new pattern over time. Um, again, it doesn't mean you even have to uh to ship those or turn them on to users, but being able to build it out that way, I think is handy. Um another example is uh and I'll talk about uh maybe without the details. We had a service where um there's a lot of legacy for this service and basically uh making code changes in there was quite difficult. Um arguably like a little bit archaic. the performance of that uh service like basically was being like squeezed to the max and like there wasn't really a lot more to get out of it.

So um it's not that conceptually the um the responsibility of that service there was no more performance to gain but how it was built they're you know they were at the limits of it especially given the constraints. So new service was introduced and new service introduced to address performance like primarily it's like it is being built because we have performance constraints as a performance critical system. Let's start building it up from scratch. And so what was done was that as this new service is being built up incrementally scenarios were brought over to that new service. it would get built and then traffic would be uh directed to that. So method calls, traffic, however you want to interpret that would be moved over to this new service. Um it allows for for testing incrementally. It allows for for making sure that the patterns are the right thing.

Cool. And then like okay, let's do the next scenario and the next one and the next one. And so over time, all of the, you know, sort of legacy code paths, they go from being 100% called, come on, buddy. Let me in over here. Um, they go from being 100% called, you know, down to zero over time. and just keep moving more and more scenarios over to the new service until it gets to the point where you can turn off the old one. Now, is there overhead to developing this way? And the answer is yes. Yes, there is overhead to doing it. But like is it does that afford you flexibility to go to do things without having to take all of your resources off without having to drop life support for something that's literally you know uh the reason users pay or the reason your entire platform is used the reason your entire product or service is used.

So those were two and like I mean the the second one I just described was a full a full rewrite but I feel like done incrementally in a in an intelligent way. The first one I explained that was a desktop application. Um I think there were elements of it that had the right intention but in um in practice it was a full rewrite from scratch and uh it was an all or nothing kind of approach and it in both cases these are both successful rewrites uh measured by the uh the end result but oh man um I would say and I would actually say both of these are were you know even though one was done incrementally probably uh it was probably underestimated how long it would take. The first one I described is certainly uh underestimated for how long it would take. So yeah, just my my reminder to you is like, you know, think about finding ways to do these things incrementally.

And um you might find that maybe you get far enough along and um maybe parts that you want to rewrite, you're like, wait, no one uses this, maybe you deprecate them instead of rewriting them, right? Like you you will learn things as you go, but incrementally is uh the way that I recommend if you can. So, um, I'm almost at the office. Friendly reminder, if you want questions answered, leave them below in the comments. I'll try my best. You can go to go to code commute.com. Submit stuff anonymously that way. Um, I got other channels and other content and stuff. So, if you made it this far, uh, my main channel is called Dev Leader. I have C tutorials. I have tutorials where I'm walking through AI tool usage. You can see how I'm developing things. Um my approach is uh especially with the AI tooling is to to never try to make you feel like oh just here's this one simple trick and it will you know we'll rewrite your codebase.

Uh no it's it's not meant to be like clickbait do you know this one AI tool solves everything but I'm trying out AI tools. I'm trying to show you how I'm how I'm leveraging them and then that way um take what you want from that. Right? The AI tools are changing so much that um I just think it's important that myself and for for you watching this that we're trying them out. We're learning from them. We're seeing how things go. So that's on my main Dev Leader YouTube channel. I have a Dev Leader podcast which is where I interview other software engineers and I do a live stream every Monday at 700 p.m. Pacific. So most of the topics are from code commute which you're watching right now. And that way uh if you want you can join in at the live stream and uh and talk about these things with me because I'd love to have you there.

So thank you so much for watching. I just parked. See you next time.