Indiana Jones and the Temple of Legacy Code

00:11 William Brander

Picture this. You're standing in a cave, it's dark, the only light source you've got with you is a burning torch in your hand, but somehow with this light source, you can see all the way to the end of the cave. It's about 25 meters long, and at the end is a pedestal with a golden artifact at the end. Whoever your lighting engineer was on set must've been amazing because this little torch isn't going to make that much light in the room.

It's quiet. You can just barely hear your own breathing. Every now and then you get little flicker of flames on the torch as you make your way down the corridor. The air is moist, it's practically damp, and every step you take, you can feel the moss under your feet. You get to the end of the cave, right up to the pedestal, and as you're about to reach for it, you put your foot down and you feel a click and you think, I did so well avoiding all of the other traps. How could I mess it up now? And as the arrows come storming into you, the last thought that crosses your mind is what kind of absolute idiot puts an artifact at the end of a cave with Rube Goldberg-esque defenses instead of just putting it somewhere safe? And also, how is it not dusty at this point?

That's a little bit like dealing with legacy code. If you ever have to read somebody else's code, the immediate reaction is almost always, what the are they doing? It's not usually a pleasant experience, especially when you try and find out who wrote the code and you find out that it was you a few months ago. This is a comic by Abstruse Goose. I absolutely love their stuff. Unfortunately, they've taken the website down, but all of their comics are absolute gold, and you can find it on archive websites.

When you review code, especially legacy code, sometimes the only metric that you've got to tell whether or not code is good or bad is the what fucks per minute ratio. A low number of WTFs means that the code is generally fine. Where a high number of WTFs, maybe I wrote it, I don't know. When it comes to reading other people's code, it's not great a lot of the time. It's even worse when you're dealing with legacy systems. I assume the fact that we're all here, we've had to deal with or are currently dealing with legacy systems at the moment. My condolences, we can also turn this into a therapy session if you'd like. That would work too.

Before we get into more legacy stuff, I want to take a step back and discuss Indiana Jones. So I love LucasArts. I grew up in the golden era of computer games. All of the characters on this screen, every single one, has had a deep meaningful impact on my life. Specifically Guybrush Threepwood. I love Monkey Island so much I've got a tattooed on my leg. I've got two pairs of Monkey Island shoes, this is my second one. I've got custom shoes for a lot of LucasArts characters.

I love LucasArts characters. And LucasArts was my first introduction to Indiana Jones. This is a game called Indiana Jones and the last Crusade, the action game, not to be confused with Indiana Jones, the last Crusade, not the action game. This was released in 1989, and I remember the day that I first played this game. We went to my dad's friend's house here. I grew up on the south coast of South Africa in a small town called East London. If you're London native, you can picture your East London and just transpose it down there. We've got the same sensibilities, the same terrible life choices and questionable fashion.

And we went to my dad's friend's house and he had a farm and it was a massive property. And I remember he had an entire section of his house dedicated to computers. And for me as a young child, this was amazing. This was mind-blowing. Who can have this much space dedicated to computers? He obviously didn't let me mess about with all of them, but he sat me down in front of one computer, took out a floppy disk, put it in, turned the computer on and pushed the turbo button and said, here, play this. And I did. And it was this game. And I remember thinking that my entire life up until that point was leading to this one moment, this moment in time when I played this game because up until then I didn't realize that computer games could be so absolutely shit.

This was horrible. This was not a good game. This was clunky. The controls, you just pushed the button, you suddenly launch yourself across the screen. This intro part of the game is so difficult that I'm pretty sure it's the inspiration for all the Dark Souls games. You have to go around collecting torches because apparently Indiana Jones is deathly allergic to the dark. I don't know, it was not a pleasant game. So much so that as we were leaving my dad's friend's house, he thought he was being very generous and said to me, would you like this? And he gave me the original disc. And even back in those days, I was a very staunch anti-pirate. I wouldn't play games if I didn't have the original game while my friends were handing out floppy disks and swapping games, I made sure that I only played games where I had the original disc and it was hard to get games in East London.

So when this gentleman offered me this disc, the fact that I looked up and said, no, and got in the car and left, says a lot about the quality of this game. It was not great. And unfortunately it soured a lot of Indiana Jones for me. So when I say I love every character in this image, what I mean is they've all had an impact on my life, including this asshole. And yes, I know this is not the same Sprite used in the actual game. You can actually tell this one is Indiana Jones. This one looks like Robert Baden-Powell has decided to go spelunking in a cave for some reason. I don't know how this is Indiana Jones at all, besides the hat. Up until this point, or not up until this point, because of this, I wasn't interested in watching the Indiana Jones movies when they came up.

The character wasn't exciting to me. But a couple years ago I went to my friend's house and his dad was there and the two of them had bonded over the Indiana Jones movies as the child was growing up and they were discussing them and having a great debate about something to do with the time travel and Pythagoras, I don't know, whatever. So I, of course, interjected myself because I had an opinion about everything, and they realized that I had never watched any Indiana Jones movies and they were decidedly upset by this and they set about remedying this. So for the next hour and 48 minutes, we didn't watch Indiana Jones. We debated what the order to watch it should be, but after we eventually settled on it, we decided we were going to watch Indiana Jones and the Temple of Doom. And I was kind of excited about this because a lot of people like the Indiana Jones franchise, right, so I must be missing something.

And we've got this lead character who's adventurous, he does these amazing things, he isn't just a guy in a suit, he's an action adventure person, and he's smart, he's a professor, he's Dr. Jones and he's ridiculously attractive. Good old Dr. Jones, oh sorry, wrong Dr. Jones, this Dr. Jones. So I watched this movie and honestly it sucked pretty bad. Between the ridiculously ludicrous cold open that is no point in the entire movie besides setting up Indiana Jones as a kidnapper and the plot holes and not to mention the problematic scenes. Has anybody watched this movie? We remember the dinner scene, right? Wow, that has just thinly veiled racism right there. Look at us, we don't eat eyeball soup or monkey brains or beetles. We're definitely better than all of these people. This movie was not good. I watched the other movies with them afterwards and some of them were nice, they were actually enjoyable. Unironically Raiders of the Lost Ark is a pretty good movie, but Temple of Doom was shit.

And here's the thing with Temple of Doom though, they didn't set out to make a terrible movie. They had a script, they had budget constraints, they had resources and they had a dream, an ambition, a goal, and they tried their best to meet that. And I'm sure that if I had watched this movie when it came out, I would've felt a lot differently about it than watching it now. Very much like legacy systems. Nobody sets out to write a terrible system. It just happens over time. There's nothing you can do to stop it. It's like the law of entropy. But when you look back at the system after 10, 15, 20 years, that system is going to feel dated. It's going to feel awkward to work with and you're going to start calling it a legacy system even though it was a good system when it came out. The same with the movie.

So when we talk about legacy, we often throw around descriptors like it's convoluted, it's brittle, it's a ball of mud, we don't know what's going on, but these descriptors don't really help us. I mean everybody knows when they're working in a legacy system, right? There's a feeling to it. It's like holding a fitted sheet. Everyone's got their own way of doing it and generally it works, but when you fuck it up, you've messed it up. And legacy systems have that feel, you know when you're working in the legacy system. But the descriptors don't necessarily help us get better at working in legacy systems.

There are attributes that can help, and there's four specific attributes that are important and we're going to cover three today. The first that we're going to cover is the understanding that you've got of the system. Then you've got the supportability of the system and the life cycle that the system is currently in. So understanding is a measure of how much knowledge you've got about the various moving parts in the legacy system. Oh, I need to change this bit of code over here, that means it's probably going to have an impact over here. I know that because I have a high degree of understanding of the system.

In Indiana Jones and the last crusade, this was the golden chalice, whatever it's called, the cup, holy grail, this was the holy grail, that's the one. And the big, bad, evil guy didn't realize this was the holy grail because it looks like it's made for, I think it was shepherds. So he drank from this one because it looks like it's made for kings. He didn't have the understanding of what the holy grail actually was.

The next attribute is supportability, and this is what enables us to make changes to the system with confidence. So it's all the guardrails that you've got around yourself, being able to know that, oh, I've made this change, I broke something. Or I made this change and I've realized that I've broken something, how do I go back to a previous state? Supportability is all the tooling around the legacy system that gives you the confidence to make changes to it. And then the life cycle of the system is are you actively working on the system or are you just fixing it for critical bugs? Are you generally in maintenance but maybe just adding a few features every now and then? The life cycle of the legacy system has an impact on the techniques we use when we work with a legacy system and if we take the first two, so if we take the understanding and the supportability of a legacy system, we can create a general rough plot of this.

So if we have the understanding of the system on the x-axis, the supportability and the y-axis, we can take the system that we are working on and put it somewhere in one of these quadrants to get a feel for what we're doing. So if you've got low understanding and low supportability of the system, this is not a good place to be, because every change you do is probably fraught with a lot of, oh crap, we broke this and we need to now go and fix that bug. Or okay, we've made this change but it works on my machine but it doesn't work on John's machine. How do we ... There's a lot of brittleness in this quadrant. If we increase the supportability but we still don't have understanding of the system, yes, you can ship it and you've got a bit more confidence in being able to ship it, but you still might make changes to the system that impact other things, then you might have to regress that and undo those releases.

High understanding low supportability means you're quite confident to make the changes in the system, but the system sort of fights you along the way. You have to do a lot of work to get the release out. Once you've got the release out, you might realize you've made a regression somewhere in the previous behavior. And then of course in the top right quadrant it's still legacy, but it feels like the changes that you're able to make here are meaningful. They have impact in improving the system that you're working with. It's probably still going to feel legacy for a very long time and trauma has a way of making you feel that it's legacy even after somebody else wouldn't see the system as legacy, but this is where you're actively looking to be because you can make improvements to the system.

So let's take a look at how we can improve the supportability of our systems. Now, when you get given a legacy system, you often don't get it in a state where it's compile able, even. You could just get given a bunch of source code files and say, here you go do something with this. The first thing that we need to do is we need to improve how that works. We need to be able to have reproducible builds. So step one, write down how to build the project, how to build the solution. It sounds obvious, but if you've got a system and it takes you a day to work out how to build the thing, write it down, put it down, say here are the dependencies, once you've got these dependencies installed, this is then the next step, this is how you compile it. That's already a vast improvement on just being given source code files that you have to work with.

Once that's in place, you can start looking at locking down specific dependency versions. This is less of a problem with Java systems or .NET systems because the version ranges and dependencies only came recently, but if you're using a node system, for instance, lock down, use a package lock file, keep those versions specific so that the bill that you do is consistent. Once you've got that, stop relying on specific machine state. So if you've got one machine, that's the only one that can do the final release because that's the one that's got the advanced installer thing on and only that machine can compile the thing, step away from that. Ideally what you would end up with is something like dev containers or NixOS or something like that where you've got your entire environment set up and anybody can come, check out your repository, start up a dev container and start working straight away.

It doesn't always happen, but it's nice to shoot for these if you can. So once you've got the build working in a way that somebody can check it out and they can run it, the next step is have that build be repeatable, put it in a script somewhere, for instance. So have a build.ps1 or a build.sh. Have the system be able to compile once you've got the code that somebody can check it out and start working straight away, it means that the feedback cycle from when you make a change or when somebody else makes a change to realizing that the change has worked or is broken, something else is shortened. And at this point, the next logical step is of course, make sure that you've got this on CI. Once you've got it on CI, you've got a lot of safety in this space. A lot of legacy systems that you might work with, you might already be in this state, that's great, well done.

We've got a repeatable build and we know that when we check something in, the code is being compiled and we've got an asset that we can now take and deploy at the end of that. From here to improve further supportability, we need to move to tests. If you don't have tests in your system, the very first thing to do is just add a test. I honestly don't care what kind of test are you adding, whether it's a unit test, an integration test, an acceptance test, it doesn't matter. Just have something, have a testing framework there. A lot of systems, especially legacy ones, they don't lend themselves to testability. So spin up something like Playwright, which would actually launch the application, go through the system, interact with it as if they were a user and then use that as your tests. Just have something.

Once you've got some tests, you of course include them in your CI and at this point we can get a lot of traction on the test because now we can go and add as many tests as we want and we know that the tests will work and we can ask our closest friendly agent to go and add tests to our code. Our tools are actually pretty good at adding tests to code that you don't understand. The question then becomes what are those test testing and are they testing something meaningful or is it just nonsense tests that are green for the sake of being green? So from here, the next thing to do is to evaluate those tests and you can use something like mutation testing to do that.

So if you're in the .NET space, something like Stryker is a great way to do this. What mutation tests do is that they look at the code and then they look at the tests. So you might have a bit of code like this which returns true if a customer is a high value customer or false if they're not. Stryker would go and look at this code and say there's a bunch of branches here. And by branches I mean decision choices that the code is making. So if the customer's total purchases is more than 100, great. Next line if the total purchases is less than 10 and the total spend is more than 1000. So they might look at this and say, well, what if you changed this around instead of being less than 10, we said greater than 10. Would the tests still pass or would one of them fail?

Because if you change the branching logic in your code and the tests still pass, guess what, you don't have a test that's testing the right thing. And in fact, I was all gun-ho about this example until I ran Stryker on it and then actually found a missing case for the previous line, not the one that I actually wanted it to find anyway. So in this case, they changed the code to greater than equal to 100 instead of greater than, and I didn't have a test that covered that. So if you've got an AR tool generating a bunch of tests that you mutation testing is a nice way, or not nice, it's an easy way to go and validate whether those tests are testing meaningful things or not.

The next thing we can do to improve the supportability of the system is to introduce observability. Now this is the first part of the talk where we are actually going to make changes to the code that's running the legacy system. Observability is a great way to see what the system is doing, but it's also sometimes quite hard to add to an existing system. So the first way we add this is we start with the simplest possible flow that we can find. You add observability around this and at this point the intention is not to test that flow or to see how that flow works, because we've chosen the smallest one on purpose. The intention here is to get the observability pipeline working in your system. So you start with the easiest. Indiana Jones is of course a professor, so he teaches and teaching is probably the easiest thing he does because at least nobody's shooting at him, which is ironic given what happens in America in schools and guns.

But anyway, so once you've got observability in your system, then you start working to add observability to the flow that you're currently fixing. You're not going to be given a legacy system and told, hey, yes, six months, go and mess it out and see what you want. You're going to be asked to fix a bug or add some functionality or make some other impact in the system. So once you've got the smallest possible flow added, add the observability to the flow you're currently working on because once you've got observability, there are amazing tools that take, especially if you're using open telemetry, that take the output of that and will give you a visual flow of what the system is doing and you can see, oh, this is why we've got into this state of here, which means I need to fix this. And then the observability on the next run once you've changed that should look different. Observability is super powerful for building a knowledge of how the system is changing and having that feedback loop given earlier to you is fantastic for the supportability of a system.

So for supportability, take the time to spend the investment on having repeatable builds, tests, observability, and make sure that all of this is a cohesive package that runs on CI. Because once you've got that, you can have a lot more confidence that even if you don't understand the impact of the changes you're making in the system, that hopefully something will be able to catch it for you. And if you do break something and you didn't catch it, change the rest so that you do catch it in the future. By doing all of this, we take all of the low supportability systems and slowly start ratcheting them up so that got a higher degree of supportability for these systems and we can make changes faster and iterate on them.

Let's have a look at understanding of the system and this is the part where it gets interesting. Flash fun. So the first thing we're going to do is we're going to continue that observability frame. We're going to continue to add observability, because observability, while it helps, show you that things are changing in your system early, the fact that you add an observability helps you reason about the system as well. So continue to add additional observability steps specifically at important boundaries because when you've got a span and your span does its thing and you hand off to the next span, if you include that causal link between the two, you've immediately made the decision that, okay, this boundary ends here, this boundary starts here. Now I can start exploring this boundary. It's a technique to explore how the parts of the system interact with each other. And at times you can shift those boundaries because as you develop more understanding of the system, maybe you realize, wait, this always precedes this, which means maybe we can bring that into this span. Observability is another great way to explore how a system works internally.

Let's have a look at git, though. So get is a fun one. Indiana Jones often finds themselves in situations like this where he's surrounded by ancient artifacts, an amazing treasure, and every time this happens, he yet again reminds us that he's actually the world's worst archeologist because he ignores all of the priceless treasure and goes straight for the MacGuffin, like a kid that's found chicken nuggets instead of vegetables or something. He's a terrible archeologist, he really is.

So if like Indiana Jones, we look at this and we see this is a giant pile of treasure, that's wonderful. As developers, we have repositories and just like the pile of treasure on the left-hand side, our pile of treasure is our git repository on the right-hand side. There's a lot of value in these git repositories that we can get out. So while the treasure, the pile of treasure is made up of individual little pieces of treasure that are valuable on their own, the git repository, well it's the same. Each commit is meaningful, the history of the repository changes as we go through all of the commits. The way that the system has changed is meaningful. It has a story to tell you that can tell you why the system is like the way it is.

Now, this is a screenshot of the commits for NServiceBus. I work for particular software, we make NServiceBus. This isn't necessarily interesting, but we are going to use NServiceBus a little bit for some explanations of the next few slides. Looking through the git history like this, I can't tell anything that's meaningful at this point. I can see we're doing a lot of rebasing, but yeah, cool. It doesn't tell me a real story. We can look deeper though. So our git repository is essentially a database and because it's a database we can query it, we can run git log on a repository and it'll go and give us all of the history for that repository, starting from the beginning all the way to the end. This is even less usable than the gitk screen where you can see the commits going like that. But it's a start.

What we can do is we can change our query to say, well give me all of the files that have changed, but just give me the names of the files. I don't really care about each commit message. And that gives us just the files. Now we starting to get somewhere because what we can do at this stage is we can group them so we can sort them by the file name, then count the commits per file name and then output that which would then give us a list of the most changed files in our repository over time. And maybe you've got a suspicion that if a file is changed a lot, maybe there's something meaningful about that file. Maybe there's a lot of bugs in that file. So by looking at this and we run this query again Nservice, we can see there's a bunch of changes to csproj files and solution files and acceptance test files. That's not particularly interesting because especially if you are using legacy systems, the csproj files used to change regularly.

So let's look at our C# file. And here's our first one that might be interesting, the unicast bus. It's changed 312 times over the lifetime of this repository. Maybe there's something there. So let's go look. We open up our IDE and we go to the unicast folder and we look for our unicast bus file, which doesn't exist. Well, that was a waste of time, but the reason it doesn't exist, well I know why it doesn't exist, but let's try and find out why it doesn't exist. So we can go back to our git repository and we can say, well give me git log -- the file name and it'll tell you everything that changed with that file name, or you can use gitk -- the file name, and it'll give you, here's all the changes for that specific file.

It doesn't matter what it says up there. The only part that's interesting is that this file was deleted in 2015 already. So looking for files that were changed the most is possibly not helpful. You're going to get a lot of false positives. Okay, new plan, what we'll do is we'll query within the timeframe of the last year. We want to find the most interesting file within the last year. So we run this against NServiceBus and we get a bunch of csproj files, some text files, and I think the only sort of interesting file is end point creator, which has only changed 12 times in the last year.

This isn't going to be a great demo. Unfortunately, well, fortunately for me, unfortunately for you, the audience with this specific talk, I know that particular spends a lot of time improving the code that we have in our repositories. We've got dedicated teams that take time out of delivering features and add in support to go and clean code up, which means that a lot of the things that are problematic are being continually refactored and improved and are particularly proud to say that it's in a completely backwards compatible manner. So if you've got an NServiceBus 10 system version 10, you can send a message and it'll get processed by a 9, 8, 7 all the way down to I think version three of NServiceBus. It's complete backward compatibility while improving the quality of the code. So it's going to make a terrible demo for this. I tried to find something else.

I went to GitHub and I was looking for specifically a repository that was .NET that was actively developed but had a long history. So NServiceBus has been around for 20 years. Is the git history of that. Still actively developed and I wanted people to know it, hopefully. So I found this. Does anybody know this? Yes, okay, we've got some. This is Jellyfin. This is fantastic. It combines all of your different media sources that you can use this as a media player and you can watch Indiana Jones and the Temple of Doom if you hate yourself for whatever reason. But the code is available on GitHub and it met all of my requirements. It's a large-ish code base. It's actively developed. It's got a long history so we can maybe find something interesting. Let's run the same query against Jellyfin and we get this. So now we're looking for all of the changes within the last year. We want the file that's changed the most and right at the top here we get base-item-repository.cs, which has been changed 81 times in the last year. That's almost twice a week. Maybe there's something interesting there.

Let's have a look. So if we look at the file, there's a bunch of commits to it. Does anything stand out to you with this? I can tell you what stands out to me. It's all of these. Oops, too many clicks. Too many clicks. It's all of these. Fix, fix, fix, avoid, fix, clean up, improve. There's a lot going on here. Now we got lucky that we found a file with so many fixes in it, because there's probably something in this file that we need to be very cautious and careful about. If there wasn't all of these commits, we might still want to look for files like this. So how can we use our git repository to find files that have got a lot of fixes in them? We can do that by changing our query. So when we query, we just add a filter that says I only want all of the commits that somewhere in the commit message have got bug fix, bug, hotfix, patch, regression. Maybe clean up would be something you would look for.

And if we ran this script, we'd get a different result. So now we are looking specifically for files that have got a lot of fixed commits in them, and there's our friend BaseItemRepository right at the top again with 46 bug fix commits over the past year. One a week. I sure hope they've got tests for this file now. I don't think they do actually. Haven't really checked. But at this point these are some strong signals that maybe it's time to look at this file and see what is happening here. Why is this file represented so often in recent history for bug fix commits? Why is this such a contentious file? Maybe there's something we need to do about this. Other techniques we can use to do this, by the way, there will be a written test for these queries at the end.

So if you don't pass a test, you have to give a green card. I don't make the rules. It's weird. No, I'm kidding. Just at the end that's got all of the scripts that I regularly use for these things. So this script over here will go and run through or this query will go and run through all of the commits in the repository and it'll take the additions and the removals of each file and give you a measure of how much of that file has been overwritten within the past year. If a file has been largely overwritten, again, I sure hope there's tests for that so that they're not just making changes for the fun of it. And if we run that, we get, here's our BaseItemRepository brand again. Roughly 60% of that file has been rewritten within the last year. That's a chunk.

I wonder why it's having so many bug fixes coming through. But these are all ways that you can find little signals where it might be interesting to go and look at more details for that file. I've been picking on the BaseItemRepository file because it's appeared at the top a lot and I'm very glad that Jellyfin had a file that they're so generously prepared exactly for this demo for me because couldn't have asked for a better one. But it doesn't necessarily mean that there's something wrong with that file. The fact that people are changing it could mean that it's improving, that they're working on improving it and me spending more time looking at that file might be a red herring. Maybe they've already got a lot of it in hand. Conversely, if a file hasn't been changed a lot in the past year in the repository, it doesn't mean that that file is fine.

Maybe that file is just too convoluted and too difficult that people do anything but work on that file. So these aren't going to tell you that file's a stinking pile of poo go work on that. These are just going to tell you, hey, here's a signal. Maybe go and investigate that. See if you can build the context of why that file is like this. If you know that the file is like that, what can you do to make sure that it doesn't happen again? Are there tests for the file? There's some other tools you can do around this. So instead of just playing git you, there's one of my favorites git-of-theseus, which is the ship of theseus where if you continue to replace parts of the ship, is it still the same ship. Wonderful metaphor for software development, and you can do the same thing for code.

If you've got an original bit of line of code, is it still the same system if you replace that line of code. If I run git-of-theseus for the lines of code that are still present on Jellyfin we get this graph, which I suppose doesn't tell you much, but you can compare it to other repositories that you do know and see if that gives you some indication. So for instance, I know the NServiceBus repository and the service control repository very well, which means if I compare it to the Jellyfin one, maybe I can get something interesting out of that. And it seems like there's less code that remains in the repositories I know that I don't think are legacy anymore compared to Jellyfin. In fact, these big drop-offs that you see on the NserviceBus ones, those are where we explicitly remove the old code after we've done all of the backwards compatibility migrations, the big chunks of code that gets deleted, it's cathartic. I can only ...My favorite saying is the only bug-free code is deleted code. So any chance I get where I can delete a chunk of code, it's such a great feeling.

Other things you can do with git-of-theseus is you can slice the code by cohorts. So you can say, well, code added in 2002, how long does that code stay around for? Fun fact about this graph, it actually looks like the code added in 2020, 2021 in 2022 seemed to stay around longer than other bits of code. I wonder if anything happened in that time period that would've made us more productive.

I don't know. Some other tools you can use, hercules, there's a very Greek thing going on with these tools. Hercules does similar things, but it takes a bit longer to process the repository. In fact, for Jellyfin, it took just over a day on my laptop to process a repository. But once you've got that, you've got other options you can break the changes in the repository downwards. You can see who's doing commits and how much of their code is staying around at what point in time.

Because if you're just picking up a legacy system, now maybe you want to know that you shouldn't go and ask, who's that first guy, Luke, Duke, Luke Luke. You don't want to go ask Luke Luke because he hasn't made any changes in a really long time. But perhaps you would go and ask Mathau because he seems to be doing a lot of work now. So you can use this to find who's working on the system, who's got understanding that you can go and ask about this and there's a bunch of other tools for hercules. Spend a few minutes looking at the repository. There's a bunch of stuff that might be helpful that helps you build understanding of what's happening in your system.

We can also refactor our systems and we can spend the time to do that. So if I were to look at the base item repository class, this is what it looks like. There's 2,600 lines in this file. Boy, I wonder why there's so many bug fixes here. Now, if I had to look at this file, I wouldn't really know where to start. What I could do is I could go and just refactor the file, take a whole bunch of stuff and say, these methods seem like they're of a similar theme. So let me take all of them, extract them somewhere else and see what breaks. Oh, that breaks, great. I can fix that. Why do I have to pass the file system into this? That's weird. The fact that you're changing things and moving things around is a way to explore what the system is doing.

Most importantly for me at this stage, while I'm refactoring for understanding purposes, I'm not committing anything. Once I've done this exercise, I'm going to abort that, throw all those refactorings is a way because I've probably broken something. But the act of going through the refactoring process, looking at the file, seeing why these are grouped together, why I can't just take this and move it somewhere else, helps me build a mental model of what else is happening in this file. It's super valuable. Jimmy Bogard has a talk called The Long Lost Art of Refactoring. It's not this talk. Unfortunately, The Long Lost Art of Refactoring wasn't recorded anywhere, but that whole talk is about how you can use refactoring as a way to explore code bases. Some of the techniques he covered in that talk are covered in his domain-driven refactoring tool. You can go watch that and get a poor man's version of The Long Lost Art of Refactoring as well.

The act of refactoring can sometimes be more meaningful than the refactored output, especially as a way of building an understanding of code. There's also a hidden coupling that we can find within our repositories. So change coupling is a fun little one because if you've got coupling with code, it means I've got this bit of code and it calls this code. So if I change the constructor of this one, I need to change the code over here so that it calls the right constructor. Great, that's easy, our ideas and compilers can tell us that we've broken something super convenient, barely even a problem. Change coupling though is sometimes called hidden coupling because what it means is we're looking for files that have changed and they always seem to change together, but there isn't a direct link between the files that we can easily find. For instance, if we make a commit to the BaseItemRepository class, maybe there's another file that almost always changes with the BaseItemRepository class, but it's not compile or linked together by the compiler, so we wouldn't necessarily pick that up.

There's tools that we can use to find this. Codescene is one that's quite nice with this. Codescene has a way of exploring code bases that also is built up of your git repository history and it's got this coupling diagram that you can look at. So we can go and see, okay, well when I changed the ready group request, almost always the buffer group request is changed. That sounds right, and if you look at those files, there's no direct link between those files. So maybe there's some DTO aspect that they've been used to do that. If we do go and look for our BaseItemRepository class, we can see that when we change BaseItemRepository 20% of the time that we change that, the base item entity file is also changed. And this is another example where there isn't a direct compiler link, those two files.

So if I was just using the IDE to try and find cupping, I would've missed this. I'm of course not saying that everybody should go out and buy Codescene. It is not necessarily a good fit for teams. In fact, I don't think it's a good fit for teams at all, I think it's good for individual people on teams, perhaps one or two people that are responsible for the code quality and when you do use it, just be aware that some of the heuristics that it uses to determine whether code quality is improving or decreasing is potentially not something you might agree with. So it's a tool, use it, don't use it, whatever. If you don't want to use Codescene, you can also look at CodeMart, which Adam Tornhill, he made CodeMart and then decided, wow, this is a fantastic idea. I can commercialize this and turned it into code scene.

So CodeMart is still available and it's got the change coupling. It's got some of the functionality that Codescene has. Obviously he's not improving the open source on very much though. But CodeMart can give you a lot of this hidden coupling ability to find. You can also dig for it yourself. So you can give it yes another script, you can give it a file name and you can go and ask your git database, hey, what changes when I changed this file? Now in this screenshot, I should have run this within the last year, but I didn't. So I ran it for the entire history of the repository. So this is pretty meaningless, but this is for the base item entity, BaseItemRepository class. When it changes 0.1% of the time, the migrate library DB changes, whatever. If I had it within the last year period, it might be a bit more meaningful.

We can also look for co-paired files. So if I don't have a starting file, I just want to find the other files that change when this file changes. And if we run this query against the Jellyfin repository, we get our base item. Whenever that changes, the folder class changes. What? That's weird. That is weird, actually. Also, when the base item changes, the library manager changes. So there's a lot of interplay between what the base items do and something with folder or library management. When the iLibraryManager class changes, I can see that the library manager would change and also the compiler would tell me that, but there's other bits of code that the compiler wouldn't tell me, but the git history would tell me. So we can use this to dig through that.

At the end of this, all we do is we're trying to improve the understanding of the code so that we know when I change this, I need to be very careful about this change because this file has got a lot of bug fix commits in it or maybe even before you make that change, first go and add a test or multiple tests or something so you've got a baseline. Use these signals as ways to decide whether or not what you need to change is something that you're comfortable changing or whether you need to be more considered your approach to that file. Ultimately what we're doing is we take in our understanding of the system and we're trying to improve that so that we are at this corner again and combining that with improving the supportability of our system, we slowly start moving things up so that we're at the point where we can effectively make changes to this legacy system.

We're building up our understanding of why the system is the way it is and how we can make changes safely to it so that we can improve the situation. That's all a long-winded way of saying, well, how do we actually make changes? This is one of my favorite scenes in the Indiana Jones movies. The gentleman in the black there has got a sword and he's doing all these cool little tricks, it looks super impressive and Indiana Jones just wasn't having any of it and he shot him instead of using his whip I think was the original intention. Apparently this wasn't scripted. So apparently India was supposed to go and do something, have a real fight with this gentleman, but Harrison Ford had a stomach bug that day and just couldn't cope and they rolled with it. That's great. So how do we deal with legacy systems without turning to day drinking? Well, you can still turn to day drinking if you want, but we don't want the legacy system to be the reason you day drink. You should day drink for emotional reasons, not work reasons.

Remember this last attribute, this life cycle that we didn't cover now or didn't cover earlier? Let's explore that a little bit more now. So a system can be in multiple stages of life cycle. It can either be a system that you are looking after and you're only fixing critical bugs in the system. You are only making sure that any changes are made are being made to meet compliance requirements or something like that. The next phase is the system could be under maintenance where you might have to add some functionality every now and then, maybe add additional reports, fixed bugs, but you're not actively working on the system to make new functionality. That would be the third category where the system is under active development, where you are still going to have to work on the system for years to come and you're going to have to continually drag the system kicking and screaming out of the legacy quadrant so that it's a pleasure to work with.

The reason that the life cycle of the system is important is because the techniques you use are only applicable in some life cycles versus others. As an example, prefer adding functionality over changing functionality. This is an example where for instance, let's say we had our GetItemValueNames method from our BaseItemRepository class in Jellyfin. This is what it would look like. It goes and gets a list of item values and then gives you the name, whatever that means.

Somebody might come and say, hey, I want to sort the names please, and you go, not a problem. Here's the code. All I do is an add an order by over here, job done. This legacy stuff is easy. Next ticket please. I chose this example specifically because hey, there was a bug because somebody changed the ordering of this and it broke other stuff. So prefer adding of a changing would mean that instead of going and changing this method, you would go and duplicate that method and have a separate method, get sorted of item value names method, and then call this method rather than call in the original because now the code, the place where you are making that call, that's the only one that's changed.

If the system was in active development, you should never ever do this though because you are duplicating a lot of code, even if it's under maintenance, probably still not a great idea.

If it's under critical support only, by all means. This is appropriate for that because you are not going to risk changing anything else in the system and having a new bug created. We can take a slightly different approach with this. So we can change at the edges rather than internally. So similar to where we had the get sorted item value names, we can go and dig through where this is eventually called so we can blow up the stack and we can find that this is eventually triggered by calling this line, GetStudioNames. So studio is the part that needs to be sorted, not the rest of it. Okay, so to change at the edge, this is where we would make that change. We would take that list and do the order by here. Of course now we're doing the order by in memory rather than passing them off to the database. Again, it depends on the context of the file and the state that it's in. This one might be better for maintenance than critical, but again, I wouldn't do this for active development because it's probably not a good technique to use.

Slice by capability. This is an example of one that I would highly recommend doing if you're doing active development on the system. So slice by capability says, well, I need to fix this. I need to change the sort of the studio names. Rather than just making the change, why don't I start refactoring the way that studios are managed? So if we look at the code, we've got an I item repository that gets passed into this method and that I item repository is the one that returns the list of studio names. Well, why don't we create a new interface I studio repository and have just studio specific stuff there so that we can now only have studio specific rules? There might be a duplicated code, but at this stage you are refactoring towards a different capability. Once you've got a capability that's isolated, it means that it's safe to make changes there because you're not going to make changes to other parts of the system and there's a bunch of other patterns.

The intention of going through these is rather not to go and look at the specific patterns, you can go and find anywhere, you can ask your local friendly agent to tell you how to do it, but be aware that the life cycle that the system is in is going to impact whether or not the technique that you're using is going to hurt you in the long run or is fine. Ultimately, what this is all about is risk management. Understanding that if I'm making this change and the system is in this state, what can I do to mitigate the risk? Do I have the support ability around me to roll back if I make a mistake? Do I have sufficient understanding of the system? Have I built enough understanding to make this change safely? And is the system in the part of its life cycle where I can make this change safely as well and that the cost of this change is appropriate.

In Raiders of the Lost Ark, you see Indiana Jones doing this amazing refactoring job where he swaps the idol out for a bag of sand and he very clearly thinks, okay, I need to take a little bit more sand out of the bag before I can swap them out. I mean he was wrong, but he thought about it, proving yet again that he's the worst archeologist in the world.

So rather than taking a leap of faith when you make changes in code, evaluate, first of all, whether or not the supportability is there, if you've got sufficient understanding and if the technique you're about to apply is appropriate for the life cycle that the system is in. Because by doing that you measure or you manage the impact of the change. If you get that change wrong and it breaks, how long will it take you to fix that? And that's essentially what working with legacy systems is, and you will get it wrong sometimes, just like Indy did. He refactored, yeah, and thought he got away with it really well until he released this into production and it ended up with a boulder chasing him down the hall and a dead coworker. Hopefully none of your commits kill coworkers. Unless you want them to, I don't know. Your job is your job, whatever.

Legacy systems are only called legacy because they've got value. Nobody would use a legacy system if it was a horrible system and it didn't do anything for them. And when you're working on these systems, it's important to spend the time and the emotional investment to remember that there is value in the systems. We use legacy as a bit of a derogatory term and it really shouldn't. It's just a natural state of systems. If you've got an amazing system now in 10 years time, you're probably going to look back at the code and cringe.

Ultimately, there's a quote about horses, I don't know who said it, but it felt appropriate for the adventure theme. The gist of it is working on bad systems now, it's forcing you to learn skills that are going to make you a better developer in the future. Being forced to make changes to systems that are brittle, that are convoluted, that are described as legacy, is equipping you with better skills to be able to make better systems in the future. So don't think of legacy systems as something awful. And I hope that when you get presented with your next legacy system you don't think to yourself, well, every moment of my life has been leading up to this point because I didn't realize the system could be so shit.

It happens. There's a bunch of references. If you scan the QR code, it'll take you to a page with all of those references there, so you don't need to go and copy those individually if you want. I do have time for questions, otherwise we've got one more slot and then NDC is over. Thank you very much.

Any questions? No? Excellent, get out of here.

Indiana Jones and the Temple of Legacy Code

About this video

🔗Transcription

Indiana Jones and the Temple of Legacy Code

About this video

🔗Transcription

Additional resources