Jeroen’s weblog

When doing software maintenance on large systems, detecting defects is typically more complicated than when creating software from scratch: often it’s not possible to add useful safety nets such as unit-tests because of design choices in the existing system that make this impractical or because of size/time constraints with regards to the project. A useful technique is formal code inspections, not only because of its effectiveness in finding defects but also because it serves a purpose in program comprehension.

In What We Have Learned From Fighting Defects, Shull et al. document the results of several workshops in the area of defect reduction. One of their resulting statements is: Reviews catch more than half of a product’s defects regardless of the domain, level of maturity of the organization, or lifecycle phase during which they were applied. Which is quite a lot compared to other techniques (numbers from Code Complete 2, page 470) such as unit-testing (30%), regression testing (25%) and doing a full system test (40%). The only ones possibly scoring better are prototyping (65%) and high-volume beta tests (75%). The former however is impractical when doing maintenance and the latter takes much more effort than code inspections.

Formal code inspections have been in use for over thirty years. Michael Fagan published his paper Design and code inspections to reduce errors in program development in 1976, after having applied the techniques at IBM in the years before. While a very interesting read, the process description Fagan provides is a bit dated. A good alternative is A Guide to Code Inspections (PDF) by Jack Ganssle. It consists of a process description, some considerations and a form to use when doing inspections.

Now if this isn’t incentive enough to start doing inspections, consider the additional advantage during maintenance: as I wrote about before, reading code is about half the effort in software maintenance, which means it’s easily the biggest part (since implementing, testing, deploying, etc. is all in the other half). When you modify code and have it inspected by others, not only are you getting the typical defect reduction associated with inspections, the ones doing the inspections are also reading the modified code and acquiring knowledge about the system’s structure and functionality. Don’t make this an explicit goal however: the strength of inspections lies in the total focus on finding errors. Learning about the system is a built-in side effect.

Important to note however that this may still be useful if you already do pair programming. As Arie van Deursen notes in Program Comprehension Risks and Opportunities in Extreme Programming, the focus of pair programming (implementing functionality) is quite different from that of code inspections (finding and removing errors), so the idea of reviewing code all the time during pair programming, although useful, may not provide the same level of defect reduction compared to actually doing additional explicit code inspections.

In Jeff Moad’s 1990 Maintaining the Competitive Edge, for which I can’t find an online version unfortunately, he predicted that in 1995, 95 percent of software life-cycle costs would be devoted to maintenance. In 2000 Len Erlikh wrote in Leveraging Legacy System Dollars for E-Business that by then 85 to 90 percent of the IS budget is typically spent on operating and maintaining legacy systems. A good overview of the numbers is maintained by Jussi Koskinen on his page on Software Maintenance Costs. Even though it’s an understudied area, the existing numbers clearly show an upward trend (even though Moad was overestimating it a bit): in the 70s and 80s maintenance was already in the 50-70 percent of total software costs. Starting with the 90s it’s 75 percent and up. In Modernizing Legacy Systems, Seacord et al. refer to this as the legacy crisis.

So does this mean that every hour spent writing new code causes people to spend up to nine hours fixing bugs? Not really. I’ve written about the types of maintenance before and a very important thing to understand is that fixing bugs is rarely the major activity during maintenance. The four types of maintenance can be divided into two groups: corrective and non-corrective. The most influential study in this area by Lientz and Swanson (comparing the software maintenance activities of 487 organizations in Software Maintenance Management) states that only about 25 percent of maintenance is spent on correcting errors. The rest is spent on enhancements. So every hour you spend writing new code may result in two hours of fixing bugs and around seven hours of enhancements (although most of that time is spent on things such as figuring out how to modify the system, testing and deploying).

In other words, high maintenance costs are not typically the result of bad software. In fact, the better your software is, the more maintenance it will probably require, because people tend to use good software and as a result of that the software will have to evolve. The reason for this, the First Law of Software Evolution tells us:

This need for continuing adaptation and evolution is intrinsic to E-type applications and software. It
is, in part, due to the fact that development, installation and operation of the software changes the
application and its operational domain so creating mismatch between the two. Evolution is achieved
in a feedback driven and controlled maintenance process.

So because we keep getting better at making useful software, people start using software more and more. This increasing usage has an ever-growing impact on the operational domains of the software (which will end up being the entire world I suppose) triggering more and more requests for enhancements. The legacy crisis then, ever-increasing software maintenance, is not a crisis that can be averted or reversed: it is an essential property of developing, deploying and operating useful software systems. And as we’ve learned from Fred Brooks, there’s No Silver Bullet for the essential complexities in software engineering.

What this means is that we shouldn’t try to avoid maintenance, because we can’t. Instead we should embed maintenance into our software processes so that development flows directly into it. A good way to ease into maintenance is thinking of initial development as maintenance, except that you’re starting with a system consisting of zero lines of code, no bug reports and the requirements as a list of requests for enhancements. We already use the same practices and the same tools for doing development and maintenance. So instead of having this 90 percent part of all costs being something nobody is prepared for, let’s just pull in the other 10 percent and call it 100 percent software engineering.

Little under a month ago, a kick-off event was being held as the official start of the Netherlands Institute for Research on ICT (NIRICT), which included a talk by David Parnas (best known for laying the foundation of object orientation) called Software Documentation: The Research Topic that Computer Science has Neglected. He argues that in order to make any real progress in the area of software quality, software documentation must improve radically. One of the issues he mentions is that reviewing, testing and inspection can only be reliably performed if you have accurate and thorough documentation. (Read the presentation for some of the other issues, it’s quite interesting.)

At the same time, most agile methodologies have been saying that documentation is a thing of the past. In Extreme Programming Explained, Kent Beck states that XP relies on “oral communication, tests, and source code to communicate system structure and intent”, instead of on design documentation. This approach has some problems however, as Lionel Brand pointed out in his 2003 keynote at the European Conference on Software Maintenance and Reengineering: how do you know that the tests you have are complete? How do you know whether everyone working on the project has intellectual control over the technical design? With documentation and reviewing you can find these things out much more easily.

In software maintenance, having good documentation is invaluable when attempting to understand a system you need to maintain that was developed by someone else. The most important part for maintenance is ofcourse the architecture, so having good architecture documentation is essential. To evaluate provided documentation, consider that good software architecture documentation is the following things:

• Up-to-date — obvious perhaps, but something that must always be considered. As a survey by Andrew Forward and Timothy Lethbridge shows, documentation is rarely updated. Technical documents that are not up-to-date are basically exceptionally boring works of fiction, in other words, useless.

• Written by the right people — sometimes organizations let interns or people from outside the project write documentation because the developers didn’t want to. This doesn’t work because as far as architecture is concerned, only the developers that came up with it can properly recognize and understand what is part of it and what is not (as I wrote about before in You can’t measure architecture.) In a 2003 IEEE Software article called Who Needs an Architect?, Martin Fowler cites Ralph Johnson as writing “In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’” So in order to have any chance of getting the message across, those expert developers should have written the documentation.

• Complete — for this you can usually only hope that this follows from the documentation being both up-to-date and written by the right people, since it’s not easily determined if you’re new to a system that needs to be maintained. Personally, I expect several things to be present in an architecture document to be at all usable to facilitate initial understanding of a system. First is a high-level view of the entire system showing all components (usually a diagram of some kind). In addition to this, the following questions must be answered for every component: What are its responsibilities? Why does it exist (in other words, why isn’t it part of another component)? What is its interface? What does this component depend on? These are very broad questions but they must all be answered in detail in order for the document to be worth anything in trying to understand the system it documents.

You could argue that the source code can satisfy these requirements and the first two properties are in fact always satisfied by the source code, but unfortunately, no matter how good your naming convention or directory layout is, the third property is never satisfied by the source code for non-trivial, real-world systems. So architecture documentation is a necessity, at least if you expect maintenance to be required at some point. About the odds of that happening, it’s easy to reference Meir Lehman’s First Law of Software Evolution: “[A real-world] program that is used must be continually adapted else it becomes progressively less satisfactory.”

After I claimed that you can’t measure architecture a couple of weeks ago, I’ve been reading up again on the topic of architecture metrics to see if my view, which I’ve held for a while now, is still valid. Unfortunately I’ve had to conclude that it is, at least with regard to software evolution and maintenance (which is what I’m mostly interested in). The problem being that some architecture metrics are actually quite promising, but not capable of measuring what you want to know when modifying existing systems.

A post by Grady Booch pointed me to an interesting paper on architectural complexity (ACM member log-in required unfortunately) by Mohsen AlSharif, Walter Bond and Turky Al-Otaiby. Two other papers on the same subject I read were a fairly classic paper by Jianjun Zhao and a recent thesis by Matti Kinnunen. That last one has a good discussion of some basic issues, such as the definition of software architecture and the aspects of a system that relate to its complexity.

I’ve been applying the proposed metrics on some existing systems to see if they could somehow help me to gain a better view of the issues related to modifying the systems. All three approaches were actually quite good at calculating the complexity of the architectures (for instance, the systems that were assessed as being highly complex also came out of the measurements that way). This is very valuable when doing initial development: if you wonder what approach to choose when selecting an architecture, being able to take some measurements is a great way to help you in tweaking designs. But in maintenance, things work differently.

Let me explain: suppose you have two systems that perform the same function. You don’t know either system and you’re supposed to modify both to add a required piece of functionality. You calculate the architectural complexity (using any of the available architecture metrics) and come up with one system having a much higher complexity than the other. Does this mean it’s going to be more expensive to implement this feature in the system with the much higher architectural complexity? Unfortunately, it doesn’t.

The system with the highest complexity may be a large blob of functionality that is complex because it has so many different inputs and/or outputs, or may even be a totally over-designed system that has incredibly deep and pointless class hierarchies or convoluted interfaces. It’s all possible. But what you don’t know is: is the design prepared for the change you want to make? Because if it’s a terrible mess, but it does have a wonderful way to add the required functionality, then the overall complexity is completely irrelevant (with regard to this single modification). And the same applies to systems that have a low architectural complexity: the system may be very elegant and simple, but if the original designers never expected this change, then the architecture may end up being completely inappropriate for the system with the modification made. This could mean that modifications may be required in every component of the system.

This makes sense because an architecture is selected based on requirements. But when the requirements change, there’s no way to tell whether the selected architecture is still appropriate. Even if the requirements specify that this change has to be prepared for, you won’t find out if this is actually the case, until you attempt to implement it. So yes, you can measure a system’s architecture, but when doing maintenance, the numbers don’t tell you what you need to know.

In my experience people tend to see metrics as some kind of gimmick: fun to let loose on your software, but not something that is actively used to make estimations, track progress or assess situations. It’s a shame, because a typical project (especially once it’s been around for a while and a couple of releases have been made) is a big source of measurable information waiting to be extracted and used. The trick ofcourse, is knowing what to measure and how to interpret the results.

One of the more useful metrics I’ve been using recently is code development complexity. This metric doesn’t look at the actual code itself, but at the distribution of modifications across the system, to predict the complexity in modifying it. The paper that proposed and first described this metric is called Studying the Chaos of Code Development by Ahmed Hassan and Richard Holt. It basically states that two factors can be used to measure the complexity of a piece of software: the amount of modifications that have to be made for each piece of functionality that is added and the distribution of these changes across all parts of the system.

This means that if you’ve added a set of new functions to a large system and all you’ve done is modified the same one or two classes for each change, the system is probably not very complex because most of it has never changed. Also, the design is apparantly appropriate given the types of changes that have to be made. An example of this is a system that uses the strategy pattern and that basically adds new implementations of an algorithm: the system itself doesn’t change, it just gets a new class every now and then. However, if a strategy is not used, new algorithms would have to be added to the user interface (for selection), to the business logic (the actual algorithm implementation) and maybe even to the storage (to determine how it deals with storing results).

Code development complexity is a great metric to see whether the way the system was designed is correct with regards to the required evolution over time. Furthermore, in the cases where it’s not, the actual data used to calculate the metric (which files are modified) are exactly where you have to look to get the complexity down: if a couple of classes are modified everytime something new is implemented, it may be wise to investigate what the cause is. Some things may be hardcoded there that don’t have to be.

To calculate it (unfortunately I haven’t found any tools to automate this yet, I only have a couple of scripts I’ve written that are tied to Subversion and a custom format for log messages that I use on projects), you basically need the following:

• A repository containing as much of the project’s history as possible, preferably everything since initial development started.
• A method to determine whether a change to a file in the repository was to fix a bug, do some general maintenance or add functionality (because changes of the last type are all you want to measure, initially).
• A coding standard and/or development environment that forces files to contain logical parts of a system. Java forces you to use a file per class, Visual Studio automatically generates a file when you add a class, which is good.

Then you have to extract all the information from the repository, choose good parameters for the algorithm (such as how long each iteration lasts, I usually pick release dates) and calculate all the information, then plot the results and voila: an overview of the evolution of your project’s complexity over time. I’ve done this on a couple of projects now and am really happy about the results: two of the systems were developed initially by myself and the results very much reflected my own opinion of where the bottle necks in easy modification of the system were. They also provided a good view of what’s been going on in one case where other developers had modified the system.

Makes me wonder why none of the source control vendors implement this kind of stuff, especially now that they’re all integrating their bugtrackers anyway. Just shows how little people really care about metrics.

This is the third and final part in a small series on organizing maintenance activities. The other parts were about understanding (what to do) and modifying (the system). I actually feel bad about dealing with testing in the final installment since I don’t want to give the impression that the end is where testing belongs. As I’ve explained before, the waterfall model is a great model for explaining the various activities involved in creating software, but it’s a lousy model to base your development process on. Because of this, I’ll be discussing things that are relevant across all parts of the maintenance process.

Green-Red-Green-Refactor

I’m a big fan of automated testing. In fact, I’m a big fan of automating anything that has to be repeated a lot, because it saves you a lot of time and prevents small errors from slipping in. When being assigned a maintenance project and receiving the system to maintain, the best thing that can happen to you is that it comes will a full suite of unit-tests. Unfortunately, this is rarely the case. But since you want to automate testing as much as possible, the first step in maintenance should be to create some unit-tests. (See an earlier post for the types of maintenance.)

• For corrective maintenance, create a unit-test that exposes the bug or incorrect behavior. This is basically in line with what the traditional test-driven development techniques tell you to do.
• For perfective maintenance, first create a unit-test that tests some of the existing functionality related to the request (sharing some of the implementation for instance) and make sure it succeeds (green). Make a copy of the unit-test and modify it to test the desired new behavior that’s not implemented yet (red). The reason for starting with a test for existing functionality is because existing systems tend to be quite a bit bigger than the typical components you’re working on when doing initial development. Because of this you want to make sure the test you’ve written is correct and this is a lot easier when you can start with a related and succeeding test.
• Adaptive maintenance requires more testing: changing the environment is a pretty big step. The quickest way to test this is to create a small set of extremely large grain tests: together they should touch as much code as possible, so that you can quickly identify that there is a problem when porting code. The same is true for preventive maintenance.

In initial development the goal is to test as little as possible in a test so that a failing one not only identifies that there is an error, but also to a high degree of precision where the error is located. Unfortunately it’s rarely possible to create such a set for an existing system that doesn’t already have a large suite of unit-tests, so we have to forget about that aspect of unit-tests and accept that all they can do is validate that the system works. Which is pretty good in its own right.

Acceptance testing

As part of figuring out how to implement a change request or additional functionality, create an acceptance test for the modified system. This acceptance test should clearly define how the system should function after the modification has been performed. This is actually something that’s easier during maintenance than during initial development: when discussing functional requirements for a system that doesn’t exist yet with a user, it’s very difficult to make sure you’re both talking exactly about the same thing (for instance because of implicit requirements). In maintenance however, there is already a fully fledged and functional prototype: the current system! This means that you can talk about how the changes impact the current system and talk about what should be different instead of talking about how something should be that doesn’t exist at all yet.

Part of creating this acceptance test is also using the system yourself. I’ve seen a lot of maintenance work being doing directly from a requirements document or a bug tracker: developers that just browse to the code where the functionality is located and modify some stuff, not thinking about how the modification will impact the entire system and then cross out the requirement in the document or set the bug to completed in the tracker. This is a source for new change requests however, so it’s a matter of days or weeks before you find yourself confronted with the same or a similar change request. Instead, use the system, get an idea for what the user wants differently and after implementing it, verify that you’ve achieved this (using the acceptance test).

This is the second part in a small series on organizing maintenance activities. The first part dealt with understanding the system to maintain. This part is about actually performing the modifications. How you go about this has a lot to do with the stage the system you’re modifying is in, as I wrote before, there’s a big difference between the so-called evolution and servicing stages in a system’s lifecycle. Although unfortunately, you’ll often be expected to evolve an application when it’s only realistically possible to service it. I will focus on this particular type of maintenance in this post.

The defining characteristic of software evolution as opposed to software servicing is the implementation of new user requirements. If the application is expected to actually change or enhance its functionality, it’s still evolving as opposed to just being kept running or serviced. Evolution is only really possible if the original development team is still available (and preferably involved in doing the changes) or if you’ve had enough time to study the application’s architecture in such a way that you fully understand all design choices and their implications. If this is the case it should be fairly straight-forward to modify the system, given that the requested modifications are possible in the context of the application’s architecture. If they’re not, then there’s no trick to make this work: you can either attempt to reject the request, explain that implementing this in such a way that the application remains evolvable in the future will be a very large change (at the architectural level) or implement it anyway knowing that this will damage the architecture and push the system into the servicing stage.

In the third and unfortunately usual case, there are some important things to pay special attention to, since they’ll make your work easier in the future when you need to perform additional maintenance. Note though that these are not necessarily good tips when doing initial development where you can still make fairly large changes to the general design. In those cases there are often more elegant solutions. (Also note that component can refer to class, object, assembly, package, etc.)

• Keep the abstractions consistent. When modifying a component, make sure you understand what it is hiding: don’t add a method that somehow exposes everything the rest of the component was encapsulating to quickly add something new. The problem is that exposing for instance a private member that was never exposed before will suddenly allow users of the component to mess with its internal state. Future modifications can turn it into a source of bugs.

• Do not depend on state. When adding a public method (or anything else that’s public), do not depend on the state of the system outside the component. Instead, let callers provide everything it needs in parameters so that you can use guard clauses to check their validity. For instance, sometimes a method may need to serialize some of its data to a file. Unfortunately, assuming that it’s always possible to create a file in the current directory and write to it is a dangerous assumption. Instead, let the caller provide an object the method can write to.

  This ofcourse means you’ll have to make more modifications outside the new method than you might have hoped. However, this is a good thing since it will force you to consider the impact of what the method is doing at every location you’re calling it from. And more importantly, future callers will have to do the same.

• Update to current components. I recently modified a .NET application that was developed when the framework came out in around 2002. There was quite some code dedicated to multi-threading through the use of a Thread instance and a mutex for communicating status information. Since .NET 2.0 however, there’s the BackgroundWorker component that simplifies this type of work immensely. So instead of maintaining the old code, it was much easier to migrate it to use a BackgroundWorker. Don’t modify things that are working correctly, but if you have to modify it anyway, update it if there’s an easier way to achieve the same thing.

• Consider wrapping components. If a component has a large amount of callers (a high fan in) and modifications are to be performed that are only relevant to a couple of these callers , consider wrapping it for them in another component so as not to disturb the relationship of the component in question with its other callers. This doesn’t make the system any prettier, but if you have no way to easily verify all the interactions (which is often the case in large maintenance projects), then this will be a relatively safe solution.

• Adapt to the used standards. This applies to both the functionality of the code as to the style: if an entire system doesn’t throw exceptions but rather returns values to indicate success, you should either change it everywhere or use it as well in the newly added code. The same is true for coding standards: code written using a crappy coding standard consistently is easier to read than code that has several coding standards, including an exceptionally elegant one. This is in line with the consistency code quality attribute.

These are some of the things to remember when modifying a system that’s either too large to fully comprehend or too complicated to properly evolve (or both). Another important technique is using metrics, which I’ll discuss in a future post in some more detail.

Software maintenance is not a big atomic operation. In order to organize maintenance work it’s useful to identify the activities, so that you can plan them seperately and track how far you’ve progressed. Roughly, maintenance can be divided into three activities: understanding, modifying and testing. In a traditional waterfall process, they’d take place in that order but don’t take my list as a recommendation for doing it that way :-) This post will discuss the understanding part of maintenance, and two future posts will deal with the others.

Regardless of whether you’re still using the waterfall process or doing single day iterations, there’s no escaping it: you have to do some of the understanding work before you can do the others. In theory, a very high testability could let you skip understanding the system before changing it: if you know that you can always test the system’s functionality completely, you never have to worry about changes breaking anything, so you could skip trying to understand the system and randomly start modifying stuff. But even if this is possible, it’s probably not a very efficient approach since maintenance would turn into some kind of brute force attack.

At the beginning of the maintenance cycle, have a look at all requests and make sure they are both clear and prioritized. This is important because doing the wrong thing or doing unimportant stuff first has a big impact on end-user satisfaction. Often if you need more time than is originally allocated, it’s less of a problem if the most important things have been done correctly. You don’t have to study the requests in-depth, but at least read them all so you have an idea of what’s required of you. This also gives you some insight into how much time is available for each request based on the types of maintenance activities. Start with the highest priority request and get all the requests that are related to it (because they affect the same component or area of the system) and keep those together so you know what aspects of the software to pay attention to when studying it.

Next is actually using the system. Reproduce any errors that need to be corrected. If you can’t reproduce an error, invest a small amount of effort into doing some typical scenarios but if you can’t reproduce it after that, reject the request to fix it: wild goose chases are entirely unpredictable in both effort and results and are therefore a waste of time. If new functionality needs to be added to the system, use the system in the area the new functionality is to be implemented and get a feeling for how it should work. Sometimes you might even discover that the new functionality already exists but is hidden somewhere and just needs to be moved around in the user interface. All-in-all, using the system is one of the most important activities in understanding maintenance work.

As I wrote before, reading source code, comments and requirements is a typical way of getting to know the system. Stepping through the code from the start is a good way to get an overview of everything that’s going on inside it. Write down a couple of scenarios related to the requests to execute, then set a breakpoint right at the first statement in the code and start stepping through the code as you execute the scenario. Try to get an idea of how the system is organized and which different subsystems are used for what purpose. In a lot of systems that have evolved for a couple of years, the responsibilities are not as neatly divided as you’d expect (for an example, read about the algorithm implementation doing GUI work).

In Refactoring, Martin Fowler notes that he often does small refactorings in order to understand a system. It’s absolutely true that doing some refactoring to clean up small inconsistencies or clarify naming helps gain a good grasp of a system. However, make sure that you only do this if you have good test coverage in the area you’re refactoring, because otherwise you’ll just create subtle bugs that hinder getting the actual work done. This is typically an area where maintenance differs from initial development: when doing the latter you could get around this by writing tests to cover the code you’re refactoring. In maintenance however this is often practically impossible: the system is too large, already contains too many modifications, etc.

Finally, another area to look for information about where changes may have to take place are in history: if available, both bug tracking and source control histories often provide a wealth of information on what has been modified in the past, where these modifications took place and the related notes and discussions by previous maintainers. In practice you’ll notice that a lot of maintenance results from previous maintenance activities: bug fixes that introduced new bugs, added functionality that has usability problems or that has some strange interactions with existing functions to name just a few. This can give you a lot of information about the causes of problems (and possibly solutions).

In the past weeks I’ve been stressing the complexity and nature of typical software maintenance activities. Before you can typically dive into performing maintenance however, an estimation of the required effort is usually needed. Now I’m not going to go present some foolproof formula to figure out how much time and effort is going to go into maintenance (since there is no such formula and several books have been written dealing with the topic in detail) but I can discuss some of the important issues that have a big impact on the final estimation.

First of all and most importantly is the state of the software you need to maintain. A couple of weeks ago I’ve discussed these quality attributes related to maintenance.

Next, the type of maintenance you need to perform is important. The reason these types of maintenance are important for determining the amount of effort required is because their impact on the system differs from type to type. Here’s a list of types of maintenance (from the IEEE’s software maintenance glossary) and some considerations to remember when performing them:

• Corrective — modifying the system to repair hardware or software faults. Basically fixing bugs, ranging from small rounding errors to complete system crashes.

  A system is typically designed to be consistent and function correctly (try not to think about cases where this is not true), so corrective maintenance is usually just a matter of finding the coding or design error and correcting it. Ofcourse there is the possibility that a bug ends up showing that the entire architecture is broken, but that’s a rare case. So the question to ask here is: is this a subtle bug or a fundamental error in the original design?

• Adaptive — modifying the system so that it can function in a changed environment. This can mean porting an application from Windows to Linux, but also modifying an application to be able to run without administrator priviliges.

  There are two questions you need to answer when estimating adaptive maintenance. First, how much is the target environment different from the current environment? Second, how closely is the system tied to the current environment? Going from Windows Forms 1.0 to 2.0 is a reasonably small change, but porting a native Win32 application to Linux is way much more work, especially if the application uses threads heavily, since threading APIs are typically very platform-specific.

• Perfective — improving the system, both functionality (what it does) and non-functionally (how it does it). These are typically user requests.

  This type of maintenance can have a huge impact even on the architectural level: sometimes users want modifications that are completely contradictory to what a system is supposed to be able to accomplish. I once developed a test application to replace a piece of hardware while developing software to interface with it. The application ended up having to be distributed as a learning tool to customers since the hardware wasn’t finished yet at the end of the project. It was designed to have a user interface that was useful for developers to test the functionality of the software, not for end users to learn to work with their upcoming hardware. The question to ask here is: how far removed from the original functionality of the system is this request?

• Preventive — modifying the system to make it better capable of handling current and future required changes. Refactoring can often be considered a typical form of preventive maintenance.

  This is easily the most complicated of all. Assuming you need to maintain a system that was developed by someone else and time has been allocated to perform preventive maintenance, this probably means the system is of high value and additional modifications will be required in the future (since those are the reasons for investing in preventive maintenance). What is basically asked of you is to modify the system in such a way that maintenance effort will decrease in the future. This can only be achieved by making sure the system’s architectural integrity is intact and gaining an understanding of the system’s inner workings similar to that of the original developers. Only by careful inspection and working with the code extensively can this be achieved, since you can’t measure architecture.

As described, the list above basically presents the types of maintenance in increasing order of average required effort. Note the word average however, since all types have both positive and negative excesses.

Another thing to remember is that both Shari Pfleeger (in Software Engineering, Theory and Practice) and Thomas Pigoski (in Practical Software Maintenance) conclude that understanding the system takes up about 40-60% of the time required to perform a maintenance activity. So once you have a good idea of what has to be done to do the maintenance, it’s usually safe to double that estimate.

In a recent study on what documentation is used to perform maintenance two conclusions struck me as interesting:

• Engineers claim to be very interested in high-level documentation such as architecture descriptions, but when actually performing maintenance, these same high-level design descriptions are some of the least used documents.
• Besides code, comments and data model which are used the most for obvious reasons, it appears that in practice engineers consult requirements documentation when performing maintenance.

A lot of engineers are confronted with having to modify systems of all sizes that they have no prior experience with. In fact, the more comfortable you are with a system, the least documentation you’ll probably read before doing anything since you already know the code. So in the cases where you just get a login to a source control system and a repository name (or worse: a zipfile or tarball) and you have to find your way around, the first thing would be to find something looking like the entrypoint and reading code and comments along with the database scripts.

When asked to think about what kind of documentation would be useful, I too would say I’d want to have a look at the architecture. But in practice the architecture basically documents what the solution should be and how it should be implemented. However, the solution is already there, coded and ready. In my experience, the question you want answered the most when browsing endless files filled with seemingly unrelated functionality is: What problem is this code supposed to solve anyway? Enter the requirements specification.

Documentation often goes stale after a while, especially once the project is finished and people maintain the software. The documentation itself rarely gets maintained properly. So the solution to this probably doesn’t lie in creating some kind of document to hold this information or adding it to an existing document. In agile circles most practices tell you to name classes, methods, etc. in such a way that it’s easy to figure out what they’re supposed to do. This is a good start but not a complete solution: when you start to figure out what and how an entire system does its work, looking at class and method names is simply way too much work.

The thing that has worked for me on some projects is to put a couple of comments at the top of each source file (or class, if that’s not the same thing in your environment) that explains in a couple of sentences the following things:

• What problem does this code solve? This can be either directly a user requirement but can also be something that the other parts of the system depend on.
• Where does this code fit into the big picture of the system? For instance, the component this code is part of.

Optionally you may add information about dependencies, but namespaces as well as using/import statements are good hints about that already. I’ve been doing this on a couple of projects and have noticed (based on the amount of questions I’ve gotten regarding the code) that it makes it a lot easier for developers to find their way around.