When I was an undergraduate student, a joke went around that had car and computer company executives bragging about whose industry showed the greatest technological innovation. The car executive had the punchline, which went something like: if car technology had progressed at the same rate as computer technology, a Rolls Royce would cost $100, have a top of speed of 1000kph, travel 1000km on 1 litre of fuel, and explode once a month, killing everyone inside.
Decades later, the joke still works, whether the subject is spectacular global meltdowns like the CrowdStrike incident, or run-of-the-mill frustrations that might cost computer users upwards of ten per cent of their time. To be fair, mechanical systems aren't immune from failure, either, as users of Sydney's train network would know. Still, the joke says something about software industry rhetoric that lionises "fast-moving" and "innovating" over over boring attributes like stable and reliable.
I got to thinking about software stability after writing a small web application to expedite processes in a hobby organisation of which I'm a member. Once the project team had conducted internal testing and put the software into production, no one has ever reported a bug. I put this down to two factors: (a) it's a small application with a small number of users, with a small number of things to go wrong; and (b) once the software was working, we left it alone. The rest of this article looks at the academic literature on leaving stuff alone.
Software Stability
Maria Salama and colleagues (2021) begin their review of software stability by comparing the definitions of stability that exist in software development and in other scientific disciplines. The Oxford English Dictionary definition refers to the condition of being stable or in equilibrium state, resistance to change and the tendency to recover from perturbations, which are all reflected in definitions like steadfast and free from change or variation (physics), the ability of an ecosystem to resist changes and return to an equilibrium state in the presence of perturbations (ecology) and the ability of the process to perform in a predictable manner over time (business process management). Even in distributed computing, we get a definition like a measure of the ability of a mechanism to detect when the effects of further actions will not improve the system state as defined by a user-defined objective.
When we get to software engineering, however, we get not only definitions like the ability to resist changes and the ability to remain unchanged over time, but also the ability to adapt to changes while remaining intact and (one of Salama and colleagues' own definitions) the ability of the design's structure to resist or adapt to changes due to maintenance activities, where stability is defined in terms of the software's ability to support change. Salama colleagues also note that none of the definitions they reviewed concerns itself with maintaining a fixed level of operation (such as not exploding every month).
Salama and colleagues go on to a more complex taxonomy of stability and relate it to attributes such as dependability, robustness and so on, including attributes that one might think directly opposed to stability—modifiability, changeability and evolvability. Software engineers can square such qualities with stability, however, by recognising that stability can exist at several levels. Object-oriented programming, for example, teaches that the interface to an object should remain unchanged even if its internal representation changes; and "architectural stability" posits that high-level software architecture should support evolution of the software over time without itself changing. Salama and colleagues conclude with some methods proposed for evaluating stability from these perspectives, but, to my knowledge, these haven't achieved widespread use in the software engineering community.
Long vs Short Release Cycles
For some years now, the fashion in software development has been for short development cycles, aiming to deliver updates to customers as quickly as possible. In the early days of this trend, a few researchers wondered if short cycles would improve software quality because bugs could be fixed more quickly, or worsen software quality because software was rushed into service without adequate controls.
Most empirical studies suggest that short release cycles don't increase the number of bugs, but they have a variety of impacts behind the scenes. One study of twenty-five open source projects found that the time to fix bugs decreased after a transition to continuous integration (Almeida et al 2022), while study of Firefox found that the time to fix bugs increased by more than 50% and that short release cycles prioritise issues in the current release over integration of long-standing issues from the backlog (Alencar da Costa et al 2018). A study of the transition to short release cycles at ING (a bank) reported that developers found code review easier and that users perceived the software to have higher quality, but that managing dependencies became more difficult and that the level of "design debt" (costs due to the software's design being no longer fit for purpose) increased (Kula 2019).
Studies of this sort, however, say little about the impact on users of frequent updates. In one review of security patches, Nesara Dissanayake and colleagues (2022) describe managing security patches in large-scale applications as "a hugely challenging process that involves multiple stakeholders making several interdependent technological and socio-technical decisions". From the 72 papers in their review, they conclude that "50% of the common challenges [in patch management] have not been directly addressed" by the literature and that a large proportion (38.9%) of the solutions that do exist address only the vulnerability assessment phase of the patching process. In the preface to pne large study of system administrators' patching behaviour, Adam Jenkins and colleagues (2024) say that, while security-minded researchers insist that updates should be applied as rapidly as possible, preferably automatically, many system administrators regard automatic patching as "simply too risky" due to the number of patches—a la CrowdStrike—that cause as many problems as they solve .
Conclusion
The tension between advancing features and preserving stability was observed as far back as 1980 by M. M. Lehman, but advancing features receives far more attention than stability in both popular media and academic literature. Even many academic authors appear blind to the impact on users and system administrators of frequent changes.
While other engineering disciplines typically define "stability" in terms of resisting (undesirable) change or returning to equilibrium following a disruption, software engineering frequently defines "stability" in terms of software's ability to support change. Nonetheless, useful conceptions of "stability" exist, such as "architectural stability", which capture something like the stability of a physical artefact expected to function for a long period of time but that may require periodic maintenance.
References
Daniel Alencar da Costa, Shane McIntosh, Christoph Treude, Uirá Kulesza and Ahmed E. Hassan 2018. The Impact of Rapid Release Cycles on the Integration Delay of Fixed Issues. Empirical Software Engineering 23, 2018, pages 835-904.
Carlos D. A. de Almeida, Diego N. Feijó and Lincoln S. Rocha 2022. Studying the Impact of Continuous Delivery Adoption on Bug-Fixing Time in Apache's Open-Source Projects. Proceedings of the 19th International Conference on Mining Software Repositories, 2022, pages 132-136.
Nesara Dissanayake, Asangi Jayatilaka, Mansooreh Zahedi and M. Ali Babar 2022. Software Security Patch Management—A Systematic Literature Review of Challenges, Approaches, Tools and Practices. Information and Software Technology 144, April 2022, article 106771.
Adam D. G. Jenkins, Linsen Liu, Maria K. Wolters and Kami Vaniea 2024. Not As Easy As Just Update: Survey of System Administrators and Patching Behaviours. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, 2024, article 972.
M. M. Lehman. Programs, Life Cycles, and Laws of Software Evolution. Proceedings of the IEEE 68(9), September 1980, pages 1060-1076.
Elvan Kula, Ayushi Rastogi, Hennie Huijgens, Arie van Deursen and Georgios Gousios 2019. Releasing Fast and Slow: An Exploratory Case Study at ING. Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2019, pages 785-795.
Maria Salama, Rami Bahsoon and Patricia Lago 2021. Stability in Software Engineering: Survey of the State-of-the-Art and Research Directions. IEEE Transactions on Software Engineering 47(7), 2021.
.jpg){kind=link}