Odi's astoundingly incomplete notes
New entries | Codein software: old != stable
We have now all learned that Debian ships a horribly outdated xscreensaver. We have seen similar things in SuSE, so it's not just Debian. And we have also heard that the package maintainers rather paper over that fact instead of shipping a current version.
Debian has that concept where they bless some random version of a software as "stable". Everything else is automatically declared "unstable". Stable for them means: the package maintainers backport some patches they deem "important" to that old version.
It was also the time of dial-up Internet, static web pages, no mobile phones, POP accounts, news on paper. No forums. Just mailing lists with no easy way to search. You were basically alone with an old book on the C language.
That basically meant, ever growing code bases became a mine field. It was unmaintainable. It started bitrotting everywhere. Litte changes here and there could easily break code in interesting places.
It was the time of Never Touch a Running System. It was the time of everybody being scared of updates. Change freezes over new years'. Extensive testing before some little change could make it into production. Literally months from writing code to deploying it into production! Then if something want wrong, exceptions to that rule to bring a dearly needed fix in quickly.
Huge projects have finally become maintainable. Also structure intrinsic to a language like the package concept of Java, or namespaces in C# help enourmously with organizing large code bases so that you can find your way around. Garbage collection eliminates the need to think too much about memory allocation, which would otherwise be real PITA and a source of many problems in a large system.
This change also leads to greatly increased development speed. The amount of changes that go into projects each day has exploded in the last decade. If you look into a project like OpenStack, the sheer amount of concentrated development power is simply mind boggling.
We have finally come to the conclusion that life is a lot easier if you can get changes into prod as fast as you can, instead of waiting for months. That's the complete opposite of the old paradigm. Yes, that will release bugs into production sometimes. But this way they also get noticed very quickly, and can be fixed speedily. No big deal. Everybody is happy. How many updates of your apps on your smart phone do you get per day? Notice something? Online services like Facebook or Netflix simply don't have maintenance windows. They are permanently online every single millisecond. They release new versions of code several times a day. Nobody even notices.
By the way, have you noticed how people seem to be so much happier about a fixed bug than about a completely new feature? New feature means they need to adapt and learn something new. Fixed bug means they can finally do what they meant to do.
Suddenly those package maintainers find themselves in a situation where they need to decide which patch of possibly hundreds each day to backport. And they face a large code base that they are very unlikely to understand as they are not developers of that software. So how can they actually do this job? Short answer: they generally can't. There are also numerous examples of when package maintainers failed. They thought they could just "fix" something in OpenSSL better than the developers. Surprise, turns out it was a security disaster.
Obviously that concept of keeping an old version forever is not useful anymore. It needs to be replaced. Some Linux distributions like Gentoo have long adopted a rolling release model with no major releases. This fits the new paradigm perfectly. It allows following upstream releases very closely.
Of course simply adopting a rolling release model is no guarantee for uptodate packages either. Also Gentoo has stable/unstable branches (keywords), which means whether integration of the package within the soup is considered complete. Gentoo took 6 years to deem Apache 2.4 "stable".
Also it is insufficient to update only certain packages of a system. Systems must keep current on all packages. For instance only updating apps and keeping an outdated kernel is therefore unsafe.
This has also immediate effects on backward compatibility. It should allow the kernel to break user space from time to time by introducing an incompatible change. Of course this would trigger a cascade of updates to user space. But in a rolling release model this should be a supported scenario. Either adapt to the change or become obsolete!
Debian has that concept where they bless some random version of a software as "stable". Everything else is automatically declared "unstable". Stable for them means: the package maintainers backport some patches they deem "important" to that old version.
Where does that paradigm come from?
From the last century. No, really! There was a point in software history where available system memory became large. So software started becoming larger, simply because you could do more stuff. At the same time the development environments and development processes lacked badly behind that evolution. Can you remember Pair Programming, eXtreme Programming? People were either using Visual C++ 3, or vi to write code. Some even Notepad. There was no infrastructure like static code analysis, continuous integration, basically no unit tests, no coverage analysis. The most you could get was syntax highlighting and CVS.It was also the time of dial-up Internet, static web pages, no mobile phones, POP accounts, news on paper. No forums. Just mailing lists with no easy way to search. You were basically alone with an old book on the C language.
That basically meant, ever growing code bases became a mine field. It was unmaintainable. It started bitrotting everywhere. Litte changes here and there could easily break code in interesting places.
It was the time of Never Touch a Running System. It was the time of everybody being scared of updates. Change freezes over new years'. Extensive testing before some little change could make it into production. Literally months from writing code to deploying it into production! Then if something want wrong, exceptions to that rule to bring a dearly needed fix in quickly.
Fast-forward to today
We have so much infrastructure that crunches the hundreds of thousands of lines of code without us even looking, that points out when something breaks. We have found ways to manage stunning amounts of code without getting lost. We have continuous integration with solid code coverage.Huge projects have finally become maintainable. Also structure intrinsic to a language like the package concept of Java, or namespaces in C# help enourmously with organizing large code bases so that you can find your way around. Garbage collection eliminates the need to think too much about memory allocation, which would otherwise be real PITA and a source of many problems in a large system.
This change also leads to greatly increased development speed. The amount of changes that go into projects each day has exploded in the last decade. If you look into a project like OpenStack, the sheer amount of concentrated development power is simply mind boggling.
Current is the new stable
Today bugs are fixed quickly. Often it takes mere minutes after a bug report until the bug is fixed in the code. And mere hours or days until it goes into production.We have finally come to the conclusion that life is a lot easier if you can get changes into prod as fast as you can, instead of waiting for months. That's the complete opposite of the old paradigm. Yes, that will release bugs into production sometimes. But this way they also get noticed very quickly, and can be fixed speedily. No big deal. Everybody is happy. How many updates of your apps on your smart phone do you get per day? Notice something? Online services like Facebook or Netflix simply don't have maintenance windows. They are permanently online every single millisecond. They release new versions of code several times a day. Nobody even notices.
By the way, have you noticed how people seem to be so much happier about a fixed bug than about a completely new feature? New feature means they need to adapt and learn something new. Fixed bug means they can finally do what they meant to do.
Suddenly those package maintainers find themselves in a situation where they need to decide which patch of possibly hundreds each day to backport. And they face a large code base that they are very unlikely to understand as they are not developers of that software. So how can they actually do this job? Short answer: they generally can't. There are also numerous examples of when package maintainers failed. They thought they could just "fix" something in OpenSSL better than the developers. Surprise, turns out it was a security disaster.
When software dies
Also software is not maintained forever. Sometimes development simply stalls. Because everybody left the project and had children. Or one of various other reasons. That's the time when also users of that software need to find something else. They will need to stop using it. Software is not alone. It runs embedded in a soup of other software. Sooner or later it will become incompatible with the rest of that soup and actually stop working anyway. Or security issues that were found pile up and make this piece of software a dangerous item.Obviously that concept of keeping an old version forever is not useful anymore. It needs to be replaced. Some Linux distributions like Gentoo have long adopted a rolling release model with no major releases. This fits the new paradigm perfectly. It allows following upstream releases very closely.
Of course simply adopting a rolling release model is no guarantee for uptodate packages either. Also Gentoo has stable/unstable branches (keywords), which means whether integration of the package within the soup is considered complete. Gentoo took 6 years to deem Apache 2.4 "stable".
Who doesn't move dies
All the above has interesting consequences if software is never updated. Such as in embedded systems. Such systems become dangerous items with time and must be disposed of.Also it is insufficient to update only certain packages of a system. Systems must keep current on all packages. For instance only updating apps and keeping an outdated kernel is therefore unsafe.
This has also immediate effects on backward compatibility. It should allow the kernel to break user space from time to time by introducing an incompatible change. Of course this would trigger a cascade of updates to user space. But in a rolling release model this should be a supported scenario. Either adapt to the change or become obsolete!
Add comment