My reference to Adam Savage in Part III was not just incidental. I think it is a very profound one, especially in software development.
Adam Savage, in a later podcast (unfortunately, I was unable to find it) explains how the phrase “Failure is always an option” represent a fundamental fact about scientific enquiry. Unlike we see in many movies where mad scientists work like crazy and then be heart broken after their experiment “fails”, scientific enquiry, especially that of experimental enquiry thrives on failure. There might be a favored outcome for an experiment, but if the real outcome is different, it provides data. Failures in many cases provide vastly more data than success. Even when one gets expected results, it will most likely be falsified later by someone else.
It is hard not to notice the similarity between this and software development.
Every claim about a software is eminently falsifiable.
While, as a software user, many of us are faced with the mysterious ways of working of an application. But, this is neither due to a supernatural intervention or from a Heisenbergian uncertainty. It is just simple classic phenomenon of not having enough information about the inner workings of the software. However, while developing a software, one cannot really appeal to ignorance. Many things in software development resembles Murphy’s law in steroids. Things are guaranteed to go wrong, and they will always go wrong in detectable ways.
Software development is always considered and thrived to be an engineering discipline. This is why we try to create one engineering process after another to make it behave more like other engineering projects. But, the history of these tight engineering controls is at best dismal, and even when they worked, they did it by curtailing innovation and creativity to the extreme. Agile/Extreme programming in many ways was a response, resistance movement if you will, against this tyranny of process. It concentrates on the human element (like the Dow chemicals commercial – but then they went and bought Union Carbide), and creativity. Instead of trying to control and limit change, agile methodologies embrace change.
Successful software development demands a lot of intellectual commitment from the people involved. It is more like pursuing a scientific experiment. Here, we have this hypothesis. What is the best way, in terms of representational accuracy, maintainability, and overall usability to model it! There is always a multitude of choices. Optimality of one of these choices is unlikely to be clear in the beginning of the process. So we have to start with hypotheses and empirically prove that the assertion is either true or false. Irrespective of which answer we get, the data we collect during this process define the problem in a better light. We get to define more variables and get the values for more constants. May be we have to go back to the proverbial “drawing board” and adjust our original hypothesis, except that the drawing board here is a constant addition to workspace. One difference from a scientific experiment though is that, at the end of even a partially successful experiment, we get something more tangible.
This is the spirit of scientific enquiry. This is why I think software development should be treated less like an engineering discipline and more like a research activity.
Agile in many ways does this. It unseats many of the mechanistic visions of earlier methodologies. By focusing more on the team dynamics and accepting change – constant change – as a welcome phenomenon. I am sure many of you are familiar with the old adage that the cost of change in software development increases exponentially as it progresses, which results in the axiom that we should try to reduce change, and capture as much as possible in the beginning. This is a wrong premise. Usual dynamic is that the user will find more things to change as the feature/component nears completion. Users may find many of their original assumptions were not accurate. We can always shut the user down until we announce that everything is done, and then tell them to live with what they have just like they are with their last home improvement job. There was a time when this would have worked. But users now understands more about software and its nature. We can no longer afford to blame the user for everything that went wrong… “they changed the specification, they don’t know what they want!”
There are some parallel efforts at resurrecting the engineering credentials of software. One such attempt is Intentional Programming. One assumption I had earlier was that the current problems in software development is just because it is a new industry and will eventually find its true calling. However, the nature of software, that of modeling real world scenarios, makes it unlikely that this will happen soon. The complexity of human society, individuals, interactions, even that of our artificial systems like banking and finance are so great, and our ability to model them, or even understand them is still in a very very early stage. Software, which tries to create virtual worlds, information models about them, and sometimes even helps create this understanding is bound to be complex and tentative.
That takes us to the next parallel between software development and scientific enquiry. Tentativeness of the solutions we create. There are so many factors that will reduce the overall usability of the system and create obsolescence from change in practices, advent of new hardware or software technology, changes in social expectations etc. Even when we successfully produce a model that satisfied the requirements, one has to constantly question the viability of that model. This could be a new human computer interaction paradigm like multi-touch or Kinect, ubiquity of small form factor devices, change in financial regulations, expectation of connectivity with the rest of the digital world, disappearing boundaries of office and home etc. Just like there are no sacred theories or laws in science, there are no sacred software. There are no eternal killer feature. There are no “only I can do this”es.
What this means is that, if I don’t poke holes in my model, someone else would. And if that someone else is a customer with the cudgel of a Service Level Agreement, it could bring us a world of hurt. So, the best way is to do this proactively. The main function of a software developer is to think about how to break what we have done, how to negate the hypothesis, how to falsify what we just proved.
So, go ahead, break your Fitnesse tests. Break the build and if you cannot fix it within the day, buy Donuts (or Parippuvata) for the whole team. As long as you take the code to a better place, it will all be forgotten.
The Salim Nair Band 