I quite clearly recall a conversation with my PhD co-supervisor, Paul Larson (now at Microsoft Research), when I was a graduate student about documenting failures and not only successes in Computer Science. Paul felt quite strongly about this, and I agreed with him. A significant reason behind the need to document failure is that Computer Science, in ways similar to Engineering, is a field in which the same ideas are re-used and re-invented over and over again.
Documenting failure is also a significant recommendation of Donald G. Reinertsen in his book Managing the Design Factory, a book that I have mentioned previously in this blog. Here’s an excerpt from Reinertsen’s book (page 79):
The fallacy in thinking that high first-pass success optimizes the design process lies in underestimating the important information generation that occurs with failure. This tendency to treat failure as the enemy is relatively new; people have not always considered failures to be negative. For example, consider the statement of Robert Stephenson, one of the great engineers of the early industrial revolution. As Henry Petroski points out in his book “Design Paradigms” [nb. a followup book is now available, Success Through Failure: The Paradox of Design], Stephenson strongly advocated discussing failure in engineering literature. In 1856 he wrote: “A faithful account of those accidents, and of the means by which the consequences were met, was really more valuable than a description of the most successful work”.
Why the need to document failures? Here is a subsequent excerpt from the same chapter:
This [handling of failure] is difficult for us to do well because we have strong human bias to value successes more than we value failures. In most organizations failure is stigmatized and nobody wants to be associated with it…..Unfortunately this produces some dangerous side-effects. Since improbable failures have high information content, it is important to communicate information about failure quickly and widely throughout the organization. To the extent that we hinder the flow of this information, we will force people to reinvent failures that we have already experienced, and that generates no useful new information.
Precisely this question – why don’t we as a scientific community document failure – was the subject of a fascinating program on CBC Radio’s Ideas that was broadcast this past Monday, 15 September. On this program, entitled Science at the Summit, noted researcher and University of Toronto president David Naylor interviews a panel of leading scientists about this question, and the many implications that lie behind the stigma of research failure. It is a compelling program, and I recommend listening to the show’s podcast which will likely be available on the CBC Ideas website at the end of September.