I agree completely – unfortunately in commercial development the “typical client” does not. So, we address the problem in our engagement model and our service level agreement – where we can set a risk/reward for reaching the project goal and a clear path for adapting the project baseline to meet the realities of discovery.

The point is to first – expose the problem which is what I did here in the article. Then, with everyone aware of the issues, we can approach the simple fact that specifications and our joint view of the project goals will evolve during development and they must. Understanding that – we can provide a framework for change and risk sharing that will guide us through the project.

The big risk is to pretend to predict the unpredictable by executing an experienced guess, methodologists are focused on the “silver bullet approach” pretending one process fits all, leading to use a methodology wrongly and skipping the adoption/tayloring step.

Predictability is desirable but it leads to situations where people build a plan early and they attach blindly to it, without considering all the real execution inputs and the plan begins to display something completely different to what the reality is, and once you notice this, the projects looks like a complete fail.

The recommendation is to focus on adaptability instead of predictability. Hope this makes sense. I wrote an article about it: “Software development, analogies and something more” it is in spanish, but I will publish it in english soon. http://www.sg.com.mx/content/view/775

But, in the type commercial development we do – we are in actuality a “software factory.” We also use Lean concepts which are in fact “translated” from manufacturing techniques and are quite in line with Agile concepts.

None of this changes the simple fact that we need to reliably estimate projects and control delivery to reach estimates or to acknowledge change and adapt the project baseline. Scrum works well when it establishes responsibility and collaboration with the client team. It is not as clear as to handling the change process in a contractual framework. That is a big part of what we address as our Service Level Agreement and I will write more about that in future articles.

The fundamental problem with software development management lies in the assumption that it can be managed using the same principles applied to manufacturing, namely deterministic process controls. Software construction takes place in very complex and unpredictable social contexts, far different than manufacturing context, production lines.

The resolution of most software construction problems is a creative process where the right solution emerges as the result of trial and error; even the simplest projects require research, learning and adjustments. Although it is expected that two items manufactured on a production line are practically the same, the same is not true in software construction; given the same problem, two competent software engineers are likely to produce different solutions, both addressing requirements just fine; how can we model that?

The alternative is the usage of empirical process controls. There is nothing new about this approach; it has been maturing since the early 70′s; it gained momentum in the 80′s with the Japanese auto industry applying it to new product development; and, it caught on with the software industry starting in the 90′s with the efforts of Ken Schwaber, Mike Beedle, and others. It is often referred to as SCRUM in the context of Agile practices. I’m a convert.