Time, Cost, Quality and Agile Consulting

Types of Consulting Engagements

“Sometimes it's a little better to travel than to arrive” 
― Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values

In my software consulting experience I have been engaged in many different kind of projects, but in all cases they fall into two main categories:

1. Time and Material (T&M)
2. Fixed Price (FP)

In most situations it happens than to decide that option #2 is achievable, then a quantity of T&M analysis must be performed in advance, in order to define the context and the scope for a possible, successive FP engagement. That's not always possible: for example, when the project is part of a public tender, you have to bid for the lowest possible price, trying to balance the need for adding a good amount of contingency, staying into the safe path, without self-sabotaging the possibility of winning the tender.

What marks the difference between T&M and fixed price? In T&M a customer is basically paying for your time, because deliverables and scope can't be clearly set in advance, or because it's already established that requirements are going to change in a way that a Fixed Price engagement is out of question, because it's too risky. A Fixed Price project is based on a set of much more stringent assumptions, in terms of context, requirements, functional and technical, which (hopefully) allows for a very accurate estimate of deliverables.

Usually customers are more keen on FP because, of course, they think it will constraint the final price by putting much more responsibilities on the consultant, while T&M seems a way to create a continuos stream of expenses. However, in reality, there is a more fundamental law which regulates any kind of software project, despite the rules of engagement, and it is related to the existing and unavoidable strong relationship among three distinct, fundamental quantities: Time, Cost and Quality.

The Basic Conjecture of Time, Cost and Quality

“When analytic thought, the knife, is applied to experience, something is always killed in the process.” 
― Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values

It's almost incredible how many people actually think they can leverage Fixed Price to be in total control, at the same time, of these three quantities:

1. The elapsed time spent from start to finish, so the final delivery date;
2. The total cost, in terms of direct money and indirect materials;
3. The overall quality of the final product.

This assumption has been historically proven false by practice, for any non-trivial software project or consulting engagement. It's a conjecture and not a theorem or a physical law, but reality has taught me that, in software development and software consulting, it is possible to accurately be in control of only two over three of these quantities.

Some examples are needed: if a project has a fixed delivery date and a fixed price, then the only left quantity one can possibly control is quality. Would it maybe explains why so many FP projects suffer from poor perceived quality?

On the other hand there is another fundamental speculation which states that "Nine women can't deliver a baby in one month". It means that, if delivery dates and quality are fixed, one is tempted to keep adding resources, in terms of people and infrastructure, loosing control about costs. In practice this tactic even leads to also increasing delivery time, because adding people on a late project usually delays it even more.

A third case is when we try to fix both cost and quality, but then we accept that elapsed time can't be predicted accurately. This is the case, for example, of companies trying to outsource development to offshore facilities, where cheaper labor force can easily be hired. Statistically this strategy has led many projects to both an indefinite development time but also poorer quality.

It's all about one single truth: software development is inherently not a traditional engineering activity. Actually, Programming is Gardening, not Engineering. 

Agile Development and Agile Consulting

"Simplicity is the ultimate sophistication". ~ Leonardo da Vinci.

So, at first sight, it seams there is no escape from the T, C & Q rule. But we are not doomed. In fact, as I wrote before, the rule applies for any non trivial software project. So the trick here is: to transform big projects or big consulting engagements into a finite sequence of very focused, well defined, little activities or mini-engagements. This is why time-boxing or feature-boxing usually work effectively, and that's what actually Agile Methodologies are, more or less, trying to achieve: transforming complexity into something more manageable and predictable, by splitting big activities into little, possibly trivial, short tasks, which can be handled in few hours or days by very few people.

I think that agile methodologies can and should be successfully applied also to pure software consulting, so to the kind of engagement usually performed in T&M. The main pillars of this strategy are nothing new and can be summarized as:

• Focus on User Stories;
• Short iterations, usually no more than two or three weeks long;
• Continuos Integration and Continuos Delivery of valuable pieces of software;
• Acceptance tests at the end of each iteration or when a single deliverable is ready.

As a side note: if you are strictly required to be on-site then that is not a Fixed Price project by definition! Fixed Price engagements MUST be off-site, exactly because you don't want to waste time renegotiating the scope each single moment. It is necessary to have customers involved daily and keep things very flexible, but asynchronous interruptions must be avoided at all costs. The main objective is to understand what final users want and adapt when requirements are changing, but this must be addressed by the process, not individuals. T&M is very different: as customer pays for your time he is entirely entitled to interrupt you and change tasks even in the middle of them. That's why committing on any detailed deliverable in a T&M assignment is extremely dangerous.

The Need for Good Architectural Decisions

"We are searching for some kind of harmony between two intangibles: a form which we have not yet designed and a context which we cannot properly describe." ~ Christopher Alexander.

The missing piece, for a Use Case made of several User Stories, is a comprehensive and reasonably complete Technical Architecture. In other words, I do believe in that kind of bottom up, emerging software design coming from an Agile, iterative process, but I think this must be developed within the frame of a clear up-front Architecture.

I mean that refactoring code and design is not only necessary, but even desirable. Then, in my experience, refactoring wrong initial architectural decisions can be extremely expensive and usually leads to big failures. I strongly believe that the role of an experienced architect is key to produce quality software systems, and this fact sounds to me to be often too underestimated in the field of Agile Methodologies. Do not fool yourself by believing your so-called "rockstar developers" (horrible term!) alone can also actually imagine, design and implement a complete and working technical architecture.

Speaking of consulting, even a short T&M engagement should be performed in the context of a well designed architecture, because even the best expert can be unable to deliver anything useful if the architectural context is broken. Focus your first steps at customer site on two main things: understanding their existing architecture and development process, and start fixing them if they are clearly broken. Otherwise the risk of failing and not get paid will be too high, despite the fact it's T&M or FP.