Over the past few years I’ve seen an increase in articles and posts about whether or not to do estimation (of cost, schedule and effort) for software development projects. This is especially true when agile/iterative methods are used to develop software for which requirements are not readily known in advance. There are actual “movements” set up to prove that estimating in and of itself is bad for software development. At the same time, I’ve worked done more and more work for clients related to software benchmarking (to find best-in-class methods, tools, and combinations to develop software) and estimation (including price-to-win estimating.) I’m now convinced that “To estimate or not to estimate?” is simply the wrong question – or at least a premature question for many companies.
Estimation is often viewed as fundamental to software development (and any other development projects or programs) as are ingredients to cooking or oxygen to life. While we might wish to discard or discredit the practice of estimation as an inconvenience and even the reason for software “failures” –(Sidenote: The Standish group’s annual CHAOS reports cite lack of “on-time” and “on-budget” software delivery as rationale for declaring project failure; both of which would disappear as factors if estimating was eliminated) – the truth is that C-level executives need a level of confidence (based on estimates) to bound their investment in new initiatives, no matter how much faith or confidence the executives have in the development teams’ ability to deliver. In my humble opinion, project managers MUST develop skills to do solid, reliable project estimates if they are to survive (and thrive.) But this is where things often fall apart – estimation is not seen as a discipline based on solid data (in part, because some organizations do estimating haphazardly based on bad data, poor models, flawed assumptions, premature input values taken as fact, among other factors.)
This does not include those organizations where the mere notion of projects (being a temporary endeavor intended to deliver an identified product, outcome, or service such as a piece of software) is like a foreign language. When I teach courses according to the Project Management Institute’s Project Management Body of Knowledge (PMBOK(R)), it’s not uncommon to find IT pros who profess that project management is not needed because their work is bounded solely by calendar months and the number of full-time-staffers. The idea that work should be managed towards a specified outcome (with goals, objectives, timelines, milestones, deliverables and a formal end) just doesn’t fit into their paradigm, even for those involved in developing advanced technology solutions. I’m excluding these companies because projects (and estimating cost and schedule) are actually beyond their comprehension, as is productivity, project comparisons or process improvement.
Given the premise that “to estimate or not to estimate” is the wrong (or at least a premature) question – then what are the right ones? Here’s a short list:
- If we do an estimate, do we know what are the correct input variables (and values) we should use? (i.e., Some idea of scope, non-functional requirements, constraints, goals, project environment, etc.) Garbage in equals garbage out.
- When estimating, do we have access to correct and appropriate historical data on which to rely? (i.e., does the historical completed project information accurately depict what actually happened on the project? Often up to 40% of true project work effort is not recorded – or it is recorded inconsistently.) Incomplete or incorrect historical data make for poor comparisons, and even worse estimates.
- Are the estimating models we propose, appropriate for the industry and application? (i.e., in construction, it would be folly to use a home building model for a hospital construction or bridge construction project, so too with software.) Every model, no matter how advanced, needs to be calibrated for the organization using it.
- Do we know enough about the object of estimation? (i.e., if it is simply an idea about an outcome without any idea of component programs or projects, a “guess”timate or rough-order-of-magnitude may be the only possibility until more data are known.)
- Are the estimating exercise/practices paid “lip service” by management? (i.e., does management summarily cut every estimate in half, or dictate due dates that override those of professional estimators?)
- Does the organization take (software) measurement seriously? (i.e., how are project measures and metrics collected – if adhoc, inconsistent, without formal processes or procedures to validate the quality of project data, then estimating will likely be equally inconsistent)
These are just a few of the important questions that need to be addressed – before we attempt to estimate and rely on the results of the practice. When estimating is done without proper planning, discipline and consistency, the results will be unreliable and even worse, downright wrong.
In IT as in life, if you’re going to invest in an endeavor (such as estimating), take the time to do it right the first time, or don’t bother doing it at all. And that, really answers the question of “to estimate or not to estimate.”
What other questions are critical to ask? What do YOU think?