Programming is a gas. I don't mean programming is entertaining and exciting (even though sometimes it really is); I mean more along the lines of the physical state of matter. Programming is a gas. You see, a gas having perfect mobility and infinite expansion will take the size and shape of the container you put it in.
Parkinson's Law states that "work expands so as to fill the time available for its completion." I'm taking this one step forward and saying that "software development expands so as to take the size and shape of the constraints placed upon it."
I've worked with tens (if not hundreds) of software engineers and I can tell you one thing with certainty: ceteris paribus, the more the engineer charges for services, the better the result. Now, obviously, some engineers misrepresent their skill level intentionally or unintentionally, but . . . on average . . .
So, if this is the case, you can put several of these programmers together and derive that the more you pay a software consultancy, the better the result as well. I know to some, this probably just sounds obtuse. You might be saying to yourself, "well, some companies just have higher profit margins than others." 'tis true; some programmers have higher profit margins.
If you've been at the top of the game for 10 years, you deserve more than someone who just started programming (or someone who has stagnated for 10 years) because you can apply all of your experience to each new project. Similarly, a software consultancy with premium engineers (no matter how old the organization) can earn a higher margin because its engineers produce such valuable results.
Understanding this stipulation (whether you agree or not) is pivotal to the rest of this post, so I'll very briefly summarize what I'm saying: you get what you pay for!
So, why do so many projects turn sour? If you're a software engineer, think of the last 10 projects you worked on. If you're not an engineer, think of the last 10 projects your company had someone else work on. What's the first step of all of these projects? Almost uniformly, it's an RFP and if not an RFP per se, it's some kind of competitive bidding process.
That's where the situation starts going south — before the project even gets kicked off! Have you ever had a project where the vendor wasn't selected almost exclusively based on cost? When was the last time someone said, "whichever vendor comes back with the best feature ideas will get the contract" or "whichever vendor has the lowest turnover rate and the highest employee satisfaction will get the contract?" In fact, in a perfect world, I believe you could select a vendor based on its mission statement and core values (and perhaps a brief summary of its gestalt experience).
Almost every project I've been on has had two constraints . . . money and time. Before a pen ever touches the signature line on the contract, the project is doomed. The shape of the project is defined by money and time, not by desired features.
Thus, when a developer looks at the project, it is going to take the shape of money and time; however, to the customer, the project is expected to be shaped by the desired features. More often than not, the money x time shape fits quite neatly inside the feature shape.
So, what exactly am I proposing? I'm proposing that the team (whether it is internal or external) should be chosen based on values and trustworthiness. A team you can trust is much more valuable than a team which is cheap. Take the team you can trust and then define the shape of your project based on the features you want to see. When you ask the team for an estimate, it's not to compete for the project; rather, the estimate is so that you'll know if you have the budget for the project in the first place.
If you do it this way, you'll get an honest estimate. Many software companies will low-ball estimates because they know that once you're committed to the project, you'll dedicate whatever additional resources are necessary to complete it. Ultimately, it'll end up costing more in the long run because of the way each "phase" will build on the "phase" before it compared to a solution that was well architected from the get-go.