Agile - Should we all?

In the world of software it seems that it should be expected, no, encouraged that people come up with new ideas to make software more useful. Computing systems are no longer simple or can be conceived of in one effort.

With that idea in mind, why doesn't every project run in an agile manner? Well, I think it's down to the fact that people want to be able to budget, and constrain their costs. That's also of high priority, and completely understandable as an approach.

What happens then? Well, it seems that projects often have the higher levels of management, and those in charge of the purse strings winning the standoff. What happens then, is that the project becomes one that has all requirements defined up front, so that they can be detailed, and costs attached to them.

What happens now? Well, unfortunately, the prject is now forced into a waterfall type, up front design project. What that means is that either the client can't change their mind, and so can't evolve their ideas, or the project has to do work for new features without a budget. Is that good? From the point of view of the project is probably is, but the end product is going to suffer as a consequence.

Can we run a project an constrain it to a budget, and a project end date? You could do, but you would probably have to put limits on a budget for each iteration, which will in turn put a limit on the features. That limitation is imposed by the budget, so there's little we can do to limit that situation. However, what we do by running the project in this way is to allow the project to evolve and improve. Now, that's a good thing isn't it?

Why is it then, that most projects are not agile projects?