Agile - Should we all?

In the world of software it seems that it should be expected, no, encouraged that people come up with new ideas to make software more useful. Computing systems are no longer simple or can be conceived of in one effort.

With that idea in mind, why doesn't every project run in an agile manner? Well, I think it's down to the fact that people want to be able to budget, and constrain their costs. That's also of high priority, and completely understandable as an approach.

What happens then? Well, it seems that projects often have the higher levels of management, and those in charge of the purse strings winning the standoff. What happens then, is that the project becomes one that has all requirements defined up front, so that they can be detailed, and costs attached to them.

What happens now? Well, unfortunately, the prject is now forced into a waterfall type, up front design project. What that means is that either the client can't change their mind, and so can't evolve their ideas, or the project has to do work for new features without a budget. Is that good? From the point of view of the project is probably is, but the end product is going to suffer as a consequence.

Can we run a project an constrain it to a budget, and a project end date? You could do, but you would probably have to put limits on a budget for each iteration, which will in turn put a limit on the features. That limitation is imposed by the budget, so there's little we can do to limit that situation. However, what we do by running the project in this way is to allow the project to evolve and improve. Now, that's a good thing isn't it?

Why is it then, that most projects are not agile projects?

New Architecture for legacy systems

I don't know what it is, but so many architects that I meet in the various projects in which I work are so focussed on writing new solutions with new technology, that they often completely miss the problem in the first place.

I've seen enthusiastic architects often propose solutions that replace the easy part of the solution, and make it "better" (when often it isn't broken) and then not even attempt, or only partially attempt to address the real problem.

Our anti-pattern is to replace all this stuff we currently have with new stuff, and still have the same fundemental problems as before.

Why instead, can't we take much more of an agile, iteratative approach, and replace things little by little, and slowly move away from one solution to a more mature one, without once putting all our eggs in one basket with a particular "buzz" technology?

What this whole process often involves is moving from one bad decision, to a new bad one, or worse still starting implementing a solution with a new technology, that by the time it is finished, is an old technology, and we iterate with the same.

To my mind, architecture is more about an approach, a pattern, that is agnostic of the technology that may be used by it. If we work toward a solution that looks like a pattern but instead uses a particular technology that completely undoes that pattern by coupling everything through the technology, then it seems a completely useless and expensive exercise.

A big area that seems to suffer from this particular ailment, is SOA and the web service based solutions. Often, I hear that the reason for choosing a web service based solution is because of a need to make that solution available, particularly to other platforms. Later, I find out that the solution is intranet based, and could have been implemented a lot better without the need at least initially for web services at all. We then get hacks based on that concept to achieve various things, largely borne out of the need for "cool" technology. Things such as WSE are such an example. Whilst there is defintely a place for these technologies, that unemotional choice is often not correctly made, in favour of a curriculum vitae boosting entry instead.

There really is a good case for technology aware-but-agnostic architects out there, my experience is that they are few and far between. An architect should be exploring all possibilities, rather than a set that suit their purposes, or career.

Unit testing, Test Driven Development (TDD)

It still amazes me that there are so many projects out and about that have lots of lines of code, interoperating, and communicating systems, and no unit tests.

I don't know how projects get out in time, or even slightly slipping without having any confidence as to how much of the solution actually works.

I would have expected all greenfield projects, and some new additions to software to be able to be built using TDD, and be able to be refactored, and for delivery of the project to be able to be done with confidence.

I know that it often scares people to be using MVC and MVP patterns and dependency injection, but I know that it would scare me more to be in charge of a project that didn't have any unit testing.

Another thing, of projects that I have worked on that have had unit tests, they're often not unit tests, and are instead system or integration tests. Who cares what they are, right? If you run an integration test, and the server goes does, does that mean that your code has failed to pass its requirements? Will it take you an age to figure out if the problem is environmental including lots of debugging, and so on? Quite possibly, as there's so much involved in the real system, so many dependencies. Ultimately you need those tests, but to be able to write solid code, unit tests are what you need.

What else intrigues me is that you hear so many people talking Agile, and yet so few actually show any evidence of being capable of doing it. It seems implausible to run an agile project without unit tests to constrain your planned and unplanned changes to code.

I heard often of people complaning that test driven development takes a lot of overhead, well yes it does, but in the long term, that will mean less time to finish the project, unless it is the most simple project you've ever seen, and even then, I think you could argue a good case for TDD.

I know there are people out there who use Agile and TDD type methodologies, please show yourselves, and make yourselves known to me, and to the world at large!