Even when a good governance structure is in place the structure of an IT project is more important than is often assumed. The structure can make estimating difficult, and make it impossible to deliver whilst retaining the confidence of the management and external stakeholders. Yet the structure is often assumed without proper debate - either with the customer or with the project team.

Many organisations and project managers still like to use the so called 'waterfall' method of controlling IT projects, because they believe it gives them greater commercial control, and allows fixed cost projects. This method splits up the total project into a series of logical steps :

- Work out what items should be done

- Estimate what these will cost.

- Agree these definitions and costs.

- Undertake formal design.

- Build the items.

- Test the items.

- Deploy and use the items.

The waterfall method can appear to offer a clear logical path from inception through to completion. The commercial control supposedly occurs because the facts are determined before agreement is reached, thereby allowing the following steps to be controlled within an agreed financial framework.

If the project uses 'well known' technology or is very similar to previous projects then the waterfall method can be successfully used. Where there is a substantial part that is new, or uncertain some serious issues exist with this approach.

In some cases the proportion of the total project costs and time that need to be spent before sufficiently precise estimates are generated can be substantial. Clients and suppliers can disagree whether this should this be free or paid for. In many cases a messy compromise is arrived at where the costs of the business analysis and estimation will be paid for ( or at least negotiated about ) after it is decided whether the project is to be accepted.

In other cases it isn't certain that people are good at thinking through complex systems with sufficient accuracy to make the design 'fit for purpose', prior to some good 'proofs of concept'.

Quite apart from the difficulty of identifying the 'facts'; the fixing of features up front can lead to an 80/20 split where 80% of the costs produce 20% of the benefits and vice versa. This should give most clients cause for concern.

There can also be significant change in the business requirements during a project of 2 or 3 years duration. This has encouraged the move to Rapid Application Development ( RAD )

Good estimates are difficult

It is difficult to produce good estimates. Increasing the granularity can lead to estimating safety margins being accumulated ( if say the minimum estimating unit is 1 day - then something that took 1 hour would be recorded as 1 day giving a 7 hour accumulation per task of this length).

Estimates need to be distinguished from use of historical data to give timings. But the latter need to be very carefully annotated and used to ensure they are being used properly.

Perhaps the thought that 'estimation should be democratically controlled, and execution involve authoritarian methods rather than as often happens the other way around'; is one of the most useful maxims here.

Acceptance of the facts

It is very difficult to know the extent to which the ultimate customer actually has bought into the design and estimates being produced. There may in fact be no acceptance other than of 'that is the way our supplier chooses to do things' giving the opportunity for later disputes.

In some projects it can be very difficult to establish who is entitled to sign off documents as correct on behalf of the organisation. This makes any approach other than time and materials problematic.

Rapid Application Development ( RAD )

Rapid development techniques are now accepted in many cases. These methods involve using a number of iterative prototypes are built to give the customer greater control over the finished result. Thus the analysis, design, build and test sequences of the whole project is split into a number of successive cycles.

There are, however, unanswered questions about the range of IT projects for which RAD is suitable, and issues about whether all of the suggested elements of RAD are as important as one another. For instance some approaches time box 3 to 6 week periods, whilst others simply allow incremental delivery over a number of months. Yet others regard the complete incorporation of automated unit testing into the coding ( build ) stage as perhaps the most important contribution.

There are numerous variations on RAD, including pairing - where two programmers work together with one writing tests and the other writing code to satisfy those tests. There is still a real need however to prioritise the efforts.

Some of the serious criticisms of RAD are that it can allow scope creep, lack of rigour, and cost overruns. It can be particularly difficult to stop 'gold plating' on particular easily understood elements (e.g. the user interface) at the expense of underlying functions that are more complex and less easily explained or understood by business people ( when the latter are involved in the assessment of each iteration ).

Here again, therefore, we encounter evidence that the structure of the project can have substantial impact on the viability of the development.

There are a number of problems therefore with both traditional and RAD methods of controlling custom and or complex IT projects. Many projects would be improved if expert help and more time were spent restructuring the project at the start to help the IT supplier clarify the design choices and the IT client clarify the business requirements. Whilst this does involve an explicit acceptance that money will be spent 'investigating' and 'researching'; this is actually nothing more than bringing present good practice out into the open.