The different types of project review each have their own characteristics and benefits. For any review however it is important to decide what the overall purpose is, and who should gain what from the output. This step is missed out in many cases and the design of the review is not given sufficient attention. Standard methodologies for carrying out project reviews can be helpful, but must be supplemented by intelligent thought! In the author's experience a review should consider both the project management standards and the subject matter of the project. Mistakes in either or both of these can lead to disaster, and it can take considerable skill and knowledge to uncover the truth.

If you are responsible for a project's success then you need the truth, and you are responsible for setting the principles and scope that will govern how effective the review will be in uncovering that truth. In particular you need to pick the reviewer so that you understand what bias is likely to be reflected in the results. Bias can arise from background, review methodology or political interests; and may not be intentional.

Bias due to interests

The review may been commissioned, designed or just operated by people who wish to prove certain results and disprove others. It may be that some wish to prove a particular methodology works, and therefore the review concentrates on the extent to which that methodology has been completely and fully employed. This may be completely different from assessing whether the project itself is likely to succeed or fail. The Prince2 manual for instance has a comprehensive healthcheck. This will check for the complete use of Prince2 but the questions are mostly of the 'is there...item X' type, and there are very few questions asking about the quality of any of the items whose existence is being checked. Jobs and career progress may be at stake and informal alliances may be formed to ensure that the "correct results" are obtained by the review.

Bias due to role

The UK government OGC gateway review is an externally focused health check which concentrates on the extent to which the project is still required and could deliver the benefits sought by stakeholders. It is also quite reasonably designed to protect a purchaser ( because the UK government almost always has this role ), rather than the creator or supplier of the project. This affects the questions and can lead to a situation in which the project is affirmed as meeting all the criteria for the purchaser but is doomed as a result of events being suffered by the supplier ( e.g. commercial losses, staff turnover etc. )

Bias due to reviewer's background

There can be subtle issues arising from the background of those carrying out the project review. Some reviewers will be much more technologists than project managers, whilst others will be the reverse. Although it makes competent practitioners much harder to find there is a good case for insisting on both sets of skills.

Bias due to distance

Reviews are sometimes undertaken by those who inhabit a management world of reports. If they then communicate with project managers and others who are somewhat distant from the actual tasks then the overall effect may be that the blind are leading the blind. Remember that real people, somewhere, should be actually doing the work to deliver the tasks required by the project. A competent reviewer needs to find them, understand them and discover what they think. This necessitates not only people and managerial skills but also some understanding of the skills employed by the project staff.

Checklists

These are useful for ensuring appropriate coverage but it is equally important that the reviewer can think freely and explore the implications of the information that the project team give. This information is normally messy and ill organised. Coherent results pointing in a single direction are unlikely and the reviewer needs to be able to cope with

So what should the balance be ?

Project Management skills in medium or large projects are certainly critical as the need to be organised is a critical success factor. 'Ready aim fire' expresses the simple truth that without those competent to first aim the project team members in the correct direction there is little chance of the target being hit. It is important that clear business justification and rules for project control are created and maintained during the life of the project. Appropriate risk management and planning are also vital to enable the overall project to achieve its intended results.

That the reviewer also possesses a reasonable level of subject related understanding is also critical so that those who are undertaking the work can be questioned rigorously about the quality and assumptions underlying the work they are undertaking. This isn't about the facility to do the work, but the ability to discuss designs, build dependencies, test conditions and the like in the detail that will establish whether the sequence of work intended is viable.

Reviews can be very important, but deciding who should carry them out, and how they should be undertaken is not simple. The most important determinant of success is your choice of reviewer.