Project Audits

Most of the processes your organization uses have audits and so should your project management processes. It's just as important to audit your projects as it is to audit your quality assurance processes. The PMBOK describes the audit as a "structured independent review to determine if project activities comply with organizational and project policies, and procedures." There are many different views and approaches to project audits out there but we're going to hue to the PMBOK approach with this article. This approach imitates the approach used for financial audits where the auditors are a firm contracted for the purpose. While it is possible to bring outside auditors in to perform your project audit, it is also possible to identify and train internal auditors. Should your organization boast a Project Management Office (PMO), or Project Management Center (PMC), that group should provide the service.

Keep the objective of the audit in mind when conducting your project audit. The objective is not to determine the health of the project, or identify corrective actions that will improve project performance, but to determine whether the project is being managed correctly. A project audit may be performed during the project life cycle, or after the project has been completed. A mid-project audit has the advantage of potentially correcting project management mistakes for the audited project whereas an audit at the end of the project won't help the current project but the audit points will benefit the entire life cycle of future projects.

The audit should start with any standards or guidelines that have been established for projects undertaken by the organization. These may be established by a PMO or PMC, or by the organization. An examination of the documented processes and plans for the project will tell you whether standards are being met and guidelines followed. The auditor should also examine how well the project is following the project plans and processes.

The Report
The output from the project audit should be an audit report which captures all the observations from the audit. The report should capture each point from the audit. Each point should be uniquely identified and the description of the point, recommendations for correction, and degree of severity should accompany the point. The degree of severity might be a number on a scale of 1 to 10, or an ordinal indicator such as (L)ow, (M)edium, (H)igh, or (E)rror, (I)mprovement, or (S)uggestion. The report should also have a comments section where your general observations on the project are recorded. You may be called upon to give the project a pass or fail in which case you need to clearly capture the criteria for pass or fail. Make the pass or fail dependent on the total score being below a set point when you use numerical scores for audit points. Make the pass or fail dependent on a maximum number of Errors when that approach is used. The pass or fail criteria should be agreed upon with the people commissioning the audit and be clearly spelled out in your report. This report is what you will present to the stakeholders who commissioned the audit and the project manager.

Starting Out
The project manager for the audited project should be engaged from the outset. This should not be an adversarial process, the results of the audit should not be used to punish the project manager unless the project manager has deliberately contravened standards and guidelines. The audit should be used as a learning opportunity and the project manager should facilitate the auditor's access to all the project documentation they require. Start the audit with a meeting with the project manager and sponsor and spell out the purpose of the audit (to improve project management performance across the organization) and the how the audit will be performed. Detail what you expect from the project manager, what the audit will produce (the audit report), and how that will be shared.

The auditor's first step should be to identify all the processes and procedures that should govern the management of the project. This information should be available from the PMO, PMC, or project manager. The first document to review will be the Project Charter, Statement of Work (SOW), or preliminary Scope Statement. This document should provide you with an overview of the key deliverables and milestones set for the project. The Project Charter should also tell you the approach the project manager intends to use to deliver on the charter objectives. This information should be used as a benchmark to measure the effectiveness of the project processes and procedures. Do they support delivery of the objective on or before the deadline? An absence of either a charter or an SOW should be noted; any project should have one or the other, or both before initiation unless some other document has been substituted in its place.

WBS and Schedule
Work Breakdown Structure (WBS) and schedule have been included in this section because of the prevalence of MS Project, which combines these 2 plans. MS Project is by no means the only tool available to perform these functions but it has become so prevalent that we're assuming that that's the tool used for the project being audited.

The key deliverables from the Project Charter should be captured in the WBS/Schedule and the work should be broken down starting from these. The key milestones should also be captured in the MS Project file. If the project is being governed by a SOW rather than, or in addition to, a charter the deliverables and milestones from the SOW should be the high level schedule entries. When you identify any missing deliverables or milestones, check with the project manager to determine if they are represented by a different entry. Frequently terminology is abbreviated in these documents such that the deliverable or milestone they were derived from is unrecognizable. Check for missing milestones in the same fashion. Make note of any deliverables or milestones that were missed in the MS Project file.

Detail planning of the entire project may not be present in the MS Project file if the audit is being performed in the middle of the project and the project is being managed using a "rolling wave" or iterative approach (this should be noted in the Project Charter or SOW). Use judgment when determining the state of completeness of the MS Project file. Work for the next iteration or phase must be planned before the next gate review but how far ahead this happens will be at the project manager's discretion.

Work that is planned must have planned start and end dates and should have a person, or people, assigned to the task. They may have predecessors, lead or lag times and a number of other attributes supported by MS Project. Work in the MS Project file that does not have a person assigned should be reviewed with the project manager and noted if no person has been assigned. The MS Project file should be kept up to date. Work that is completed should be marked as 100% complete in the file and work that is partially completed should have its degree of completeness recorded. Look for slipped start dates, these will be dates in the past with 0% completeness. Tasks with slipped start dates should be re-scheduled with new forecast start and end dates. Note that the MS Project file may not be set up to capture planned and forecast dates. Tasks with slipped end dates should also be reviewed. Any tasks with end dates in the past that are less than 100% complete should be reviewed with the project manager to determine whether the task is 100% complete and simply not updated in MS Project, or whether the PM missed updating MS Project. The MS Project file will never be completely current with the project, but it should be no more than a day or two out of date. An over-abundance of out of date information in the MS Project file should be noted in your report.

Gate Reviews
These may be called something else by the performing organization such as Phase Exit reviews, Business Decision Points, Kill Points. Each of these that the project has passed should be accompanied by artifacts. The following is a list of artifacts that may be produced:

o A list of criteria for passing the gate
o The gate decision
o A list of meeting attendees
o A list of decision makers
o The process for reaching a decision
o A pass or fail for each gating criteria
o The decision
o Actions assigned during the meeting and their status

Each gate review should have, at minimum, a list of the criteria and whether each passed or failed and a copy of the decision. Any criteria that failed should have an action to correct it. These actions should be captured in an Action Register or Issue Log, along with any other actions identified during the meeting. The project manager may have used a separate register or log to track these items, or they may have used the project's log or register but the items related to the gate should be identified with the gate. Items that are not closed should have a forecast close date in the future. Note those that don't and review these with the project manager. Items which have been neglected should be noted.

The gate held to close out the project should be accompanied by a letter or e-mail of acceptance from the customer or client. This acceptance may happen during the Gate Review meeting where the project is undertaken for an internal client, for example when the IT organization creates a financial system for the Finance department. The acceptance of the system by the support organization should be clearly spelled out in the gate communication in that case.

Project Plans
The project manager may have captured all the plans for the project in one document, or they may have captured them in several documents. Areas of the project which may be planned are:

o Change
o Communications
o Cost
o Human Resources
o Procurement
o Quality
o Risk
o Scope or Requirements

These plans should be reviewed to determine the degree to which they follow the approach described in the Project Charter. Discrepancies between any of these plans and the planned approach should be reviewed with the project manager. Review the plans for the actions they call for and then compare these actions to the MS Project file. Have the actions been captured in tasks in that file? For example, when the Risk Management plan calls for a workshop to identify and quantify risks during the project planning phase, has that workshop been scheduled? If the workshop was scheduled, was it held and does the file show it as 100% complete? Work should be scheduled in the MS Project file for any actions called for by the plans. Review any actions that aren't supported by planned work with the project manager and note any failures.

Change Register/Change Requests
Most projects will have a process that governs how requested changes to the project are managed and most should have a register to record the change requests the project receives. Projects that aren't supported with a change register will have the requests archived. The register and individual change requests should record the nature of the change, some sort of business case for the change, the date the request was received, the proposed solution, the impact of the change on budget, schedule, quality, and scope objectives, whether the request was approved or rejected, who made the decision, and a planned implementation date for approved change requests. Change requests should also be accompanied by the communications called for by the Change Management Plan. Check that each request was accompanied by the appropriate communication and make note of any that were not. The paper trail for rejected change requests will end here.

Change requests which were approved may or may not change one of the 4 baselines for the project. Change requests which don't change any of those baselines can be ignored by the audit. Any change requests which call for a change to one or more of the baselines will also change one or more of the plans: MS Project (schedule, new work, changed work, deleted work), Communications Management (new, changed or deleted communications), Cost Management (budget), Human Resources (additional workers, workers removed from the plan), Procurement (a change to the outsourced work or list of vendors), Requirements Management (new, changed, or deleted requirements), and Risk Management (a change to the risks being managed by the project). A change that changed any of the baselines should be reflected in a change to one or more of the plans and a new project baseline. Any changes which should have triggered a change to the plans and baselines and did not should be reviewed with the project manager.

Risk Register
Risks should be captured and managed using a register. The following information may also be captured there:

o A description of the risk event
o Whether the risk is accepted or mitigated
o How the risk will be mitigated (if it is to be mitigated)
o The action or actions required for mitigation
o Any contingency plans to address consequences of the risk event
o MS Project file id of the action
o Status of the risk

Review the risk register to determine whether it is current with the project or not. Risks should be reviewed periodically (in accordance with the Risk Management plan and project schedule), new risks identified, obsolete risks retired and status updated. Check the register against the Risk Management plan to determine if the planned activities have taken place. Check the work in the MS Project file where the register refers to it to ensure that actions called for by the Risk Register have been completed. Review any discrepancies with the project manager.

Action Registers or Issue Logs
Issues and actions that occur throughout the project should be captured and tracked in an action register or issue log. These documents should tell you the nature of the issue, the action identified to address it, the person responsible for the action, and the planned completion date of the action. Review the register or log to ensure that all actions have been completed when you are performing the audit at the end of the project, or that actions planned before today's date have been completed in the case where the audit occurs during the project. Review any slipped actions with the project manager to determine whether the action hasn't been completed or whether it was completed and the log or register was not updated. These documents should be kept up to date so if they are more than a week out of date you should make note of the fact.

Drafting the Report
Audit points which were recorded during the course of the audit should be transcribed to the report. A description of the audit point should be captured along with the information described in the Report section of this article. Use your judgment when assigning a severity rating to the point. You should have some criteria in mind for a numerical rating - what constitutes a 10, an 8, a 1, etc. Points which you identify as errors will need to be fixed ASAP if the audit was performed mid project, or on the next project. Errors, Improvements, and Suggestions should be accompanied by a description of the action which would correct or improve project management performance.

Your comments should capture your overall observations of the project and any suggestions for improvement. Now is the time to evaluate the project manager's performance in the area of keeping plans up to date. You may want to identify a separate audit point to cover this area, or simply pass comment in the comments section of your report. Projects where the plans and schedules are badly out of date should prompt a suggestion for improvement.