July 26, 1999

To:Audit Committee

From:City Auditor

Subject:Works Best Practices

Recommendations:

It is recommended that:

(1)the Commissioner, Works and Emergency Services, establish for each consultant contract award, a performance management process, incorporating appropriate parameters and relevant, measurable and objective benchmarks; and that any future consulting contract awards recommended to the Works Committee and Council include a description of the consultant performance management criteria and process to be used;

(2)prior to or in conjunction with awarding any future consulting contracts, the Commissioner, Works and Emergency Services report to the Works Committee on the total cost of consulting services required for this project, including those consulting services required in the implementation of process control systems and other facilities;

(3)prior to or in conjunction with awarding any future consulting contracts, the Commissioner, Works and Emergency Services clarify how the consulting services to be provided in Phase 2 of the Works Best Practices program differ from those rendered during the planning and design stage (Phase 1) as well as from those consulting services to be procured for the construction and implementation component;

(4)any quality assurance activities deemed necessary in Phase 2 of the Works Best Practices program be performed either by Works staff or, if a consulting firm is used, that the firm be independent of the consulting consortium and report directly to Works staff;

(5)in view of the magnitude of the Works Best Practices program, the City Auditor work with Works staff during the implementation phase of the program to ensure that appropriate performance management criteria are established, and report independently to Council on any concerns or issues, including areas relating to value for money.

(6)this report be forwarded to the Works Committee for consideration.

Background:

On March 2, 3, and 4, 1999, City Council had before it a report from the Works and Utilities Committee entitled Works Best Practices Program and Projected Staffing Levels. In considering this report, Council requested that the City Auditor report through the Audit Committee on the advisability of performing a value for money audit on consultant services for this program, and the costs thereof.

Comments:

The Works Best Practices (WBP) program is a long term improvement initiative undertaken by the Water and Wastewater Services Division of the Works and Emergency Services Department. The WBP program is an expansion of the Water Pollution Control Division's (former Metro Works Department) Strategic Information Systems Plan (SISP), initiated in 1995 to address strategies for the replacement of aging and obsolete systems and equipment in the wastewater treatment plants.

A Request for Proposal (RFP) was issued in 1995 to engineering consulting firms to assist in the implementation of the SISP project. In February 1996, the former Metro Council approved the award of a contract for $14.4 million to a consortium of 10 consulting firms, led by EMA Canada. The agreement with EMA Canada, who was responsible for the overall management of the services provided by the contract, including the procurement and payment of specialized technical services provided by the other consulting firms in the consortium.

To further maximize the effectiveness and efficiency of the Division's operations, the planned technological improvements of the SISP plan were broadened to include the application of best practices in the industry in areas such as total productive operations, workforce flexibility, unattended operations, and program-driven maintenance. The essential elements of the WBP program were redesigned work methods in all areas of the business, with specific emphasis on labour-intensive operations and maintenance activities, fully supported by integrated operations, maintenance and management systems.

In June 1997, Metro Council approved the amendment of the contract with EMA Canada to incorporate additional work activities associated with the WBP program, specifically to extend the program to the Water Supply and Solid Waste Management Divisions. Council also approved a $6.1 million increase in the value of the contract with EMA Canada. In December 1998, City Council approved a further contract increase of $979,000 to complete Phase 1 activities.

A competitive process was not followed with respect to the above noted contract extensions which increased the contract value from $14.4 million to $21.5 million. Works staff felt that, based on the work already completed and the knowledge obtained by the EMA-led consortium, and to ensure project continuity and consistency among the divisions, it would be more cost effective to retain EMA as opposed to issuing a new RFP. They also indicated in their report to Council that, in light of the projected operating savings from the WBP program, the three to four month delay caused by the issuance and processing of a Request for Proposal would have resulted in between $10 million and $15 million in permanently lost future operating cost reductions. While we agree that extending the contract to EMA Canada would take advantage of EMA's acquired knowledge and better ensure process continuity, we are not convinced that the permanently lost savings from the delay caused by requesting proposals for the additional services required, is a reasonable argument.

Given the nature of the services provided, the requesting of proposals from qualified consulting firms is an important means of ensuring that value for money is achieved. Consequently, considering the significant value of the contract extension, a competitive process with specific deliverables defined should have been followed for the additional work activities relating to the Works Best Practices program. This would have at least provided a level of assurance that the amounts charged for the additional services and deliverables required, were reflective of the market.

As at December 31, 1998, approximately $21 million in consulting fees have been incurred in Phase 1 of the project, relating mainly to preliminary planning and design work. The costs of the various consulting services are provided below:

ActivityCost

Overall Project Management$2,195,963

Quality Assurance 635,400

Organization Architecture 2,104,900

Integration Architecture 933,700

Technology Architecture 2,328,813

Work Management Practices 1,133,250

Systems Design & Implementation Plan 798,078

Process Control System Design 4,308,400

Corporate Administrative Work plan 122,825

Performance & Operations Management Design 1,044,100

Industrial Waste Requirements 257,600

Laboratories Requirements 516,550

Redesign Workshops 2,381,017

Expenses 412,250

Subtotal19,172,846

GST 1,342,099

Total20,514,945

Payments to the consultant were based on specified deliverables. Approximately 200 deliverables have been signed off to date. Some of the major deliverables from Phase 1 of the WBP program include: overall project plan; technology infrastructure report; preliminary design of organizational structure; performance and operations management requirements and design document; installed network and communications components; preliminary process control systems design for wastewater treatment plants; process control systems technical standards; best practices training and documentation; and facilitation of business process redesign workshops.

The 1999-2003 Capital Works Program includes $110 million for the WBP program. Approximately $70 million of this amount relates to the replacement and modernization of aging and obsolete equipment and facilities, which Works staff indicate would have been required regardless of the WBP program. Included in this $70 million is the cost of consulting services required to implement the various process control systems and facilities. The balance of the $40 million is mainly for general consulting services relating to the WBP program, $21 million of which has already been spent for preliminary design and planning activities. Works staff indicate that the remaining $19 million is required in Phase 2 of the program to provide program integration and coordination services such as program management, technology integration, process control system implementation management, organizational design and integrated work area implementation.

Given the magnitude of consulting expenditures incurred and to be incurred on this program, it is important that the costs of all consulting services be identified and the need for these services justified. It is also important that the responsibilities of the consultants and the services to be provided are clarified relative to services provided in Phase 1, as well as those consulting services directly related to Phase 2 implementation activities, so that Council can clearly distinguish the need for the different consulting services required in the various components of the project.

Works staff estimate that upon full implementation of the WBP program and using 1996 as the base year, the Division will realize annual net operating savings of $36 million, mainly in the form of staff reductions, reduced hydro consumption and lower chemical requirements. In order to determine the extent to which these estimated savings are realized, it is imperative that proper mechanisms be put in place to account for the savings that are directly attributable to the program. Works staff indicate that they are in the process of developing a benefits tracking system to account for the savings related to the program.

Conclusion:

Implementation of the WBP program represents a significant investment by the City. The acquisition of external resources to implement this program was deemed necessary because not all of the required technical skills and industry expertise were available internally. Further, in-house resources were required to maintain current operations and could not be redeployed on a full time basis to work on the WBP program. It was also felt that an independent consultant would provide a more objective assessment of the current operations. Given the magnitude and complexity of the project, Works staff decided that the use of consulting services was necessary and provided the best opportunity for the timely completion of the project and successful implementation.

Phase 1 of the project involved the delivery of various soft services relating to preliminary planning and design activities. Based on the information available, it would be possible to ascertain whether the specific deliverables were achieved and at the budgeted cost. However, because of the intangible nature of these deliverables and the lack of appropriate benchmarks for measuring performance, it would be difficult to assess the value of the consulting services provided, especially since the benefits of the project will not be known until the project is substantially completed.

The consulting consortium included a quality assurance consultant, whose responsibility was to ensure compliance with the pre-established process and procedures. This consulting firm was a member of the consulting consortium and was paid by EMA Canada, the lead consultant. To be effective, the quality assurance function should be independent of the consulting consortium and report directly to Works staff.

City Council recently reaffirmed its commitment to the WBP program, with the expectation that the significant investment in the program will be recovered through future savings, mainly in the form of lower staffing levels. Council's recent decision to not layoff any staff directly resulting from the WBP program implementation until after December 31, 2000, will impact on the Works and Emergency Services Department's ability to achieve the planned operating cost reductions within the scheduled time frame. Redeploying staff to other divisions will be difficult as these divisions are also downsizing. Consequently, the department's ability to downsize prior to the year 2001, will be limited to attrition or other programs such as voluntary separation and early retirement.

The fact that the initial consulting services were procured following a competitive process, provides some level of assurance that the consulting fees relating to the SISP component of the project reflect the market. However, a competitive process was not followed for the additional services required for Phase 1 of the WBP program. As such, it would be difficult to determine whether a fair and reasonable price was charged for the deliverables relating to this component of the project since the market was not tested. A competitive process has been followed for the procurement of program integration and coordination consulting services for Phase 2 of the WBP program.

Based on our preliminary review, it is estimated that it would take approximately three months to complete a value for money audit on consultant services for the Works Best Practices program. Due to the intangible nature of the deliverables, any assessment of the consultant's services would be to a large extent subjective. This combined with the fact that the program is still in the design stage and many of the benefits have yet to be realized, would make it difficult to perform an effective audit on the consulting services rendered to date. However, given the significant dollars being spent on consultant services for this program, it is essential that Works staff provide Council with a complete and clear picture of the different types of consulting services required. It is also important that appropriate benchmarks are developed against which to measure the performance of each consultant and that a performance management process be incorporated into the consultant contract. Given the magnitude and complexity of this project, Audit Services will be working with Works staff to assist in developing performance management criteria and to ensure that significant issues are addressed and reported to Council as necessary.

Contact Name and Telephone Number:

Tony Veneziano, Senior Audit Manager, 392-8353

Jeffrey Griffiths, City Auditor

dl

A:\WBPP4.RPT