Works Best Practices Program -
Recommendations of Audit Committee
The Works Committee recommends the adoption of the recommendations of the Audit Committee embodied in the
following communication (July 20, 1999) from the City Clerk, subject to:
(1)deleting Recommendation No. (2); and
(2)adding to Recommendation No. (3) at the end thereof the words "including value for money":
Recommendations:
The Audit Committee recommends that:
(1)the report (July 12, 1999) from the City Auditor be amended by deleting the words, "and report directly to Works
staff" and inserting in lieu thereof the words, "be accountable to the City Auditor and report concurrently on any issues to
both the City Auditor and Works staff" in Recommendation No. (4), and that the report, as amended be adopted;
(2)the City Auditor perform an analysis of the consultant services for the Works Best Practices Program to date,
including value for money of Phase 1 of the program; and
(3)the City Auditor not be precluded from commenting on Phase 1 of the Works Best Practices Program while following
through with any of the recommendations outlined in the report.
Background:
The Audit Committee, on July 19, 1999, had before it the report (July 12, 1999) from the City Auditor respecting Works
Best Practices, and recommending that:
"(1)the Commissioner of Works and Emergency Services establish for each consultant contract award, a performance
management process, incorporating appropriate parameters and relevant, measurable and objective benchmarks; and that
any future consulting contract awards recommended to the Works Committee and Council include a description of the
consultant performance management criteria and process to be used;
(2)prior to or in conjunction with awarding any future consulting contracts, the Commissioner of Works and Emergency
Services report to the Works Committee on the total cost of consulting services required for this project, including those
consulting services required in the implementation of process control systems and other facilities;
(3)prior to or in conjunction with awarding any future consulting contracts, the Commissioner of Works and Emergency
Services clarify how the consulting services to be provided in Phase 2 of the Works Best Practices Program differ from
those rendered during the planning and design stage (Phase 1) as well as from those consulting services to be procured for
the construction and implementation component;
(4)any quality assurance activities deemed necessary in Phase 2 of the Works Best Practices Program be performed either
by Works staff or, if a consulting firm is used, that the firm be independent of the consulting consortium and report directly
to Works staff;
(5)in view of the magnitude of the Works Best Practices Program, the City Auditor work with Works staff during the
implementation phase of the program to ensure that appropriate performance management criteria are established, and
report independently to Council on any concerns or issues, including areas relating to value for money; and
(6)this report be forwarded to the Works Committee for consideration."
During consideration of this matter, the Committee also had before it a communication (July 19, 1999) from Toronto Civic
Employees' Union, Local 416.
Ms. Gina Gignac, National Representative, Toronto Civic Employees' Union, appeared before the Audit Committee in
connection with the foregoing matter.
The Committee's recommendations are noted above.
(Report dated July 12, 1999, addressed to the
Audit Committee from the City Auditor)
Recommendations:
It is recommended that:
(1)the Commissioner of Works and Emergency Services establish for each consultant contract award, a performance
management process, incorporating appropriate parameters and relevant, measurable and objective benchmarks; and that
any future consulting contract awards recommended to the Works Committee and Council include a description of the
consultant performance management criteria and process to be used;
(2)prior to or in conjunction with awarding any future consulting contracts, the Commissioner of Works and Emergency
Services report to the Works Committee on the total cost of consulting services required for this project, including those
consulting services required in the implementation of process control systems and other facilities;
(3)prior to or in conjunction with awarding any future consulting contracts, the Commissioner of Works and Emergency
Services clarify how the consulting services to be provided in Phase 2 of the Works Best Practices Program differ from
those rendered during the planning and design stage (Phase 1) as well as from those consulting services to be procured for
the construction and implementation component;
(4)any quality assurance activities deemed necessary in Phase 2 of the Works Best Practices Program be performed either
by Works staff or, if a consulting firm is used, that the firm be independent of the consulting consortium and report directly
to Works staff;
(5)in view of the magnitude of the Works Best Practices Program, the City Auditor work with Works staff during the
implementation phase of the program to ensure that appropriate performance management criteria are established, and
report independently to Council on any concerns or issues, including areas relating to value for money; and
(6)this report be forwarded to the Works Committee for consideration.
Background:
On March 2, 3, and 4, 1999, City Council had before it a report from the Works and Utilities Committee entitled "Works
Best Practices Program and Projected Staffing Levels", In considering this report, Council requested that the City Auditor
report through the Audit Committee on the advisability of performing a value for money audit on consultant services for
this program, and the costs thereof.
Comments:
The Works Best Practices (WBP) Program is a long-term improvement initiative undertaken by the Water and Wastewater
Services Division of the Works and Emergency Services Department. The WBP Program is an expansion of the Water
Pollution Control Division's (former Metro Works Department) Strategic Information Systems Plan (SISP), initiated in
1995 to address strategies for the replacement of aging and obsolete systems and equipment in the wastewater treatment
plants.
A Request for Proposals (RFP) was issued in 1995 to engineering consulting firms to assist in the implementation of the
SISP project. In February 1996, the former Metro Council approved the award of a contract for $14.4 million to a
consortium of 10 consulting firms, led by EMA Canada. The agreement with EMA Canada, who was responsible for the
overall management of the services provided by the contract, including the procurement and payment of specialized
technical services provided by the other consulting firms in the consortium.
To further maximize the effectiveness and efficiency of the Division's operations, the planned technological improvements
of the SISP plan were broadened to include the application of best practices in the industry in areas such as total productive
operations, workforce flexibility, unattended operations, and program-driven maintenance. The essential elements of the
WBP Program were redesigned work methods in all areas of the business, with specific emphasis on labour-intensive
operations and maintenance activities, fully supported by integrated operations, maintenance and management systems.
In June 1997, Metro Council approved the amendment of the contract with EMA Canada to incorporate additional work
activities associated with the WBP Program, specifically to extend the program to the Water Supply and Solid Waste
Management Divisions. Council also approved a $6.1 million increase in the value of the contract with EMA Canada. In
December 1998, City Council approved a further contract increase of $979,000.00 to complete Phase 1 activities.
A competitive process was not followed with respect to the above-noted contract extensions which increased the contract
value from $14.4 million to $21.5 million. Works staff felt that, based on the work already completed and the knowledge
obtained by the EMA-led consortium, and to ensure project continuity and consistency among the divisions, it would be
more cost-effective to retain EMA as opposed to issuing a new RFP. They also indicated in their report to Council that, in
light of the projected operating savings from the WBP Program, the three to four-month delay caused by the issuance and
processing of a Request for Proposals would have resulted in between $10 million and $15 million in permanently lost
future operating cost reductions. While we agree that extending the contract to EMA Canada would take advantage of
EMA's acquired knowledge and better ensure process continuity, we are not convinced that the permanently lost savings
from the delay caused by requesting proposals for the additional services required, is a reasonable argument.
Given the nature of the services provided, the requesting of proposals from qualified consulting firms is an important
means of ensuring that value for money is achieved. Consequently, considering the significant value of the contract
extension, a competitive process with specific deliverables defined should have been followed for the additional work
activities relating to the Works Best Practices Program. This would have at least provided a level of assurance that the
amounts charged for the additional services and deliverables required, were reflective of the market.
As at December 31, 1998, approximately $21 million in consulting fees have been incurred in Phase 1 of this project,
relating mainly to preliminary planning and design work. The costs of the various consulting services are provided below:
ActivityCost
Overall Project Management$2,195,963
Quality Assurance635,400
Organization Architecture2,104,900
Integration Architecture933,700
Technology Architecture2,328,813
Work Management Practices1,133,250
Systems Design and Implementation Plan798,078
Process Control System Design4,308,400
Corporate Administrative Work Plan122,825
Performance and Operations Management Design1,044,100
Industrial Waste Requirements257,600
Laboratories Requirements516,550
Redesign Workshops2,381,017
Expenses412,250
Subtotal19,172,846
GST1,342,099
Total20,514,945
Payments to the consultant were based on specified deliverables. Approximately 200 deliverables have been signed off to
date. Some of the major deliverables from Phase 1 of the WBP Program include: overall project plan; technology
infrastructure report; preliminary design of organizational structure; performance and operations management requirements
and design document; installed network and communications components; preliminary process control systems design for
wastewater treatment plants; process control systems technical standards; best practices training and documentation; and
facilitation of business process redesign workshops.
The 1999-2003 Capital Works Program includes $110 million for the WBP Program. Approximately $70 million of this
amount relates to the replacement and modernization of aging and obsolete equipment and facilities, which Works staff
indicate would have been required regardless of the WBP Program. Included in this $70 million is the cost of consulting
services required to implement the various process control systems and facilities. The balance of the $40 million is mainly
for general consulting services relating to the WBP Program, $21 million of which has already been spent for preliminary
design and planning activities. Works staff indicate that the remaining $19 million is required in Phase 2 of the program to
provide program integration and coordination services such as program management, technology integration, process
control system implementation management, organization design and integrated work area implementation.
Given the magnitude of consulting expenditures incurred and to be incurred on this program, it is important that the costs
of all consulting services be identified and the need for these services justified. It is also important that the responsibilities
of the consultants and the services to be provided are clarified relative to services provided in Phase 1, as well as those
consulting services directly related to Phase 2 implementation activities, so that Council can clearly distinguish the need for
the different consulting services required in the various components of the project.
Works staff estimate that upon full implementation of the WBP Program and using 1996 as the base year, the Division will
realize annual net operating savings of $36 million, mainly in the form of staff reductions, reduced hydro consumption and
lower chemical requirements. In order to determine the extent to which these estimated savings are realized, it is
imperative that proper mechanisms be put in place to account for the savings that are directly attributable to the program.
Works staff indicate that they are in the process of developing a benefits tracking system to account for the savings related
to the program.
Conclusion:
Implementation of the WBP Program represents a significant investment by the City. The acquisition of external resources
to implement this program was deemed necessary because not all of the required technical skills and industry expertise
were available internally. Further, in-house resources were required to maintain current operations and could not be
redeployed on a full-time basis to work on the WBP Program. It was also felt that an independent consultant would provide
a more objective assessment of the current operations. Given the magnitude and complexity of the project, Works staff
decided that the use of consulting services was necessary and provided the best opportunity for the timely completion of
the project and successful implementation.
Phase 1 of the project involved the delivery of various soft services relating to preliminary planning and design activities.
Based on the information available, it would be possible to ascertain whether the specific deliverables were achieved and at
the budgeted cost. However, because of the intangible nature of these deliverables and the lack of appropriate benchmarks
for measuring performance, it would be difficult to assess the value of the consulting services provided, especially since the
benefits of the project will not be known until the project is substantially completed.
The consulting consortium included a quality assurance consultant, whose responsibility was to ensure compliance with the
pre-established process and procedures. This consulting firm was a member of the consulting consortium and was paid by
EMA Canada, the lead consultant. To be effective, the quality assurance function should be independent of the consulting
consortium and report directly to Works staff.
City Council recently reaffirmed its commitment to the WBP Program, with the expectation that the significant investment
in the program will be recovered through future savings, mainly in the form of lower staffing levels. Council's recent
decision to not lay off any staff directly resulting from the WBP Program implementation until after December 31, 2000,
will impact on the Works and Emergency Services Department's ability to achieve the planned operating cost reductions
within the scheduled timeframe. Redeploying staff to other divisions will be difficult as these divisions are also
downsizing. Consequently, the department's ability to downsize prior to the year 2001 will be limited to attrition or other
programs such as voluntary separation and early retirement.
The fact that the initial consulting services were procured following a competitive process, provides some level of
assurance that the consulting fees relating to the SISP component of the project reflect the market. However, a competitive
process was not followed for the additional services required for Phase 1 of the WBP Program. As such, it would be
difficult to determine whether a fair and reasonable price was charged for the deliverables relating to this component of the
project since the market was not tested. A competitive process has been followed for the procurement of program
integration and coordination consulting services for Phase 2 of the WBP Program.
Based on our preliminary review, it is estimated that it would take approximately three months to complete a value for
money audit on consultant services for the Works Best Practices Program. Due to the intangible nature of the deliverables,
any assessment of the consultant's services would be to a large extent subjective. This combined with the fact that the
program is still in the design stage and many of the benefits have yet to be realized, would make it difficult to perform an
effective audit on the consulting services entered to date. However, given the significant dollars being spent on consultant
services for this program, it is essential that Works staff provide Council with a complete and clear picture of the different
types of consulting services required. It is also important that appropriate benchmarks are developed against which to
measure the performance of each consultant and that a performance management process be incorporated into the
consultant contract. Given the magnitude and complexity of this project, Audit Services will be working with Works staff
to assist in developing performance management criteria and to ensure that significant issues are addressed and report to
Council as necessary.
Contact Name and Telephone Number:
Tony Veneziano, Senior Audit Manager, 392-8353
(A copy of the communication from the Toronto Civic Employees' Union, Local 416, referred to in the foregoing
communication has been forwarded to all Members of Council with the agenda for the Works Committee meeting of
October 6, 1999, and a copy thereof is on file in the office of the City Clerk.)