Report 2018-108 All Recommendation Responses

Report 2018-108: California High‑Speed Rail Authority: Its Flawed Decision Making and Poor Contract Management Have Contributed to Billions in Cost Overruns and Delays in the System's Construction (Release Date: November 2018)

Recommendation #1 To: High-Speed Rail Authority, California

To ensure that the change orders it approves are necessary and that their costs are appropriate, the Authority should adhere to the guidance and estimates the oversight firms provide to it. If the Authority chooses to deviate from the oversight firms' recommendations, it should clearly document why it made those deviations.

1-Year Agency Response

In September, we provided examples of change orders where the Authority adhered to the PCM guidance, which is the recommendation, as well as the various levels of the Authority's Delegation of Authority for Design-Build Contracts, up through Business Oversight Committee approval.

Regarding Business Oversight Committee approval of change orders, the example change orders the Authority provided and references in its response did not include change orders that went before the committee, and therefore do not include documentation of the Committee's review for conformance with the oversight firms' recommendations. Although the change orders provided did not require the Committee's approval, each change order that does require Committee approval includes an accompanying Business Case that shows the PCM's recommendations, thus enabling the Committee to verify if the recommendations have been adhered to.

Also, in June 2019 the Authority's Board of Directors (Board) updated the CEO's delegation for contract management through delegation to manage contingency following the approved program schedule (baseline). The Authority's Delegation of Authority for Design-Build Contracts aligns with the delegation to the CEO approved by the Board.

Also, the Authority's Delegation of Authority for Design-Build Contracts requires appropriate review by state managers before any change order is approved. All change orders are submitted by the PCM, with one to four state managers (or the Business Oversight Committee for change orders greater than $20 million) reviewing the change order before it is approved. Thus, appropriate review of change orders by state management, including reviewing adherence to PCM recommendations, has been implemented.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

In our 6-month assessment, we noted that the Authority still had not demonstrated the Business Oversight Committee's review of change orders for conformance with the oversight firms' recommendations. As part of its 1-year response, the Authority for the first time provided an example of the committee using the Business Case tool it created for this purpose--though the change order did not deviate from the PCM recommendation.

The Authority also provided data indicating that, for the first six months after implementing the new procedures, it did not deviate from PCM-recommended dollar amounts for any change orders among its three construction projects. This kind of monitoring--and thorough follow-up when any such deviation occurs--will be critical to ensuring the issues we identified in our audit do not recur. This is especially true given the fact that, as we mentioned in our previous assessment, the Business Oversight Committee does not review all change orders.

6-Month Agency Response

The Authority provided documentation to show the Business Oversight Committee charter was revised and the Business Case template was updated to identify if there is a disagreement with the PCM. We also provided examples of change orders where we adhered to the guidance from the PCM.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

There has been no change in the status of this recommendation since the Authority's 60-day submission in January 2019. In our response to that submission, we acknowledged the charter revisions the Authority mentions here, but explained that because the Authority reported at the time that it had not diverged from any oversight firm recommendations, we were not yet able to determine that the Authority is adhering to its new requirements. The Authority indicated to us in June 2019 that it still has not diverged from any oversight firm recommendation since the time of our audit report; as such, our assessment remains the same as our assessment of its response at 60 days. Further, the oversight process the Authority has put in place does not apply to all change orders--only those that come before the business oversight committee. The example change orders the Authority provided and references in its response did not include change orders that went before the committee, and therefore do not include any documentation of the committee's review for conformance with the oversight firms' recommendations. As such, we are not able to assess the performance of any additional oversight structure not already in place at the time of our audit.

60-Day Agency Response

The Authority revised the Business Oversight Committee charter to require documentation when a governance committee overrules the PCM's recommendation. The revised Business Oversight Committee charter has been approved.

The Business Case, which is the document that is prepared for requesting a change to the Baseline or a proposed change order that requires approval by the Business Oversight Committee, has been updated to document the PCM's recommendation and cost estimates along with an explanation of any deviations. The Business Case has been implemented.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided the revised Business Oversight Committee charter, which states that the committee is to ensure that the Authority adheres to the oversight firms' guidance and should clearly document the reasons for any deviation from those recommendations. However, the Authority told us that it has not yet used the new change order proposal document for a case wherein the proposed change deviates from the oversight firm recommendation. Therefore, we are not yet able to determine that the Authority is adhering to this new requirement. Once we are able to review instances in which the Authority has appropriately justified deviations from oversight firm recommendations, we will be able to conclude that this recommendation has been fully implemented.

Recommendation #2 To: High-Speed Rail Authority, California

Before executing its next construction contract, the Authority should establish formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays. At a minimum, these prerequisites should identify specific benchmarks related to land acquisition, utility agreements and relocations, and agreements with external stakeholders, including impacted local governments and other railroad operators.

Annual Follow-Up Agency Response From November 2020

The Authority has established formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays. The prerequisites identify specific benchmarks related to land acquisition, utility agreements and relocations, and agreements with external stakeholders, including impacted local governments and other railroad operators. The Authority is using the prerequisites to plan and prepare for new construction beyond the existing Central Valley construction projects.

The provided Stage Gate Procedure, which has been incorporated into the Program Controls Manual, provides guidance to ensure that each construction project passes through a robust review process before the project can advance to the next stage.

Moreover, the diagrams in the procedure illustrate the process of ensuring that certain preconstruction activities (ROW, utility relocation, environmental and other governmental permits) are sufficiently advanced to enable construction to start without a contractor being subsequently impeded or delayed.

In addition, as evidence that the Authority is using the Stage Gate Procedure to plan and prepare for new construction beyond the existing Central Valley construction projects, the provided Bakersfield Extension (LGA) schedule has been developed (specific dates within the schedule may change, depending on funding availability). The schedule is evidence of a project-specific plan that ensures preconstruction activities are sufficiently advanced prior to proceeding to the next stage of project delivery. Adherence to the plan will ensure that construction contractors are not impeded or delayed by Authority-caused delays.

This recommendation to establish formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays has been fully implemented.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

We evaluated the Authority's stage gate process during previous reporting cycles. At that time, the Authority did not provide evidence of using the process for a project that would be applicable to all relevant components of our recommendation. As part of its current response, the Authority has for the first time demonstrated use of the stage gate process on such a project, albeit one that is not finalized. Specifically, the schedule indicates that the Authority will complete 85 percent of needed land acquisition before directing the contractor to begin construction, and that all utility relocations will be complete soon thereafter. Although the Authority notes in its response that specific dates may shift, if the Authority adheres to these thresholds, it should help prevent further avoidable cost overruns like those we identified during our audit.

1-Year Agency Response

The Authority provided documentation in September related to the Track and Systems procurement to demonstrate the utilization of the stage gate process, including the risk-informed contingency assessment. This documentation is evidence that the stage gate process, which establishes formal prerequisites for beginning construction to prevent avoidable cost overruns and project delays, has been used (as applicable to the Track & Systems contract).

California State Auditor's Assessment of 1-Year Status: Partially Implemented

We evaluated the Authority's creation of its stage gate process as part of our 6-month assessment. At that time, we concluded that the Authority's new policy indicates that a detailed plan should be in place to ensure prerequisites are met before procuring a construction contractor; however, the policy the Authority provided did not itself include such a plan or any of the specific thresholds listed in the recommendation. At the time, the Authority explained to us that it would not be able to develop such a plan until the path of its next construction segment,including environmental clearance, has been finalized.

In its 1-year response, the Authority indicates that it has implemented the stage gate procedure with the track and systems contract it is preparing to procure. However, the documentation the Authority provided in support of its claim of full implementation acknowledges that key aspects of the stage gate process are not relevant because the work being contracted will take place on top of current construction in the Central Valley. As such, the use of the stage gate process for this construction, while potentially beneficial, does not address the types of delay and costs issues we discuss in our audit report and does not demonstrate that the Authority has established the benchmarks that our recommendations references would be required "at minimum" for implementation. In order to demonstrate full implementation of this recommendation, the Authority will need to plan and prepare for new construction beyond the existing Central Valley construction projects.

6-Month Agency Response

The Authority has developed a stage gate process to ensure that each construction project passes through a robust review process before the project can advance to the next stage.

Moreover, the stage gate process ensures that certain pre-construction activities (right-of-way, utility relocation, environmental and other governmental permits) are sufficiently advanced before construction can start, this will prevent any future contract from being subsequently impeded or delayed by Authority activities.

Seven gates have been identified for a typical design-build project, with key deliverables and governance committee approvals for each gate to be successfully completed before the project can proceed to the next stage.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Our review of the stage gate process the Authority has developed indicates that a detailed plan should be in place to ensure prerequisites are met before procuring a construction contractor; however, the documentation provided does not itself include such a plan or any of the specific thresholds listed in the recommendation. The Authority explained to us that it will not be able to develop such a plan until the path of its next construction segment, including environmental clearance, has been finalized. As such, the Authority has not provided any additional data about anticipated thresholds or risk analyses since the time of the its 60-day response. Without developing and setting actual thresholds to ensure the Authority does not once again prematurely begin construction, which are central components of the recommendation, the Authority has not yet substantiated full implementation.

60-Day Agency Response

The Authority developed draft Baseline project work plans composed of discretely defined tasks. These tasks are linked together based on project delivery sequencing. Work plans are a tool to identify critical junctures and project milestones, or benchmarks including land acquisition, utility agreements and relocations, and agreements with external stakeholders, which will be used to make recommendations to the Business Oversight Committee and Project Delivery Committee - and then to the Board -- prior to execution of future construction contracts. The draft work plans are being circulated for review.

California State Auditor's Assessment of 60-Day Status: Pending

The Authority provided a schedule for an upcoming system segment that included the same types of sequencing analyses we reviewed during our audit and which we discuss on pages 30 and 31 of the audit report. The draft documents also included benchmark figures for some but not all portions of the segment, and some but not all of the issue areas we identify in our response. Further, it is not yet clear what analyses the Authority conducted to arrive at these specific benchmarks. When the Authority claims full implementation, we will be looking for evidence of relevant and complete risk assessments for all applicable benchmarks the Authority has set. We will also look for documentation in Authority policy that it is committed to meeting those prerequisite benchmarks before executing its next construction contract.

Recommendation #3 To: High-Speed Rail Authority, California

To better position itself to complete the three Central Valley projects by the December 2022 federal grant deadline, the Authority should improve its monitoring and evaluation of the oversight firms' risk assessment processes and should take steps to ensure that these processes are consistent across the three projects by May 2019.

Annual Follow-Up Agency Response From November 2020

The Authority has improved its monitoring and evaluation of the oversight firms' risk assessment processes and has taken steps to ensure that these processes are consistent across the three projects.

Risk update meetings and workshops for each of the Central Valley construction packages have been held and continue, resulting in updated project risk registers. The Authority has also hired a new program risk manager who is using the risk registers as a tool to mitigate major program and project risks.

The workshops, coupled with risk guidance provided by the program risk manager, have resulted in risk management consistency across all three projects. As documentation that the workshop process has been completed and that risk management consistency across all three projects has been achieved, the summaries of the quantitative schedule risk analysis performed for each project have been provided. Similarly, a quantitative cost risk analysis has also been performed for each project.

Regarding improving its risk management program overall, the Authority has reviewed and refreshed the entire risk exposure to the program during the previous six months and has developed a revised Estimate to Complete (ETC) for the 2020 Business Plan. The ETC is supported by detailed risk assessments, both qualitative and quantitative.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

In our one-year assessment of this recommendation's status, we noted that the Authority had not provided documentation of the Central Valley risk workshops. In its most recent response, the Authority provided documentation that workshops had taken place. The Authority also provided updated risk assessments for each of the three construction projects. It will be important for the Authority to continue to monitor the consistency and quality of the risk assessments for each of the three distinct projects as it continues to try and complete those projects before the December 2022 deadline.

1-Year Agency Response

Risk update meetings and workshops have continued, resulting in updated project risk registers. Updating the risk registers is a continuous process as project development and schedules are advanced. The Authority has also hired a new program risk manager who will focus on using the risk registers as a tool to mitigate the major program and project risks.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

The Authority has still not provided documentation that it completed the specific Centrally Valley risk workshops that it informed us were delayed at the time of its six-month response. Although we acknowledge risk assessment for each individual construction project is an ongoing process, our recommendation focused on ensuring consistency across all three projects, which the Authority's response does not fully address. Once the Authority provides documentation that the workshop process is complete and we are able to review evidence of the actions it produces for ensuring consistency across the projects, we will evaluate whether the recommendation has been fully implemented.

6-Month Agency Response

The Authority has hired an additional risk management resource to head program risk management. The Authority has updated its PCM Manual to provide more explicit guidance on risk management.

Also, the Authority has developed and updated risk management-related governing and guiding documents to ensure consistency of risk management processes across construction projects. The documents include the updated Risk Management Plan that defines roles and responsibilities within program and project teams for development of risk registers and mitigation plans, and a Risk Management Manual that further delineates the responsibilities for the PCM team in ensuring that risk management processes are implemented as well as for the Authority to ensure consistency of risk assessments across all projects. The entire risk team, including both resources at the program level and from the Central Valley construction projects, has collaboratively worked on the risk management-related documents. The Risk Management Manual also includes more guidance on Monte Carlo analysis.

The entire risk team is having risk update meetings and risk workshops to provide consistency for all program elements to update the project risk registers including cost-risk evaluation of the three construction projects. Upon updating the project risk registers, the major focus moving forward will be mitigation of risk events. Each risk workshop will have a narrative report and the accepted mitigation steps that will be tracked very closely in the Risk Management System.

Furthermore, the Program Controls Manual was updated to include a process for preparation of the estimate-to-complete in the Cost Forecasting Procedure.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

The Authority provided documentation demonstrating the changes it describes in its response, which it made effective on May 15, 2019. If adhered to, these new practices should help the Authority ensure that its oversight firms are providing it with the best and most up-to-date information about risk to the Central Valley construction projects, as well as the associated costs. After following up with the Authority, it informed us that the important risk workshop work for the three Central Valley construction projects is still ongoing. Once that process is complete and we are able to review evidence of the actions it produces for ensuring consistency across the projects, we will be able to evaluate whether the recommendation has been fully implemented.

60-Day Agency Response

The Authority has hired additional risk management personnel to oversee the Central Valley design-build construction projects. Also, the Authority is advancing its Cost-Risk Evaluation of these projects ensuring risk management and mitigation, and estimate-to-completion consistency, across the three construction projects.

The Authority is enforcing all policies and procedures related to PCM oversight. The PCM Manual is being revised to be more explicit on risk management. The revised draft PCM Manual is currently circulating for review and comment.

The Program Controls Manual is being updated to include a risk management section and will be circulated for review and comment.

California State Auditor's Assessment of 60-Day Status: Pending

Because oversight firms already played a significant role in the risk management process at the time of our audit, this recommendation focuses on the need to improve the Authority's monitoring and evaluation to ensure the quality of the oversight firms' performance in this area. The Authority provided draft procedures for conducting oversight firm evaluations, as well as an example evaluation form. We reviewed completed versions of these evaluation forms during our audit and determined that the Authority should take steps to ensure their consistency across the three oversight firm contracts, which it has not yet demonstrated. We will be looking for further evidence that the Authority has taken these steps, as well as the other improvements the Authority mentions in its response, when the Authority reports full implementation of this recommendation.

Recommendation #4 To: High-Speed Rail Authority, California

To enable policymakers and the public to track the Authority's progress toward meeting the federal grant deadline of December 2022, the Authority should, by January 2019, begin providing quarterly updates to the Legislature detailing the progress of the three Central Valley construction projects using an earned value model that compares construction progress to the projected total completion cost and date. The Authority should base these updates on the most current estimates available.

Annual Follow-Up Agency Response From October 2023

Working with our federal partners, we completed a thorough and lengthy negotiation of our grant agreement scope and schedule resulting in an amendment to our federal grant, which preserves the federal funding and extends the period of performance from December 2022 to December 2028. Amendment 7 to Agreement Number FR-HSR-0009-10-01-06 is included with our status submission. The updated schedule and budget was highlighted in the 2023 Project Update Report submitted to the Legislature in March 2023. Quarterly reporting to the Legislature on the Authority's progress toward meeting the December 2028 completion date will commence for the reporting of Quarter 4 of 2023 and be submitted in January 2024.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

At the time the Authority ceased its quarterly reporting, we were still working with Authority staff to ensure that its reporting sufficiently enabled policymakers and the public to track the Authority's progress. In implementing Recommendation #5, the Authority has secured later deadlines for the completion of the project, but our recommendation remains relevant. The Authority indicates that it will restart its reporting in January 2024.

Annual Follow-Up Agency Response From November 2022

Working with our federal partners, we completed a thorough and lengthy negotiation of our grant agreement scope and schedule resulting in an amendment to our federal grant, which preserves the federal funding. Amendment 7 to Agreement Number FR-HSR-0009-10-01-06 will be provided upon finalization. One of the key areas of agreement was to generally conform the ARRA Amendment 7 to Amendment 4 to Agreement Number FR-HSR-0118-12.

As reported to the Board in the October 2022 meeting, our construction schedules are being finalized and construction is anticipated to be completed for Construction Package 4 in spring of 2023, and in mid-2026 for Construction Packages 1 and 2-3.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

This recommendation remains partially implemented. Although the amendment the authority references in its response, once finalized, may change the current deadline for completing construction on the system's initial segments, it does not in and of itself remove the need for regular, quality reporting to the Legislature. This is consistent with our previous assessment of the authority's progress. Once the amendment is in place, we will reassess the authority's plans for its reporting.

Annual Follow-Up Agency Response From October 2021

Working with our federal partners, we completed a thorough and lengthy negotiation of our grant agreement resulting in an amendment to our federal grant. Amendment 4 to Agreement Number FR-HSR-0118-12 dated June 15, 2021, has been submitted separately. One of the key areas of agreement was an updated deadline of December 2026.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

After submitting its response, the Authority clarified that it has amended one of its two relevant federal grant agreements and is working with the federal government to amend the other. Therefore, it still must amend the other grant before the relevant 2022 deadline will be affected.

If and when that happens, there will continue to be value in the Legislature and other stakeholders being able to monitor the Authority's progress toward completion of the Central Valley projects, including its progress toward meeting its new deadline. As we explain in our previous assessments of this recommendation, the public reporting the Authority has performed to date does not allow policymakers or the public to easily determine whether progress is accelerating or decelerating. Therefore, we continue to assess this recommendation as partially implemented.

Annual Follow-Up Agency Response From November 2020

The Authority provides quarterly updates to the Legislature detailing the progress of the three Central Valley construction projects using an earned value model that compares construction progress to the projected total completion cost and date. The Authority bases these updates on the most current cost estimates available.

The Authority's presentation of data within its status reports shows the trends of construction progress (whether accelerating or decelerating) by using historical information. The ARRA Status Report provided includes a spending rate chart comparing the previous 12 months of spending to the previous 3 months of spending, as well as the required spending to meet the ARRA requirements. Moreover, the monthly Central Valley Status Report provided includes fully detailed graphs that include historical data and forecasts of project cost and progress. These graphs include earned value charts for overall program and ARRA scope of work, historical labor utilization, historical and planned expenditures, right of way parcels delivery status (planned and actual), and planned and actual structures and guideway construction status. This detailed data allows policymakers and the public to easily determine the status and progress of the projects as the ARRA deadline approaches.

Specifically, regarding tracking progress towards meeting the ARRA deadline within the reports, please see "Spending Rate to Date versus Spending Rate Required to Meet Baseline Completion Dates ($M/month)" on page 5 of the ARRA Status Report and "Remaining Expenditures to Achieve ARRA (DB + CP 5) - $ Millions" on page 11 of the Central Valley Status Report.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

The Authority continues to use the same reporting format that we evaluated during the six-month and one-year reporting cycles. As we concluded then, the Authority's current presentation of the data does not allow policymakers or the public to easily determine whether progress is accelerating or decelerating as the Authority approaches the federal grant deadline of December 2022.

In its one-year response, the Authority stated that it would "work with the Chair of the Peer Review Group and key Legislative staff to develop a clear and concise presentation of the historical information". However, no changes have been made. The Authority's reference instead to separate and distinct projections contained in other reports is not a replacement for providing needed clear and unambiguous information to policy makers.

1-Year Agency Response

The Authority provided the third quarterly report to key legislative staff on October 1, 2019. The report uses the same template as the prior quarter, developed in consultation with the Peer Review Group and other key stakeholders, for comparability.

The Authority has continued to collaborate with the Chair of the Peer Review Group on any refinements to the Quarterly ARRA Report that may be implemented. Moving forward, as a sufficient history of costs and performance is established through the report, the Authority will work with the Chair of the Peer Review Group and key Legislative staff to develop a clear and concise presentation of the historical information.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

The Authority notes in its response that its quarterly report from October 2019 uses the same template as the previous report, which we evaluated as part of the six-month status of this recommendations. As we concluded then, the Authority's current presentation of the data does not allow policymakers or the public to easily determine whether progress is accelerating or decelerating as the Authority approaches the federal grant deadline of December 2022. The Authority notes it is still working on including historical information.

6-Month Agency Response

The Authority is committed to producing a quarterly report that achieves the recommended purpose - to enable policymakers and the public to track the Authority's progress in meeting the federal grant deadline. To that end, following the release of the first Quarterly ARRA Report in January 2019, and with the valuable feedback received from key legislative staff and the State Auditor, the Authority has worked collaboratively with the Chairman of the Peer Review Group to develop a draft report template to be used moving forward. The draft report template contains clear and digestible dashboards illustrating, for each of the Construction Packages, the original and current budget (including cost risks), original and current schedule, and spending rate averages achieved in the previous three and twelve months compared to the spending rate required to meet established completion dates. The draft report template also includes a schedule status on all Phase I environmental documents.

On April 3, 2019, the draft report template was provided to key legislative staff and the State Auditor for review and consideration. The Authority and Chair of the Peer Review Group met with key legislative staff and the State Auditor on May 16, 2019 to review the report template and gain consensus on form and frequency, with the next Quarterly ARRA Report to be released on July 1, 2019 and every quarter thereafter. Minor technical revisions were made to the draft report template based on the feedback received during the May 16, 2019 meeting. While the draft report template largely addressed the elements requested, it was agreed amongst the meeting participants that additional collaboration would occur over the next quarter to further augment the report template for historical costs and performance.

Given the additional elements being developed over the next quarter, this recommendation is partially implemented with anticipated full implementation on October 1, 2019.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

The Authority's July 2019 quarterly report improved upon the version we reviewed in January by including more up-to-date cost and progress data for the Central Valley construction projects. However, the Authority's current presentation of the data does not allow policymakers or the public to easily determine whether progress is accelerating or decelerating as the Authority approaches the federal grant deadline of December 2022. This is a core purpose of the recommendation we made to the Authority. In its response, the Authority indicates it will be working to include additional historical cost and performance information.

60-Day Agency Response

Using information from the monthly PDSR, and in consultation with the Peer Review Group, the Authority developed a Quarterly ARRA Status Report - including earned value, cost variance and schedule performance index. Additionally, to avoid confusion and redundancy, this new Quarterly Program Status Report incorporates elements of the biannual Peer Review Group report which will therefore be superseded by this new quarterly status report. The Quarterly ARRA Status Report includes detailed information on the progress of the three Central Valley construction projects. The Authority will post the new Quarterly ARRA Status Report to its website making it available to the public and the Legislature by the end of January 2019.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

A report provided by the Authority on January 31, 2019 includes point-in-time information about the progress of the construction projects in the Central Valley, using components of an earned value model. However, the model on page 16 of the report, which is intended to demonstrate the Authority's progress toward meeting the federal grant deadline, is based on data from June 2018, and therefore does not represent an update to the analysis we provided in our audit report. Similarly, the report uses the Authority's June 2018 baseline budget as the measure of total expected costs for the projects, even though the report also lists "risks that may affect schedule and cost." Our recommendation is that the Authority should base its update reports on the most current estimates available. Therefore, going forward, we expect the Authority to base its model on more up-to-date information about actual progress, as well as its best, most recent estimate of project costs to the extent that those estimates diverge from the current baseline. Additionally, to allow policymakers to track progress over time, the Authority's future reporting should also depict historical costs and performance in order to demonstrate whether progress is accelerating or decelerating as the Authority approaches the federal grant deadline of December 2022.

Recommendation #5 To: High-Speed Rail Authority, California

To ensure that it is adequately prepared if it is unable to meet the federal grant deadline of December 2022, the Authority should, by May 2019, develop a contingency plan for responding to such a scenario.

Annual Follow-Up Agency Response From September 2023

As noted in our response to Recommendation 4, Amendment 7 of our federal grant generally conforms to Amendment 4 to Agreement Number FR-HSR-0118-12, which implemented the contingency plan and addressed the risk of being unable to meet the prior deadline.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The Authority has now amended two federal grant agreements for its Central Valley projects that extend those agreements' deadlines to December 2026 and December 2028, respectively.

Annual Follow-Up Agency Response From November 2022

As noted in our response to Recommendation 4, Amendment 7 of our federal grant generally conforms to Amendment 4 to Agreement Number FR-HSR-0118-12, which implemented the contingency plan and addressed the risk of being unable to meet the prior deadline.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

We will assess the status of this recommendation when the authority and has finalized the amendment it refers to in its response.

Annual Follow-Up Agency Response From October 2021

As noted in our response to Recommendation 4, our amended federal grant extended the deadline to December 2026, which implemented the contingency plan and addressed the risk of being unable to meet the prior deadline.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

After submitting its response, the Authority clarified that it has amended one of its two relevant federal grant agreements and is working with the federal government to amend the other. Therefore, we have evaluated the status as partially implemented.

Annual Follow-Up Agency Response From November 2020

The Authority is undertaking an enhanced risk assessment of the assumptions in the Business Plan, to be issued December 15, 2020. We are looking closely at how we are doing against the ARRA deadline. As we have expressed in the past, the Authority will require the cooperation of the FRA to achieve the federal December 2022 construction deadline and/or to explore alternatives to that deadline.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

As we have stated previously, although we remain concerned about the Authority's ability to meet its December 2022 construction deadline, we acknowledge that the Authority will require the cooperation of the FRA in order to explore alternatives to that deadline. Given the possible financial implications for the State if the Authority does not meet the deadline, the Authority should continue attempts to reengage the FRA in order to develop contingencies for such a scenario.

1-Year Agency Response

We have addressed the risks associated with the FRA's disengagement and the effect of the delay in the FRA's action on NEPA Assignment (which had been pending since July 2018) in the Authority's May 2019 Project Update Report to the Legislature. On July 24, 2019, the Authority announced that the State of California had received approval from the FRA to assume its federal environmental review responsibilities under NEPA through a program known as NEPA Assignment. The Authority and FRA have held ongoing constructive dialog related to NEPA Assignment.

The Authority will require the cooperation of the FRA to achieve the federal December 2022 construction deadline and/or to explore alternatives to that deadline.

California State Auditor's Assessment of 1-Year Status: Pending

We discuss the Authority's May 2019 Legislative report in our six-month assessment to this recommendation. As we stated then, although we remain concerned about the Authority's ability to meet its December 2022 construction deadline, we acknowledge that the Authority will require the cooperation of the FRA in order to explore alternatives to that deadline. The Authority confirmed for us that, notwithstanding the NEPA issue it reports in its response, its attempts to engage the FRA about contingency planning around the December 2022 deadline have remained unsuccessful. Given the possible financial implications for the State if the Authority does not meet the deadline, the Authority should continue its efforts to reengage the FRA in order to develop contingencies for such a scenario.

6-Month Agency Response

Following receipt of the FRA's February 19, 2019 letter notifying the Authority of its intent to terminate Cooperative Agreement No. FR-HSR-0118-12-01-01 effective March 5, 2019, the FRA disengaged from all communication with the Authority. Therefore, the Authority is presently unable to perform the necessary consultation with the FRA and evaluate collaboratively the alternatives for contingency planning should the Authority fail to achieve the required scope of work by the current December 2022 grant performance period. The Authority continues full implementation of its "Get to Work plans" in order to accelerate construction performance and achieve the current federal grant performance period deadline. The Authority is eager to resume our long-standing partnership with the FRA.

California State Auditor's Assessment of 6-Month Status: No Action Taken

The Authority addressed the FRA's disengagement in its May 2019 project update report to the Legislature, and provided us documents indicating that the FRA has cancelled regular standing meetings with Authority personnel. Although we remain concerned about the Authority's ability to meet its December 2022 construction deadline, we acknowledge that the Authority will require the cooperation of the FRA in order to explore alternatives to that deadline. Given the possible financial implications for the State if the Authority does not meet the deadline, the Authority should continue its efforts to reengage the FRA in order to develop contingencies for such a scenario.

60-Day Agency Response

The Authority continues to routinely update the individual project risk registers in coordination with the FRA on a quarterly basis. In doing so, the Authority has established project controls allowing construction performance monitoring to achieve the federal grant deadline. If, by July 2019, progress in the Central Valley is not tracking to the "Get to Work plans", the Authority will consult with FRA to evaluate options and the possibility of seeking an amendment to the federal grant deadline. Monitoring Central Valley construction from present to July 2019 will allow 2019 construction to fully initiate to evaluate progress against planned project schedule milestones.

California State Auditor's Assessment of 60-Day Status: Pending

The Authority's response states that it continues to take steps to improve the likelihood that it will meet the federal grant deadline of December 2022. However, as we discuss on pages 32-33 of our audit report, meeting the federal deadline will require significant increases in the rate of construction progress in the Central Valley. As such, to ensure it is prepared if it is unable to meet the federal grant deadline, the Authority should as soon as possible, and by May 2019, develop a contingency plan.

Recommendation #6 To: High-Speed Rail Authority, California

To improve its contract management, increase accountability, and justify the significant amount it pays for contracted services, the Authority should, by May 2019, prioritize contract management efforts and reduce the frequency with which contract management responsibilities shift among Authority staff by establishing a formal process for hiring and assigning full-time, experienced contract managers. These contract managers should have duty statements reflecting their contract oversight responsibilities, and they should report to supervisors who understand those responsibilities and have extensive knowledge about the contracts' deliverables. In addition, those supervisors' duty statements should clearly lay out their responsibility for addressing any contract manager noncompliance with the Authority's contract management policies and procedures, whether reported by CMSU or identified by another means.

6-Month Agency Response

The Authority has updated and finalized the Human Resources Hiring manual to include language regarding hiring contract managers. A process was developed that requires the Authority to justify the reason for contract management responsibilities to shift among Authority staff. In addition, the organization chart has been updated to denote positions that are contract managers. Duty statements for both contract managers and their supervisors have been approved and finalized. Contract manager training specifically for supervisors was created and contract manager supervisors have been trained.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

We reviewed materials the Authority provided to support its implementation of this recommendation, and confirmed the revisions to its hiring manual and the existence of contract management training for supervisors. We also reviewed a selection of duty statements for contract managers and their supervisors and confirmed that, in contrast to what we found during our audit, those duty statements include clear statements about contract management and supervisory responsibilities. The Authority provided evidence that it has assigned full-time contract managers to the largest and highest risk types of contracts we reviewed during our audit, and has indicated to us that it will continue to assess the need to assign additional full-time contracts managers as it enters into new contracts or amends existing contracts.

60-Day Agency Response

The Authority has drafted a formal process for hiring and assigning full-time experienced contract managers to reduce the frequency with which contract management responsibilities shift among Authority staff. This formal process emphasizes contract management experience/skills as well as desirable contract manager qualifications. The draft contract manager hiring process is circulating for comment and finalization.

All new advertised positions that require contract management specific skills are specified within the duty statement. The Human Resources Office will review all job advertisements to assure compliance with the formal process. Furthermore, the Authority will denote contract management positions on the organization chart. The updated organization chart is being circulated for comment and finalization.

All existing duty statements have been reviewed and modified to reflect contract management/oversight responsibilities for all contract managers and their supervisors. Contract manager supervisors' duty statements address their responsibility to hold their contract management staff accountable for compliance with the Authority's contract management policies and procedures. Revised duty statements are being circulated for comment and finalization.

Contract manager supervisors will attend contract management training to ensure that the contract managers they supervise are adhering to the Authority's policies and procedures. The Authority will create a separate contract management training specifically for supervisors by January 2019.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Authority provided draft documentation related to the required duties and qualifications of contract managers, as well as formal processes for transferring responsibility for contracts among contract managers. It also provided draft training materials for staff that supervise contract managers. To fully implement this recommendation, the Authority must finalize its hiring process, assign contract managers official positions on its organizational chart, adjust reporting relationships as needed, and provide and document relevant training for supervisors.

Recommendation #7 To: High-Speed Rail Authority, California

To improve its contract management, increase accountability, and justify the significant amount it pays for contracted services, the Authority should, by May 2019, require CMSU to establish a schedule to monitor individual contract manager compliance and report annually the results of this monitoring to Authority executive leadership. To help ensure the integrity of its oversight role, CMSU should be composed of state staff in place of RDP consultants.

1-Year Agency Response

The Authority requested and received 35 positions - seventeen related to performing contract assessments and administration and 18 positions related to accounting and budget areas. Based on the recruitment plan to replace 35 consultant positions with state staff, the Authority believes that any actual or perceived conflict of interest will be eliminated.

Four of the seventeen contract assessment and administration positions have been filled. Interviews for the two remaining CAB positions were held in October and are anticipated to be filled by late November. Job bulletins for the remaining eleven contract administration positions have been released and it is anticipated that interviews will be conducted in November/December and the positions filled in the first part of the year.

Of the 18 positions related to accounting and budgets, 10 of the 18 positions have been filled. Interviews for three positions were held in September and October and are in the final steps of the hiring process. The remaining five have either posted job bulletins or will be posted soon with interviews anticipated in early December.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

In December 2019, the Authority provided an updated organizational chart showing that five of the seven positions in the CAB/CMSU are filled by state staff, including the director position. If the Authority uses these individuals in the roles previously performed by RDP consultants, it will help ensure the integrity of the unit's oversight functions.

6-Month Agency Response

The Authority has continued to augment staff to perform contract assessments. Three existing staff within the Authority have been temporarily reassigned to the Contract Administration Branch (CAB), formally known as CMSU. These staff support the assessment process, update policies and procedures and assist in the contract management training. Additionally, in order to reduce the reliance on consultant staff in the short-term, the Authority identified vacant positions that could either be reclassified or moved temporarily to CAB. These positions have been advertised to fill the vacancies.

The Authority has also submitted a Finance Letter for inclusion in the May Revise of the Governor's 2019-2020 Budget requesting 35 positions. Seventeen positions are related to consultants performing contract assessments and contract administration and 18 positions are related to positions in the Accounting and Budget branches.

In addition, the CEO has directed an enterprise-wide review to determine if consultants are temporary, of a particularly urgent nature, or of such a highly-skilled or technical nature not available in civil service.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

The Authority provided a calendar indicating that CMSU (now referred to as the Contracts Administration Branch, or CAB) conducted contract manager assessments between November 30, 2018 and February 27, 2018. However, the organizational chart the Authority provided as part of its response indicates that despite the temporary reassignment of three state staff to CAB, the unit is still composed primarily of RDP contractors. Additionally, although the Authority provided evidence that it has advertised positions that will replace the RDP contractors with all state staff, it has not yet filled these positions. Further, the Authority's records show that RDP consultants performed more than three-quarters of the contract manager assessments that the Authority reports having conducted in response to our audit recommendations, an approach the Authority indicated to us that it took in order to expedite the assessments. Nonetheless, these include assessments of managers responsible for specific contracts that we concluded in our report could represent conflicts of interest if the staff performing the assessment were RDP consultants. As such, despite its claim of full implementation, the Authority has not yet demonstrated that it is able to effectively staff and operate its contract management oversight group with state staff, which we believe is necessary and prudent.

60-Day Agency Response

The Authority has developed a schedule to perform assessments of contract managers' compliance with all contracts which will be completed by November 2019. The first 23 such assessments were completed through December 2018. An executive report will be prepared no less than annually. The assessments are performed by RDP staff with state staff oversight, as well as review and approval of the completed assessment.

The Authority concurs that CMSU should be composed primarily of state staff. The Authority has temporarily transferred internal state personnel to augment the CMSU and drafted an interim staff augmentation request to sister departments. The draft request is being circulated for coordination and review. In the interim, any assessments performed by contracted RDP staff will be reviewed and approved by state staff.

A Budget Change Proposal Concept for the 2020/21 fiscal year to request staff augmentation in order to remove contracted RDP consultants and replace with state staff is currently being prepared.

California State Auditor's Assessment of 60-Day Status: Pending

The Authority provided a schedule indicating that it planned completion of the contract manager assessment process by December 2019. It did not provide results of the assessments its response indicates the Authority has already completed, but indicates that it will report on those results to its executives annually. The Authority also did not provide details about its staff augmentation request or Budget Change Proposal. As we emphasized on page 85 of our audit report, we believe the Authority should move to secure professional state staff for contract management oversight positions sooner than fiscal year 2020-21.

Recommendation #8 To: High-Speed Rail Authority, California

To improve its contract management, increase accountability, and justify the significant amount it pays for contracted services, the Authority should, by May 2019, hold contract managers accountable for performing the duties that the Authority's policies assign to them. Specifically, CMSU and, to the extent necessary, contract managers' supervisors should require and review evidence from contract managers demonstrating their approval of deliverables, detection and resolution of contractor performance issues, and assessment of contract amendments for merit. The Authority should not accept observations and reports from its contractors or the RDP consultants in place of this evidence.

6-Month Agency Response

The CAB has conducted and completed assessments for all 195 current active contracts. CAB assessors evaluated documentary evidence for compliance with Contract Management Procedure requirements. All non-compliance issues were reported, tracked and resolved. CAB will continue to assess active contracts to ensure contract managers are following the policies and procedures established by the Authority.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority provided records indicating that staff and RDP consultants completed assessments of 63 contract managers and the contracts assigned to them. Upon our request, the Authority provided examples of its assessment documents; these assessments indicate that reviewers determined whether contract managers were maintaining certain, but not all, documentation that Authority policies require--such as invoice logs--and issued non-compliance findings when they were not. We also reviewed documentation indicating that the Authority made contract managers' supervisors aware of instances of noncompliance. The Authority informed us that it will begin following up on the results of the initial assessments in Fall 2019, and that this second phase will seek to ensure that contract managers are filing all required documentation and following all required Authority contract management processes when developing and using that documentation. Going forward, the Authority should not only conduct this scheduled follow-up review but continue to assess contract manager performance on a regular basis as part of holding them accountable for their responsibilities.

60-Day Agency Response

As of December 31, 2018, the CMSU has completed 23 assessments of contract managers' performance and the remainder are scheduled. These assessments are following a formal process requiring and reviewing evidence from contract managers demonstrating their approval of deliverables, detection and resolution of contractor performance issues, and assessment of contract amendments. Once an assessment is issued, it will be tracked to closure/resolution to verify the implementation of any required corrective action. Corrective actions will be tracked by the Quality team and reported to executive management.

California State Auditor's Assessment of 60-Day Status: Pending

This recommendation pertains to the ongoing evaluation of contract managers' compliance with Authority policies and procedures. The Authority provided a draft document summarizing the process by which ongoing evaluations will take place. The document states that a compliance assessment schedule will be developed and will include assessments of each contract and each contract manager within the assessment period, which it indicates is at least once every calendar year. Although the Authority indicates it has completed 23 assessments, it has not provided documentation of those assessments. As part of our assessing this recommendation's implementation, we will review the schedule, assessment tools used, as well as the results of actual assessments. In doing so, we will look for involvement on the part of contract managers' supervisors, such as reviewing evidence from contract managers demonstrating their approval of deliverables, detection and resolution of contractor performance issues, and assessment of contract amendments for merit.

Recommendation #9 To: High-Speed Rail Authority, California

To prevent the inappropriate use of contractors to perform state functions, the Authority should develop procedures by May 2019 for evaluating whether new and existing administrative duties should be assigned to contractors or to state employees.

1-Year Agency Response

Procedures to determine whether a new or existing position performing administrative duties should be assigned to a consultant/contractor have been established and were implemented. An additional enterprise-wide effort to align Authority staff to ensure that form fits function is currently in progress. The evaluation utilizes objective criteria on how to best achieve proper alignment of state and consultant staff to functionality, as well as considering the right balance of resources. However, this additional effort does not affect the Authority's implementation of this recommendation.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

In addition to the changes it made to its CMSU staff as part of implementing recommendation #7, the Authority provided documentation that it has replaced significant portions of its finance consultants with state staff. Although the Authority confirmed it did not use its new policies and procedures as part of this process, it confirmed that it will do so for all administrative duties going forward.

6-Month Agency Response

The Authority has created procedures, as well as a new form to be completed, for evaluating whether administrative duties should be assigned to a consultant/contractor or state employee. The procedure includes instructions for contract managers to provide adequate justification for utilizing consultants/contractors.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

The Authority has implemented procedures requiring that contract managers of professional service contracts work with procurement staff to ensure compliance with state law prior to awarding a new contract. The procedures also set forth actions that contract managers must take in order to modify existing contractors' duties. The Authority notes in its response to recommendation #7 that the CEO has directed a review to determine if existing consultants are temporary, of a particularly urgent nature, or of such a highly-skilled or technical nature not available in civil service. We confirmed with the authority that this review is ongoing.

60-Day Agency Response

The Authority's Administration Office has drafted procedures for evaluating whether new or existing administrative duties should be assigned to contractors or to state employees based on Government Code 19130. The draft procedures are being circulated for review and comment.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided draft language it intends to include in its hiring manual which requires that contract managers of personal services contracts must ensure compliance with state law regarding the use of state employees. It also provided a draft form that contract managers must complete to certify that the use of outside consultants is appropriate. Going forward, as part of determining whether the Authority has fully implemented this recommendation, we will be looking for additional assurance that the drafted language is suitably broad to cover all relevant contracted duties, as well as that the Authority has instituted an effective process for reviewing contract managers' determinations.

Recommendation #10 To: High-Speed Rail Authority, California

To ensure that contract managers' invoice reviews are complete and that invoiced costs are allowable under contract terms, the Authority should amend its applicable procedures by May 2019 to require contract managers to document their review of invoiced rates and expenses.

6-Month Agency Response

The Contract Invoicing and Payment Policy, as well as the corresponding Procedure, have been revised and finalized to require Contract Managers to be prepared to justify and/or provide backup documentation which validates the work performed.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority provided the revised procedures it references in its response.

60-Day Agency Response

The draft revision of procedures referenced in the initial response will be initiated in January and then circulated for comment and finalization.

California State Auditor's Assessment of 60-Day Status: Pending

The Authority provided a draft of the revisions to its existing procedures related to invoice review. However, these revisions are not responsive to the recommendation that the Authority require contract managers to actively document their reviews of invoice materials. When we shared our concerns with the Authority, it responded that it would take our comments into consideration when performing further reviews of the procedures.

Recommendation #11 To: High-Speed Rail Authority, California

To ensure the consistency and effectiveness of its efforts to monitor the performance of the oversight firms with which it contracts, the Authority should develop a formal methodology by May 2019 for using the performance evaluation tool it has implemented. This methodology should include procedures for assessing the sufficiency of the oversight firms' review and approval of invoices for construction contracts.

6-Month Agency Response

The Authority's Project and Construction Management Consultant Performance Evaluation Policy POLI-CMAN-01 and Procedure PROC-CMAN-01 were issued in May.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority implemented procedures that lay out processes by which PCM contract managers are to assess the PCMs' performance on at least a quarterly basis. The procedures set forth performance categories and rating criteria to guide contract managers in using the templates we discuss in our report, and prescribe meetings to help ensure consistency across the three active construction projects.

60-Day Agency Response

The Authority has developed a specific policy and procedure that provides consistent guidance to contract managers, including an enhanced performance-based evaluation dashboard for its PCM firms. This is currently in draft form and will be completed and implemented by May 2019. Once completed, the existing policy and procedure (POLI-FIS-034 and PROC-FIS-034) will be revised to reference the newly adopted policy and procedure. The Contract Performance Monitoring and Reporting Policy POLI-FIS-034 and Procedure PROC-FIS-034 are being revised to document this formal methodology. The draft revision is being circulated for comment and finalization.

In addition to the Authority's PCM Manual, a PCM Contract Management Manual, that includes the PCM's processes and roles and responsibilities for evaluating design-builder invoices has been developed. The Authority's contract manager and quality team for each Construction Project evaluates the PCMs for compliance with this process. Authority Contract Managers check this each month as part of their final invoice approval.

California State Auditor's Assessment of 60-Day Status: Pending

The Authority provided the draft language for the revision to its oversight firm manual, which states that the Authority is committed to providing a consistent method and process for evaluating the oversight firms' performance. However, the Authority did not provide any of the guidance for contract managers that it describes having developed in its response, nor any accompanying policy revisions. As part of our future evaluation of the status of this recommendation, we will expect to review those documents as well as additional evidence that the Authority's contract managers are using the methodology and that the methodology accounts for the oversight firms' reviews of construction invoices.

Recommendation #12 To: High-Speed Rail Authority, California

To ensure that the oversight firms' spending is reasonable, the Authority should develop a formal process by May 2019 for tracking any out-of-scope work that the oversight firms perform. To reduce the likelihood that its contracts with the oversight firms run out of funds prematurely as a result of this additional work, the Authority should also develop a formal process for amending the oversight firms' contracts contemporaneously to change orders that significantly extend the timelines or increase the scope of work of the construction contracts that oversight firms oversee.

6-Month Agency Response

The Authority's PCM Manual has been revised and finalized to include the PCM's responsibility to monitor potential out-of-scope work prior to starting any such work. The analysis for any amendment shall be in accordance with Contract Change Management Procedure, Appendix D (PROC-CADM-07). Resulting amendments will comply with the requirements of the Program Delivery Committee and the Business Oversight Committee.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority's new procedures in this area direct the PCMs to monitor the construction contractors for disputes or other changes that may create out-of-scope work for the PCM to oversee. If it identifies such work, the PCM must report to the Authority within 60 days regarding the need for a contract amendment.

60-Day Agency Response

The Authority revised the PCM Manual to reflect their responsibility to identify, manage and request amendments for out-of-scope work. This revision assigns the Authority PCM contract managers' responsibility for tracking and identifying any potential out-of-scope work and the necessity to amend contracts at each PCM work plan revision and approval. The draft revision is being circulated for comment and finalization.

The Business Oversight Committee is providing oversight of the necessity and business case to amend PCM contracts. The Program Delivery Committee is focusing on key construction milestone timely achievement and decision-points, including when, or if, PCM contracts require amendment.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided draft language for revisions to its contract manual for oversight firms that summarizes the process for identifying the impacts to contracts for out-of-scope work and corresponding requests for amendments. As part of our future review of this recommendation's implementation status, we will be looking for procedures related to the timeliness of this process, which is a key element of the recommendation. Additionally, the Authority states that the Business Oversight Committee (committee) is providing oversight of amendments for these contracts. However, as we discuss in the audit report, although the committee has been in place since September 2017, we identified concerns with an oversight contract amendment approved by the committee in March 2018. It is not entirely clear from the Authority's response specifically how the committee's current role is related to the Authority's efforts to fully implement this recommendation. Therefore, in its future responses, it is important for the Authority to specify the role of the committee in fully implementing the recommendation and to provide documentation demonstrating its role in effective implementation.

Recommendation #13 To: High-Speed Rail Authority, California

To help improve the effectiveness of its sustainability policy, the Authority should revise the policy by May 2019 to more clearly differentiate between the construction and operation phases of the high-speed rail system. Further, it should ensure that each objective in each section of the policy is associated with quantifiable metrics for evaluating implementation.

6-Month Agency Response

The Authority has completed the revision of its Sustainability Policy, including distinctions for each commitment as to construction or operations, and presented these revisions to the Board. The Board adopted the revised policy April 16, 2016. The Authority has posted the updated implementation plan which matches each commitment with a quantitative metric. http://hsr.ca.gov/docs/programs/green_practices/sustainability/Sustainability_implementation_plan_SUMMARY.pdf

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority has revised its sustainability policy to clearly distinguish which goals and objectives pertain to the construction phase versus the eventual operation of the high-speed rail system. Further, it has identified both key indicators and quantitative metrics for its policy objectives in order to to allow the Authority to evaluate implementation.

60-Day Agency Response

The Authority has begun review and revision of its Sustainability Policy. In tune with Authority governance, the Project Delivery Committee was notified of the requirement to revise its Sustainability Policy to more clearly differentiate between construction and operations. A formal business case will be submitted to the Project Delivery Committee in January 2019.

As an interim step, the Authority has revised the implementation plan to match each existing policy objective with a quantitative metric, posted on our website as noted above. The implementation plan will be revised subsequent to the policy revision.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority shared draft revisions to its sustainability policy that include additional differentiation between the construction and operations phases of the system with respect to the policy's objectives. Going forward, we will monitor the finalization and adoption of these revisions, as well as ensure the Authority has developed quantitative metrics for each; as we explain in our audit report, we expect the Authority to revisit its most recent revisions to the metrics in its implementation plan, which the Authority notes in its response, once the revisions to the policy are in place.

Recommendation #14 To: High-Speed Rail Authority, California

To allow it to evaluate the sustainability of the high-speed rail system's construction, the Authority should, by May 2019, perform and document a review of its compliance with its existing quality controls related to ensuring the validity and completeness of contractor-reported data. The Authority should also establish a formal process to perform such reviews periodically.

6-Month Agency Response

The Authority has completed a quality assurance review of data quality control. In addition, the Authority has completed PROC-PLAN 03 Procedure for Assuring Data Quality in Environmental Management and Mitigation Application (EMMA) Sustainability Records which indicates that the Quality team will conduct periodic quality assurance audits on the implementation of the data quality process.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority provided the procedures it references in its response. The procedures require periodic internal audits of staff's implementation of applicable procedures for assuring the quality of the Authority's environmental data. The Authority also provided documentation that it conducted one such audit between February and March 2019.

60-Day Agency Response

The Authority has continued its continuous quality review of contractor submittals. The Authority has drafted a quality assurance process that details how periodic reviews of its compliance with the quality controls related to validity and completeness of contractor-reported data should be conducted. The draft is being circulated for review and comment.

Quality team resources have been requested to perform an initial quality assurance (estimated for 1st quarter 2019) and will be scheduled periodically to verify that the established controls are functioning as expected.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided a draft policy stating that it will conduct quarterly reviews related to the validity and completeness of contractor-reported data, which will include periodic field inspection of construction equipment. As the Authority notes in its response, the results of any such reviews are still forthcoming. We will review those results as part of assessing the future implementation of this recommendation.

Recommendation #15 To: High-Speed Rail Authority, California

To help ensure that it meets its sustainability goals, the Authority should comprehensively compare the three construction projects' performances to their construction contractors' original baseline estimates on a quarterly basis. It should perform the first of these comparisons no later than May 2019.

6-Month Agency Response

The Authority has completed quarterly reports for each construction package that illustrate actual criteria air pollutants, greenhouse gas emissions, water consumed, and diversion rates and weight of construction waste. This quarterly reporting includes comparison of the actuals against estimates for criteria air pollutants, greenhouse gas emissions, water and construction waste. The reports also include illustration of contractors' required performance for reporting and absolute waste diversion rates. These reports are posted to the Sustainability SharePoint site and distributed to the executive team and Program Delivery staff.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

We reviewed the report the Authority provided and found that it compares the construction contracts' performance to baseline estimates or project goals for key environmental impacts, such as greenhouse gas emissions and other air pollutants.

60-Day Agency Response

The Authority has continued with collection of data from all active construction projects. The Authority has assembled reports for each construction package that illustrate the actual air quality and greenhouse gas emissions, water consumed, and diversion rates and weight of construction waste. This report includes comparison of the actuals against estimates for key impact topics for the construction packages. For air quality and greenhouse gas emissions, a projection to construction completion is included as a scenario for total emissions. The reports also include illustration of contractors' required performance for reporting and absolute waste diversion rates. Charts that were available were included in the December program delivery status reports, and future complete quarterly reports will be provided on the Authority sustainability site.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided draft analyses comparing the three construction projects' emissions of carbon dioxide and other pollutants to construction contractors' original baselines and actual construction progress. As part of assessing the recommendation's status in May, we will revisit these analyses and also determine whether the Authority has begun conducting equivalent analyses for other environmental impacts, such as water use, which we discuss on page 64 of our audit report.

Recommendation #16 To: High-Speed Rail Authority, California

To help ensure that its contractors' proposed environmental impacts are reasonable and to measure the progress of its sustainable construction efforts over time, the Authority should, by November 2019, identify and track standardized measures—such as project miles—that will allow it to compare construction impacts across the high-speed rail system's different construction projects.

6-Month Agency Response

The Authority has completed its update of the construction emissions database (model) with information from the adopted 2018 Program Baseline and relevant, validated data from construction. The Authority will use this database, in part, to develop targets for upcoming procurements.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

In addition to updating its model, the Authority developed a methodology establishing how it will use data from the model to develop estimates for future construction, against which it can manage and evaluate contractor performance.

60-Day Agency Response

As noted in the Audit response, the Authority has identified and tracked standardized metrics related to construction efforts. The Authority has a model that assesses program environmental impacts using standardized metrics, including tons of CO2e, kgCO2e/kg, kgCO2e/gallon, kgCO2e/kWh, normalized by miles and construction typology. The Authority has completed an update to this model with the refined project lengths and infrastructure typologies detailed in the adopted 2018 Program Baseline and is carrying out a series of reviews and stress testing of formulas to confirm fidelity. This review and testing involves incorporating relevant, validated data (e.g., tons of CO2e and environmental product declarations for actual materials installed) tracked on each construction package, among other refinements. As testing is completed, this updated model will be used to support the development of targets for future construction contracts. The model structure can also be used to illustrate and compare different construction contracts.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided documentation of its efforts to revise its existing model, work it indicates is ongoing. Once the Authority reports that it has completed the process of refining the model to compare construction environmental impacts and set targets for new construction, we will review the final model and any targets the Authority has set.

Recommendation #17 To: High-Speed Rail Authority, California

To increase the transparency of its reporting, the Authority should, by May 2019, expand its quarterly small business, DVBE, and DBE utilization reporting to account for the total value of all its contracts and to identify the reasons it has exempted specific contracts.

6-Month Agency Response

The Authority has posted small business, DVBE and disadvantaged business contract information on the Authority's website. The report identifies, for fiscal year, State commitments of total State contract dollars, State small and micro business dollars, State small and microbusiness percentage and state DVBE percentage. In addition, the report identifies federal expenditures of total active contract value to date, applicable contract dollars, total expended by prime contractors, total small business/disadvantaged business enterprise/DVBE/microbusiness expenditures to date and the applicable percentage. http://hsr.ca.gov/docs/programs/small_business/Total_Active_Values_Quarterly_Report_April_2019.pdf A policy has also been developed that defines which contracts are exempt from small business, DVBE, and or disadvantaged business' goals.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

The Authority provided an official version of the small business, DVBE, and disadvantaged business policy it submitted in draft form at the time of our 60-day review. This policy specifies the types of contracts that it exempts and includes a process for approving the exemption of any other contract from the program. The Authority also posted a report that contrasts total active contract dollars with the amount subject to state and federal utilization goals.

60-Day Agency Response

The Small Business Program has collected data related to the total value of all current contracts and contracts that require small business, DVBE, and/or disadvantaged business. They are also working to post the information to the Small Business Program section of the Authority's website.

The Authority is modifying the current annual report to reflect quarterly information. A policy has been drafted that will clearly specify which contracts are exempt from small business, DVBE and/or disadvantaged business according to state and/or federal regulations, policies, and guidelines.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Authority provided a draft of its revised small business, DVBE, and disadvantaged business policies, which specifies which types of contracts are exempt under state guidelines and establishes requirements for excluding any other type of contract from the program's utilization goals. The Authority also provided a draft template it plans to utilize to calculate utilization rates based on total contract value. When the Authority has completed those calculations and updated its reporting to include this additional information, we will evaluate the status of this recommendation to determine whether it is fully implemented.

All Recommendations in 2018-108

Agency responses received are posted verbatim.