Report 2011-116.2 All Recommendation Responses

Report 2011-116.2: Department of General Services: Strengthening the Division of the State Architect's Workload Management and Performance Measurements Could Help It Avoid Delays in Processing Future Increases in Workload (Release Date: May 2012)

Recommendation #1 To: General Services, Department of

To better gauge the timeliness of its plan review and better communicate with design professionals, the division should develop goals for the time spent on the plan review phase, in the style of those used by the Office of Statewide Health Planning and Development, and measure and report its success at meeting these goals.

1-Year Agency Response

This recommendation has been fully implemented. On its website, the DSA provides information to school districts regarding the anticipated finish date of the plan review activity. As an enhancement to this information sharing process, the DSA has developed additional reports that measure anticipated and actual plan review finish dates. This information was posted to the DSA's website in May 2013 (See Attachment 1).

California State Auditor's Assessment of 1-Year Status: Fully Implemented

The Division of the State Architect (DSA) provided a report showing the number of projects where the DSA completed plan review before or after the anticipated date. The report, Attachment I referenced in the response, is available at the following link: http://www.dgs.ca.gov/dsa/AboutUs/metrics.aspx.

6-Month Agency Response

According to the department, the division is actively developing reports that measure anticipated and actual plan review finish dates. The department stated that the division is reviewing data to ensure the validity of the information and is preparing metrics on the success of meeting plan review goals. The division plans to publish this information in January 2013. (See 2013-406, p. 45)

California State Auditor's Assessment of 6-Month Status: Pending

Recommendation #2 To: General Services, Department of

In order to avoid delays in plan review, the division should develop a policy that defines when it will expedite plan review using its statutory authority to contract for additional plan review resources.

Annual Follow-Up Agency Response From October 2013

On November 30, 2012, DSA issued a policy which specifically addresses the use of its contracting authority to expedite plan review. DSA's intent in developing the policy was to provide guidance on the use of consultants to assist in ensuring that projects are reviewed within a timeline that meets the needs of a school or community college district (project owner). The policy includes guidelines which provide for the consideration of various issues such as budget constraints, workload demands and available staff resources in determining if consultants will be used to perform the plan review function.

To provide a project owner with relevant information on the plan review function, DSA publishes information on the average time taken to assign projects to plan review staff after receipt of a complete application, referred to as bin time. Subsequently, after the project has been assigned to plan review staff, a timeline is estimated for completion of the plan review phase of project review. In general, DSA moves projects to plan review in the order projects are received by the division. However, in some instances, the division gives priority to projects that have a specific turnaround identified by the project owner, such as a funding deadline.

Over the last few years, DSA generally has been able to complete the plan review function in a timely manner without the use of consultants. However, DSA management is continuing to actively monitor plan review function metrics, such as a maximum six week goal for bin time, to ensure that timely actions are taken to meet expected project turnaround times. These actions could involve expediting plan review through the reassigning of staff resources or contracting with consultants.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The division provided a copy of the policy referenced above. Additionally, in response to another recommendation, the division created a report showing its performance against anticipated plan review finish dates. Such information should allow the division to make informed decisions about when it needs to use its contracting authority to expedite plan review.

1-Year Agency Response

As noted in our six-month status report, this recommendation has been fully implemented. Specifically, on November 30, 2012, the DSA issued a policy which specifically addresses the use of its contracting authority to expedite plan review. The policy includes guidelines which provide for the consideration of various issues such as budget constraints, workload demands and available staff resources in determining if consultants will be used to perform the plan review function.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

On November 30, 2012, the division published a policy to provide guidance regarding when it will use consultants to conduct plan review of school projects. The policy states that the division will use consultants for plan review when the division is unable to meet the project owner's need for turnaround time, plan review cannot be accommodated in some other way, such as by transferring plans to another region, and resources exist for consultants. The policy outlines some steps the division will take to determine whether to use consultants, however, according to the division, it does not typically request that project owners provide a turnaround time for projects. Further, the policy does not tie the division's use of consultants to any kind of metric.

6-Month Agency Response

On November 30, 2012, the division published a policy to provide guidance regarding when it will use consultants to conduct plan review of school projects. The policy states that the division will use consultants for plan review when the division is unable to meet the project owner's need for turnaround time, plan review cannot be accommodated in some other way, such as by transferring plans to another region, and resources exist for consultants. The policy outlines some steps the division will take to determine whether to use consultants, however, according to the division, it does not typically request that districts provide a turnaround time for projects. Further, the policy does not tie the division's use of consultants to any kind of metric, such as that which will be developed according to Recommendation 1.1.

(See 2013-406, p. 46)

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Recommendation #3 To: General Services, Department of

To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should provide current information on its performance, by phase, at the time of the reporting period.

1-Year Agency Response

The DSA has changed its methodology for calculating time for plan review activities to provide additional transparency to stakeholders and management. The DSA now calculates performance as projects complete each phase of plan review instead of completion of the entire plan review process. In addition, any phase that starts and ends on the same day is now calculated as "1" day instead of with a zero value.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

The Division of the State Architect (division) now produces two versions of its "Plan Review Scorecard," one which reports on the status of plans that have finished review in a given month, and another which reports on plans still in progress. Further, the scorecards each provide details on the individual projects used to calculate the performance metrics. This allows stakeholders the ability to examine how an individual project fares in relation to the whole or to analyze the data using an alternate methodology.

6-Month Agency Response

According to the department, the division will change its methodology to capture relevant information upon completion of each plan review phase in January 2013. (See 2013-406, p. 46)

California State Auditor's Assessment of 6-Month Status: Pending

Recommendation #4 To: General Services, Department of

To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should exclude zero values from its calculations related to projects that did not have activity in a particular phase.

6-Month Agency Response

The division no longer issues monthly plan review workload reports,

but its monthly plan review scorecard excludes zero values by design. The report includes only projects with approved plans and thus every plan in the report will have completed each phase of the plan review process. (See 2013-406, p. 46)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

Recommendation #5 To: General Services, Department of

To more accurately report on its plan review activities to stakeholders and provide relevant information to management, the division should exclude projects from client phase calculations that were not returned to the division for back check within the division's deadlines.

1-Year Agency Response

DSA has issued a new policy that eliminates the six-month maximum (maximum allowable before being voided) for projects that have not been returned for back check. The new policy provides that regional managers have discretion in voiding an application based on the individual circumstances of a specific project (See Attachment II, Revised IR A-17).

California State Auditor's Assessment of 1-Year Status: Fully Implemented

In its six-month response to this audit, the Division of the State Architect (division) stated it changed its calculation of plan review processing time to use a maximum of 365 days for projects that have not been returned for back check. However, this response indicates a change in policy to allow managers discretion to void applications based on project circumstances. The intent of our recommendation was to ensure that the division's performance reporting followed it policies as written. By changing the policy, the division addresses the recommendation.

6-Month Agency Response

According to the department, the division changed its calculation of plan review processing times to use a maximum of 365 days for projects that have not been returned for back check. It said the division would begin using the new calculation in its November 2012 metrics' reports. However, the division's November 2012 plan review scorecard still included projects that had not begun back check within the division's one-year deadline. (See 2013-406, p. 46)

California State Auditor's Assessment of 6-Month Status: Pending

Recommendation #6 To: General Services, Department of

To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it receives all relevant plan changes.

Annual Follow-Up Agency Response From October 2013

In previous status reports, DSA noted that regulatory and procedural changes had been made that result in a more simplified and streamlined process for approving construction change documents. The regulations no longer refer to change orders or field changes whose tracking led to some of the concerns developed by the State Auditor. Instead, they require design professionals to submit all construction change documents related to the structural safety, access compliance, or fire and life safety portions of the project for division review and approval.

As noted in DGS' updates to BSA Report No. 2011-116.1 which focused on DSA's construction oversight and project close-out functions, DSA has taken numerous actions to ensure that adequate oversight is performed of project inspectors during construction. These actions include initiating a training program that focuses on ensuring that consistent construction oversight is being provided by its field engineers, including actively monitoring that construction is only conducted in accordance with approved construction documents. The training contained modules that address overseeing project inspector performance and record keeping during construction. The training sessions were conducted in August 2012, November 2012 and February 2013.

Recently, DSA also implemented a system that allows project architects/engineers to submit construction change documents electronically by establishing folders within DSAbox (DSA's cloud based internet solution). This system will assist in ensuring the timely submittal of plan changes for approval.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The division provided evidence that it trained staff in monitoring construction and specifically in monitoring project inspectors. Additionally, the division's site visit process includes a job file review. The checklist includes a review of addenda and change orders. While this control addresses our recommendation, we did not conduct a follow-up to test implementation of this control.

1-Year Agency Response

DSA has made changes to its regulations that result in a more simplified and streamlined process for approving construction change documents. The regulations no longer refer to "change orders" or "field changes" whose tracking led to some of the concerns developed by the BSA. Instead, they require design professionals to submit all "construction change documents" related to the structural safety, access compliance, or fire and life safety portions of the project for division review and approval.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

The division issued a revised interpretation of its regulations regarding construction changes. That interpretation outlines a process for design professionals to submit certain construction changes to the division for approval prior to the start of construction. Further, a division policy states that the project inspector must monitor changes and report those that require division approval. However, the interpretation did not describe how the division intends to ensure it receives all relevant construction changes. According to the division, it is implementing a process for electronic submittal of construction change documents that will allow all involved parties access to information on the project. Additionally, the division stated it is implementing a process to audit project inspector records; however, as of September 2013 the division has not provided evidence that it has completed these steps.

6-Month Agency Response

The division issued a revised interpretation of its regulations regarding construction changes. That interpretation outlined a process for design professionals to submit certain construction changes to the division for approval prior to the start of construction. However, the interpretation did not describe how the division intends to ensure it receives all relevant construction changes. According to the division, it is implementing a process for electronic submittal of construction change documents that will allow all involved parties access to information on the project. Additionally, the division states it is implementing a process to audit project inspector records, in part to ensure the division has received all construction changes requiring approval. (See 2013-406, p. 47)

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Recommendation #7 To: General Services, Department of

To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it reviews and approves all relevant plan changes before the start of related construction.

Annual Follow-Up Agency Response From October 2013

As noted under the previous recommendation, DSA has taken numerous actions to ensure that projects are constructed in accordance with approved construction documents. These robust actions should ensure that all relevant plan changes are identified and reviewed in timely manner.

On June 1, 2013, an inspection card process was also implemented for all new construction projects. The process allows for verification of structural integrity and fire and life safety at the completion of each applicable phase of a project. The implementation of this process included DSA providing detailed instructions related to a project inspector's role in ensuring that construction is conducted in accordance with approved construction documents. The inspectors are held responsible and accountable for complying with the requirements.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The division provided evidence that it trained staff in monitoring construction and specifically in monitoring project inspectors. Additionally, the division's site visit process includes a job file review checklist. The checklist includes a review of addenda and change orders. The division also provided evidence of its new requirements on project inspectors and its inspection card process. While these controls address our recommendation, we did not conduct a follow-up to test the implementation of these controls.

1-Year Agency Response

In May 2012, the DSA initiated the first phase of implementing the new requirements which involved the creation of a process for K-12 school districts and community college districts to self-report the final costs of their construction projects (See DGS Status Report, dated July 27, 2012, Attachment I, DSA Bulletin 12-01). In October 2012, the DSA initiated the second phase to simplify the processing of construction change documents. This phase requires K-12 school districts and community college districts to submit only those changes that affect the structural safety, access compliance, or fire and life safety portions of a project. Any other construction change documents are not required to be submitted to the DSA unless specifically requested by the division (See DGS Status Report, dated December 5, 2012, Attachment II, DSA Bulletin 12-03).

California State Auditor's Assessment of 1-Year Status: Partially Implemented

In a procedure adopted in November 2012, the division requires design professionals to attest to the fact that all changes to structural, access, or fire and life safety portions of a project have received division approval. Additionally, the division requires that project inspectors monitor changes to plans and notify design professionals if any changes appear to require division approval. The new procedure also outlines the division's process for recording approval of plan changes. Specifically, the division will retain a copy of approved changes as will the project inspector. However, the division has not yet completed a process to ensure it receives all relevant construction changes prior to the start of construction.

6-Month Agency Response

In a procedure adopted in November 2012, the division outlined its process for recording approval of plan changes. Specifically, the division's policy states it will retain a copy of approved changes as will the project inspector. The division will also require design professionals to attest to the fact that all changes to structural, access, or fire and life safety portions of a project have received division approval. Additionally, the division will require that project inspectors monitor changes to plans and notify design professionals if any changes appear to require division approval. However, as described in Recommendation 1.4.a, the division has not yet completed a process to ensure it receives all relevant construction changes prior to the start of related construction. (See 2013-406, p. 47)

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Recommendation #8 To: General Services, Department of

To appropriately oversee changes to approved plans, the division should develop policies and procedures to ensure that it documents its approval of all relevant plan changes.

6-Month Agency Response

In a procedure adopted in November 2012, the division outlined its process for recording approval of plan changes. Specifically, the division's policy states it will retain a copy of approved changes, as will project inspectors. (See 2013-406, p. 47)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

Recommendation #9 To: General Services, Department of

To ensure that the division performs a final review in all disciplines, the division should require and provide a means for recording final plan review of fire and life safety and access compliance-related work in the database.

1-Year Agency Response

This recommendation has been fully implemented. At the time of the BSA's audit, the DSA's supervisors or other technical experts conducted quality-control checks (referred to as final review) in all three disciplines prior to returning plans for needed corrections to design professionals. However, a record of this activity was only maintained in the database for the structural safety discipline. Recently, the DSA revised its process and database to ensure that the final review activity for the fire and life safety and access compliance disciplines are also recorded in the project database (See Attachment III).

California State Auditor's Assessment of 1-Year Status: Fully Implemented

The auditee provided evidence of the database changes mentioned above.

6-Month Agency Response

According to the department, the division is working with information technology staff to develop additional fields in its project database to capture final review dates. The division estimates it will complete this task in June 2013. (See 2013-406, p. 47)

California State Auditor's Assessment of 6-Month Status: Pending

Recommendation #10 To: General Services, Department of

To ensure that staff are current on building standards, the division should document its staff 's participation in building standards update trainings by maintaining attendance rosters.

6-Month Agency Response

The division submitted to the state auditor instructions provided to staff via e-mail informing them that the division requires an attendance sheet be kept as a record of who attends trainings and that the division maintains this sheet in training records. Additionally, the division submitted an example of the attendance sheet used at a recent training. (See 2013-406, p. 48)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

All Recommendations in 2011-116.2

Agency responses received after June 2013 are posted verbatim.