Report 2012-122 Recommendation 18 Responses

Report 2012-122: Mental Health Services Act: The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance (Release Date: August 2013)

Recommendation #18 To: Health Care Services, Department of

Health Care Services should develop standardized data collection guidelines or regulations, as appropriate, that will address inconsistencies in the data that counties report to the State. In developing these guidelines or regulations, Health Care Services should consult with the Accountability Commission to ensure that data collected reasonably fulfill statewide evaluation purposes.

Annual Follow-Up Agency Response From October 2019

DHCS continues to work closely with counties, providing in-depth technical assistance to improve CSI and DCR data quality. Counties were issued CSI Data Quality Improvement Issue reports which identified county-specific data entry issues and the steps necessary to correct the errors identified. Forty-three counties submitted Plans of Correction and DHCS is monitoring the timelines for correction. The contractor for CSI/DCR Data Quality Improvement, MHDA, has produced quarterly reports which contain the client counts, service counts, and percent error of data submissions to support monitoring and assistance for counties in improving data submission. In addition to this, MHDA has updated the DCR User Manual and provided a technical assistance webinar for counties. MHDA also created EPLD templates for DCR data in the Structured Query Language Environment at DHCS so that DHCS staff can better monitor county data quality data. For the remainder of FY 2019-20, MHDA will train DHCS on use of the EPLD template, so county assistance can fully transition to DHCS staff. MHDA will continue to work closely with DHCS on CSI data monitoring and communicating with counties as needed. In FY 2020-21, MHDA and DHCS will continue to collaborate to transition CSI/DCR county technical assistance and training functions to DHCS staff. CSI corrective action plans will be created for counties and MHDA will conduct CSI data submission webinars for counties and provide technical assistance to vendors regarding business rules. For DCR, MHDA will provide additional training materials and quarterly trainings to counties in addition to data quality reports.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Annual Follow-Up Agency Response From August 2019

DHCS continues to work closely with counties, providing in-depth technical assistance to improve the quality of the CSI and DCR data. Specifically, counties were issued CSI Data Quality Improvement Issue reports which identified county-specific data entry issues and the steps necessary to correct the errors identified. Forty-three counties submitted Plans of Correction and DHCS is monitoring the timelines for correction. The contractor for CSI/DCR Data Quality Improvement and MHDA has produced quarterly reports which contain the client counts, service counts, plus percent error of data submissions to further enable continued monitoring and assistance for counties and their progress towards improving data submission. In addition to this, MHDA has updated the DCR User Manual and provided a technical assistance webinar for counties. MHDA also created EPLD templates for DCR data in the Structured Query Language Environment at DHCS so that DHCS staff can better monitor county progress and compliance with submitting quality data. For the remainder of FY 2019-20, MHDA will provide training to DHCS regarding the use of the EPLD template so that assistance to counties can fully transition to DHCS staff. MHDA will also continue to work closely with DHCS in monitoring the improvement of CSI data and communicating with counties as needed.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Health Care Services is taking steps to improve and standardize the data it collects from counties.

Annual Follow-Up Agency Response From November 2018

DHCS contracted with a vendor to perform data clean up on the data included in the DHCS CSI and MHSA DCR. The purpose of this project is to provide DHCS, the MHSOAC, and the counties with accurate and complete data that may be used to track performance outcomes for Community Services and Supports, Full Service Partnership clients. In addition, the contract will develop templates and provide technical assistance to counties regarding data collection to ensure that data is being submitted consistently. Counties have already received comprehensive data error reports, and technical assistance calls to support the interpretation and use of these reports are currently underway. The project will be completed by December 2020. DHCS sent corrective action plan letters to counties in October 2018 identifying the errors for FY 2015-16 and 2016-17 that need to be corrected and to ask the counties to submit a corrective action plan to correct those errors and to explain processes they will implement to ensure quality data going forward.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Annual Follow-Up Agency Response From October 2015

DHCS is planning to meet with the Mental Health Services Oversight and Accountability Commission (MHSOAC) to review the current guidance provided to counties regarding data collection and reporting to the DCR to determine the need to develop more standardized guidance. DHCS will develop more standardized guidance if it determines in consultation with MHSOAC that such guidance is needed.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

As part of the review and development of MHSA regulations, DHCS continues to consult with the Accountability Commission to determine any regulatory changes needed to ensure data are collected consistently and in support of the Evaluation Master Plan.

If needed, draft regulations will be developed in consultation with the Accountability Commission and made available for public comment in the spring of 2015, with final regulations adopted during the fall of 2016.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

DHCS continues to work with the Accountability Commission on the development of MHSA regulations. As part of the review and development of MHSA regulations, DHCS continues to consult with the Accountability Commission to determine any regulatory changes needed to ensure data are collected consistently and in support of the Evaluation Master Plan. Given the significant time required to develop the audit regulations, the development of the data collection regulations have been delayed. DHCS expects to begin developing regulations during the summer of 2014 with final regulations to be adopted beginning in the winter of 2014.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

DHCS will consult with the Accountability Commission to determine if regulation changes are necessary to ensure data are collected consistently and in support of the Evaluation Master Plan. If needed, draft regulations will be made available for public comment in the spring of 2014. We anticipate that final regulations will be adopted during the fall of 2014.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2012-122

Agency responses received are posted verbatim.