Report 2014-111 All Recommendation Responses

Report 2014-111: California Department of Public Health: It Has Not Effectively Managed Investigations of Complaints Related to Long-Term Health Care Facilities (Release Date: October 2014)

Recommendation #1 To: Public Health, Department of

To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by January 1, 2015, Public Health should establish and implement a formal process for monitoring the status and progress in resolving open facility-related complaints and ERIs at all district offices. This process should include periodically reviewing a report of open complaints and ERIs to ensure that all complaints and ERIs are addressed promptly.

1-Year Agency Response

In July 2015, CDPH provided to all district offices an Open Complaints Data (OCD) Query Tool.

The OCD Query Tool has:

- current data - refreshed weekly

- a summary table showing Open Complaints by District Office and SFY Received

- a summary table highlighting data clean-up issues

- an exportable detail file for taking action on the open complaints and clean-up issues

OCD resources include:

- User Instructions

- Data Dictionary

- Feedback Survey

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Public Health provided documentation showing periodic review of district offices' status and progress in resolving open facility-related complaints and ERIs.

6-Month Agency Response

On May 8, 2015, CDPH posted district office -specific data to the stakeholder website page. CDPH

Branch Chiefs use this district office-specific data with the district office managers to manage

performance.

http://www.cdph.ca.gov/programs/Pages/CHCQPerformanceMetrics.aspx

California State Auditor's Assessment of 6-Month Status: Pending

Although Public Health provided a link to its website where it posts district office-specific data, it did not provide documentation to demonstrate that it established and implemented a formal process for monitoring the status and progress to ensure that all complaints and ERIs are addressed promptly.

60-Day Agency Response

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of long-term care health facility complaints and ERI investigations for the first quarter of fiscal year 2014-2015.

By January 31, 2015, CDPH will provide district-specific data to the district offices to use as a management tool. CDPH will work with the district offices to monitor these performance metrics.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #2 To: Public Health, Department of

To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by January 1, 2015, Public Health should improve the accuracy of information in the spreadsheet that PCB uses to track the status of complaints against individuals and review the reports of open complaints to ensure that all complaints are addressed promptly.

Annual Follow-Up Agency Response From September 2016

The Professional Certification Branch (PCB) modified the data collection process to improve tracking of timeliness of open investigations and continues to use the data to monitor timeliness of open investigations. PCB also upgraded an entry-level position to an analytical position to analyze the data entered and retrieved from the spreadsheet and to reconcile data from the spreadsheet with the database and reports. This analyst is distinct from staff that enters the data. In addition, the number of people authorized to enter data was reduced; training regarding the relationship of data to produced reports and database and need for accuracy was provided; a full journey level analyst was hired to provide administrative assistance and the Intake Manager now provides oversight of data contained on spreadsheet and randomly audits entries monthly to promote accuracy. Furthermore, reports were created to better monitor aging and quarterly reports are published on internet to identify volume, timeliness, and existing workload. Public Health to provide documentation to substantiate that this is fully implemented.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

PCB modified data collection process to improve tracking of timeliness of open investigations and continues to use the data to monitor timeliness of open investigations. PCB modified data collection by adding columns in the spreadsheet to include: Appeal, Drop Down Menu allows a Yes or No response to indicate an appeal was received from subject of the complaint, Appeal Status, Depicts final status of appeal and whether proposed decision was altered by Dept. (Altered, Denied, Granted, Settled, or Withdrawn), Final Decision Outcome, Reflects outcome (action) listed in Final Decision issued by Dept. following the hearing, Final Decision Date, Identifies the date of the Final Decision issued by Dept. following the hearing, Type of Finding (A/N/M, 1AN, 2AM, 3NM, 4ANM). For those investigations that result in a Finding being included on the State Nurse Aide Registry, this field identifies type of finding (abuse [A], neglect [N], misappropriation [M], or a combination of findings). Columns were added to identify type of finding included on the State Nurse Aide Registry and information related to the receipt and outcome of a request for an appeal. PCB upgraded an entry-level position to an analytical position to analyze the data entered and retrieved from the spreadsheet and to reconcile data from the spreadsheet with the database and reports. This analyst is distinct from staff that enters the data. In addition, the number of people authorized to enter data was reduced; training regarding the relationship of data to produced reports and database and need for accuracy was provided; full journey level analyst was hired to provide administrative assistance and the Intake Manager now provides oversight of data contained on spreadsheet and randomly audits entries monthly to promote accuracy. Furthermore, reports were created to better monitor aging and quarterly reports are published on internet to identify volume, timeliness, and existing workload.

California State Auditor's Assessment of 1-Year Status: Pending

Public Health did not provide documentation to substantiate its claim that the recommendation is fully implemented. Specifically, it did not provide any documentation to demonstrate that it provided training to staff to emphasize the need for accuracy of the data. Public Health also did not provide documentation to demonstrate its claim that the intake manager now provides oversight of data contained in its tracking spreadsheet and randomly audits entries monthly to promote accuracy.

6-Month Agency Response

PCB modified its data collection process to improve tracking & monitoring of the timeliness of open

investigations.

PCB modified the data collection by adding additional columns in the spreadsheet to include:

- Appeal

o Drop Down Menu allows a Yes or No response to indicate an appeal was received from the subject of the complaint

- Appeal Status

o Depicts the final status of the appeal & whether the proposed decision was altered by the Department (Alternated, Denied, Granted, Settled, or Withdrawn)

- Final Decision Outcome

o Reflects outcome (action) listed in Final Decision issued by the Department following the hearing

- Final Decision Date

o Identifies the date of the Final Decision issued by the Department following the hearing

- Type of Finding (A/N/M, 1AN, 2AM, 3NM, 4ANM)

o For those investigations that result in a Finding being included on the State Nurse Aide

Registry, this field identifies the type of finding (abuse [A] neglect [N] misappropriation [M] or a combination of findings)

Columns were added to identify the type of finding included on the State Nurse Aide Registry & information related to the receipt & outcome of a request for an appeal. PCB upgraded an entry level position to an analytical position to perform an analysis of the data entered & retrieved from the spreadsheet & to reconcile the data from the spreadsheet with the information found in the database & reports. In addition, the number of people authorized to enter data was reduced; training regarding the relationship of the data to the produced reports & database & need for accuracy was provided; the full journey level analyst hired to provide administrative assistance & the Intake Manager provide oversight of the data contained on the spreadsheet & randomly audit entries on a monthly to promote accuracy. Furthermore, reports were created to better monitor aging & quarterly reports are published on the internet to identify volume timeliness & existing workload.

California State Auditor's Assessment of 6-Month Status: Pending

Health Care Services' PCB indicated it has modified its process for tracking and monitoring data for its open investigations and creates reports of aging complaints. These reports are published on its website. However, Health Care Services has not yet provided documentation to support that it has improved the accuracy of its tracking spreadsheet.

60-Day Agency Response

PCB has modified its data collection process to improve tracking of the timeliness of the open investigations. Additionally, PCB clearly defined the data elements that are collected and recorded in the tracking spreadsheet and restricted edit permissions. On August 26, September 19, and November 13, 2014, PCB issued emails to all Investigations Section staff to with instructions for improving and monitoring monthly the accuracy of the data in the master spreadsheet.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #3 To: Public Health, Department of

To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should establish a specific time frame for completing facility-related complaint investigations and ERI investigations and inform staff of the expectation that they will meet the time frame. Public Health should also require district offices to provide adequate, documented justification whenever they fail to meet this time frame.

Annual Follow-Up Agency Response From October 2020

The California Department of Public Health (CDPH) follows the complaint investigation timeframe established by Senate Bill 75, Statutes of 2015. Compliance with the requirement to complete all long-term care complaint investigations within 60 days of receipt can be monitored at https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/FieldOperationsComplaints_ERIs.aspx.

CDPH is working hard to investigate all FRIs; however, we disagree with the recommendation to establish timelines for ERI investigations. CDPH is implementing strategies to eliminate backlogged complaints and ERIs with the goal of investigating and closing the backlogged cases by the end of 2021. Federal requirements on initiating and completing entity-reported incident investigations vary, based on facility type and how the intake is prioritized. CDPH continues to work on the timeliness of these investigations through enhanced monitoring of workload activities, public reporting of workload performance, and improved district office implementation.

In response to the open complaints against individuals, there are no state or federal requirements to complete investigations about certified individuals within specified timelines. The Professional Certification Branch (PCB) within the CDPH continues to closely monitor the investigations performance metrics. PCB has investigation policies and procedures and PCB posts their performance metrics on CDPH's website on a quarterly basis.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From November 2018

The California Department of Public Health (CDPH) follows the complaint investigation timeframe established by Senate Bill 75, Statutes of 2015. Compliance with the requirement to complete all long-term care complaint investigations and all long-term care entity-reported incidents within 90 days of receipt can be monitored at https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/FieldOperationsComplaints_ERIs.aspx.

CDPH disagrees with the recommendation to establish licensure timelines for ERI investigations. Federal requirements on initiating and completing entity-reported incident investigations vary, based on facility type and how the intake is prioritized. CDPH continues to work on the timeliness of these investigations through enhanced monitoring of workload activities, public reporting of workload performance, and improved district office implementation.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

We assessed the status of this recommendation as partially implemented because Public Health revised its complaint investigation policies and procedures to reflect the revised time frames specified in SB 75, statutes of 2015. However, SB 75 does not require such time frames for ERI investigations and Public Health does not agree with our recommendation to establish time frames for ERI investigations. Public Health's website shows that during fiscal year 2017-18, it completed 28 percent of ERI investigations within 90 days and 43 percent within 180 days. Its website also shows that of the 10,705 ERI investigations that were open during the quarter ending June 30, 2018, more than 6,000 ERI investigations had remained open for more than 180 days. As we state on page 32 of our report, we believe that Public Health's lack of accountability has contributed to its district offices' failure to complete investigations within reasonable time.

Annual Follow-Up Agency Response From December 2017

Public Health policies for complaint investigations aligned with Senate Bill 75, Statutes of 2015. As of July 2017, Public Health is 93% compliant with the requirement to complete all immediate jeopardy long-term care complaint investigations within 90 days of receipt (see https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/FieldOperationsComplaints_ERIs.aspx).

As part of process improvement and performance management efforts in this area, in August 2016, Public Health issued policy and procedure 100.3.75 regarding the requirements for complaint investigations and the formation of complaint teams (Complaint Team PP, attached). The complaint team at each district office comprises Health Facilities Evaluator Nurses who investigate complaint uninterrupted from start to finish.

As noted in our initial response Public Health CDPH disagrees with the recommendation to establish timelines for ERI investigations. CDPH continues to work on the timeliness of these investigations through enhanced monitoring of workload activities, public reporting of workload performance, and improved district office implementation.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

We assessed this recommendation as partially implemented because Public Health said it disagrees with part of the recommendation regarding establishing timelines for ERI investigations. As we state on page 32 of our report, we believe that Public Health's lack of accountability has contributed to its district offices' failure to complete investigations within reasonable time.

Annual Follow-Up Agency Response From September 2016

Public Health developed and implemented policies and procedures with targeted time frames to ensure investigations are conducted timely across priority levels that lack them.

Additionally, Public Health developed complaint teams at each Public Health District Office to ensure complaint time frames are being met, increase efficiencies, and manage complaint workload. Public Health holds District Administration and District Managers monthly meetings to inform staff of the policies and procedures and expectation that they will meet the time frame.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Although Public Health has revised its policies and procedures to reflect the new timelines required by Senate Bill 75, Statutes of 2015, for facility-related complaint investigations, it did not provide documentation to demonstrate that it has established time frames for ERI investigations. When we followed-up, Public Health staff acknowledged that it has not established time frames for ERI investigations.

1-Year Agency Response

On June 24, 2015, SB 75 chaptered, which created complaint investigation completion timeframes that will be implemented on a phased in basis over the next few years. Specifically SB 75 requires:

- L&C to complete long-term care (LTC) IJ level complaint investigations that are received on or after July 1, 2016 within 90 days of receipt.

- All other LTC complaints received between July 1, 2017 and July 1, 2018, must be completed within 90 days of receipt.

- After July 1, 2018 complaint investigations must be completed within 60 days of receipt.

- These time periods may be extended up to an additional 60 days if the investigation cannot be completed due to extenuating circumstances.

- Any citation issued must be completed within 30 days of the investigation.

- CDPH to annually report data on department's compliance with the complaint investigation completion timelines beginning in 2018-2019.

- If CDPH does not meet the timeframes we must document the extenuating circumstances explaining why and provide written notice to the facility and the complainant, if any, of the basis for the extenuating circumstances and the anticipated completion date.

CDPH is revising its complaint investigation policies and procedures to reflect the revised timeframes. The revision will be published by the end of 2015.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

We assessed the status of this recommendation as partially implemented because Public Health states that it is revising its complaint investigation policies and procedures to reflect the revised time frames specified in SB 75, statutes of 2015. Additionally, we followed up with Public Health to clarify why it indicated the status of this recommendation as "will not implement." Public Health indicated although SB 75 establishes time frames for completing investigations of complaints against facilities, ERIs were not part of SB 75 and Public Health does not plan to establish time frames for ERIs. As we state on page 32 of our report, we believe that Public Health's lack of accountability has contributed to its district offices' failure to complete investigations within reasonable time.

6-Month Agency Response

On May 8, 2015, CDPH posted district office-specific data to the stakeholder website page. CDPH Branch Chiefs use this district office-specific data with the district office managers to manage performance.

www.cdph.ca.gov/programs/Pages/CHCQPerformanceMetrics

California State Auditor's Assessment of 6-Month Status: Will Not Implement

Public Health does not provide any reason for why it will not implement our recommendation to establish a specific time frame for completing facility-related complaints and entity-reported incident (ERI) investigations. As we state on page 32 of the report, we believe that Public Health's lack of accountability has contributed to its district offices' failure to complete investigations within reasonable time. Public Health's response indicates that in May 2015 it posted district office-specific data to the website and that branch chiefs use this data with district office managers to manage performance. However, without first defining specific time frames that it considers to be timely, it is unclear how district office managers will manage performance and effectively ensure that staff promptly complete all investigations.

60-Day Agency Response

CDPH is committed to respond to facility-related complaints and ERIs in a timely manner. Rather than establish specific time frames for investigations at this point. CDPH has developed performance metrics that promote staff accountability without compromising the quality and the thoroughness of the work.

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of long-term care health facility complaint and ERI investigations for the first quarter of fiscal year 2014-2015.

By January 31, 2015, CDPH will provide district-specific data to the district offices to use as a management tool. CDPH will work with the district offices to monitor these performance metrics.

California State Auditor's Assessment of 60-Day Status: Will Not Implement

Public Health fails to provide any reason for why it does not agree with our recommendation to establish a specific time frame for completing facility-related complaints and entity-reported incident (ERI) investigations. As we state on page 32 of the report, we believe that Public Health's lack of accountability has contributed to its district offices' failure to complete investigations within reasonable time. Public Health states that it will provide data, including data on timeliness, to its district offices as a management tool. However, without first defining what it considers to be timely, the steps that Public Health outlines will be ineffective in ensuring that district offices promptly complete all investigations.

Recommendation #4 To: Public Health, Department of

To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should develop formal written policies and procedures for PCB to process complaints about certified individuals in a timely manner. These policies and procedures should include specific time frames for prioritizing and assigning complaints to investigators, for initiating investigations, and for completing the investigations. Public Health should also inform staff of the expectation that they will meet these time frames. It should require PCB to provide adequate, documented justification whenever PCB fails to meet the time frames.

Annual Follow-Up Agency Response From November 2018

The Provider Certification Branch (PCB) performance metrics are posted at https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/PCB_MetricsDashboard.aspx. The metrics show that in state fiscal year (SFY) 2017, PCB received an additional 332 complaints when compared to SFY 2016. The number of open complaints increased from 246 at the start of SFY 17 to 467 at the end of SFY 17. The average age of the open complaints fluctuated from 61 calendar days at the start of SFY 17 to 90 calendar days at the end of SFY 17. As of September 1, 2018, PCB had no open complaints prior to SFY 17, with the exception of four complaints with law enforcement holds. PCB will continue to review policies and procedures for any quality improvement opportunities.

There are no federal requirements to complete investigations within specified timeframes nor has the Centers for Medicare and Medicaid Services (CMS) reported any concerns to the California Department of Public Health (CDPH) with PCB investigations. Given that enhanced oversight and interventions have led to sustained, significant reduction in open complaints as well as continually improved timeliness of complaint investigations since the issuance of findings for this audit in SFY 14, the CDPH has determined that specified timelines are not necessary.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

Public Health's website shows that of the 467 open investigations within PCB during the quarter ending June 30, 2018, 197 were open for more than 90 days, including 56 that were open for more than 180 days. As we state on page 33 of the report, it is especially important for Public Health's PCB to develop formal written policies and procedures, including the establishment of specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints about certified individuals in a timely manner.

Annual Follow-Up Agency Response From December 2017

Public Health has developed investigation policies and procedures and continues to revise them as needed. The Provider Certification Branch (PCB) performance metrics are posted at https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/PCB_MetricsDashboard.aspx The metrics demonstrate that PCB continues to reduce the number of open complaints and decrease the time taken to complete a complaint investigation. PCB has reduced the number of open complaints from 761 at the start of state fiscal year (SFY) 16 to 246 by the end of SFY 17. PCB continues to reduce the average calendar days to investigate a complaint from186 at the start of SFY 16 to 61 by the end of SFY 17. As of August 1, 2017, PCB has no open complaints prior to SFY 17 with the exception of four complaints with law enforcement holds from SFY 16 that prevent them from being completed at this time. PCB will continue to review policies and procedures for any quality improvement opportunities. There are no federal requirements to complete investigations within specified time frames nor has the Centers for Medicare and Medicaid Services (CMS) reported any concerns to CDPH with PCB investigations. Given that enhanced oversight and interventions have led to significant reduction in open complaints as well as continually improved timeliness of complaint investigations, CDPH has determined that specified timelines are not necessary.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although Public Health indicates the status of this recommendation as fully implemented, Public Health's response indicates that it disagrees with establishing specific time frames for investigations by its statement that it has determined that specified timelines are not necessary. Further, Public Health's website shows that as of June 30, 2017, it still had 246 open complaints, 47 of which have remained open for more than 90 days. As we state on page 33 of the report, it is especially important for Public Health's PCB to develop formal written policies and procedures, including the establishment of specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints about certified individuals in a timely manner.

Annual Follow-Up Agency Response From September 2016

As noted in our previous response, PCB has developed investigation policies and procedures and updates them as needed. PCB's performance metrics are posted at http://www.cdph.ca.gov/programs/Pages/CHCQPerformanceMetrics.aspx. The metrics demonstrate that PCB continues to reduce the number of open complaints and decrease the time taken to complete a complaint investigation. PCB reduced the number of open complaints from 1,097 at the start of fiscal year (FY) 2014-15 to less than 600 in the third quarter of FY 2015-16, even as the number of new complaints continues to increase. Further PCB reduced the average days to investigate a complaint from receipt to completion from 285 in the second quarter of FY 2013-14 to 133 in the third quarter of FY 2015-16. As of September 1, 2016, PCB has no open complaints from prior to 2015 and only five open complaints from 2015. Law enforcement holds are delaying PCB's completion of the 2015 investigations. These improved outcomes are the result of increased staffing, quarterly performance reporting and monitoring, and enhanced management oversight. PCB expects to continually reduce the number of open complaints and improve the timeliness of complaint investigations. Further, as the audit report noted, there are no federal requirements to complete investigations within specified time frames nor has the Centers for Medicare and Medicaid Services (CMS) reported any concerns to CDPH with PCB investigations. Given that enhanced oversight and interventions have led to continually improved timeliness of complaint investigations, CDPH has determined that specified timelines are not necessary.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

As we state on page 33 of the report, it is especially important for Public Health's PCB to develop formal written policies and procedures, including the establishment of specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints about certified individuals in a timely manner. Although Public Health's data show a decline in the backlog of open cases, as reported in its third quarter metrics for fiscal year 2015-16 posted on Public Health's website, the data also show that it closed 820 cases, or 68 percent of the 1,195 complaints closed during the period, more than 90 days after receipt of the complaint. We continue to stand by our recommendation that Public Health's policies should include specific time frames for prioritizing and assigning complaints to investigators, for initiating investigations, and for completing the investigations.

1-Year Agency Response

PCB's documented policies and procedures are completed and PCB will update them any time procedures are revised. The attached "PCB Intake Staff Services Analyst Procedure" and "PCB Program Technician Procedures" are samples of PCB procedures. Additional PCB policies and procedures total hundreds of pages; we can provide additional documents at your request.

CDPH undertook a quality improvement project to address the timeliness of complaint investigations; the same process is applicable to ERIs. Using a "plan, do, check, act" continuous quality improvement cycle, in September 2015 we implemented the revised process in selected district offices ("do" phase). We will review the effectiveness of the revised process and revise if needed and roll out to all the district offices ("check" and "act").

CDPH disagrees with establishing specific timeframes for investigations, but continues with our commitment to improve upon the timeliness of investigations.

As seen in our performance metrics posted on our website, trends continue to show a reduction in the amount of open investigations as well as improved timeliness of investigations.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Although Public Health indicates the status of this recommendation as fully implemented, Public Health's response states that it developed policies and procedures, but that it disagrees with establishing specific time frames for investigations. Public Health does not provide a reason for why it disagrees with establishing specific time frames for investigations, but states that it is committed to improving timeliness of investigations. However, as we state on page 33 of the report, it is especially important for Public Health's PCB to develop formal written policies and procedures, including the establishment of specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints about certified individuals in a timely manner.

6-Month Agency Response

PCB has completed documenting many of its policies and procedures and is currently working with the Results Group to identify methods to enhance efficiencies for the investigative process before finalizing the documented procedures. CDPH is awaiting the final report from the Results Group. Findings from the report may result in modifications to existing policies.

CDPH disagrees with establishing specific timeframes for investigations but continues with our commitment to improve upon the timeliness of investigations.

As seen in our performance metrics posted on our website, trends continue to show a reduction in the amount of open investigations as well as improved timeliness of investigations.

California State Auditor's Assessment of 6-Month Status: Pending

Public Health's response indicates that it has not yet finalized the documented procedures for PCB. We look forward to receiving documentation of the procedures once Public Health finalizes them. Additionally, Public Health does not provide a reason for why it disagrees with establishing specific time frames for investigations but states it is committed to improving timeliness of investigations. However, as we state on page 33 of the report, it is especially important for Public Health's PCB to develop formal written policies and procedures, including the establishment of specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints against certified individuals in a timely manner. Without defining what it considers to be timely completion of investigations, it is unclear how Public Health will ensure that PCB complies with federal regulations.

60-Day Agency Response

PCB continues to develop and implement written policies and procedures for investigating complaints against certified individuals. In September 2014, CDPH began work with The Results Group to review the current processes, practices, policies, and data technology. The Results Group has documented the existing processes and by December 31, 2014, will provide PCB with recommendations to enhance efficiencies, data collection and maintenance, and timeliness.

CDPH disagrees with establishing specific timeframes for investigations.

CDPH is committed to investigating complaints against certified individuals in a timely manner. Rather than establishing specific time frames for investigations at this point, CDPH has developed performance metrics that promote staff accountability without compromising the quality and the thoroughness of the work.

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of complaints against certified individuals for the first quarter of fiscal year 2014-2015.

California State Auditor's Assessment of 60-Day Status: Pending

Public Health fails to recognize the importance of our recommendation. As we state on page 33 of the report, it is especially important for Public Health's Professional Certification Branch (PCB) to establish specific steps and time frames for completing those steps because federal regulations require Public Health to investigate complaints against certified individuals in a timely manner. Without defining what it considers to be timely completion of investigations, it is unclear how Public Health will ensure that PCB complies with federal regulations.

Recommendation #5 To: Public Health, Department of

To ensure that district offices address ERIs consistently and to ensure that they investigate ERIs in the most efficient manner, Public Health should assess whether each district office is appropriately prioritizing ERIs. Specifically, it should determine, on a district-by-district basis, whether district offices' assigning ERIs a priority level that requires an on-site visit is justified. This assessment should also determine whether each district office is prioritizing ERIs appropriately when determining that on-site investigations are not necessary.

1-Year Agency Response

CDPH continues to review complaints and ERIs for appropriate prioritization level and timely onsite visits through by a quarterly lookback review of a sample of complaints and ERIs. We began this review with Los Angeles County and expanded it to all district offices beginning with the first quarter of FY 2015/16.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

CDPH continues to review complaints and ERI for appropriate prioritization level and timely onsite visit. This review is currently conducted for LA County District offices. After April 2015, the review will expand to the other district offices ensuring all district offices are reviewed each quarter.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #6 To: Public Health, Department of

To ensure that district offices address ERIs consistently and to ensure that they investigate ERIs in the most efficient manner, Public Health should use the information from its assessment to provide guidance to district offices by October 1, 2015, on best practices for consistent and efficient processing of ERIs.

Annual Follow-Up Agency Response From September 2016

On August 4, 2016, Public Health released updated policy and procedure for the Abbreviated Standard Survey (federal complaint process) in Skilled Nursing/Nursing Facilities. These policy and procedures provide specific guidance to all district offices on timelines, investigation, documentation, and completion using best practices for consistent and efficient processing of entity-reported incident (ERIs). These policies include quality measures for continuing monitoring and evaluating performance according to these policy and procedure.

Licensing and Certification's district office supervisors received training on these updated policy and procedures between January-July 2016.

This topic was also discussed at an all-state District Administrator/District Manager meeting in August 2016, which reinforced the role of the supervisor in triage and review of prioritization as well as the quality of the process for timely completion.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

CDPH undertook a quality improvement project to address the timeliness of complaint investigations; the same process is applicable to ERIs. Using a "plan, do, check, act" continuous quality improvement cycle, in September 2015 we implemented the revised process in selected district offices ("do" phase). In December 2015, we will review the effectiveness of the revised process and revise if needed and roll out to all the district offices ("check" and "act").

California State Auditor's Assessment of 1-Year Status: Pending

The supporting document that Public Health provided focuses on improving timeliness of complaint investigations. Public Health has not yet provided documentation to support that it has issued guidance to district offices on best practices for consistent and efficient processing of ERIs.

6-Month Agency Response

CDPH continues to review complaints and ERI for appropriate investigation. L&C's Policy and Procedure for complaints was used as the basis of the study's criteria.

This review is currently conducted for LA County District offices. After April 2015, the review will expand to ensure all district offices are reviewed each quarter. Sample selection includes only completed and closed complaints and ERIs.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #7 To: Public Health, Department of

To ensure that district offices address ERIs consistently and to ensure that they investigate ERIs in the most efficient manner, Public Health should review periodically a sample of the priorities that district offices assign to ERIs to ensure compliance with best practices.

Annual Follow-Up Agency Response From November 2018

The California Department of Public Health conducts monthly data analysis of Entity Reported Incidents (ERIs) and complaints reviewed by district office supervisors in a peer review process by having every supervisor review the priorities assigned to the complaint/ERI previously reviewed by the front-line supervisor. Complaints or ERIs must have different original and reviewing supervisors to ensure peer review is complete. The data is shared with district office management staff as well as CHCQ leadership. Additionally, the Quality Improvement team provides biannual reviews of 180 total complaints and ERI samples derived from all district offices (9 per office) to review quality of work and prioritization of complaints and ERIs. In 2018, supervisors were given training on assigning priority levels through the "Training Supervisor Training" and "New Supervisor Academy".

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From December 2017

Public Health continues to conduct periodic reviews of ERIs and complaints for quality and adherence to policy. In September and November, 2016, the Quality Improvement (QI) team conducted the Abbreviated Survey Review project. We selected 19 abbreviated surveys for review, see report attached. We found all 19 to comply with state and federal case closure requirements.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

The documentation that Public Health provided does not support its claim of full implementation. Specifically, the documentation provided does not show that Public Health is reviewing a sample of the priorities that district offices assign to ERIs. Without doing so, Public Health cannot ensure that district offices address ERIs consistently, efficiently, and comply with best practices.

Annual Follow-Up Agency Response From September 2016

Public Health continues to review complaints and ERIs for appropriate prioritization level and timely onsite visits through by a quarterly lookback review of a sample of complaints and ERIs.

Training presented during the Center's Supervisor Academy as well as the Quarterly Training Supervisor Academy in 2016, included detailed curriculum, instruction and practical examples for triaging complaints/entity-reported incidents (ERIs). Additional content included evaluation of the timeliness of the onsite visit investigation

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Public Health provided documentation that it identified best practices for prioritizing ERIs, provided the training to its staff in January 2016, and updated its policies in August 2016. However, it did not provide documentation that subsequent to its training and updated policies, it has performed periodic reviews of a sample of the priorities that district offices assign to ERIs to ensure compliance with best practices.

1-Year Agency Response

CDPH continues to review complaints and ERIs for appropriate prioritization level and timely onsite visits through by a quarterly lookback review of a sample of complaints and ERIs. We began this review with Los Angeles County and expanded it to all district offices beginning with the first quarter of FY 2015/16.

California State Auditor's Assessment of 1-Year Status: Pending

As we noted in our public reasoning for recommendation #6, Public Health has not yet provided documentation to support that it has issued guidance to district offices on best practices. Until it issues such guidance, it cannot review periodically a sample of the priorities that district offices assign to ERIs to ensure compliance with best practices.

6-Month Agency Response

CDPH continues to review complaints and ERI for appropriate prioritization level, timely onsite visit, and investigations. This review is currently conducted for LA County District offices. After April 2015, the review will expand to the other district offices ensuring all district offices are reviewed each quarter.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #8 To: Public Health, Department of

To protect the residents in long-term health care facilities from potential harm, Public Health should ensure that its district offices have adequate staffing levels for its licensing and certification responsibilities, including staffing levels that allow prompt investigations of complaints. Specifically, Public Health should continue working with CalHR to complete the reclassification of district offices' investigator supervisor and manager positions and then quickly fill the vacant positions at district offices.

Annual Follow-Up Agency Response From October 2020

1. The 2020-21 Budget Act added 115.6 positions to be phased in over three years; 20 positions will be phased in over fiscal year 2020-21, 40 will be phased in over fiscal year 2021-22, and the final 55.6 will be phased in during fiscal year 2022-23. The additional staffing will:

- Improve processing times of complaint and FRI investigations;

- Improve responsiveness to long-term care complaints;

- Complete mandated workload efficiently, including licensure workload for changes of service and change of location etc.

2. By October 2019, the Center for Health Care Quality had recruited and filled a number of vacant Health Facility Evaluator Nurse (HFEN) positions, which reduced the vacancy rate to 2.9%. CDPH was authorized an additional 75 surveyor positions, which increased the vacancy rate to 6.9% and we are working to fill those positions now. The new positions will fulfill the need for staff at district offices.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From September 2019

The Health Facilities Evaluator Nurse (HFEN) applications have increased in general throughout the state and HFEN vacancy rates have fallen in all areas of the state in general. However, the San Francisco Bay Area has proven to be the most challenging in meeting program goals due to the highly competitive labor market for nurses in the economic region. The San Francisco Bay Area's HFEN vacancy rates in June were as follows:

East Bay: 30% (18/60 PYs)

San Francisco: 17% (6/35 PYs)

Santa Rosa: 22% (9/41 PYs)

In addition, supervisor vacancies as of June were as follows:

East Bay: 22% (7/9 PYs)

San Francisco: 83% (1/6 PYs)

Santa Rosa: 14% (6/7 PYs)

The Center for Health Care Quality (CHCQ) is contracting with a consultant who will develop strategies for improving hiring practices in high-vacancy districts, such as the San Francisco Bay Area, and CHCQ has been investing significantly into recruiting in the Bay Area. Currently 60 percent of the web traffic to the predominant recruiting website (www.CDPHnurse.org) originates from the Bay Area. CHCQ continues to expand outreach efforts in the Bay Area and refine messaging to increase numbers of applications and improve conversion rates. To ensure that workloads are addressed, CHCQ has offset the vacancies through several methods: 1. Borrowing surveyors from other districts, 2. Increasing the classifications we hire as surveyors, including vocational nurses and AGPAs, as we have determined surveyors do not need to be an RN. Public Health hopes to expand the workforce and free up registered nurses to do more work at their level, such as investigative work requiring RN-level of care. Public Health is working with the Department of Health Care Services to merge duty statements for nurses and supervisors and ensure that they are in the same bargaining unit. The nurse classification series revision package was sent to CalHR in April 2019.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Public Health's response does not support its claim of full implementation. Specifically, as Public Health notes in its response, a few of its district offices continue to have high vacancy rates. Further, Public Health did not provide any documentation to support that it has offset the vacancies through 1) borrowing surveyors from other districts; and 2) increasing the classifications, including vocational nurses and AGPAs, it hires as surveyors.

Annual Follow-Up Agency Response From November 2018

In 2017, Center for Health Care Quality (CHCQ) hired consultants to identify improvements in recruiting and reduce delays between candidate selection and approval for hire.

The average time to process new hires after selection was reduced by 56%; 27 days in 2016 to 12 in 2018. The processing time for Health Facilities Evaluator Nurses (HFENs) improved from an average of 43 days in 2016 to 15 days in 2018, a 65% decrease. The overall vacancy rate decreased by 53%, from 17.4% in July 2016 to 8.2% in July 2018. The HEFN vacancy rate in 2016 and 2018 were 16.48% and 3.22%, respectively, an 80.5% decrease.

CHCQ improved the hiring time and decreased the vacancy rate by:

- Staff participated in statewide job fairs hosted by medical and educational institutions

- California Human Resources (CalHR) eliminated the medical clearance requirement for HFEN candidates

- Sent recruitment postcards to thirteen counties

- Ongoing ads in the Nursing Voice and local newspapers near district offices

- Started using Indeed online ads, and created a virtual job fair for HFEN hiring

- Switched to biweekly ads in the state's Examination Certification Online System

CDPH's annual estimate of workload and staffing via the November Estimate process (https://www.cdph.ca.gov/Programs/CHCQ/LCP/Pages/EstimateReports.aspx) projects programmatic workload functions and positions needed to perform these functions.

At CalHR's direction, CDPH discontinued its reclassification project for investigator supervisor and manager positions in 2016 due to CalHR's Civil Service Initiative (CSI). CSI reduced the overall number of civil service classifications through consolidation. CDPH's proposal included an increase in classifications, conflicting with CSI efforts. CalHR didn't support CDPH's reclassification and did not schedule additional meetings with the Union to adopt the proposed changes. CDPH decreased vacancy rates for HFEN and Health Facilities Evaluator II Supervisory classifications.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

The documentation provided by Public Health shows that overall the district offices had eight vacant supervisory positions of 102 positions and 20 vacant staff HFEN positions of 621 positions as of September 2018. However, some district offices still have large number of vacant supervisory and staff positions. Specifically, 4 of 6 supervisory positions at the San Francisco district office and 4 of 7 supervisory positions at the Santa Rosa district office were vacant as of September 30, 2018. Similarly, 14 of 41 HFEN positions at the Santa Rosa district office and 15 of 60 HFEN positions at East Bay district office were vacant. As we state on page 39 of the report, until Public Health ensures that it has the necessary staffing levels at each district office to address adequately the district office's workload related to licensing and certification of long-term health care facilities, including prompt investigations of complaints, Public Health is hampered in its ability to ensure the safety, health, and well-being of residents living in these facilities.

Annual Follow-Up Agency Response From December 2017

CalHR, as part of civil service improvement initiatives, is focused on reducing the number of civil service classifications through classification consolidation efforts. Since the Public Health reclassification proposal does not reduce the number of classifications in the series, CalHR is not acting on the proposal. Despite the reclassification not moving forward, Public Health has taken numerous other recruitment actions to reduce vacancy rates.

Public Health has successfully streamlined the hiring process by eliminating the health questionnaire with approval from CalHR. Public Health has all complaint team supervisors positions filled. The vacancy rate for the HFEN classification has dropped in 8 of the past 10 months, from 17% to 12% (72 vacancies out of 620 positions) as of September 30, 2017. Also as of September 30, 2017, 43 of 49 Health Facilities Evaluator Manager I and Manager II positions are filled, a vacancy rate of 12%.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Public Health's previous response stated that fiscal year 2015-16 Budget Act authorized 77 health facilities evaluator nurse (HFEN) positions and it had filled 21 positions and had 17 pending hires. Although it currently claims that it has streamlined the hiring process and reduced vacancy rates, it did not provide any supporting documentation.

Annual Follow-Up Agency Response From September 2016

Annually, the Center for Health Care Quality (CHCQ) allocates district office surveyor positions based on a workload needs analysis to ensure staff allocations are appropriate to promptly address all licensing and certification workload, including the investigation of complaints.

Public Health continues to look for ways to expedite the hiring process to fill vacant positions. Public Health's Human Resources Branch staff and CHCQ meet at least monthly to address barriers to hiring. In addition, CHCQ executed two consultant contracts in April 2016 to provide recommendations to improve CHCQ recruitment and onboarding practices.

To date, Public Health has filled 136 of the 240 approved positions from the 2015-2016 Budget Act.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

The 2015-16 Budget Act increased CDPH's positions by 240 to complete licensing and certification workload. Of these 240, on July 1, 2015, 77 health facilities evaluator nurse (HFEN) position were authorized. As of November 6, 2015, CDPH has 21 completed and 17 pending hires of HFENs

The Center will address ongoing recruiting, onboarding and retention issues through two consultant contracts. The Center expects to execute a recruitment contract, and an onboarding and retention contract by December 2015.

CDPH continues to work with CALHR to complete the reclassification of district office investigator, supervisor, and manager positions.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Public Health identifies this recommendation as fully implemented. However, Public Health's response states that it continues to work with CalHR to complete the reclassification of district office investigator, supervisor, and manager positions. Therefore, we have assessed this recommendation as partially implemented.

6-Month Agency Response

Based on the L&C Program November Estimate, the January 2015 Governor's Budget includes a request for additional staff and funding to complete licensing and certification workload.

CDPH continues to work with CALHR to complete the reclassification of district office investigator, supervisor, and manager positions.

CDPH has established a workgroup to look into ways for expediting filling vacant positions at district offices.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

On November 19, 2014, CDPH posted on our website "Vacancy Rates by Position Classification for the Center of Health Care Quality" with data as of September 30, 2014.

CDPH will continue to post this information quarterly.

As of December 1, 2014, CDPH completed examinations for Health Facilities Evaluator II (Supervisor), Health Facilities Evaluator Manager I, and Health Facilities Evaluator Manager II. Certification lists for these classifications are now available and district offices are recruiting for these positions.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #9 To: Public Health, Department of

To protect the residents in long-term health care facilities from potential harm, Public Health should ensure that its district offices have adequate staffing levels for its licensing and certification responsibilities, including staffing levels that allow prompt investigations of complaints. Specifically, Public Health should complete by May 1, 2015, a staffing assessment to identify the resources necessary for district offices to investigate open complaints and ERIs and to promptly address new complaints on an ongoing basis. Public Health should use this assessment to request additional resources, if necessary.

1-Year Agency Response

The 2015-16 Budget Act increased CDPH's positions by 240 to complete licensing and certification workload. Of these 240, on July 1, 2015, 77 health facilities evaluator nurse (HFEN) position were authorized.

CDPH's request for these positions was based on an analysis completed and documented in L&C's November Estimate.

http://www.cdph.ca.gov/pubsforms/fiscalrep/Documents/LC%20November%20Estimate%20for%202015-16%20final%2001-08-15.pdf.

Appendix C, beginning on page 16, describes our detailed methodology for determining total position needs.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

The Governor's 2015-16 Budget, updated at the May Revision, includes a request for funding for 237 state positions and $14.85 million for LA County to conduct L&C state and federal work. These requests were based on an analysis completed and documented in L&C's November Estimate.

http://www.cdph.ca.gov/pubsforms/fiscalrep/Documents/LC%20November%20Estimate%20for%202015-16%20final%2001-08-15.pdf

California State Auditor's Assessment of 6-Month Status: Pending

The link included in Public Health's response shows the method Public Health used to determine the staffing needs. However, the estimates included in the referenced document is at the department level and not at the individual district office level. Further, Public Health did not provide any data or calculations to support the estimates included in the referenced document.

60-Day Agency Response

CDPH's annual estimate of the workload and staffing needs of the Licensing and Certification Field Operations Division will be released with the Governor's Budget on January 10, 2015.

California State Auditor's Assessment of 60-Day Status: No Action Taken

Public Health did not provide any documentation to support its annual estimate of the workload and staffing needs to be released with the Governor's Budget on January 10, 1015. As a result, it is unclear whether Public Health completed a staffing assessment to identify the resources necessary for district offices to investigate open complaints and ERIs and to promptly address new complaints on an ongoing basis.

Recommendation #10 To: Public Health, Department of

To protect the residents in long-term health care facilities from potential harm, Public Health should ensure that its district offices have adequate staffing levels for its licensing and certification responsibilities, including staffing levels that allow prompt investigations of complaints. Specifically, by January 1, 2015, Public Health should establish a time frame for fully implementing the recommendations that its consultant identified related to the processing of complaints about long-term health care facilities.

1-Year Agency Response

The 2015-16 Budget Act increased CDPH's funding for 240 positions and $14.85 million for LA County to conduct L&C work.

The Center for Health Care Quality has developed a staffing model to identify the needs of each district office, and has used this model to allocate the new 240 new positions.

CDPH's work plan for implementing the consultant's recommendations is available at the link below. The work plan includes a timeline for each recommendation.

http://www.cdph.ca.gov/programs/Documents/Amended%20August%202015%20Remediation%20Work%20Plan%20Update.pdf

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

The Governor's 2015-16 Budget, updated at the May Revision, includes a request for funding for 237 state positions and $14.85 million for LA County to conduct L&C state and federal work. These requests were based on an analysis completed and documented in L&C's November Estimate.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

CDPH agrees with this recommendation but not the January 1, 2015 timeline.

On November 7, 2014, CDPH posted on our website a work plan for implementing the assessment report recommendations.

The work plan includes anticipated timelines for the initiation and completion of each recommendation. On November 13, 2014, CDPH held a stakeholder meeting to discuss and receive feedback on the work plan. CDPH will update the work plan monthly with progress reports and any changes to the timeline.

California State Auditor's Assessment of 60-Day Status: Pending

Public Health did not provide any documentation to support the activities it outlines in its response. It also does not specify when it expects to establish a time frame for fully implementing the recommendations that its consultant identified in August 2014. We believe that Public Health should establish a time frame as soon as possible.

Recommendation #11 To: Public Health, Department of

Public Health should take steps to ensure that PCB has the resources necessary on an ongoing basis to complete investigations of complaints against individuals. Specifically, Public Health should assess whether the temporary resources it has received are adequate to reduce the number of open complaints to a manageable level. This assessment should also determine whether permanent resources assigned to PCB are adequate to address future complaints. Public Health should use this assessment to request additional resources, if necessary.

Annual Follow-Up Agency Response From September 2016

As previously stated, the Professional Certification Branch continues to reduce the number of open complaints to a manageable level while adequately addressing newly received complaints. http://www.cdph.ca.gov/programs/Pages/CHCQPerformanceMetrics.aspx

An analysis of actual data regarding the number of complaints received, and investigations pending and completed in each of the last five full fiscal years was used to project data for the current and next three fiscal years. Based on the trends of the actual data and these projections, current staffing levels are adequate to address future complaints.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Public Health provided a budget change proposal (BCP), approved by the Department of Finance, requesting to convert 18 existing limited-term positions in its PCB to permanent positions and requesting two additional positions in its Office of Legal Services to improve the timeliness of investigation of complaints against caregivers. Public Health's BCP states that augmenting the existing analysts with position and spending authority by converting the 18 two-year limited-term positions will allow the PCB to improve the timeliness of complaint investigations from greater than one year to less than three months by fiscal year 2018-19. It also states that adding the two attorney positions to serve as the PCB's house counsel and litigation support will better represent Public Health at administrative appeal hearings.

1-Year Agency Response

The performance metrics posted on our website indicates that PCB continues to reduce the number of open, aged complaints despite the increase of received complaints. As of September 30, 2015, all complaints received in fiscal year 2013/2014 were complete with the exception of three investigations with law enforcement barriers. There are 408 open complaints remaining from fiscal year 2014/2015 and 348 open complaints from fiscal year 2015/2016. This number of open, pending complaints is manageable with the temporary resources.

http://www.cdph.ca.gov/programs/Pages/CHCQPerformanceMetrics.aspx

California State Auditor's Assessment of 1-Year Status: Pending

Although the data that Public Health references on its website show that it reduced the number of open complaints, it did not provide an assessment to determine whether the permanent resources assigned to PCB are adequate to address future complaints.

6-Month Agency Response

As of April 1, 2015, PCB has filled all 18 new positions established in the Budget Act.

According to the performance metrics posted on our website, trends show that PCB continues to reduce the number of open, aged complaints. As of March 31, 2015, all complaints received in fiscal year 2012/2013 have been completed. There are 276 open complaints received in fiscal year 2013/2014 and 618 open complaints received in 2014/2015.

CDPH will continue to monitor and report progress of PCB's investigation as well as assess the staffing needs of PCB to perform this work.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

As of December 1, 2014, PCB has filled 15 of the 18 new positions established in the Budget Act and is recruiting to fill the remaining three positions.

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of complaints against certified individuals for the first quarter of the 2014-2015 state fiscal year.

As of December 5, 2013, PCB has further reduced the number of open, aging complaint investigations to 17 received in fiscal year 2012/2013, 609 received in fiscal year 2013/2014, and 409 received in fiscal year 2014/2015.

California State Auditor's Assessment of 60-Day Status: Pending

Public Health's response does not discuss whether it assessed the temporary resources and whether they are adequate to reduce the number of open complaints to a manageable level.

Recommendation #12 To: Public Health, Department of

To ensure that its district offices properly investigate complaints and ERIs, Public Health should make certain that all district offices follow procedures requiring supervisory review and approval of complaint and ERI investigations. If the district offices do not have a sufficient number of supervisors to review investigations they did not conduct, Public Health should arrange to assist the districts until such time that they do have a sufficient number of supervisors.

Annual Follow-Up Agency Response From September 2019

The previous method for monthly metrics was for districts to self-report its data on review and approval of complaints and Entity Reported Incidents (ERI) investigations. The current method is that each district submits its data to CHCQ. Staff analyze the metrics and notify the respective branch chief of the results. The branch chief discusses any deficiencies with the district manager and together they create an action plan for meeting the threshold.CHCQ's training section recorded and dispersed a training covering FRI/Complaints. All Health Facilities Evaluator Nurses (HFENs)were required to complete this training. CHCQ tracked the training through its training platform. The link for the training is as follows:

https://www.proprofs.com/training/course/?title=untitled-course_5c90035f66e0d_213638

To ensure procedural consistency, in January 2019 the CHCQ training unit presented at a District Administrator / District Manager meeting a PPT of data collected from January-September 2018 on state and federal complaint/incident survey review criteria. Any criterion that do not score an 85% compliance rate or greater have an associated action plan. CHCQ leadership approves an action plan and it, along with related findings, is sent to all district offices and branch chiefs. At the time of the April meeting, only one criterion was below the 85% threshold:

Federal Criteria 1: Was a sufficient sample chosen to evaluate the complaint/incident

Investigation mirroring the time and conditions of the incident: 72% of sample reviews met this criterion.

Action Plan: This component was addressed in the training and SEQIS is reviewing how the question regarding this data is worded to ensure we are getting an accurate response from the offices. All other criteria was met. Staff in CHCQ are monitoring the districts' compliance with all criteria and threshold levels, and reporting any deficiencies to the branch chief overseeing the respective district(s).

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Public Health provided documentation showing that it is ensuring districts are requiring supervisory review and approval of its investigations, as we recommended. Specifically, it provided an example of a district's survey tool that, among other items, measures the frequency that a district's complaint and facility related incident (FRI) investigations have supervisory review and approval. The survey tool indicates that districts scoring less than 85 percent on any of its metrics must include an action plan to remedy issues identified. Public Health also provided an example of its summary metrics report, incorporating results for all of its district offices.

Annual Follow-Up Agency Response From November 2018

California Department of Public Health (CDPH) continues to work diligently to reduce the vacancy rate for Health Facilities Evaluator II (HFE II) Supervisors, who are important to the complaint and Entity Reported Incidents (ERI) (now called Facility Reported Incidents [FRI]) review process. As of September 30, 2018, 94 out of 102 HFE II positions were filled for a vacancy rate of 7.9%, as compared to September 2017 where 88 out of 103 HFE II positons were filled for a vacancy rate of 14.6%.

CDPH initiated a monthly review of complaints and FRIs for compliance with policies and procedure in July 2017. The protocol describes in detail the monthly reporting process. The summary report reflects the data obtained from district office surveys. A total of 896 complaints and/or FRI samples were reviewed from all district offices. The report captures information from July 2017 through March 2018; results for April, May and June 2018 are pending. Page three of the summary document shows 83% compliance with sending a supervisory peer-reviewed complaint/FRI. The second chart shows 68% compliance with sending one complaint/FRI for sample review. This number will increase proportionately, as the HFE II supervisor vacancy rate decreases. The data collected in this project was presented to district office management and CHCQ leadership groups on June 20, 2018.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Public Health provided documentation showing that it has a process for reviewing the investigations of complaints and FRIs completed by district offices, and that it has generally made progress in hiring additional supervisors for its district offices. Public Health's response and the documentation provided shows that not all districts have met Public Health's internal goal of supervisors reviewing a minimum of 85 percent of district office investigations. In follow up discussion regarding its response, Public Health indicated that branch chiefs and other staff meet with district offices that are not meeting this goal, and that they create an action plan to assist the district get back to at least the minimum goal. However, Public Health did not provide documentation of those action plans.

Annual Follow-Up Agency Response From December 2017

Public Health redirects resources to district offices that demonstrate a need for support and an ability to hire health facilities evaluator supervisors. Public Health continues to reinforce the supervisor's role in reviewing and approving investigation findings. A standardized report template specifically identifies all complaint types and volume and allows each branch chief to discuss workload priorities and policy and procedure compliance to ensure supervisors are available to review and approve the documents. District offices have multiple supervisors to provide review of complaints and ERIs. The supervisor position vacancies are declining; as of September 30, 2017, 16 of 110 positions are vacant (15% vacancy rate); the functional vacancy rate is approximately one supervisor vacancy per district office. Public Health collaborated with the Human Resources Branch to increase the frequency of the supervisor's eligibility exam processing so that Public Health can quickly promote eligible candidates who are ready for a supervisor position. The exams dates are listed on the department's website.

Public Health continues to conduct periodic reviews of ERIs and complaints for quality and adherence to policy. In September and November 2016, the Quality Improvement (QI) team conducted the Abbreviated Survey Review project. We selected 19 abbreviated surveys for review, and found all 19 to comply with state and federal case closure requirements.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Public Health has not made certain that all district offices follow procedures requiring supervisory review and approval of complaint and ERI investigations. Although Public Health's response states that it continues to conduct periodic reviews, it only demonstrated reviews of compliance in the fall of 2016. Further, based on the documentation provided from those compliance reviews, Public Health reviewed only one sample at each district and therefore we do not believe this is a sufficient review for the basis of its conclusion that it has fully implemented this recommendation to make certain that district offices follow procedures requiring supervisory review and approval of complaint and ERI investigations.

Annual Follow-Up Agency Response From November 2016

Public Health has filled most of the Health Facilities Evaluator Nurse supervisor positions. District offices are 81% staffed and recruiting and hiring is ongoing.

On August 4, 2016, Public Health released updated policy and procedures for the Abbreviated Standard Survey (federal complaint process) in Skilled Nursing/Nursing Facilities. These policy and procedures provide specific guidance to all district offices on timelines, investigation, documentation, and completion using best practices for consistent and efficient processing of entity-reported incident. These policies include quality measures for continuing monitoring and evaluating performance according to these policy and procedure.

Licensing and Certification's district office supervisors received training on these updated policy and procedures between January-July 2016.

This topic was also discussed at an all-state District Administrator/District Manager meeting in August 2016, which reinforced the role of the supervisor in triage and review of prioritization as well as the quality of the process for timely completion.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

CDPH continues to remind supervisors of their review obligations, most recently in the District Administrator/District Manager Academy in August 2015. We are developing a sign-off sheet to document supervisory review as part of the complaint investigation documentation. By January 31, 2016, we will prepare a District Office Memo communicating this new procedure.

The 2015-16 Budget Act increased CDPH's funding for 240 positions and $14.85 million for LA County to conduct L&C work. The new positions included 24 new supervisors. CDPH has scheduled new supervisor academies for January, March, and June 2016 for the newly hired supervisors to assist with their orientation and staff development.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

CDPH continues to review complaints and ERI for appropriate prioritization level, timely onsite visit, and investigations. These reviews are currently conducted for LA County District offices. After April 2015, the review will expand to the other district offices ensuring all district offices are reviewed each quarter.

The Governor's 2015-16 Budget, updated at the May Revision, includes a request for funding for 237 state positions, including 24 new supervisors, to conduct L&C state and federal work.

In addition, CDPH is developing and implementing plans to assist district offices that have vacancies of supervisors by redirecting work to neighboring offices.

California State Auditor's Assessment of 6-Month Status: Pending

Public Health has not yet provided documentation of the actions noted in its response.

60-Day Agency Response

On October 28, 2014, CDPH issued a reminder to all district office managers during the face-to-face District Administrator/District Manager meeting about the importance to investigate properly complaints and ERIs. Additionally, it included a web link to the most current complaint policies and procedures, which include supervisory review.

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015.

By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #13 To: Public Health, Department of

To make certain that its district offices comply with federal requirements regarding corrective action plans, Public Health should establish a process for its headquarters or regional management to inspect district office records periodically to confirm that they are obtaining corrective action plans according to the required time frame and verifying that facilities have performed the corrective actions described in the plans when required.

Annual Follow-Up Agency Response From November 2018

Between July 1, 2017, and June 30, 2018, California department of Public Health (CDPH) completed 7,355 surveys with violations in Long-Term Care (LTC) facilities. Of the 7,126 Statements of Deficiencies that were sent to providers, CDPH received 6,730 (94%) Plan of Corrections (POCs) back from the provider. Of these, 2,846 (42%) were back from the provider within 10 days. CDPH conducted one or more revisit surveys, to confirm that facilities properly implemented their proposed corrective actions, for 7,027 (96%) of these surveys. CDPH developed a monthly report for headquarters managers to use to confirm that district offices are obtaining corrective action plans according to the required timeframe. CDPH delivered a training in October 2018 for branch chiefs on the new monthly report.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Public Health provided documentation showing a process for reviewing a sample of district investigations in September and November 2017. As part of of its review, Public Health assessed the acceptability of the plans of correction. Public Health also provided documentation showing its periodic monitoring of district office compliance with federal requirements regarding plans of correction.

Annual Follow-Up Agency Response From December 2017

The Statewide electronic plan of correction (ePOC) training and outreach efforts to all District Offices was successfully completed by the June 2017 deadline. Public Health completed the training for the Los Angeles County District Offices by July 2017. As of September 12, 2017, CDPH has completed over 1,700 ePOC surveys. District offices continue to add additional facilities which will provide broader data collection and timeliness management. Public Health conducts revisit surveys to confirm that facilities properly implement their plans of correction and will implement compliance testing to track the timeliness of POC submissions.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Annual Follow-Up Agency Response From September 2016

In January 2016, Public Health's Licensing and Certification Program (L&C) joined other states in participating in Centers for Medicare/Medicaid Services (CMS) web-based electronic plan of correction (ePOC) program. The ePOC program is designed to allow providers, Public Health, and the CMS to view corrective action plans issued by state regulators and respond and upload documents that facilities provide in response to corrective action items. The ePOC program documentation includes the statement of deficiencies (CMS 2567) issued by State staff and facilities documented response(s). Public Health's District Offices (DO) serving all 58 California counties are phasing into this program and all will be using this system by June 2017.

L&C Training Unit conducted a webinar to providers and Public Health staff covering ePOC process. Participating in the ePOC program will strengthen Public Health's ability to monitor and review DO's timely and more efficiently document evidence of completion.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

CDPH developed criteria for reviewing plans of correction and verification of implementation. The criteria can be found on page 43-44 of the "LTC Abbreviated Survey P&P." This review was added to the Abbreviated Survey Review for LA County in April 2015 (fourth quarter 2014/15). Starting in October 2015, this review expanded to statewide.

California State Auditor's Assessment of 1-Year Status: Pending

Pages 43 and 44 of the Abbreviated Survey Policies and Procedures describe district offices' responsibilities in obtaining and reviewing corrective plans from LTHC facilities. However, our recommendation was for Public Health to establish a process for headquarters or regional management to inspect district office records periodically to confirm that the district offices were obtaining corrective action plans according to the required time frame and verifying that facilities have performed the corrective actions described in the plans when required.

6-Month Agency Response

CDPH continues to review complaints and ERI for appropriate prioritization level, timely onsite visit, and investigations. This review is currently conducted for LA County District offices. After April 2015, the review will expand to the other district offices ensuring all district offices are reviewed each quarter.

CDPH has developed and added criteria for reviewing plans of correction and verification the provider implemented their plans of correction to the LA County monthly reviews.

California State Auditor's Assessment of 6-Month Status: Pending

Public Health has not yet provided documentation of the actions noted in its response.

60-Day Agency Response

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Public Health did not provide documentation of the actions noted in its response. Additionally, it indicates the recommendation is not yet fully implemented.

Recommendation #14 To: Public Health, Department of

To ensure that it has closed complaints and ERIs appropriately, Public Health should take steps by April 2015 to verify that complaints that its field operations branch closed administratively were closed appropriately. For example, it could request the district offices to verify that the closures were appropriate.

Annual Follow-Up Agency Response From October 2019

In August 2013 the Center for Health Care Quality (CHCQ) within the California Department of Public Health (CDPH) began a clean-up project to close backlogged cases. Those lacking a complete investigation report were re-analyzed. Often records were no longer available for review and, due to lack of evidence, the files were closed by the field office. CDPH's retention policy is five years. All 258 files specified by CSA were older than five years from initial receipt of the Entity Reported Incidents, now called Facility Reported Incidents (FRI), or complaint. The QI specialists pulled the specified files and spent three months analyzing the records on a case-by-case basis. The 258 files were intakes received from 2001-2008; All 258 files were administratively closed by the field office where the initial report/complaint was received; The majority were closed by the field office from December 2013 through January 2014; None were administratively closed by headquarters; Some files were investigated or re-investigated by the field office six years after the incident; All the files were initiated and investigated prior to the state dual enforcement guidance.

CHCQ developed a process to upload justifications for FRI administrative closures into a departmental database. Feedback and comments from district managers in field offices were incorporated into the process. Staff presented a draft at the May 2, 2019, DM/DA meeting. We then streamlined the process and finalized for use. This new process is integrated into the curricula for the New Supervisor Academy and New Surveyor Academy in advance of the final adoption of the Complaint P&P.

California State Auditor's Assessment of Annual Follow-Up Status: Resolved

The agency response incorrectly indicates that these complaints were administratively closed by the field offices. As we stated on page 48 of the report, Public Health's interim deputy director stated that the field operations branch "...made a strategic decision to close these cases administratively because it believed, due to the completion dates recorded in the database, that the risk was low that any of the cases were actually still open after five years or more." The interim deputy director further stated that "...staff at headquarters relied on the information in the database to close the complaints and ERIs, without reviewing the file or verifying with the district offices that they had completed the investigations and sent out all the necessary notifications."

Nevertheless, we consider that Public Health has resolved this issue because it states that CHCQ re-analyzed the administratively-closed complaints and ERIs that lacked a complete investigation. Public Health provided us with a summary of the results of its analysis indicating that the complaints it closed administratively were done appropriately. For example, it states that 113 of the 258 complaints it administratively closed were unsubstantiated claims due to a lack of evidence, and that another 99 of those administratively-closed claims were unsubstantiated due to no identified violations. Public Health also identified 14 claims that it could not locate the records for review, and another three claims were substantiated with state violations. Public Health correctly indicates that these files, administratively closed in 2013, were filed with the agency between 2001 and 2008. All of these files are now well beyond the agency's 5-year record retention policy.

Annual Follow-Up Agency Response From November 2018

The California Department of Public Health monitors closed complaints and Entity Reported Incidents (ERIs) for appropriate closure within the Prioritization Project. Quality Improvement (QI) Specialists perform an annual review of the prioritization of intake samples and compare their determination with that of the district office. If a disparity exits, a second QI reviewer evaluates the packet. Fifteen samples were identified as "Administrative Review/Offsite investigation," and nine showed a discrepancy between QI Specialists and the district offices regarding prioritization. The intake form may omit pertinent information supporting the decisions. A report of the evaluation discrepancies is sent to the district manager, administrator, and supervisors to review and make corrective actions. QI specialists have collected data for May 2018 and July 2018 but have not yet analyzed the data.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Public Health's response and documentation provided do not specify whether the ERIs and complaints its QI unit reviewed were for those that its field operations branch closed administratively (as opposed to those that the district offices closed after performing a review of the ERI or complaint). As such, Public Health has not demonstrated it has verified that complaints that its field operations branch closed administratively were closed appropriately.

Annual Follow-Up Agency Response From December 2017

Public Health continues to monitor the entity-reported incident (ERI) process for quality improvement. From March 28 to April 7, 2017, Quality Improvement (QI) team and Los Angeles (LA) Monitoring Unit [LAMU] supervisors conducted quality review of 20 ERI/Complaint abbreviated survey packets/samples (five samples each district office). The result of these reviews determined 8 of 20 samples were closed appropriately. 12 of 20 samples were determined requiring further investigation to ensure the allegations were completely investigated before closing the cases. The result of these reviews was discussed with the District Office's managers.

The QI team and LAMU staff continue implementing the State Observation Survey Analysis (SOSA) of the federal recertification survey process. There were total of 14 SOSA surveys (State DOs: nine surveys; LA DOs: five surveys) conducted in 2016; and ten SOSA surveys (State DOs: six surveys; LA DOs: four surveys) conducted in 2017. The results of the SOSA were presented to the District Managers/Administrators in writing and discussed in teleconferences or face-to-face meeting.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

Neither Public Health's response nor the documentation provided specify whether the ERIs and complaints its QI and LAMU units reviewed were for those that the district offices closed administratively (as opposed to those that the district offices closed after performing a review of the ERI or complaint). As such, Public Health has not demonstrated it has verified that complaints that its field operations branch closed administratively were closed appropriately.

Annual Follow-Up Agency Response From November 2016

On August 4, 2016, Public Health's Licensing and Certification Program (L&C) released updated policy and procedures for the Abbreviated Standard Survey (federal complaint process) in Skilled Nursing/Nursing Facilities, which includes guidance for administrative review and closure.

Additionally, L&C Quality Improvement Unit (QI) implemented a State Observation Survey Analysis (SOSA) of the federal recertification survey process where a team "shadows" the survey team which may include complaint/entity-reported incident (ERI) investigations.

The QI team conducts one SOSA per month for each district office including LA County. During the process, complaint investigative reports, including those administratively closed, are reviewed for quality control and appropriate use of administrative off-site closure. The SOSA includes a sample selection of nine closed complaints/ERIs, at least one is an administrative closure. The results of the SOSA are presented to the District Manager/Administrator in writing and discussed in a teleconference. The SOSA surveys are ongoing for all district offices and represent an opportunity for continuous process improvement over complaint processes.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Although Public Health provided documentation supporting its proposed actions going forward to help prevent inappropriate closures of complaints, it did not address whether it verified the complaints that its field operations branch closed administratively were closed appropriately. When we followed up with staff, Public Health told us that, because of its four year record retention policy, Public Health cannot perform a review of the 258 complaints we identified that were closed administratively.

1-Year Agency Response

CDPH will develop criteria to evaluate the appropriate use of administrative closure by end of November 2015. Starting third quarter FY 2015-2105 (January-March 2016) CDPH will review a sample of closed complaints and ERIs to evaluate the appropriate use of administrative closures and present findings for any additional training necessary. Based on our first review, we will determine the need for, and frequency of, any ongoing sampling and review.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

By June 2015, CDPH will develop criteria to evaluate the appropriate use of administrative closure and implement reviews in LA County by July 2015. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide to include assessment of appropriate administrative complaint closures.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Beginning in September 2014, CDPH Branch Chiefs initiated visits to each district office to ensure compliance with policies and procedures including those related to complaint and ERI closure and will continue this during their quarterly visits. During these visits, the Branch Chiefs discuss complaint and ERI investigation monitoring with district office administrative, supervisory, and management staff.

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide, including assessing whether any administrative complaint closures were appropriate.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #15 To: Public Health, Department of

To improve oversight of its district offices' complaint and ERI investigation process, Public Health should increase its monitoring of the district offices' compliance with federal and state laws as well as with its policies. For example, Public Health could accomplish this by directing its regional managers to spend more time at the district offices to enforce district office compliance with policies, or by directing its quality improvement section to review a random sample of investigations for quality and adherence to policy. Public Health should further establish a formal process to review periodically LA County's compliance with the terms of its contract, including compliance with the terms for investigating complaints.

6-Month Agency Response

CDPH Branch Chiefs have increased their visits to district offices, and increased regional meetings to ensure compliance with policies and procedures related to complaint and ERI closure. During these visits, the Branch Chiefs discuss complaint and ERI investigation monitoring with district office administrative, supervisory, and management staff.

CDPH has implemented a monitoring unit in LA County that consists of a former District Office Manager (retired annuitant), a Health Facility Evaluator Supervisor, and 2 Health Facility Evaluator Nurses. This unit ensures LA County's compliance with the terms of its contract, including compliance with the terms for investigating complaints. In addition, weekly calls occur between L&C Headquarters and LA County officials to monitor workload progress and provide guidance and direction as needed.

CDPH continues to review complaints and ERI for appropriate prioritization level, investigative process, and adherence to regulatory requirements and policy.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Beginning in September 2014, CDPH Branch Chiefs initiated visits to each district office to ensure compliance with policies and procedures including those related to complaint and ERI closure and will continue this during their quarterly visits. During these visits, the Branch Chiefs discuss complaint and ERI investigation monitoring with district office administrative, supervisory, and management staff.

CDPH has developed criteria for reviewing complaint and ERI prioritization and quality of investigations. Beginning May 2014, CDPH has used these criteria to conduct monthly reviews of a sample of complaint and ERI investigations in LA County. CDPH prepares a quarterly report of these reviews. These reviews will continue in LA County until April 2015. By April 2015, CDPH will add criteria for reviewing plans of correction and supervisory review to the LA County monthly reviews. By October 2015, CDPH will use the criteria to monthly review a sample of the investigations in district offices statewide.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #16 To: Public Health, Department of

To better protect the safety of residents in long-term health care facilities, Public Health should direct its district offices to comply with required time frames for initiating and closing completed investigations. If a district office lacks sufficient resources to initiate or close investigations within those time frames, Public Health should arrange to assist that district until such time that the district complies with the statute.

1-Year Agency Response

On May 8, 2015, CDPH posted district-specific data to our website. Subsequent reports will continue to report district office specific details of the complaints and entity reported incidents volume, timeliness, and disposition. CDPH Branch Chiefs use this district-specific data as a management tool and will continue to work with the district office managers to monitor these performance metrics, including meeting required timeframes.

As documented in our benchmark report to CMS, if a district office lacks sufficient resources to initiate or close investigations within those time frames, CDPH Branch Chiefs will collaborate to assist that district until such time that the district complies with the statute.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

We reviewed Public Health's website and verified it is providing district-specific data on the timeliness of completing complaints investigations. Public Health also provided documentation showing it has provided assistance to district offices experiencing workload issues.

6-Month Agency Response

On May 8, 2015, CDPH post district-specific data to our website. Subsequent reports will continue to report district office specific details of the complaints and entity reported incidents volume, timeliness, and disposition. CDPH Branch Chiefs use this district-specific data as a management tool and will continue to work with the district office managers to monitor these performance metrics.

California State Auditor's Assessment of 6-Month Status: Pending

Public Health's response states that it posted district-specific data on its website about volume, timeliness, and disposition of complaint and ERI investigations. However, Public Health does not address our recommendation that it direct its district offices to comply with required time frames for initiating and closing completed investigations and provide assistance to districts unable to meet those time frames.

60-Day Agency Response

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of long-term care health facility complaint and ERI investigations for the first quarter of the 2014-2015 state fiscal year.

By January 31, 2015, CDPH will provide district-specific data to the district offices to use as a management tool. CDPH will work with the district offices to monitor these performance metrics.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #17 To: Public Health, Department of

To make certain that it complies with statutory time frames for adjudicating appeals related to individuals, Public Health should establish a process to monitor its contractor's performance with contract terms.

1-Year Agency Response

CDPH has developed a tracking log to monitor the contractor's performance and updates the log monthly.

In addition, CDPH has scheduled quarterly meetings with DHCS to review the status of the hearings. Meeting dates as follows:

- October 6, 2015

- January 14, 2016

- April 6, 2016

- July 6, 2016

- October 5, 2016

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

CDPH has developed a tracking log to monitor the contractor's performance. This log is updated monthly.

California State Auditor's Assessment of 6-Month Status: Pending

Although Public Health's response states that it has developed a tracking log to monitor the contractor's performance, Public Health did not provide the tracking log and does not describe what is included in the tracking log or how the log will be used to monitor its contractor's performance with the contract terms. Also, in August 2015, Public Health indicated to us that it plans to set up quarterly meetings with its staff to review the tracking log but has not set the dates of the meetings.

60-Day Agency Response

The statutory provision that governs Administrative Hearings for CDPH is Section 131071 of the Health and Safety Code, which states that notwithstanding any other provision of law, CDPH will conduct hearings pursuant to the Administrative Procedures Act and Section 131071. Those provisions do not designate specific deadlines for setting or conducting hearings.

CDPH will review and monitor the contract.

California State Auditor's Assessment of 60-Day Status: No Action Taken

As we state on pages 56 and 57 of the report, Public Health's interpretation of the law is incorrect for two reasons. First, rules of statutory construction provide that significance should be given to every word in a statute, which must be read in the light of its historical background and evident objective. The statutory requirements concerning time deadlines for hearings affecting these individuals specifically state that Administrative Procedures Act (APA) procedures apply unless those procedures conflict with the specific statutory provisions governing appeals by nurse assistants and home health aides. Because the statutory time deadline for hearing an appeal clearly conflicts with otherwise applicable APA provisions, we conclude that the deadline supersedes the APA.

Second, when two laws upon the same subject are passed at different times and are inconsistent with each other, the one last passed must prevail. In this case, the pertinent section referring to the APA was enacted in 2007 and has not been amended since. The section of state law prescribing the time frames for Public Health was last amended in 2013, at which time the Legislature declined to remove the 60-day time requirement, thereby evidencing an intention to preserve this provision.

Recommendation #18 To: Public Health, Department of

To ensure that the Legislature promptly receives information about the timeliness of Public Health's complaint processing related to long-term health care facilities, Public Health should continue to include all of the statutorily required information in its annual report and submit it by the due date.

6-Month Agency Response

On May 8, 2015, CDPH post district-specific data to our website. Subsequent reports will continue to report district office specific details of the complaints and entity reported incidents volume, timeliness, and disposition.

In February 2015, CDPH posted its annual Fee Report for FY 15/16 on the website. The report contains all statutorily required information.

The Legislature is informed via the Fee Report, posted at:

www.cdph.ca.gov/pubsforms/fiscalrep/Documents/LicCertAnnualReport2015.pdf

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

On November 7, 2014, CDPH posted on our website performance metrics on the volume, timeliness, and disposition of long-term care health facility complaints and ERI investigations for the first quarter of fiscal year 2014-2015.

By January 31, 2015, CDPH will provide district-specific data to the district offices to use as a management tool. CDPH will work with the district offices to monitor these performance metrics.

In February 2015, CDPH will release its annual fee report containing all of the statutorily required information, as defined above.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2014-111

Agency responses received are posted verbatim.