Skip Repetitive Navigation Links
California State Auditor Logo COMMITMENT • INTEGRITY • LEADERSHIP

California Department of Corrections and Rehabilitation
It Has Poorly Administered the Integrated Services for Mentally Ill Parolees Program, and With Current Funding Cuts, It Must Find Ways to Transition Parolees to County Services

Report Number: 2020-103

Appendix

Scope and Methodology

The Audit Committee directed the California State Auditor to conduct an audit of Corrections to determine whether it has appropriately managed the integrated services program. Specifically, we examined Corrections’ oversight and program management as well as certain issues related to program outcomes. Table A lists the objectives that the Audit Committee approved and the methods we used to address them.

Audit Objectives and the Methods Used to Address Them
AUDIT OBJECTIVE METHOD
1 Review and evaluate the laws, rules, and regulations significant to the audit objectives. Reviewed relevant state laws, regulations, and other background material applicable to the program.
2 Identify and analyze the roles of state-level entities related to the program and evaluate Corrections’ oversight of the program to determine whether the State’s oversight is adequate.
  • Reviewed laws and reports that the Council on Criminal Justice and Behavioral Health (council) and the California Rehabilitative Oversight Board produced.
  • Interviewed Corrections’ staff and the executive director of the council. We determined that Corrections has sole responsibility for the program, including necessary oversight.
  • Reviewed Corrections’ contracts with the program’s providers and determined the types of oversight Corrections was required to perform.
  • Interviewed Corrections’ staff to understand the oversight activities that Corrections has performed.
  • Obtained and reviewed various documents, including emails from the program manager, for the past year to understand her level of communication with the providers. We also reviewed reports the providers prepared and summaries of Corrections’ site reviews.
3 Identify the program’s annual appropriations for the past 10 fiscal years. For the past five fiscal years, break down expenditures by categories covering contracts for health care providers, housing, and administration.
  • Reviewed laws, historical budget documents, and Corrections’ accounting data to obtain this information. Due to records retention policies, not all information was available for the 10‑year period. Thus, we could not include the entire 10-year period for appropriations.
  • Interviewed Corrections’ staff and reviewed relevant Corrections data and documents to evaluate the program’s staffing levels.
  • Determined the number of program slots for participants authorized in state law and in program contracts.
4 Determine who refers participants to the program, what training they receive, and what criteria they use when referring participants.
  • Interviewed Corrections’ staff.
  • Reviewed law, program policy, contracts, and other relevant information related to the program and participant eligibility.
  • Reviewed staff training materials and related information to ascertain how specific the training was to program eligibility.
5 Analyze the selection process for program participants, including the following:
  1. Identify criteria used to select participants, including an individual’s risk of homelessness and housing status, and how the criteria are weighted.
  2. Determine whether Corrections uses evidence-based tools in its selection process.
  3. Determine what data, such as housing status, Corrections collects for consideration in selecting participants.
  • Reviewed relevant state and federal laws to identify eligibility criteria.
  • Interviewed Corrections’ and contracted providers’ staff about the processes they follow to refer individuals to the program and enroll them.
  • Randomly selected two case files from each of five senior psychologists, for a total of 10 case files. Reviewed the case files to determine whether Corrections considered the applicable criteria (mental illness and homelessness) when issuing a referral, and whether support for a referral decision was clearly documented in the case file.
  • Determined how Corrections weighs referral criteria and whether Corrections uses evidence‑based tools in its selection process. We determined that Corrections does not have a process to weigh one eligibility criterion over another, nor does it use evidence-based tools in its referral process.
6 Determine whether Corrections adequately collects, tracks, and analyzes program data related to housing status, the services participants receive, and recidivism. To the extent possible, determine the number and cost of services provided through the program that could have otherwise been funded through Medi-Cal.
  • Interviewed Corrections’ staff and reviewed data and information sources maintained by Corrections that track information related to program participants’ eligibility, referrals, and participation.
  • Evaluated the sufficiency of these sources for conducting analysis of the performance or effectiveness of the program.
  • Reviewed and analyzed documentation from Health Care Services related to county mental health plans and the billing process for related mental health services.
  • Interviewed Corrections’ and provider staff to determine the obstacles to billing Medi-Cal for the program.
  • Calculated the amount Santa Clara County has received in Medi-Cal reimbursements for program services in the past three completed fiscal years.
7 To the extent possible, for the past five fiscal years, perform the following related to recidivism rates:
  1. Calculate the recidivism rate for participants.
  2. Determine how many eligible parolees were unable to participate because of a lack of program space, and calculate their recidivism rates.
  3. Compare the recidivism rates of both groups and summarize the types of crimes for which recidivists were convicted.
  • Interviewed Corrections’ staff to identify relevant data sources.
  • Reviewed data sources from Corrections and Health Care Services. We also obtained and reviewed invoices from the contracted providers that they submitted to Corrections for reimbursement for program services and housing. We determined that Corrections does not maintain sufficient or consistent information on program participants, services, and outcomes. In addition, the provider information was in a mix of formats, some electronic and some hard-copy documents. Corrections does not have a complete or consistent dataset to support the analysis specified in this objective.
8 To the extent possible, determine the following about participants:
  1. The number of participants who were homeless when they first enrolled in the program and the number who became homeless while participating in the program.
  2. The number of participants who receive housing assistance under the program by the type of housing provided, such as housing in treatment settings, time-limited housing, or permanent housing. Determine the number of participants who share a bedroom.
  3. The number of participants who received interim interventions, such as shelter stays or bridge housing, and whether and when these participants were eventually connected to permanent housing.
  4. The dropout rate for the program, the reasons that participants dropped out, and the average length of participation for those who dropped out.
  5. The number of participants who remained in the program until they dropped out.
  6. The number of participants who are Medi-Cal eligible and the number enrolled in Medi-Cal.
See Method described in Objective 7.
9 Determine who refers participants to housing and evaluate how payments are made to housing providers.
  • Interviewed Corrections staff and provider staff.
  • Reviewed provider contracts and identified the required referral process and requirements for housing subsidies and payments.
  • Evaluated Corrections’ process for monitoring housing payments and determined its sufficiency.
  • Reviewed information collected about program housing services in monthly and quarterly reports and evaluated its sufficiency.
10 For a selection of program providers, determine the extent to which they connected participants to local housing programs.
  • Interviewed provider staff, including administrators and housing coordinators.
  • Randomly selected nine cases of participants who completed the program from Quality Group Homes–Fresno, Telecare–Los Angeles, and Turning Point. Reviewed case files to determine the participants’ housing situations upon being discharged from the program.
11 Evaluate Corrections’ and legislative efforts to improve the program’s outcomes. Determine what steps Corrections has taken to coordinate with counties, the outcomes of those steps, and plans to improve coordination with counties in the future.
  • Reviewed laws and bills as well as contemporaneous documentation from Corrections detailing the Legislature and Corrections’ efforts to improve the program.
  • Interviewed Corrections’ program administrator to document Corrections’ plans for improving coordination with counties.
  • Analyzed Corrections’ and University of California, Los Angeles’s, reports on the program.
12 Review and assess any other issues that are significant to the audit. In response to the Budget Act of 2020, which eliminated funding for the program, we performed the following steps:
  • Interviewed Corrections’ staff.
  • Reviewed Corrections’ training materials, policies, and procedures to identify existing processes for connecting parolees to other mental health and housing services. Analyzed the processes we identified to understand potential risks and weaknesses in those processes.
  • Inquired with Corrections for reviews on the effectiveness of parole agents and staff in parole outpatient clinics at connecting individuals to county services. We also searched public websites for any external reviews.
  • Reviewed county mental health plans and contacted staff in the behavioral health departments of Orange, Riverside, San Joaquin, and Shasta Counties. We asked for assistance in identifying county services for parolees with a mental illness because the integrated services program did not operate in these counties. Interviewed staff about potential risks and weaknesses of the county services. San Joaquin did not acknowledge our inquiry.
  • Contacted staff in behavioral health departments in Fresno, Kern, Los Angeles, Sacramento, San Bernardino, and San Diego Counties to obtain their perspectives on the elimination of the program. Interviewed staff about the potential risks and weaknesses of replacement services. Los Angeles and Fresno did not acknowledge our inquiry.
  • Interviewed staff at the County Behavioral Health Directors Association to obtain their perspective on the elimination of the program.

Source: Analysis of Audit Committee’s audit request number 2020-103, state law, and information and documentation identified in the column titled Method.


Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of computer-processed information we use to support our findings, conclusions, or recommendations. In performing this audit, we relied on electronic data files that we obtained from Corrections. These files included lists of referrals maintained by senior psychologists and data exports from Corrections’ parolee tracking system. Because Corrections does not maintain a comprehensive database of all program referrals and participants, our analysis of these files was limited to verifying that we had received the information we requested. We did so by confirming with the agency staff that the number of records in the files we received was correct. We cannot ensure the completeness of the files we obtained; however, there is sufficient evidence in total to support our audit findings, conclusions, and recommendations.





Back to top