Report 2019-116 Recommendation 12 Responses

Report 2019-116: Juvenile Justice Crime Prevention Act: Weak Oversight Has Hindered Its Meaningful Implementation (Release Date: May 2020)

Recommendation #12 To: Mendocino County Probation Department

To adequately assess the effectiveness of its programs at reducing juvenile crime and delinquency, Mendocino should collect data on all participants in each JJCPA program and for each service it provides.

Annual Follow-Up Agency Response From October 2021

Mendocino County does not agree with the evaluation from our year 1 response in May of 2021 that the recommendation was only Partially Implemented. We feel we have Fully Implemented. As evidenced by the supporting documents we provided at that time, Mendocino is collecting a large amount of participant data for each JJCPA program and service. As we mentioned, the process of data collection isn't a stagnant effort, but rather an on-going process that continues to evolve.

The limited space provided here for response does not allow full explanation of the circumstances regarding our assessment. We are submitting a detailed letter regarding the status of our implementation separately as a supporting document. For the reasons stated in this supporting letter, we request that the status of this recommendation be amended to Fully Implemented.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Although Mendocino disagrees with our May 2021 assessment, it did not provide evidence at that time to support its claim that it had fully implemented our recommendation. Mendocino provided more recent documentation to support that it is now collecting data on all the participants in the Foundations program. Additionally, Mendocino indicated that the implementation of its IMPACT program has been delayed because of the pandemic, but it has developed a template that it will use to collect participant data. Finally, Mendocino's GREAT program provides presentations to groups of individuals who are not involved with the justice system, so it is reasonable for Mendocino to not collect participant-specific data.

1-Year Agency Response

Mendocino County Probation has 2 partners providing services for out-of-custody youth - Mendocino County Youth Project (MCYP) & GEO Reentry. Working with our partners, we have established data collection requirements for JJCPA-funded services. Data is provided to Mendocino County Probation on a monthly basis. Provided are copies of data provided by these partners for billed months. MCYP: 1)Foundations Monthly Reports for October 2020-February 2021, 2)Dual Diagnosis 602 Services for September & October 2020. MCYP lost data for several months prior to October 2020 due system issues. They implemented a new fiscal/case mgmt. system & was limited on the data they were able to rebuild. Also provided are copies of data from GEO Reentry for July 2020-April 2021. Our CMJJP also includes the IMPACT & GREAT programs, provided by Probation staff in coordination with Mendocino County schools. Implementation was delayed due to COVID-19 & remote education; so, we do not have data to provide. Implementation targeted for July 2021. Data collection efforts are an on-going process that evolves as we evaluate our programming. We continue to work with our providers & staff on type of data which will assist in our analysis. Data requirements also change as programming needs change. We have written language into our contracts with our providers, requiring annual meetings for data review and potential change in trends for youth treatment & service needs. Due to COVID-19 pandemic during the current & prior fiscal year, many programs were adversely impacted by reductions or cancellations of services; most notably for programming in school settings. We anticipate the effects of the pandemic through this fiscal year and into the next, hoping as restrictions ease we will be able to increase services to the youth of our community. We continue to work with our outside providers and our internal staff to evaluate & implement, when possible, alternative service delivery methods.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Based upon our review of supporting documentation provided by Mendocino, we determined that it is not collecting data on all the participants in several of its programs, including Foundations, the IMPACT and GREAT programs. Further, Mendocino indicates in its response that it includes language in its contracts with providers that requires annual meetings for data review. However, Mendocino stated in a follow-up email that such language is not yet included in all of its current contracts with providers. Therefore, we have assessed the status of this recommendation as partially implemented.

6-Month Agency Response

Mendocino County worked with our Community Based Organization (CBO) partner, Mendocino County Youth Project (MCYP), to establish data requirements for program and service participants. We are also collecting data from our internal staff that are, or will be, providing programming/services. Attached are copies of the "Accounting of JJCPA-YOBG Expenditures" sections for the two (2) programs that are JJCPA-funded that reflect current data collection. Data collection efforts are not stagnant, but rather an on-going process. Our data collection and analysis activities will continue to evolve as we evaluate our programming. We continue to work with our providers on the type of data which will assist in our analysis. We have, however, established the basic data collection processes that we will continue to expand and develop, which we interpret as satisfying this recommendation.

It should be noted that, due to the COVID-19 pandemic during the current and prior fiscal year, delivery of many programs were adversely impacted by reductions or cancellations of services. This is most notable for programming occurring in school settings. We anticipate the continuation of the effects of the pandemic throughout this fiscal year. We continue to work with our CBO and internal staff to evaluate and implement, when possible, alternative ways to deliver the necessary services to the youth of Mendocino County.

California State Auditor's Assessment of 6-Month Status: Pending

The attachments Mendocino provided to us indicate at a high-level various data elements, including number and length of sessions and number of juveniles served. However, Mendocino did not substantiate to what extent it collects data on all participants in each JJCPA program and for each service it provides. Until it does so, we will continue to report this recommendation as pending.

60-Day Agency Response

Mendocino County is working with our Community Based Organization (CBO) partners to establish the data required for program and service participants in each program provided. We are also working with our internal staff that are, or will be, providing programming/services to develop and establish the performance criteria and data required to adequately assess the effectiveness of our programs. Several of the programs/services are a new component of our JJCPA program; therefore, we are able to develop and establish the appropriate criteria at the onset of the program/service. Some of the programs/services that are part of our JJCPA program have been delayed or otherwise impacted as a result of the COVID-19 events currently impacting the country. The Assistant Chief Probation Officer has been assigned the task of overseeing the implementation, evaluation and assessment of the various programs/services which are part of our CMJJP. We anticipate full implementation of this recommendation within 12 months; however, COVID-19 events may delay or require modification to some implementation.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2019-116

Agency responses received are posted verbatim.