Report 2017-129 Recommendation 14 Responses
Report 2017-129: Department of Rehabilitation: Its Inadequate Guidance and Oversight of the Grant Process Led to Inconsistencies and Perceived Bias in Its Evaluations and Awards of Some Grants (Release Date: July 2018)
Recommendation #14 To: Rehabilitation, Department of
If it finds errors in an evaluation that merit restarting the grant process, rescoring of applications, or convening a new evaluation panel, Rehabilitation should resolve any issues before it begins the rescoring process. It should also notify applicants to ensure that they are aware of any changes to the process due to the errors. Further, it should consider promulgating regulations and amending its grant manual to permit staff to request evaluators to rescore applications or convene a new evaluation panel when it finds issues with an evaluation.
Annual Follow-Up Agency Response From September 2023
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. After approval of the rulemaking process, the GSM will be adopted in its final form.
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. After approval of the rulemaking process, the GSM will be adopted in its final form.
- Estimated Completion Date: June 2023
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
In regard to the previous solicitation that CSA mentions in the example, we note that the Department determined that there was no evidence of evaluator bias, and there was sufficient evidence that the evaluator understood their responsibilities as an evaluator and completed their duties as an evaluator free of bias.
Consistent with the Department's commitment to improving our grant award process, the Request for Application (RFA) for the Traumatic Brain Injury Program September 2021 includes the public solicitation for evaluators. The Department will provide more guidance to the post evaluation staff person assigned to review the evaluation panel selection thoroughly. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
As we stated in our previous assessment, for its previous grant, Rehabilitation issued a public solicitation for evaluators, which included minimal qualification requirements. According to its Grant Solicitation Manual, Rehabilitation should also identify criteria that will disqualify potential evaluators who have the appearance of bias or conflicts of interest. However, Rehabilitation did not identify such criteria for that RFA. As a result, we found that one of its evaluators had at least an appearance of bias with one of the grant applicants. Aside from having evaluators self-certify that they are free from conflicts of interest or bias, Rehabilitation did not demonstrate that it has made efforts to ensure it receives the most qualified evaluators possible that are impartial and unbiased.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation is in the process of promulgating regulations, and it has amended its grant manual. Rehabilitation states in its grant solicitation manual that, in addition to the preliminary review of the evaluation process by the technical review team, it will designate staff separate from those involved in the grant process to conduct a review for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications. Rehabilitation did not demonstrate that it conducted the preliminary review, and its post-evaluation review is questionable. Specifically, we identified at least one evaluator who had, at minimum, a perceived bias, which the post-evaluation review team did not identify. Had it sufficiently evaluated the grant process, it would have identified this perceived bias, which likely would have merited the convening of a new evaluation panel.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded: Regulations - June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process concluded; Regulations June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
All Recommendations in 2017-129
Agency responses received are posted verbatim.