Report 2015-505 All Recommendation Responses

Report 2015-505: Follow-Up—California Department of Veterans Affairs: Better Collection and Use of Data Would Improve Its Outreach Efforts, and It Needs to Strengthen Its Oversight of County Veterans Service Officer Programs (Release Date: July 2015)

Recommendation #1 To: Veterans Affairs, Department of

CalVet should fully implement the recommendations from our 2009 report.

Annual Follow-Up Agency Response From July 2017

All recommendations for report 2009-108 were fully implemented in our July 2016 response. Please reference those responses and the State Auditor assessments of the implementation.

Completion Date: July 2016

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

Responses for report 2009-108 were entered as annual follow up responses in that report.

Estimated Completion Date: October 1, 2016
Response Date: July 2016

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

Responses for report 2009-108 were entered as annual follow up responses in that report.

Estimated Completion Date: December 2016
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

CalVet continues to gather demographic information and conduct needs assessments through a variety of channels including the Geographic Distribution of Expenditures report, County Veterans Services Office (CVSO) Subvention Reports, and U.S. Department of Veterans Affairs VetPop Data, while continuing to connect veterans and their families with services and benefits through ongoing strategies and partnerships. CalVet understands the significance of the myCalVet site and its potential to supply information statewide in an ongoing and cost effective way. CalVet is exploring the feasibility and cost required to use an external vendor in order to fully implement the recommendation to develop a marketing plan to analyze data from the website to target outreach. The scope of work in development and includes an analysis of the website profile data, development of a targeted outreach plan to increase awareness of benefits and services as well as encouraging veterans to create profiles regionally in the state through a digital marketing campaign.

The CalVet CVSO Auditor is currently implementing procedures for the audit functions to include periodic reviews of CVSO claims activities and College Fee Waiver Program. Claim activities and College Fee Waivers along with their supporting documentation are be verified by the CVSO Auditor on a periodic basis. The written desk procedures for these processes will be completed by December 31, 2015. District Office staff continue to provide regular training and feedback to CVSOs including feedback on corrections made to incoming claims on a monthly basis, and will continue to provide training to all CVSOs at three training conferences each year. Regulations to require CVSOs to report information on their outreach activities to CalVet are in discussion at this time.

Estimated Completion Date: April 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #2 To: Veterans Affairs, Department of

Veterans Services should develop and implement a plan by December 31, 2015, to routinely analyze and use myCalVet data to identify trends in the services veterans and their families indicate they are most interested in and incorporate the results of such analyses into its outreach efforts.

Annual Follow-Up Agency Response From July 2017

As described in the July 2016 response, CalVet continues to deploy the VSD Outreach Operations Guide (Attachment 1) and CalVet Outreach Analytics Desk Reference (Attachment 2), to compile and evaluate myCalVet data by region and category of interest on a quarterly basis. Attachment 3 shows the data for the first quarter of 2017, veterans/families expressed greatest interest in Education (e.g. GI Bill, education benefits for veterans' dependents). As a result, Veterans Services conducted extensive outreach efforts on Education also shown in Attachment 3.

CalVet is currently developing the California Transition Assistance Program (CalTAP), in an effort to streamline the delivery of electronic information to service members, veterans, and their families via the CalVet website, and more specifically myCalVet. Please reference Attachment 4, which describes the ongoing data analysis and targeted outreach that CalTAP accomplishes.

Completion Date: December 2016

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

Between January and June 2016, CalVet continuously analyzed myCalVet data, and used the results to guide its outreach efforts. This has been in accordance with the plan implemented in November 2015 (comprised of the attached VSD Outreach Operations Guide (Annex 1) and CalVet Outreach Analytics Desk Reference (Annex 2)), to compile and evaluate myCalVet data by region and category of interest ("pillars"). As shown in Annex 3 (myCalVet Data Analysis), the data showed that for Q4 CY2015 and Q1 CY2016, veterans/families expressed greatest interest in Education (e.g. GI Bill, education benefits for veterans' dependents); the sole exception was the Central Coast, which expressed greatest interest in Health Care during Q4 CY2015. As a result, Veterans Services conducted extensive outreach efforts on Education, as shown in Annex 4 (Outreach Action), during the first six months of CY2016; in addition the Central Coast LINC targeted Health Care during Q1 CY2016 efforts. A recent example of successful outreach engagement is attached as Annex 5 (In the News). To optimize efforts for the second half of CY2016, Veterans Services has improved the reporting tool used by its outreach workers to more clearly identify the links between myCalVet data results and each outreach event, as shown in Annex 6 (OOM). For the second half of CY2016, Veterans Services will continue to plan each outreach worker's monthly schedule based on myCalVet data results from the previous month and quarter. CalVet will communicate the results of these efforts in December 2016.

Estimated Completion Date: 12/01/2016
Response Date: July 2016

California State Auditor's Assessment of 1-Year Status: Not Fully Implemented

6-Month Agency Response

The Plan, implemented in November 2015, is comprised of the attached reports, VSD Outreach Operations Guide, and CalVet Outreach Analytics Desk Reference. The Plan details the specific steps taken to routinely analyze and use myCalVet data for outreach efforts focus. In addition, an overview and implementation plan for additional sources of data compilation, analyses, and resultant outreach focus are also included in the Operations Guide and relevant attachments. Data collection and compilation (per the Desk Reference) was initiated for the 4th Quarter of CY2015 (completed in December 2015), with ensuing analysis and action (per the Operations Guide) scheduled to take place the first week of January, 2016, with resulting incorporation into outreach efforts commencing throughout the remainder of Q1 CY2016, across each LINC Outreach Region.

Estimated Completion Date: December 2016
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Estimated Completion Date: December 31. 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #3 To: Veterans Affairs, Department of

To determine the most successful methods for informing veterans and their families about the myCalVet website, and to increase the number of registered users, Veterans Services should evaluate myCalVet data to identify which marketing methods were most effective in informing registered users about the website. Veterans Services should complete this analysis on an annual basis, beginning no later than December 31, 2015.

6-Month Agency Response

Calvet has analyzed data from the first year and a half of myCalVet Registered User inputs, to identify the most effective marketing methods for informing veterans and their families about the myCalVet website. See the attached report, CalVet VSD Response Narative, BSA 2015-505, Rec 2, for detailed narration of the analysis and ensuing action plan to ensure the most effective marketing methods for informing veterans and their families about the myCalVet website.

Completion Date: December 2015
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Estimated Completion Date: December 31, 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #4 To: Veterans Affairs, Department of

To enhance the effectiveness of its outreach activities, CalVet should initiate, by October 31, 2015, steps to establish a regulation, in accordance with state law, that will require the CVSOs to report information on their outreach activities to CalVet.

6-Month Agency Response

Proposed regulations requiring CVSOs to submit the number of claims filed for C&P benefits as well as outreach activities metrics were submitted to the Office of Administrative Law on October 30, 2015.

Completion Date: October 2015
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Regulations to require CVSOs to report information on their outreach activities to CalVet are in discussion at this time and the process to establish the regulations will be initiated by October 31, 2015.

Estimated Completion Date: October 31, 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #5 To: Veterans Affairs, Department of

To ensure that the funding it distributes to the CVSOs is consistent with their actual workloads, Veterans Services should, by December 31, 2015, develop and implement procedures to more thoroughly review the accuracy of the data in the CVSOs' workload activity reports by reviewing the records that are the basis of the reports.

6-Month Agency Response

CalVet monitors the quantity and accuracy of the CVSO claims through VetPro and the CalVet CVSO Auditor. The CalVet audit of the CVSO claims activities and verification of the appropriateness of college fee waivers fully meets legal requirements. The Veteran Services CVSO Auditor follows a very thorough and standardized procedure that has been solidified in a desk reference manual (completed December 23, 2015). The standardized audit process includes validation of the documents that are the basis of the workload activity reports including official forms and documents submitted for Subvention and Medi-Cal Cost Avoidance funding, as well as performing annual sample audits of both college fee waivers and other claims submitted by CVSOs for adherence to laws, policies, and procedures.

Completion Date: December 2015
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Estimated Completion Date: December 31, 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #6 To: Veterans Affairs, Department of

To ensure correctness, completeness, and consistency in its audits of the CVSOs' workload activity reports, Veterans Services should formalize and document its audit procedures for reviewing these reports by December 31, 2015.

6-Month Agency Response

The Subvention and Medi-Cal Cost Avoidance Audit Manual and Desk Procedures document was completed December 23, 2015. This document is the basis for all CVSO audit functions and ensures standardization and accountability for the bi-annual audit. The CalVet audit of the CVSO claims activities and verification of the appropriateness of college fee waivers fully meets legal requirements.

Completion Date: December 2015
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Estimated Completion Date: December 31, 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #7 To: Veterans Affairs, Department of

To improve its oversight of the College Fee Waiver Program and ensure that the CVSOs are not erroneously waiving college fees, Veterans Services should develop and implement a review process for college fee waivers by December 31, 2015.

6-Month Agency Response

The CalVet Subvention and Medi-Cal Cost Avoidance Audit Manual and Desk Procedures document was completed December 23, 2015. This document includes the procedures and requirements for performing sample audits of college fee waivers in order to ensure that CVSOs are not erroneously waiving college fees. Strategically selected samples of college fee waivers from every CVSO are reviewed to look for issues such as duplicate student claims in the same academic year, grants and denials for a single student in an academic year, and missing information on the DVS-40. Validation of the documents that are the basis of the grant or denial for the college fee waiver are also part of the audit procedure.

Completion Date: December 2015
Response Date: January 2016

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Estimated Completion Date: December 31, 2015
Response Date: September 2015

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2015-505

Agency responses received are posted verbatim.