Report 2013-102 Summary - October 2013
Employment Development Department:
It Needs to Address Data Issues to Better Evaluate and Improve the Performance of Its Employment Programs for Veterans
Our review of the Employment Development Department's (department) efforts to assist veterans in finding employment highlighted the following:
- We question the validity of California's performance statistics.
- Data from California's Base Wage File—which is used to calculate performance measures on the State's workforce development system—is problematic. We noted more than 1,400 instances where a single Social Security number was associated with 10 or more different names in a single quarter.
- The process it uses to summarize data could lead to inflated measures for wages or misstatements of who did or did not obtain employment.
- It does not periodically review data in the Base Wage File to report suspicious activity to the appropriate authorities.
- It is not always meeting its goals for the grant it receives from the U.S. Department of Labor (Labor) under the veterans grant program—Labor indicated that California is among the lowest performing states.
- It does not make program decisions based on how certain groups of veterans who have accessed the department's job assistance services fare in obtaining employment compared with nonveterans.
- Although it participates in an interagency council and is developing initiatives in cooperation with other entities to improve veterans' ability to find work, it is too soon to tell if those initiatives will be effective.
RESULTS IN BRIEF
Through its Workforce Services Branch, the Employment Development Department (department) assists Californians, including veterans, with finding employment. Funding for this work comes from the U. S. Department of Labor (Labor) via the Workforce Investment Act of 1998 (WIA) and the Wagner-Peyser Act of 1933 (Wagner-Peyser). An additional grant—the Jobs for Veterans State Grant (veterans grant)—provides funding for specialized staff to assist veterans in finding work and to conduct outreach to employers on behalf of veterans. Although all veterans receive priority for workforce services offered through the WIA and Wagner-Peyser, the veterans grant focuses on providing services to disabled and economically disadvantaged veterans.
The poor quality of the data California uses to report information to Labor on participants in its workforce development system, and the methodology the department uses to collate those data, call into question the validity of California's performance statistics. Labor compiles data from California on participants in its workforce development system to calculate three common measures Labor uses to assess the system: entered employment rate, employment retention rate, and average earnings. The computation of these measures requires California to use information from its Base Wage File, which tracks total wages paid to individuals in California. However, we noted more than 1,400 instances in the Base Wage File where a single Social Security number was associated with 10 or more different names in a single quarter—in one instance, with 162 different names.
According to the department, when it submits data for its performance metrics to Labor, the Base Wage File's wage information is summarized by Social Security number and reporting period, creating a single record for each Social Security number. This could lead to inflated measures for wages or for misstatements of who did or did not obtain employment. According to a chief of Labor's Division of Workforce Investment, Labor requires that states perform due diligence for accuracy and data integrity when using wage data for performance measurements. Labor expects states to review the data and identify anomalies. Without such monitoring, confidence in the department's data erodes.
Further, the association of different names with one Social Security number is a possible indicator of identity theft. According to the department, state law prohibits the department from sharing this information with other entities, such as law enforcement, that might investigate such cases unless the department receives a request from the affected entity. However, according to federal regulations, disclosure of this information to a public official for use in the performance of his or her duties is permissible under certain circumstances. As an entity of state government, the department has a responsibility to the State's citizens to provide information to law enforcement when the department reasonably suspects that individuals are reporting or otherwise using Social Security numbers inappropriately. Unless it periodically reviews data in the Base Wage File and reports suspicious activity to the appropriate authorities, the department is missing an opportunity to thwart potential identity theft.
Even based on the data the department submits to Labor, it is not always meeting its goals for the grant it receives from Labor under the veterans grant program. The department and Labor negotiate annual goals for the veterans grant. The department has consistently failed to achieve the negotiated goals for the entered employment rate since fiscal year 2009-10. In an April 2013 letter to the department's former director, Labor indicated that California is among the lowest performing states when measuring the rate at which program veterans enter employment. In its response, the department asserted that a new data system, implemented in March 2013, would have the capacity to record veteran outcomes more effectively. Further, the department asserted that it had begun training all veterans employment representatives to use the new system; however, we noted that as of August 2013, it had not yet done so. In September 2013 the department provided us with several draft training materials that generally focused on educating veterans employment representatives on how the three common measures are calculated and how department employees can use the new CalJOBS data system to improve their case management of veterans. A manager within the department's Workforce Services Division expected the department to finalize the training in late September or early October 2013, after we completed fieldwork for our audit.
In addition, although the department routinely reports performance data to Labor on its services to veterans, it does not make program decisions based on how certain groups of veterans who have accessed the department's job assistance services fare in obtaining employment compared with nonveterans. According to the chief of the department's Workforce Services Division, although the department maintains data for both the general and veteran populations that it serves, it does not routinely compare the two data sets to identify any disparities or to identify best practices that may help either program population become more successful. He stated that the department considers its services to veterans successful if it reaches the goals it negotiated with Labor for the veterans grant program. However, as we already noted above, the department does not always meet its goals.
Although the department primarily measures the success of the veterans grant through the three common measures it negotiates with Labor, the department does not monitor whether it has performed additional actions contained in its State Veterans' Program and Budget Plan (five-year plan) for the veterans grant. The five-year plan states that California will, for example, promote hiring and retention through developing business relationships with organizations in health care, banking, and other sectors and through assessing and targeting veteran-friendly industries for additional recruiting and marketing. Nevertheless, a veterans program support specialist within the department stated that the department has no formal data collection process for the goals it is trying to accomplish in the five-year plan. For example, we requested information from the department's central office regarding the status of the action steps in the five-year plan. The department's personnel at the central office did not have information to answer our questions and asked the field divisions to respond. The field divisions provided narrative descriptions of the work being done. However, these responses were not part of a routine reporting process that the central office could use when making program policy decisions. According to a deputy chief in the Workforce Services Division, the department does not monitor the results of these action steps at its central office. As a result, the central office does not know whether its action steps are being implemented or are effective or successful in accomplishing the plan's goals.
The department participates in the Governor's Interagency Council on Veterans (interagency council) established in 2011 and is developing initiatives in cooperation with other entities to improve veterans' ability to find work, but it is too soon to tell if those initiatives will be effective. For example, in April 2013, the department signed a memorandum of understanding with the Sacramento Area Human Resource Association (SAHRA) under which SAHRA will help department staff acquire the skills needed to better assist veterans with networking and resume writing. Further, the employment workgroup of the interagency council—chaired by the deputy director of the department's Workforce Services Branch—is exploring the creation of a California Transition Assistance Program. While service members are required to attend a federal transition assistance program, the California program would provide a refresher course on the services available to veterans and the ways to access veterans benefits. However, as of August 2013, there was no timeline for implementing the California program.
Finally, the department does not assess the success of a grant program to benefit veterans based on two of the program's goals—their average earnings over a six-month period, and their retention of employment for six months. The Veterans Employment-Related Assistance Program (veterans assistance program) provides federal funding in the form of grants to transition veterans into high-wage, high-growth occupations by training them for specific jobs in specific sectors. Grant recipients must forecast their performance goals based in part on the two measures just described. However, upon completion of the grant programs, the unit in charge of monitoring the grants does not calculate job retention or average earnings of the veterans who participated in the grant programs. According to the head of the unit responsible for monitoring these grant programs, the unit's staff does not analyze data on retention and the average earnings each grantee reports following the conclusion of the grant's active status. He stated that unit staff could obtain this information if necessary; however, doing so is beyond the scope of the unit's work. Nonetheless, as the unit responsible for monitoring the grant programs, it should track all established performance measures. If it does not do so, the department cannot demonstrate that the grant programs are successful in providing positive employment outcomes for veterans nor can it make informed decisions regarding future grants.
To improve the quality of the performance reporting it submits to Labor, the department should work with Labor to develop reasonable controls to avoid reporting overstated and inaccurate performance measures.
To help protect the State's citizens from identity theft, the Legislature should expressly authorize the department, on its own initiative, to share information from the Base Wage File with appropriate law enforcement officials when evidence exists of the potential misuse of Social Security numbers. If the department receives such legal authority, it should, at least annually, review the Base Wage File for associations of multiple names with a single Social Security number. The department should also establish a reasonable threshold for the number of associated names that will trigger further scrutiny from the department or referral to law enforcement.
To improve the department's performance on its negotiated goals with Labor, the department should, by January 2014, ensure that all veterans employment representatives are fully trained to use the new data system it implemented in March 2013.
To identify ways to better serve veterans in California, the department should assess the success or struggles of veterans within demographic categories, such as age, race, or educational attainment, by comparing veterans' performance with respect to finding employment to that of nonveterans in the same demographic categories and across demographic categories and use this analysis to inform program decisions.
To better optimize its leadership role in the interagency council's employment workgroup, the department should ensure that the employment workgroup develops a timeline for completing its action items and develops a process for measuring its success in improving employment outcomes for veterans. Specifically, the department should take the lead for establishing a time frame for evaluating tools to help assess and translate military skills into finding civilian jobs and establishing a transition assistance program.
To evaluate the success of the veterans assistance program going forward, the department should analyze the performance of the grant recipients across all three established common measures.
The department chose not to respond to three of the nine recommendations we made beginning on page 38. For those recommendations where the department did respond, it generally agreed but expressed concerns about its ability to implement certain recommendations due to either legal restrictions or limited resources.