Report 2005-133 Summary - November 2006
Department of Education:
Its Mathematics and Reading Professional Development Program Has Trained Fewer Teachers Than Originally Expected
Our review of the Mathematics and Reading Professional Development Program (program) revealed that:
- Only a small percentage of mathematics and reading teachers have completed the full 120 hours of training for their current assignments.
- School districts we surveyed cited several barriers to increased participation in the program, including teacher apathy toward attending training, concerns about funding, and a lack of training providers in close proximity. Nevertheless, school districts in counties with relatively large or small numbers of eligible teachers in various geographic regions throughout the State appear equally capable of accessing program services.
- The Department of Education (Education) has done little to actively promote the program and currently relies on school districts to navigate its Web site to learn about and apply for the program.
- Education has not ensured that program compliance audits are conducted in accordance with program statutes.
- Education's July 2005 report to the Legislature was of limited value because it lacked relevant and accurate data for gauging program outcomes.
- Education's ability to adequately track teacher participation in mathematics and reading training is complicated by the multiple funding sources involved and by reduced program-specific funding.
- The State Board of Education relied on the Sacramento County Office of Education to advertise and implement the program.
RESULTS IN BRIEF
Enacted by the Legislature during fiscal year 2001-02, the Mathematics and Reading Professional Development Program (program) is a voluntary program that aims to provide standards-based instructional training to 176,000 teachers statewide. Although the Legislature originally envisioned achieving this goal over a four-year period with annual appropriations of $80 million, several statutory changes reduced program funding, extended the program, and established caps on the number of teachers that can be trained annually. Chapter 524, Statutes of 2006, recently extended the program through fiscal year 2011-12.
More than five years after the program's enactment, our audit found that a small percentage of teachers have completed the full 120 hours of mathematics and reading standards-based training for their current assignments. Our survey of 100 school districts that participated in the program through fiscal year 2004-05, representing about 46 percent of all eligible teachers statewide, suggests data exists at school districts to substantiate that only 7,230 teachers have completed the program's training. Of this amount, roughly 25 percent likely had their training funded by sources other than the program, such as other federal and state grants. Perhaps more surprisingly, school districts representing 58 percent of the teachers in our survey reported that they could not readily tell us how many of their teachers had completed the training from program funding or other funding. This indicates that most districts and the State cannot currently identify the teachers who have received standards-based training for their current mathematics or reading assignment, regardless of funding source, nor can they identify those who still need the training.
School officials responding to our surveys of participating and nonparticipating school districts cited similar barriers to increased teacher participation in the program. These perceived barriers included teacher apathy toward attending program training, concerns about funding, and a lack of training providers in close proximity. School districts reported reluctance on the part of their teachers to attend program training, given its 120-hour duration and its focus on state-adopted textbooks rather than on the state standards themselves. In addition, school districts explained that program funding sometimes arrives at the school district four to six months after local funds have already covered the cost of training and that the payments they receive do not cover all training costs. Survey respondents also explained that training providers might not be located close enough for travel to be practical, making access to training more difficult. However, our review of program participation, summarized in Appendix C of this report, suggests that counties with relatively large and small numbers of eligible teachers appear equally capable of accessing program services.
The role of the Department of Education (Education) in administering the program essentially has been limited to forwarding school districts' annual applications to the State Board of Education (board) for approval and to processing program payments. Although not specifically required to do so in statute, Education has done little to actively promote the program. Instead, Education has relied on school districts to navigate its Web site describing various grant programs in order to learn about and apply for the program. It appears that a more concerted outreach effort is warranted, given that a small percentage of teachers have completed the training and that several of the school districts we surveyed were unaware of the program's existence or confused about the eligibility or funding aspects of the program. We also noted that Education did not take the necessary steps to ensure that the program's compliance requirements were incorporated into the audit guide proposed by the State Controller's Office and adopted by the Education Audit Appeals Panel. As a result, licensed local auditors who perform compliance audits of school districts are not aware of the requirements, and Education has disbursed about $113 million in program funds through fiscal year 2005-06 without the benefit of a local-level audit.
In addition to its limited outreach and monitoring efforts, Education's report to the Legislature in July 2005 regarding the program's effectiveness was of limited value. Although the report met the program's statutory reporting requirements, the reporting requirements themselves are insufficient to assess the program's success. In particular, Education's report lacks relevant and accurate data regarding the number of trained teachers that are currently using the training in the classroom, and it provides no correlation between teacher training and student achievement. Education's method of calculating the number of teachers receiving at least 40 hours of training is overstated because it includes duplicate counts of teachers, such as a teacher completing both the 40- and 80-hour training components. Education's report also does not include data on the overall number of eligible teachers, the number of teachers that have completed the full 120 hours of training using program funds, or the number of teachers who may have attended training but whose participation was funded through other state and federal funding sources. Information about these trainings funded by other sources may be of interest to the Legislature when making future funding decisions regarding the program. Moreover, although Education is developing a data system that will enable it to monitor teacher preparation programs, our review of the feasibility study for this project indicated no present plan to include data about the program in the system.
Education's ability to adequately track teacher participation in mathematics and reading standards-based training is complicated by the multiple funding sources involved and by reduced program-specific funding. Although Education has a process to track program payments to school districts, and can thus match program funding with the numbers of teachers participating, such a process does not exist for school districts that use other federal and state funding for the same training. Our survey results suggested that a significant number of districts use other federal and state funding to cover the costs of mathematics and reading standards-based training. Since Education does not ask school districts to report the number of teachers trained using other funding sources, it does not have a complete picture of the number of teachers that have actually completed the mathematics and reading standards-based training. Funding reductions early in the program's life may have driven school districts to seek other funding sources outside the program. We noted that the $143 million originally appropriated for the program in its first two years was subsequently reduced by about $98 million to fund other state priorities.
Program statutes require the board to approve the curricula of training providers, ensuring such curricula are aligned with the mathematics and reading content standards adopted by the board. The board relied heavily on the Sacramento County Office of Education (SCOE) when the program was first implemented. Specifically, SCOE developed criteria for evaluating training providers, created program promotional materials for school districts, and facilitated the evaluation of curricula submitted by training providers. Our audit noted that SCOE's contracts, which amounted to about $400,000, were exempt from the State's competitive bid process. Further, we noted that, although members of the board felt pressure to implement the program quickly, the former state superintendent of public instruction expressed concern with the haste of the development of the initial SCOE contract. Ultimately, to avoid a potential conflict of interest, the board cancelled an extension of the contract after SCOE hired a former board member. The board now contracts with the Orange County Department of Education (OCDE) to manage the provider approval process. Our review of the board's contracts with SCOE and OCDE revealed that the performance period for each agreement predated the Department of General Services' approval. As a result, the board exposed the State to potential liability for work performed prior to formal approval.
Given that only a small percentage of teachers have completed the full 120 hours of program training, and that teacher participation is voluntary, the Legislature should consider redefining its expectations for the program, clearly stating the number of teachers to be fully trained as well as any gains in student achievement expected. Based on how it defines the program's goals, the Legislature should consider making statutory changes to ensure that Education provides meaningful data with which to evaluate program success. Examples of meaningful program data include the following:
- Unduplicated counts of teachers who have completed the training with the aid of program and non-program funding, with a comparison of these figures to the total number of teachers who are eligible to participate in the program. Education could capture this information by modifying its claim form, adding a data collection tool similar to the one shown in Appendix D.
- Measures of the resulting gains in student achievement for teachers who have completed the program's training, such as higher student scores on standardized tests.
To remove a barrier to increased teacher participation in the program, Education should explore opportunities to expedite its payment process to school districts. One such opportunity would be to seek legislation authorizing Education to approve the annual certifications submitted by school districts instead of waiting for board approval, thus removing any payment delay caused by the need to wait for the next board meeting.
To ensure that all school districts are aware of the program and that as many teachers participate in the program as possible, Education should conduct annual outreach activities to all school districts. A component of such an outreach program should include directly informing each school district of the amount of funding for which it is eligible each year.
To ensure that required compliance audits are occurring, Education should take steps to ensure that the program's compliance requirements are included in audit guides related to the annual audits of school districts.
To ensure that it does not expose the State to potential liability for work performed before the contract is approved, the board should ensure that it obtains the Department of General Services' approval of its contracts and amendments before the start of the contract period and before contractors begin work.
Education and the board indicated that they have taken action or plan to take action to implement our recommendations.