Report 2012-108 Recommendation 5 Responses

Report 2012-108: School Safety and Nondiscrimination Laws: Most Local Educational Agencies Do Not Evaluate the Effectiveness of Their Programs, and the State Should Exercise Stronger Leadership (Release Date: August 2013)

Recommendation #5 To: Los Angeles Unified School District

To ensure that it is effectively preventing and addressing incidents of discrimination, harassment, intimidation, and bullying in its schools, Los Angeles Unified should ensure that school sites evaluate the effectiveness of the programs they choose to implement.

Annual Follow-Up Agency Response From August 2015

5A. RESPONSE: The 30-day timeline for reviewing bullying complaints is memorialized into multiple documents:

- BUL-5212.2 Bullying and Hazing Policy (Exhibit 5a1)

o Section VIII, Responding to Bullying and Hazing (page 8)

o Section X, Resolution of Incidents of Bullying (page 10)

- Bullying Complaint Response Worksheet, Attachment C (Exhibit 5a2)

- School Safety Plan, Section 4.3.1 Bullying and Hazing (Exhibit 5a3)

- The Administrators Online Certification System directs the respondents to BUL-5212.2 timeline for review and resolution (Exhibit 5a4)

5B RESPONSE: The Rubric of Implementation (ROI) is a Discipline Foundation Policy monitoring tool. The operations coordinator, director and the principal review evidence and collaboratively rate the school's implementation of the School-Wide Positive Behavior on a Likert scale (Exhibit 5b1). The composite score classifies the school as Fully Implemented, Partially Implemented or Targeted. Targeted Schools receive additional support from the local district and the School Operations. At the Spring 2015 submission, 543 schools (69.3%) were "Fully Implemented," 241 schools (30.7%) were "Partially Implemented," and 0 schools were "Targeted." (see tabs "CHART" and "Spring 14-15," Exhibit 5b2).

5C RESPONSE: The Safe School Plan is a three-volume annual assessment in which principals rate their school's progress towards implementation of 63 overarching goals on a Likert scale. Two revisions were made to Goal 4.3.1, Bullying and Hazing for 2015/16 Safe School Plan pertinent to this audit: (Exhibit 5a2):

Safe School Plan, Volume 1, Section 4.3.1, Bullying & Hazing Policy— The school and all personnel shall take proactive measures to promote respect and acceptance among students, staff, and the school community.

- Administrators review the School Experience Survey data to inform bullying prevention efforts.

- Reported incidents must within 30 days; resolution must be achieved within 60 days.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From November 2014

- The District has two external protocols by which the schools' evaluative processes are monitored:

o The District submits Exhibit 1, samples of completed biannual Rubric of Implementation (ROI). The Rubric of Implementation is a tool by which the local ESC School Operations Coordinators evaluate the effectiveness of the school's safety programs on a 4-point Likert scale across eight key features.

o The District submits Exhibit 4, the completed analysis of the bullying prompts on the 2013/14 School Experience Survey. The School Experience Survey queries staff, students, and parents on measures associated with school satisfaction and safety. The 2013/14 version added nine prompts related to bullying. The data are disaggregated to the individual school site to inform site administrators, ESC and central office as to the stakeholders' rating of the efficacy of their school's bullying prevention and intervention practices. These data are used in conjunction with iSTAR reports and bullying complaints to determine the effectiveness of school safety efforts on the Rubric of Implementation.

- The District submits Exhibit 3, the revised Bullying and Hazing Policy which denotes 30-day requirements for updates and 90-day requirements for determination of resolution. The Bullying policy directs schools to measure the effectiveness of their programs.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Although Los Angeles Unified created tools to assist school sites to evaluate the effectiveness of the programs they choose to implement, it did not substantiate that schools are using those tools.

1-Year Agency Response

- Will continue to use the biannual Rubric of Implementation, a monitoring tool to ensure that schools implement school safety programs. The ROI was revised to identify the school administrator and school operations coordinator who review the plan for effectiveness (Exhibit 1Ar).

- Has drafted a revision of the Bullying and Hazing policy to require that schools monitor, evaluate and modify their bullying prevention and intervention efforts to promote positive school climates. The bulletin also provides metrics to consider in the evaluations (Exhibit 2J).

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Los Angeles Unified has created tools to assist school sites evaluate the effectiveness of the programs they choose to implement. However, Los Angeles Unified has not provided support to show school sites are effectively using the tools.

6-Month Agency Response

Schools are expected to review all relevant forms of data, including school-based data (office referrals, incident reports, suspension rates, attendance rates, bullying complaints, graffiti, and other markers of school engagement) to determine the efficacy of their efforts. Schools describe their efforts of implementation on the annual Safe School Plan, which is filed centrally and reviewed at the ESC level.

Educational Service Center (ESC) Responsibilities

Each ESC Instructional Superintendent and Administrator of Operations is responsible for teaching, enforcing, advocating and modeling the Discipline Foundation Policy (Exhibit 2I) to ESC staff and school administrators. The Administrator of Operations will designate a team of ESC staff who will ensure the successful implementation of this policy through support and guidance. In addition to providing assistance, the ESC Operations staff, along with the Instructional Director, are responsible for:

Analyzing data, monitoring and supporting school practices in order to address situations where practices need to be strengthened.

Regularly reviewing the online Rubric of Implementation (Exhibit 1A) with the site administration and providing targeted assistance to schools as needed.

Supporting and assessing school practices in order to make suggestions to address situations where practices need to be modified. School support should include a regular review of the goals listed in the Safe School Plan.

Ensuring that all deans participate in mandatory District training on discipline and alternatives to suspension.

To comply with the California State Auditors Recommendation 3, the District:

Requires that the ESC Administrators of Operations work closely with their schools to ensure that schools design and implement practices that address the specific needs of that school community, and that those practices are regularly monitored, evaluated, and revised as necessary.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Revise the complaint log to request critical information (dates of report, closure, monitoring).

Work with ITD to facilitate school site personnel's ability to generate reports on bullying incidents that were documented in iSTAR into a searchable data set, such as an Excel file. (Currently, only central office has the capacity to generate reports.)

-Require that schools use reports in searchable complaint databases (e.g. iSTAR, LAUSD MAX, Excel) to facilitate identification of patterns.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2012-108

Agency responses received are posted verbatim.