Report 2011-120 Data Reliability Assessments

To identify the total amount paid for General Services' repairs to the State Route 710 extension project parcels and property units (SR 710 properties) for the period from July 1, 2008, through December 31, 2011.

To identify hours charged by employees and descriptions of work performed for repairs to the SR 710 properties for the period from June 1, 2011, through December 31, 2011.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We did not perform completeness testing to identify all projects for SR 710 properties and the associated expenditures in ABMS because we were not able to identify a complete list of the projects or expenditures. Specifically, General Services' personnel could not provide a way to identify projects for SR 710 properties prior to January 2010. As a result, we could not ensure we identified a complete listing of projects for the SR 710 properties and the associated expenditures.

In our assessment of the accuracy of the ABMS data, we tested only key fields related to projects that we could identify as SR 710 property expenditures. Specifically, we pulled a random sample of 29 expenditures and tested the accuracy of these data by tracing the data to supporting source documentation. We found three errors in the time cards related to the first 16 expenditures we tested, so we did not test the remaining 13 items.

To identify the number and types of State Route 710 extension project parcels and property units (SR 710 properties).

Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed completeness testing of the SR 710 properties by reconciling the Caltrans property records and acquisition maps to the Right-of-Way Property Management System and determined the data was complete.

To identify the total invoiced amount for contractors' repairs to State Route 710 extension project parcels and property units for the period from July 1, 2008, through December 31, 2011.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed accuracy testing on a random sample of 46 maintenance requests. Caltrans was not able to provide support for two sample items and we identified an additional 14 errors related to payment amounts. Due to the number of errors we identified, we stopped testing after 36 sample items. Further, because of the inaccuracies we identified in the payment amounts, we use invoiced amounts in our analysis.

Due to the number of errors we identified in our accuracy testing, we were able to determine the reliability of the data without conducting completeness testing.

To identify the rental income collected for each State Route 710 extension project parcels and property units for the period from July 1, 2008, through December 31, 2011.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed accuracy testing on a random sample of 30 rental billings and payments for tenants and found four errors in the first 11 sampled items. Due to the number of errors, we did not test the remaining 19 items.

Due to the number of errors we identified in our accuracy testing, we were able to determine the reliability of the data without conducting completeness testing.

To identify the difference between the rents that Caltrans charged its tenants and the fair market rents for the State Route 710 extension project parcels and property units and the average age of the fair market rent determinations as of February 9, 2011.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed accuracy testing on a random sample of 30 rental rates Caltrans charged tenants and found four errors in the first nine sampled items. Due to the number of errors, we did not test the remaining 21 items. We also tested the accuracy of 30 fair market rents and their determination dates by tracing the data back to supporting documents. We found errors in seven of the fair market rents and in eight of the determination dates.

Because of the number of errors we identified in our accuracy testing, we were able to determine the reliability of the data without conducting completeness testing.

To estimate the amount of forgone rent for the period from July 1, 2007, through December 31, 2011, because Caltrans rented State Route 710 extension project parcels and property units below fair market value.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed accuracy testing on a random sample of 30 rental rates Caltrans charged tenants and found four errors in the first nine sampled items. Due to the number of errors, we did not test the remaining 21 items. We also tested the accuracy of 30 fair market rents and their determination dates by tracing the data back to supporting documents. We found errors in seven of the fair market rents and in eight of the determination dates.

Because of the number of errors we identified in our accuracy testing, we were able to determine the reliability of the data without conducting completeness testing.

To identify all tenants living in the State Route 710 extension project parcels and property units and compare the listing of tenants to the California State Controller's Office's payroll records to determine if any of the tenants are state employees.

Not Sufficiently Reliable—We performed data-set verification procedures and electronic testing of key data elements and found no issues.

We performed accuracy testing on a random sample of 30 tenants and found no errors in this testing.

While performing completeness testing we identified a significant data limitation. The Right-of-Way Property Management System does not record all of the tenants living at a property. Thus, we may not have identified all of the state employees who were Caltrans' tenants.

To identify if any of the tenants in Caltrans' database were state employees from July 2006 to March 2012.

Sufficiently Reliable—We reviewed the testing of the payroll system's major control features performed as part of the State's financial audit for the fiscal years ending June 30, 2007, through June 30, 2011.