Pennsylvania Provides Customized PETI Tools to Its EETT Grantees
##AUTHORSPLIT##<--->
Pennsylvania has focused its Enhancing Education Through
Technology (EETT) grants from the NCLB Act on literacy, mathematics, and the effective use of digital tools to engage learners and enable teachers to differentiate instruction. Thus far, reports from the field indicate that progress is being made on many of these fronts.
Pennsylvania’s teachers are now gaining proficiency in redesigning learning activities to leverage the potential of technology incrementally, while greater numbers of students are increasingly engaged in learning. More adminis-trators and teachers are also starting to use real-time data to inform instruction.
Unfortunately, not everyone is there yet. We know this because the Pennsyl-vania Department of Education (PDE) is analyzing each EETT grant recipient’s readiness to use technology effectively through a research project required by NCLB as a condition for receiving funds. Through our evaluator, the Metiri Group, Pennsylvania is offering a customized online version of SETDA’s Profiling Educational Technology Integration (PETI) suite of tools to all its EETT grantees, with the expectation that resultant data will provide insights into how schools can further optimize their EETT investments. There are six research questions driving the state evaluation:
1. Type of Use. How is education technology being implemented and used in elementary/secondary schools as a result of this grant? Did specific target populations make progress?
2. Teacher Practices. As a result of this grant: Has teacher technology proficiency increased? Are teachers better prepared to use research-based methods and technology in their professional practices? Have their instructional practices changed?
3. System Capacity. Has the EETT grant program increased the capacity of school leaders and school systems to incorporate effective uses of technology into teaching and learning?
4. Progress. As a result of the grant, were there changes in: classroom practices; levels of access to technology; engagement of students in learning; or school culture, including parent/community connections?
5. Learning Outcomes. What impact did the grant have on students’ academic achievement levels? Did students acquire the technology and other 21st century skills necessary to thrive in a high-tech society?
6.State Administration. Is this program being effectively administered?
The state's evaluator cross-matched Pennsylvania’s research questions to the PETI framework, and then modified the PETI instruments slightly to align to Pennsylvania’s EETT priorities. By deselecting and selecting clusters aligned to PETI indicators, Metiri maintained the validity and reliability of the instruments, while providing Pennsylvania with a suite of tools aligned to its unique EETT program.
The District View
All administrators and teachers involved in Pennsylvania’s EETT program are asked to participate in the surveys before the “treatment” begins in the school to collect baseline data. Data collection is then repeated periodically in those same schools with trend data reported over time.
Individual survey results are kept confidential. The teacher survey takes about 20-25 minutes to complete, while the administrator survey requires about 30-40 minutes. Once the surveys are complete, an administrator can access the results in several ways. The district also has real-time access to the raw data at the click of a button, resulting in a file download in Excel along with a corresponding codebook. Thus, the district or its EETT evaluator can mine the data in ways that are meaningful to them.
The result is that all grantees have a profile of their current use of technology as well as their state of readiness to use technology effectively. At the same time, the state has an aggregate profile of those same elements - both statewide and disaggregated by community type, grade level and type of program. Furthermore, all data definitions are consistent.
Site Reviews
Each time the surveys are adminis-tered, site reviewers visit a stratified random sample of about 10% of grantees using the PETI site observation tools to serve as their visitation protocols. As with the surveys, these instruments are customized for Pennsylvania, again maintaining reliability and validity. The site visits represent a chance to collect anecdotal information that can more fully inform what is going on in individual classrooms. Here are two examples:
Asked about the impact of the EETT program, a school administrator from a small-town district in eastern Pennsylvania commented: “Prior to this grant, one of our third-grade teachers had minimal technology skills and only used technology to type worksheets for her students. Following the ILS training we offered, she immediately recognized the benefits of technology and became a leader in her building in implementing the ILS program as suggested. She followed the same path after attending the writing workshop trainings in her school. ...
“Last summer, she was introduced to Palm technology where she quickly adapted many of the writing workshop activities to a graphic organizer program (which was a part of the technology integration training) and to documents that could be accessed on a Palm. To date, her entire class of third- graders is actively and independently using their Palms to retell stories, brainstorm for their own stories, prewrite and write utilizing writing workshop strategies. Students beam files and assignments to their teacher and to each other on collaborative works. When instructed, they successfully sync to their teacher’s laptop for final printing and evaluation. Technology is a significant yet seamless part of these students’ learning process and success. ...”
Another school administrator from a small district outside of Pittsburgh reported gains in algebra scores through EETT support: "The foundation of our middle and high school math program is the integrated math series. But because this series incorporates diversified areas of course content into each chapter, students sometimes have difficulty learning one concept and then transitioning to another that’s seemingly unrelated. The cognitive tutor sequence of algebra I, geometry and algebra II has brought stability and motivation to the learning process for these students. The interaction of student with computer personalizes the learning process and provides immediate feedback. The cooperative learning component provides students the [chance] to problem solve with peers about applied, real-world situations.
“Additionally, teacher professional development is intense and provides the strategies to direct specific student learning. Our teachers also have formed a network of colleagues that affords them vital problem-solving support and technical assistance.”
The site visitation data is then entered online, aggregated, and incorporated into the school and state reports, providing further validation to the survey data. Stories from the field bring the data sets to life through examples and insights from students, educators and teachers.
The State View
The evaluation design for EETT in Pennsylvania contains data collection from multiple sources, including the PETI surveys for teachers and administrators, site visitations based on the PETI site tools, biannual reports from project directors (including evaluation data), and end-of-project reports. The PDE has access to real-time reports from the PETI instruments, including online frequency reports for each survey question, as well as data exports. Periodic reports are also customized to provide updates that triangulate the data from all sources and report results organized around the department’s research questions. Finally, Metiri supports the PDE by giving feedback to schools on their progress and recommendations for improvements or necessary midcourse corrections.
In Summary
PDE has found the PETI tools to be very useful. They provide feedback to schools regarding their readiness to use technology, systemically enabling schools to more effectively leverage EETT investments for the attainment of NCLB goals.