State-Level Studies
State-Level Studies
Evaluating State Educational Technology Program (ESETP). ESETP is a federal grant program designed to increase the capacity of states to design, conduct and procure high-quality evaluations of educational technology. The grant competition provides more than $15 million over the next three years for states to:
1. Plan and conduct a scientifically based evaluation of an educational intervention that uses technology applications to increase student achievement in one or more core academic subjects.
2. Test and document the methods, practices and instruments used to assess the impact of the intervention on student achievement.
3. Make documented information about the evaluation plan and its implementation available to other states.
The Education Department, through its School Support and Technology Programs Office and its Office of Educational Technology, made grant awards to 10 State Education Agencies. The focus of the studies is described in brief below.
1. Arkansas Education Department
Evaluation of the EAST (Environmental and Spatial Technology) Initiative; $1.8 million over three years.
The EAST Initiative involves the creation of interdisciplinary school-based technology labs that promote student intellectual growth and technology skills acquisition, as well as teacher training on facilitating student learning through service projects and teamwork. The study will involve 120 projects serving 9,000 students (55% rural, 25% suburban and 20% urban setting).
2. Iowa Education Department
Using Technology to Support the Scaling-Up of the Iowa Professional Development Model; $1.9 million over three years.
This multiagency, statewide field-research effort will focus on reading and mathematics instructional practice in the sixth, seventh and eighth grades in 150 public school districts, including 43 high-needs districts as defined by NCLB. The project will yield a research model for identifying and scaling up teacher training on best practices.
3. Maine Education Department
The Impact of Teachers' Professional Development on the Mathematics Achievement of Low-Performing Rural Students in Technology-Rich Classrooms; $1.9 million over three years.
The study will focus on seventh- and eighth-grade students in schools that serve low-income rural communities and have shown low performance in eighth-grade mathematics. The study promises to contribute to research-based knowledge of effective practices in mathematics education and technology integration, ubiquitous computing, professional development, and education in low-income rural schools.
4. N.C. Public Instruction Department
LANCET: Looking at North Carolina Educational Technology; $1.5 million over three years.
This project will study the implementation of the state's IMPACT professional development model and its effects on schools, teaching practices and student achievement. The project will develop and assess strategies for building the capacity for educators across the state to collect, analyze and use evaluation data for making decisions about technology programs, projects and practices.
5. Pennsylvania Education Department
Evaluation of Student and Parent Access Through Recycled Computers (eSPARC); $1.8 million over three years.
The eSPARC study project seeks to develop and test an evaluation model to measure the impact of educational technology initiatives. The project will randomly assign recycled computers to a sample of 400 fifth-graders and their families. The study will assess whether and how in-home computer and Internet access impact students and parents.
6. Tennessee Education Department
The Tennessee EdTech Accountability Model (TEAM); $1.7 million over three years.
The project will measure the effectiveness of an intervention to prepare school-based technology coaches to work with teachers to align technology use with the delivery of the curriculum to foster increased student achievement. The project will measure the effectiveness of the intervention in 37 schools and develop a replicable, validated evaluation protocol for use in all schools.
7. Texas Education Agency
Evaluation of the Texas Technology Immersion Pilot (eTxTIP); $1.9 million over three years.
TxTIP is a technology immersion pilot seeking to increase student achievement by providing each student with a wireless mobile computing device, software, online and other learning resources. The TxTIP evaluation will study the effectiveness of technology immersion in increasing middle school students' achievement in core academic subjects, technology proficiency and attendance, among other outcomes.
8. West Virginia Education Department
ED PACE: Educational Development for Planning and Conducting Evaluations; $1.4 million over three years.
ED PACE will assess student achievement in virtual foreign language courses as compared to the achievement of students in classroom-based foreign language courses. Over three years, the project will generate three scientifically based research models (a summative, formative and action research model) that can be replicated in other settings.
9. West Virginia Education Department
The Evaluation of West Virginia's Enhancing Education Through Technology Model School Project; $1.3 million over three years.
The study will make use of technology-based desktop meters and random interval data collection pop-up screens to document the use, time, topic and function budgets of teachers and students. These methods will yield information about classroom integration of technology as an outcome of professional development and the impact of technology integration practices on student performance.
10. Wisconsin Public Instruction Dept.
A Study of the Effectiveness of Three Models of Implementing Educational Technology; $1.6 million over three years.
The Wisconsin project will (a) identify three promising models of educational technology use in state schools from existing data; (b) implement the models using the Title II, Part D competitive grant process; (c) and evaluate the effectiveness of the models on student achievement.
State Educational Technology Evaluation Grants
The U.S. Education Department, through the School Support and Technology Programs Office and the Office of Educational Technology, made grant awards to 10 State Education Agencies. Descriptions of the studies, including the scientifically based methodologies on which the studies are based, are provided below.
Arkansas Department of Education
Evaluation of the EAST (Environmental and Spatial Technology) Initiative
$1.8 million over three years
Online: www.eastproject.org
This project will assess the nature, quality and intensity of EAST program implementation strategies and processes. It will also evaluate their relative outcomes on teachers' attitudes, classroom practices and content knowledge, as well as on students' attitudes, skills and achievements. EAST involves the creation of interdisciplinary school-based technology labs that promote student intellectual growth and technology skills acquisition, as well as teacher training on facilitating student learning through service projects and teamwork. The study will involve 120 projects serving 9,000 students (55% rural, 25% suburban and 20% urban setting). Study results will yield deeper insights into specific participant, environmental and program characteristics that appear to influence student outcomes. In addition, an evaluation sustainability study will assess the extent to which the project's dissemination and aptitude-building activities are serving the capacity of Arkansas and other states to plan, conduct and procure high-quality evaluations.
Iowa Department of Education
Using Technology to Support the Scaling-Up of the Iowa Professional Development Model
$1.9 million over three years
Online: www.state.ia.us/educate
This project will use the Iowa-adopted professional development model, which is based on best practices, as the basis for scaling up an educational intervention system using experimental, quasi-experimental and randomized classroom trials. It will seek to demonstrate that scientifically based teacher training on best practices using technology must be causally linked to the implementation of those practices in the classroom in order for the impact to be observed in student achievement in math and reading. This multiagency, statewide field-research effort will focus on reading and mathematics instructional practice in the sixth, seventh and eighth grades in 150 public school districts, including 43 high-need districts as defined by the NCLB Act. The project will yield a research model for identifying and scaling up teacher training on best practices.
Maine Department of Education
The Impact of Teachers' Professional Development on the Mathematics Achievement of Low-Performing Rural Students in Technology-Rich Classrooms
$1.9 million over three years
Online: www.state.me.us/education
The Maine evaluation uses an experimental design with randomized assignment of schools to investigate the impact of intensive, multifaceted professional development on teacher classroom practices, as well as the impact of student and teacher use of technology to enhance mathematics learning and student mathematics achievement. The study will focus on seventh- and eighth-grade students in schools that serve low-income rural communities and that have shown low performance in eighth-grade mathematics. The study promises to contribute to the research-based knowledge of effective practices in mathematics education and technology integration, ubiquitous computing, professional development, and education in low-income rural schools.
North Carolina Department of Public Instruction
LANCET: Looking at North Carolina Educational Technology
$1.5 million over three years
Online: www.ncpublicschools.org/news/03-04/103103.1.html
The LANCET project will use experimental, quasi-experimental and case study designs to study the implementation of the state's IMPACT professional development model, as well as its effects on schools, teaching practices and student achievement. The project will develop and assess strategies for building the capacity for educators statewide to collect, analyze and use evaluation data for making decisions about technology programs, projects and practices. It will also disseminate the strategies, methods, instruments and protocols used in and resulting from the project.
Pennsylvania Department of Education
Evaluation of Student and Parent Access Through Recycled Computers (eSPARC)
$1.8 million over three years
Online: www.pde.state.pa.us/ed_tech/cwp/view.asp?Q=100397&A=169
The eSPARC study project seeks to develop and test an evaluation model that can be used by local and state education agencies to measure the impact of educational technology initiatives. The project will randomly assign recycled computers to a sample of 400 fifth-grade students and their families. The study will then assess whether and how in-home computer and Internet access impact students and patents. The study will also produce and disseminate research methods and tools that can be used to measure the impact of technology initiatives across program areas.
Tennessee Department of Education
The Tennessee EdTech Accountability Model (TEAM)
$1.7 million over three years
Online: www.state.tn.us/education/acctteam.htm
This project will measure the effectiveness of an intervention to prepare school-based technology coaches to work with teachers on methods of aligning technology use to the delivery of the curriculum by using instructional materials that foster increased student achievement. The project will measure effectiveness of the intervention in 37 schools; develop a replicable, validated evaluation protocol for use in all schools, and disseminate the results and instruments nationally.
Texas Education Agency
Evaluation of the Texas Technology Immersion Pilot (eTxTIP)
$1.9 million over three years
Online: www.tea.state.tx.us/
TXTIP is a state-mandated technology immersion pilot that seeks to increase student achievement by providing each student with a wireless mobile computing device, software, as well as online and other learning resources. The TxTIP evaluation will test the effectiveness of technology immersion in increasing middle school students' achievement in core academic subjects, technology proficiency, attitudes and attendance. The evaluation will also test its effect on the school environment, personnel, as well as on parent and community partnerships. Approximately 38,000 students and 2,700 teachers in about 60 randomly assigned middle schools will participate in the evaluation.
West Virginia Department of Education
ED PACE: Educational Development for Planning and Conducting Evaluations
$1.4 million over three years
Online: http://wvde.state.wv.us
ED PACE will employ a quasi-experimental design with experimental elements to assess student achievement in virtual foreign language courses as compared to the achievement of students in classroom-based foreign language courses. Over the course of three years, the project will generate three scientifically based research models (a summative research model, a formative research model and an action research model) that can be replicated in other settings used at local, state and national levels to measure the impact of other technology-enhanced interventions on student achievement and to validate their effectiveness.
West Virginia Department of Education
The Evaluation of West Virginia's Enhancing Education Through Technology Model School Project
$1.3 million over three years
Online: http://access.k12.wv.us
The project will assess the outcomes for teachers and students of West Virginia's ESEA Title II, Part D school-based teacher trainer initiative. The study will employ an experimental research design and make use of technology-based desktop meters and random interval data collection pop-up screens to document the use, time, topic and function budgets of teachers and students. These methods will yield objective, detailed information about classroom integration of technology as an outcome of professional development and the impact of technology integration practices on student performance on West Virginia's tests of standards-based content. In its third year, data from the study will be used to explore the extent to which evaluation data is used by state policy-makers to inform decisions.
Wisconsin Department of Public Instruction
A Study of the Effectiveness of Three Models of Implementing Educational Technology
$1.6 million over three years
Online: www.dpi.state.wi.us/
The Wisconsin project will (a) identify three promising models of educational technology use in state schools from existing data; (b) implement the models using the Title II, Part D competitive grant process; and (c) evaluate the effectiveness of the models on student achievement using quasi-experimental methods to assign experimental and control groups, student portfolios, student self-reports, and standards-based knowledge assessments to measure student achievement.
This article originally appeared in the 05/01/2004 issue of THE Journal.