Extra Comp Sci Lessons Associated with Higher Assessment Scores

• By Joshua Bolkan
• 04/17/18

Students who did extra computer science activities in transdisciplinary modules scored higher on reading comprehension exams, as well as Florida standardized tests in science and English and language arts (ELA), according to preliminary results from a new study.

The findings are the preliminary results of a National Science Foundation-funded study led by Outlier Research and Evaluation from the University of Chicago and conducted at Broward County Public Schools (BCPS) using computer science education resources from Code.org.

For the study, Broward embedded "Code.org Fundamentals lessons in BCPS' nonnegotiable elementary literacy block," according to information released by Outlier. "To do so, BCPS personnel developed 'transdisciplinary' 'Time for CS' (Time4CS) modules that included science, ELA and social studies lessons and associated Code.org lessons connected with a problem-based theme. In the 2016-2017 academic year, participating schools implemented two modules for each, 3rd, 4th and 5th grade during BCPS's existing 180-minute literacy block."

More than 1,600 students taught by 157 teachers at 16 different schools were included in the study, with half of the classes using the Time4CS units and the other half serving as the control group, though the number of students assessed varied from one exam to another.

The scores of 1,607 students on Achieve 3000 literacy assessments showed that "Teacher completion of a higher percentage of 'extra' Code.org CS lessons was significantly associated with higher Achieve 3000 literacy scores under the condition where the teachers reported themselves as higher in 'resourcefulness and coping,'" according to a report on the study's preliminary findings. More experience teaching computer science was also correlated with higher scores on this exam, as were higher levels of self-reported teacher innovativeness.

For the standardized tests, the researchers used sample sizes of 1,600 students for ELA and math, and 580 students for science, as those assessments are only given to students in grade 5 at BCPS. Researchers found that higher scores on these assessments were also associated with teachers who completed extra Code.org computer science lessons if the teachers also self reported as being more resourceful and capable of coping or, again, having higher levels of self-reported innovativeness.

The results may be promising, but the researchers point out that these preliminary findings, at least, raise more questions than answers.

"So, what are we to make of these findings?" a whitepaper on the preliminary results asks. "Like any study, this one elicits a number of new and interesting questions. Why were there significant findings associated with completion of a higher percentage of 'extra' Code.org CS lessons and completing 'additional' CS activities, whereas completion of grade-level specific Code.org lessons were not associated with student outcomes? What is the role of teacher resourcefulness and coping in a classroom and how might it be related to student academic achievement? What might explain why teachers who report being more innovative have students with higher academic outcomes?"

Visit outlier.uchicago.edu to read a whitepaper from Outlier about the preliminary results. More information is also available in a Code.org blog post at medium.com.