Miami Science Museum
Miami Science Museum Home

Research

The Institute for Education Sciences funded the Miami Science Museum, in partnership with the University of Miami, to develop a comprehensive early childhood science curriculum, assessment tools and professional development program.

The overall goal of Early Childhood Hands-On Science (ECHOS®) is to investigate science as a domain for enhancing overall school readiness and to demonstrate that very young children can learn fundamental science concepts and the process skills associated with higher order thinking skills.

The fully developed preschool science curriculum consists of nine units, each containing four guided week-long science lessons that are introduced in 20-minute segments to small groups of children. A teacher assistant and/or volunteer simultaneously works with the balance of the children using related integration cards in three domains: language/literacy, math and creative arts. The iCards are short activities that are easy to set up.

ECHOS focuses on Life, Earth, and the Physical Sciences. The curriculum combines direct instruction with guided inquiry-based science experiences and exploration. Curriculum units are sequenced to present increasingly more complex science process skills in the categories of observing, describing, categorizing, predicting, experimenting and drawing conclusions.

Quasi-Experimental Study

A quasi-experimental study was conducted in 2010 with 30 Head Start Centers in Miami-Dade County to examine the efficacy of the ECHOS model program. ECHOS professional development and classroom implementation occurred over a seven-month period. In summary, the study found that:

  • Teachers who had exposure to ECHOS felt more comfortable teaching science and had more positive attitudes and beliefs toward how science can benefit preschool children.
  • Children in ECHOS classrooms obtained significantly higher scores on the direct assessment of preschool science.
  • Both teachers and children in ECHOS classrooms exhibited significantly higher levels of science-related behaviors, relative to non-ECHOS classrooms.

Results are presented below by measurement instrument.

Preschool Science Classroom Observation Tool (PreSCOT)

Science-related behavior ratings of teachers (t(28) = 0.654, ns) and children as a group (t(28) = 0.343, ns) in ECHOS and non-ECHOS classrooms were not significantly different at the beginning of the year. However, at the end of the year teachers (t(28) = -2.777, p < .01) and children (t(28) = -2.683, p < .01) in ECHOS classrooms exhibited significantly higher levels of science-related behaviors, relative to non-ECHOS classrooms.

Teachers' science-related behaviors across the school year graph
Children's science-related behaviors across the school year

Direct Assessment of Science

Multi-level analyses were conducted (using HLM6) in order to examine the effects of the ECHOS professional development program on children's science readiness. Children in ECHOS classrooms obtained significantly higher scores at the end of the year relative to children in non-ECHOS classrooms, after controlling for demographic covariates (age, sex, ethnicity) and pre-test science scores (y01 = 23.92, t = 3.48, p < .001).

Preschool Teacher Attitude and Beliefs about Science (P-TABS)

Relative to the beginning of the year, teachers in ECHOS classrooms at the end of the year felt more comfortable teaching science (t(49) = -2.137, p < .05) and had more positive attitudes and beliefs toward how science can benefit preschool children (t(49) = -2.861, p < .01). Teachers in non-ECHOS classrooms, however, showed no changes from the beginning to the end of the year.

Randomized Controlled Trial

From 2011 to 2013, the Institute for Education Sciences provided funding for the Miami Science Museum, in partnership with the University of Miami, to conduct a two-year full-scale efficacy study of the ECHOS professional development model.

  • In the 2011-2012 school year, 91 classrooms (45 ECHOS, 46 Control), with no prior experience implementing ECHOS, participated in the first year of the two-year RCT. Baseline data were collected prior to randomization.
  • Classrooms were randomly assigned to treatment group implementing ECHOS or to a business-as-usual control group.
  • Approximately ten children from each classroom were randomly selected and stratified by age and gender, to be assessed on their science skills.
  • The program serves primarily African-American (48%) and Hispanic (51%) children, ages 3-5.

Year One Findings

Preschool Science Classroom Observation Tool (PreSCOT)

Analyses at the end of the first year show preliminary evidence for the effectiveness of ECHOS on the quality of teacher's science instruction and on children's science skills. A series of repeated-measures analysis of variance (ANOVA) revealed that ECHOS teachers significantly improved their scores in use of instructional strategies associated with higher-order thinking skills, specifically predicting, investigating, and evaluating, across the year, while control group teachers did not.

PreSCOT Results Graphs

ECHOS Fidelity Observation Tool

This ten-item instrument administered fall, winter and spring was designed to measure adherence to model (5 items) & quality of implementation (5 items) using a 3-point rubric (0, 1, 2). With respect to fidelity, items were created to measure both fidelity "adherence" and fidelity "quality." Overall, teachers' fidelity to ECHOS was high. ECHOS teachers showed significant increases in total fidelity and fidelity quality from winter to spring.

Changes in Fidelity Adherence and Fidelity Quality from Winter to Spring

Fidelity Winter Score
Average (SD)
Spring Score
Average (SD)
Change over time
Average (SD)
Adherence 8.29 (1.82) 8.73 (1.51) Not significant
(F = 2.18, p = .147)
Quality 7.69 (2.34) 8.64 (2.01) Significant
(F = 7.29, p = .01)
Total 15.98 (3.95) 17.38 (3.19) Significant
(F = 5.37, p = .025)


P-TABS (Preschool Teacher Attitudes and Beliefs About Science)

Administered to teachers in fall and spring of academic year. Three subscales: child benefit, teacher comfort and teacher challenges. Over time, ECHOS teachers reported significantly greater challenges in teaching science while control teachers did not. Over time ECHOS teachers also rated science as having more benefit for children and felt more comfortable teaching science.


Teacher Science Knowledge Test

ECHOS teachers significantly improved their scores from fall to spring, while control teachers did not, on a test of general science knowledge developed by the UM research team.


Direct Assessment of Preschool Science

Approximately 900 children (randomly selected to 450 per condition) were assessed on science skills in the fall, winter and spring of the school year.

Multilevel models (time nested in children nested in classrooms) were analyzed to determine if ECHOS was associated with children's growth in science skills. All children showed significant growth in science skills (p < .001). A trend indicated faster rates of growth for children in ECHOS classrooms (p = .069)

While ECHOS did not predict gains from fall to winter, a trend indicated that ECHOS predicted more gains in science skills from winter to spring compared to control classrooms.

Year Two Findings

Does ECHOS improve Head Start teachers' science teaching?

Using the Preschool Science Classroom Observation Tool, teachers were observed in the Spring, 2011 prior to random assignment to ECHOS and Control conditions presenting a science lesson of their choice. The first time point (baseline prior to random assignment to Treatment (ECHOS versus Control) shows the equivalence of the two groups (.79 versus .81). Analysis of the four subsequent time points after random assignment and the beginning of the ECHOS program (Winter, 2011, Spring 2012, Fall, 2012, Spring 2013) were conducted using SPSS version 21 with the GLM procedure using a repeated measure design. Treatment (ECHOS versus Control) served as a between subject variable and time as a within subject variable. These data are shown in figure 1 below. ECHOS teachers had a higher mean PreSCOT score across these 4 time periods compared to Control teachers (.85 versus .74). This difference was statistically significant, F(1,64) 4.43, p = .039.

PreSCOT Year 2 Results Graph

Evaluations of ECHOS Replication - Shared by Organizations

Great Lakes Science Center, Cleveland, Ohio

PNC Grow Up Great: Year 2 Final Evaluation Report, August 2014

Executive Summary

The Great Lakes Science Center (GLSC) partnered with Catholic Charities Head Start (CCHS) to support quality science programming for young children and their families. This is the second year that GLSC has trained early childhood educators to implement the Early Childhood Hands On Science (ECHOS) curriculum from the Miami Museum of Science, and the first year of CCHS' participation.

Programming included professional development (PD) sessions and an individual classroom coaching session for each of 26 lead teachers and their assistants. Supervisors also received targeted PD sessions. Each classroom received an in-class presentation from GLSC staff and enjoyed three field trips. Families were invited to two family events, received newsletters with suggestions for at-home science activities, and a family pass to visit the science center.

The impact of this year's project was evaluated by collecting data through teacher surveys, family surveys, teacher interviews, classroom visits and feedback from project leads. Among the factors that key GLSC and CCHS staff identified as contributing to the project's success were the organization-wide buy in by CCHS staff, with an administrator who was very involved with the implementation, the purposeful alignment of the excellent ECHOS curriculum with CCHS' HighScope curriculum, and engagement of site supervisors.

The results of the project were very impressive. 80% of the teachers interviewed provided examples of their classroom science activities that were rated in the "higher quality" range, twice the rate for comparison teachers, and a marked increase over project teachers' own ratings from the beginning of the year. Classroom environment scores for science approached the "excellent" range.

Teachers reported greater confidence in their skills and knowledge related to early science teaching. They are offering quality science activities with increasing regularity, with all teachers offering science activities at least weekly and 50% reporting daily activities.

Supervisors reported providing more frequent feedback related to science by the end of the year, with half of the supervisors providing feedback monthly or more. Teachers rated the supports for their science teaching highly. Teachers are using the supports for their science teaching available through the project more frequently, with the materials provided by GLSC being used most often.

84% of project families attended a science event with their child. More than half of the project families reported doing a recent at-home science program with the child. Both of these rates were slightly lower than the rate for comparison families. However, of those families who did at-home activities, project families reported doing so more frequently.

Next year the project can build on their success by considering how to build a community of learners, including teachers and supervisors, to help support continued excellence in their early science programming.

Click here to download the complete Evaluation Report


ECHOS® is a registered trademark of the Patricia and Phillip Frost Museum of Science.

Copyright © 2014 Patricia and Phillip Frost Museum of Science