Evidence of gains in student understanding of science through the use of GEMS.

Earth, Moon, and Stars Research Studies: An extensive body of research supports both the approach used and the level of educational effectiveness achieved by the GEMS unit Earth, Moon, and Stars. This research, including related studies and publications, contains strong evidence of gains in student understanding of two importance science concepts—gravity and the spherical shape of the Earth. Based on what research has determined to be major student misconceptions, the GEMS unit was developed and tested, then studies were conducted to determine the extent to which the designed unit made a measurable difference in student learning. Subsequently, lessons learned from this research have been applied to several other GEMS units, including Moons of Jupiter and the upcoming Messages from Space: The Solar System and Beyond (in press).

The study involved the application of Nussbaum’s Earth Notions Classification Scheme to results of testing of 159 boys and girls from public schools in San Francisco, and the relation of these results to research conducted in Nepal, Israel, and Ithaca, New York. The classification scheme was itself subjected to rigorous analysis (see under "Results/Validation and Refinement of the Earth Notions Classification Scheme, page 211–217) with statistical and prediction analysis, resulting in a suggested refinement to include an additional notion between Notions III and IV in Nussbaum’s classification. Results were analyzed based on age and grade, and in relation to previous research, and confirmed that children interpret information about the spherical Earth and gravity in terms of their own models of the world, and that these interpretations, while representing reasonable "alternative frameworks" from the child’s point of view, require considerable additional learning experience to be transformed into accurate scientific conceptions.

The GEMS unit Earth, Moon, and Stars was intentionally designed and developed to help students overcome the misconceptions that were "unearthed" by the series of research studies, by engaging students in observations of the sky, and having them consider how alternative models can best explain their observations. The GEMS unit prominently includes a pre- and post-questionnaire, "What Are Your Ideas About the Earth?" which builds upon the experience of the previous studies, and in turn became a central element in the subsequent research studies of educational effectiveness of the GEMS unit. (Aside from these more formal studies, teachers are also instructed in the unit about how to use the questionnaire provided as a pre-test and post-test to assess the degree to which their students have comprehended modern scientific concepts about the Earth’s shape and gravity. The questionnaire, with actual student work, and instructions to the teacher on its use, are featured as a case study in the GEMS assessment handbook.)

Since the GEMS unit was developed, tested, and published, additional studies have evaluated the extent to which the activities in the unit succeed in overcoming student misconceptions. "Unraveling Students’ Misconceptions about the Earth’s Shape and Gravity," details a study involving 539 students from 18 classrooms in 10 different states. The experimental treatment was the GEMS unit, Earth, Moon, and Stars. The primary experiment was a treatment-group-only design, in which teachers (trained in the use of the questionnaire assessment instrument at a summer institute sponsored by NSF) administered the same test to all students before and after the treatment. The purpose was to determine the impact of the treatment on students’ understanding of the Earth’s shape and gravity concepts. Data were analyzed in three age groups (4th and 5th graders; sixth graders; and 7th and 8th graders). As expected from previous studies, on the pretest all classes displayed a wide variety of conceptions about these concepts. After the unit, however, the number of subjects who held misconceptions was far fewer. Chi-square analyses showed that a significant number of students at all grade levels shed their misconceptions concerning both the Earth’s shape and gravity. A surprising finding was that younger subjects responded more positively to the experimental treatment than older students, so that, after instruction in the GEMS unit, fourth and fifth graders were as knowledgeable as seventh and eighth graders concerning the Earth’s shape and gravity. While the GEMS unit was tested and found effective from Grades 4–8, the study suggests that this may indicate that presentation of the unit at the earlier grade levels may be particularly beneficial.

As Table 4 on page 279 of the Sneider Ohadi article depicts, percentages of students who demonstrated increased understanding of the Earth’s shape before and after the GEMS unit went from 24% to 72% for Grades 4–5; from 27% to 45% for Grade 6; and from 38% to 62% for Grades 7–8. The percentage of students understanding gravity went from 7% to 67% for Grades 4–5; from 15% to 47% for Grade 6; and from 30% to 60% for Grades 7–8. In conclusion, the authors state: "…The concepts selected for study by the students—the earth’s spherical shape and gravity—were considered by many researchers to be of fundamental importance in allowing students to understand the modern scientific explanations of a wide variety of phenomena, such as the daily cycle of the sun, phases of the moon, and seasons. These findings were bolstered by a full-experimental, control-group study… supporting the conclusion that the constructivist teaching unit—Earth, Moon, and Stars, from the GEMS series—enabled large numbers of students to unravel their misconceptions and construct a more accurate model of the world." (Emphasis added.) These studies, and additional studies and articles referenced below, have also had a significant impact on further curriculum development in GEMS, in particular on the Moons of Jupiter and Messages from Space units (see Sneider articles in collaboration with Varda Bar and others cited below on gravity in space, weight and free fall, and gravity and air). As the Sneider Ohadi study progressed, insights gained were taken into account as the GEMS assessment handbook was developed and as revisions of Earth, Moon, and Stars were published.

C. Sneider and S. Pulos. "Children’s Cosmographies: Understanding the Earth’s Shape and Gravity." Science Education 67 (2) (1983): 205-221.

C. Sneider and S. Pulos, Evangeline Freenor, Joyce Porter, and Betty Templeton, "Understanding the Earth’s Shape and Gravity," Learning ‘86, Vol. 14, No. 6, February, 1986, pages 43–47.

C. Sneider and M. Ohadi. "Unravelling Students’ Misconceptions About the Earth’s Shape and Gravity," Science Education 82 (1998) pages 265–284.

C. Sneider, Earth Moon and Stars GEMS teacher’s guide, Lawrence Hall of Science, 1986, 1989, 1992, 1994, 1996, 1998.

C. Sneider with Varda Bar and Nathalie Martimbeau, "What Research Says: Is There Gravity in Space?" Science and Children, April 1997.

C. Sneider with Varda Bar, Barbara Zinn, and Rivka Goldmuntz, "Children’s Concepts About Weight and Free Fall," Science Education, Volume 78, Number 2, pages 149–169, 1994.

C. Sneider, "Does Gravity Need Air?: A Force That Holds the Universe Together," GEMS Network News, Fall/Winter, 1993, pages 26–29.

C. Sneider, "Shape of the Earth Assessment Task for Earth, Moon, and Stars," GEMS Network News, Fall/Winter, 1993, pages 30–31.

Barber, J et al, Insights and Outcomes: Assessments for Great Explorations in Math and Science, "Case Study Using Pre-Post Testing, What Are Your Ideas About the Earth from Earth, Moon, and Stars," pages 102–109.

Nussbaum, Joseph, "The Earth as a Cosmic Body," in R. Driver, E. Guesne, A. Tiberghien (editors) Children’s Ideas in Science, Open University Press, Philadelphia, 1985.

The Galaxy Classroom Project (Pilot Program 1991-1995) funded by Hughes Aircraft and the National Science Foundation: The Galaxy Classroom Project is a multimedia, year-long program for K–5 students in classrooms nationwide. A main goal of the K–2 component is to impact student learning of the science/mathematics concepts and processes of observing, comparing, communicating, properties of solids and liquids, structure/function of living organisms. The Pilot Program consisted of a core classroom curriculum from the GEMS and FOSS projects of LHS with two series of interactive television programs designed to incorporate the science and math concepts emphasized in the classroom program. Family home activities and classroom activities involving fax and the Internet are also included. The program is currently being conducted statewide in Georgia, in selected districts in California, elsewhere in the United States, and in Canada. Since 1995, the Project revised their classroom curriculum to include only GEMS units. The GEMS units for the K–2 program include Terrarium Habitats, Liquid Explorations, and other GEMS early childhood units. The 3rd through 5th grade program focuses on Bubble-ology, Oobleck, Chemical Reactions, Investigating Artifacts, and five others. As is typical of GEMS, several of these units also have a strong mathematics component. The executive summary of the final report of the Galaxy K–2 program show student gains in learning key concepts, improvements in teacher instructional practices, and an increase in curiosity of students. The evaluation gathered quantitative data on GALAXY’S impact on student learning through pre-post tests of observation skills and an assessment of the science content presented. The report also states: "Teacher reports and evaluation results confirm that most students understood the concepts of the two GALAXY themes (recognizing and comparing the properties of various liquids, solids, and mixtures and identifying and comparing the characteristics and features of insects)." The report adds, "GALAXY first and second graders exhibited a striking and statistically significant growth in curiosity when compared to their non-GALAXY peers." (Page1, Far West Laboratory, Final Report.) The Far West Laboratory for Educational Research and Development (currently West Ed Laboratory) of San Francisco conducted the evaluation for the Galaxy Classroom Project. It was directed by Dr. Gloria Guth.

PEACHES I Project, funded by the National Science Foundation: In 1994, the Primary Education for Adults, Children, and Educators in Science (PEACHES) Program completed a 4-year teacher enhancement project funded by NSF to improve the science and math teaching skills of preschool teachers and day care providers. The project conducted two 60-hour, 4-unit courses for early childhood educators based on 10 curriculum units developed by the program that contained developmentally appropriate science/math activities for 4-6 year olds. Six of the ten units, Tree Homes, Ant Homes Under the Ground, Ladybugs, Eggs Eggs Everywhere, Penguins and Their Young, and Mother Opossum and Her Babies are part of the GEMS program—GEMS re-tests and publishes PEACHES trial versions, which then become GEMS units. In the final evaluation, Project Evaluator Dr. Bo De Long conducted a study to measure changes occurring in preschoolers’ knowledge about life science as a result of the project curriculum. Eighteen preschool-age children (from 3.4 years to 5.0 years) were administered individual interviews regarding their knowledge of and familiarity with the diets, habitats, and defense behaviors of various pond-, tree-, and ground-dwelling animals featured in the curriculum. These interviews consisted of open-ended questions designed to elicit knowledge about these animals as well as reveal the kinds of reasoning skills the children were employing when talking and thinking about life science. Students were administered the interview as a pre-test approximately one week prior to beginning the units (Tree Homes, Homes on the Ground, Ant Homes Under the Ground, and Homes in a Pond) and then again as a post-test approximately one week following completion of the units. The total time between pre- and post-tests was approximately three months. A control group from the same school was administered the same pre- and post-test separated by a similar three month interval. This group was included in order to ensure that any changes observed in the children’s quality or quantity of factual knowledge as well as changes in the reasoning skills they applied were due to exposure to the curriculum rather than to expected developmental changes or learning effects due to testing. The original control group included twelve three- and four-year-olds, divided equally by sex—however, all but four females dropped out of the study due to moving, changing schools, or entering a new class where the units were being taught.

Children’s understanding and retention of some of the content taught was measured by how well they could identify six animals presented in the units and by evaluating their responses to questions about the diet, habitats, and defense behaviors of those animals. Students were first given a dichotomous score (correct/incorrect) for their knowledge of the identity and behaviors of these six animals. The findings indicate that children exposed to the curriculum did, in fact, learn and retain content about the behaviors of the animals they are studying.

Table 6. Mean scores for content evaluation pre and post tests for treatment group and computed values of t for within-subjects matched group comparisons

Critical value of t at 17df = 2.110, p< .05*; 2.898, p< .01**; 3.965, p< .001***

Table 7. Mean scores for content evaluation pre and post tests for control group
and computed values of t for within-subjects matched group comparisons

Critical value of t at 3df = 3.182, p< .05*

Note: The accuracy of teachers’ perceptions that children became more sophisticated in some of their cognitive skills because of the PEACHES/GEMS units was also evaluated. While sufficient evidence was not found for this, the great majority of teachers reported that their students’ became better reasoners after they experienced the PEACHES/GEMS units, so this may merit further study. Insufficient evidence was also found for another teacher’s report—that the units had a positive impact on sorting and classification skills. Again, a longer-term study might reveal qualitative gains in classification skills that were not demonstrated in this short-term evaluation.

Seabrook GEMS Site Studies. Myra Luciano of the Seabrook, Texas GEMS Network Site has conducted two studies. In the first, in 1997, on the GEMS unit Build It! Festival at her elementary school, she conducted a performance-based evaluation with a random sampling of 19 second and 24 fifth graders to determine whether or not students demonstrated improved learning in the areas of patterns, shapes, and spatial sense. The compiled data shows considerable improvement in recognition of shapes and spatial sense for both grade level groups. In 1999, Ms. Luciano conducted a study on the GEMS unit Animal Defenses, with 30 1st graders in the treatment group and 16 1st graders in the control group. The treatment group of 30 participated in the Animal Defenses unit the year before, when they were Kindergarteners. The 16 in the control group had never been exposed to the GEMS lessons. Both groups were asked to record all the animal defenses they were familiar with. The data by Ms. Luciano shows that the treatment group’s median score was 4 words as compared to the median score of 1 word for the control group. The analysis by Eric Crane, School of Education, U.C. Berkeley indicates that the scores are significant if student factors such as age, grade, and teachers are taken into account. Further conversation with Ms. Luciano indicated that students in both groups were randomly distributed over three first grade classrooms at one school.

Top