Brian M. Slator
Department of Computer Science
North Dakota State University
USA
slator@cs.ndsu.edu
Donald P. Schwert
Department of Geosciences
North Dakota State University
USA
donald_schwert@ndsu.nodak.edu
Bernhardt Saini-Eidukat
Department of Geosciences
North Dakota State University
USA
bernhardt_saini-eidukat@ndsu.nodak.edu
Abstract: The Geology Explorer provides a multi-modal virtual
environment that implements an educational game for teaching principles of
geology. The game is a networked, multi-player, simulation-based, educational
environment that illustrates our role-based pedagogical approach. This takes the form of a synthetic, virtual
world (Planet Oit) where students are given an authentic experience and the
means and equipment to explore a planet as a geologist would. Each student's experience includes elements of
exploration of a spatially-oriented, virtual, world; practical, field oriented,
expedition planning and decision making; and scientific problem solving (i.e. a
“hands on” approach to the scientific method).
Students assume a role and learn about real science by exploring in a
goal-directed way and by competing with themselves and with other players.
In this
paper, data are reported from a 1999 study, in which students enrolled in a
freshman-level Physical Geology course explored the
planet for credit. These data were
collected in two forms: 1) a survey of student perceptions, positive and
negative; and 2) a data mining analysis of student histories which reveals
apparent categories of student problem-solving style. Planet Oit can be visited at http://oit.cs.ndsu.nodak.edu/
The Geology Explorer
(Saini-Eidukat, Schwert & Slator 1999; Schwert, Slator & Saini-Eidukat
1999) is a virtual world where learners assume the role of geologists on an
expedition to explore a mythical planet: Planet Oit, which is designed to
replicate the physical environments of Earth (and in the same orbit, but
directly opposite the Sun). Learners participate in field-oriented expedition
planning, sample collection, and “hands on” scientific problem solving.
To play the game, students are
transported to the planet's surface and acquire a standard set of field
instruments. They are issued an electronic logbook to record their findings
and, most importantly, are assigned a sequence of exploratory goals. The
students make their field observations, conduct small experiments, take note of
the environment, and generally act like geologists as they work towards their
goal. A scoring system has been
developed, so that students can compete with each other and with themselves.
The Geology Explorer is being developed as a synthetic environment using the freely available Xerox PARC LambdaMOO (Curtis 1997; Bruckman, 1997), which is an environment for creating text-based virtual worlds, to simulate a portion of Planet Oit. Students armed with tools and instruments created as LambdaMOO objects land on the planet to undertake an exploration exercise. They are given an authentic geologic goal, e.g., to locate and report the position of potentially valuable mineral deposits. Accomplishing each of these goals entails mastering several geologic concepts and procedures, and demonstrates student mastery of the material. The first module involves mineral exploration, where students are expected to plan an expedition, locate mineral deposits, and survive the somewhat hostile virtual environment in order to report on it (future modules on hydrology, metamorphic facies, etc. are underway). The first version of the Geology Explorer is text-based; (Slator, Schwert, & Saini-Eidukat 1999); a graphical interface was launched in Fall, 2000.
Planet Oit is designed to emulate
the geologic features and processes of Earth. The first version is based on a
realistic planetary design, consisting of a single, super-continent composed of
roughly 50 locations (Figure 1), arranged so as to be both diverse and
coherent. A variety of Earth-like
environments, ranging from tropical coastlines to volcanic calderas to
glaciated peaks, allows for multiple geologic terrains to be explored. A museum
of rocks and minerals is available at the landing site for use as a standard
reference collection. Coordination of
navigation on the planet is made possible by using directions relative to
Earth-like geographic poles (North, South, etc.).
Implemented, as well, are almost
40 scientific instruments and geologic tools (streak plate, acid bottle,
magnet, etc.), nearly 100 different rocks and minerals, and over 200 boulders,
veins, and outcrops. In the text-based
version, students use a command language, which allows for navigation,
communication, and scientific investigation while on the planet. Command verbs dictate the student’s application
of instruments (“streak,” “scratch,” “hit,” etc.) and senses (“view,” “taste,”
“touch,” etc.). Students can
communicate, and therefore work, with one another through verbal commands. An on-line "user card" listing all
these commands and functions is available at:
http://oit.cs.ndsu.nodak.edu/oit/usercard.html.
Once the layout and artifacts of
Planet Oit were implemented, the rules of the game were imposed over top. After being transported to the planet’s
surface, students are automatically assigned an initial exploratory goal and
can acquire whatever equipment they wish.
The goals are intended to motivate the students into viewing their
surroundings with a critical eye, as a geologist would. Goals are assigned from a principled set, so
as to leverage the role-based elements of the game by gradually leading
students to more difficult and interesting goals.
In order for a student to achieve
a goal (and therefore earn points), they must address a multitude of tasks
identical to those faced daily by field geologists. These include the selection and use of proper
field tools, navigation across the planet to the correct region, and
interpretation of the tests applied to the problem. As each goal is satisfactorily completed, the
student is automatically assigned new goals requiring progressively higher
levels of expertise and decision-making.
Through this practical application of the scientific method, students
learn how to think, act, and react as geologists (see Duffy & Jonassen
1992). This is a particular
strength. Student progress is tracked in
terms of goals achieved -- and students
have a self-paced, anytime/anywhere learning experience.
Physical Geology at North Dakota
State University is a large-enrollment (400+ students in one large lecture
section), 3 semester hour course. Aside
from lecture, the course content is augmented by slides, a set of course
lecture templates, a textbook, and a web resource site that includes
self-quizzes, photographs, course news, and links to related resources. Testing is by multiple-choice exams, with
students submitting their results on optical scan sheets. Nearly 100% of the students enroll in the
course to complete either general education science requirements or specific
course requirements within their majors.
Because the class is so large in enrollment, it demonstrates the obvious
impracticality of field-training each student to think and behave as a
scientist, and as a geologist.
In the Fall semester of both 1998
and 1999, the Geology Explorer was incorporated into the Physical Geology
curriculum. Data were gathered with a
view toward 1) answering several questions about student use of technology and
2) student perceptions of, and satisfaction with, the Planet Oit simulation
using an on-line evaluation questionnaire.
The primary goal was to
investigate the effect on the student experience with Physical Geology as
consequence of introducing the Geology Explorer prototype as a supplementary
resource to classroom instruction for a non-major introductory course. To do this, we implemented tracking routines
on Planet Oit in order to get statistics for time-on-task, correlations for
computer literacy and attitudes towards technology, effect on final grade. We anticipated that these data would lead to
a classification by learning style.
In the Fall, 1999, 81 students
completed a Geology Explorer assignment, scoring the required 500 points, and
then completed an on-line follow-up evaluation form. This form is web-based and requires
identification information (e.g. name, student ID number, and e-mail address),
and is composed of 35 questions about the Planet Oit experience (the full form
is on-line at http://oit.cs.ndsu.nodak.edu/~mooadmin/cgi-bin/oitform.html ).
In the post-test evaluation,
82.8% of the students said they somewhat agreed or strongly agreed they had
learned something from the game, and only five students (5.4%) disagreed or
strongly disagreed that they had learned something. At the same time, only 9.7% somewhat or
strongly disliked the concept of game, and 62.4% thought they might like to
play the game again. Meanwhile, the
students perceived the game to be at an appropriate level of difficulty, with
only 8.6% describing the game as much too complex, and 0 students believing it
was much too simple.
For the 81 students who completed
the assignment and the post-test evaluation, the average actual time on task
was 3.47 hours with a range from 0.83 hours to 8.04 hours (while the average
estimated time on task was 5.2 hours with a range from 1 to 12.5 hours). Of this group, 11 (13.6%), underestimated
time spent on the planet, 12 (14.8%), overestimated their time on task by 25%
or more, and 58 (71.6%), accurately estimated the time they spent on the planet.
Through the course of the experiment, and by
interacting with students both off- and on-line, we came to believe that
identifiable learning and problem-solving styles were being employed by the
students. Some students appeared to take
an analytical approach: frequently referencing the on-line help, conducting
sequences of experiments, and making diagnoses leading to their scoring points
in a deliberate fashion. Other students
seemed to take a pattern-matching approach: exploring far and wide in search of
outcrops that seemed to match the description of what they were looking for,
and then scoring points with relatively few experiments. There was also a small but noticeable group
taking a straight “brute force” approach, simply visiting location after
location and identifying everything there, one after another, as their goal,
eventually succeeding after many tries.
One monument to this approach was a student in 1998 who made 1,244
guesses on the way toward obtaining just five correct answers.
To investigate the nature of the
apparent trends, an analysis was conducted using logging data to count the
number of “reports” (i.e. guesses), the number of locations visited, and the
number of experiments conducted (e.g. hit, scratch, streak, etc.), for the same
81 students who had completed the game and the evaluation survey in 1999. These data are summarized in Table 1.
Reports Moves Experiments
average 42.6 139.2 73.8
st. dev 38.2 83.1 63.2
min 5 19 0
max 238 518 301
Table 1: Student Reports, Moves, and Experiments in
1999
Using these values for average and standard deviation, we developed a classification of behaviors by looking for combinations of either much higher or much lower than average activities in terms of reports, moves, or experiments, or combinations of these. These data are summarized in Table 2. There are a total of 24 clusters, each marked with a code, which represent the three significant categories.
The values in Table 2 indicate
that a wide range of approaches are supported by the Geology Explorer, a
testament to the user-centered and user-controlled nature of the
simulation. Further, almost half of the
students can be classified as consistently efficient, not only economizing on
their problem-solving effort, but doing so across all three dimensions.
Meanwhile, over half were above
normal in one or more dimension, with 17 making excessive reports (code “R,”
the “brute force” approach); 19 making excessive movements around the planet
(code “M,” the pattern matching approach), and 22 making more than the normal
number of experiments (code “E”). Note
that three students were excessive on all three dimensions, and only two
students were within 1/2 standard deviation on all three.
|
Consistently normal or below normal activity |
Consistently normal or above normal activity |
Mixed problem-solving activity |
|||
|
rme |
10 |
-ME |
5 |
-Me |
4 |
|
r-e |
8 |
--E |
4 |
rM- |
2 |
|
r-- |
5 |
R-E |
4 |
r-E |
2 |
|
-m- |
5 |
R-- |
3 |
RmE |
2 |
|
-me |
4 |
RME |
3 |
--- |
2 |
|
--e |
4 |
RM- |
3 |
Rm- |
1 |
|
rm- |
4 |
-M- |
2 |
-mE |
1 |
|
|
|
rmE |
1 |
|
|
|
|
|
R-e |
1 |
|
|
|
|
|
Rme |
1 |
|
|
|
Total (49.4%) |
40 |
Total (29.6%) |
24 |
Total (21.0%) |
17 |
Table 2: Learning / Problem-Solving Styles
Note:
R = many reports; r = few reports; M = many moves; m = few moves;
E = many experiments; e = few experiments.
Example: “-Me” means normal reporting, many moves,
below normal
experiments
(where normal is within one-half standard deviation from the mean).
We can only speculate, at this point, what these data
mean. We would seem to be seeing a great
deal of apparent variability in student style which could point to basic
differences, or which might simply be a function of pre-conceived notions on
the part of the students as to how interactive software games usually
work. That is, a certain number of
students might be tempted to “game” the system -- i.e. devote their energies
toward learning to “beat the game” rather than learning the geological content
the game is meant to convey.
Gaming the system in more-or-less
constant concern in efforts of this type and a problem that we, as designers,
must be constantly vigilant against.
However, it must be recognized these issues are not strictly confined to
computing media. Students are resourceful,
and there is a long history of anecdotal accounts of students, for example,
“cracking the code” of lab samples in order to pass a test. Anyone who has taught laboratory sections
for any length of time has similar stories on this theme.
We can make only preliminary claims, at this stage, to a definitive classification of student problem-solving categories. For example, above normal “M” in Table 2, indicating movement and exploration more than 1/2 standard deviation above the mean, might not directly indicate the “pattern matching” strategy mentioned above, but it could simply indicate a high degree of curiosity in some students. Alternatively, it could mean (modesty forbids our promoting this alternative), that our virtual environment is so exciting and filled with wonders that students are exceedingly engaged and feel compelled to see everything they can (19 students were coded “M,” we note, over 23% of the total).
By the same token, 22 students
were “excessive,” if that's the right word, in terms of code “E,” for
experimentation in Table 2. This is hard
to criticize on any level, as experimentation is, in one sense, what we hope to
teach. Perhaps these students were
repeating experiments because our logging procedures were too inaccessible and
they re-did experiments rather than refer to their logbooks. If so, this is a failing of the software that
we must try to address.
Lastly, we find that 17 students
made excessive reports. This was the
indicator that led us to this data-mining investigation in the first place: the
intuition that some students (in these data, 21%) were taking a “brute force”
approach to their assigned goals. Is
this supported? We suspect so. But there are many open questions, the primary
of which is how to encourage these students (if we're right in this
supposition), to be more analytical in their approach. One aspect we would like to track in the
future is the effect of our software tutors in terms of steering the gamers,
and the truly overwhelmed, towards more deliberative strategies.
Bruckman, A. (1997). Finding
One's Own in Cyberspace. In C. Haynes & J.R. Holmevik, (Eds.), High
Wired: On the Design, Use, and Theory of Educational MOOs). Ann Arbor:
University of Michigan Press.
Curtis, P. (1997). Not Just a
Game: How LambdaMOO Came to Exist and What It Did to Get Back at Me. in Cynthia
Haynes and Jan Rune Holmevik, Editors: High Wired: On the Design, Use, and
Theory of Educational MOOs. Ann Arbor: University of Michigan Press.
Duffy, T.M., & Jonassen,
D.H. (1992). Constructivism: new implications for instructional technology. In
Duffy and Jonassen (eds.), Constructivism and the Technology of Instruction.
Hillsdale: Lawrence Erlbaum.
Saini-Eidukat, B., Schwert,
D., & Slator, B.M. (1999). Designing, Building, and Assessing a Virtual
World for Science Education. Proceedings of the 14th International
Conference on Computers and Their Applications (CATA-99), April 7-9,
Cancun.
Schwert, D.P., Slator, B.M.,
& Saini-Eidukat, B. (1999). A virtual world for earth science education in
secondary and post-secondary environments: The Geology Explorer.
International Conference on Mathematics/Science Education & Technology
(M/SET-99), March 1-4, San Antonio, TX.
Slator, B.M., Schwert, D.P.,
& Saini-Eidukat, B. (1999). Phased Development of a Multi-Modal Virtual
Educational World. Proceedings of the International Conference on Computers
and Advanced Technology in Education (CATE'99), Cherry Hill, NJ, May 6-8.
Development of the Geology Explorer is funded by the National Science Foundation under grants DUE-9752548, EAR-9809761, and EIA-0086142. We also acknowledge the large team of dedicated undergraduate and graduate students in the computer and earth sciences who have made this project so successful. Special thanks are due to John Bauer for Java graphical client development, to Rebecca Potter for graphical development, to Otto Borchert for simulation development, to Dean Vestal, Ned Kruger, Bryan Bandli, and Jane Willenbring, for geology content development and assessment, to Mark Tinguely, who saved our world when its universe imploded, and to Dave Schmidt for the name: Planet Oit.