Res Sci Educ

DOI 10.1007/s11165-014-9412-3

The Challenge of Evaluating Students’ Scientific Literacy

in a Writing-to-Learn Context

Louisa Tomas & Stephen M. Ritchie

# Springer Science+Business Media Dordrecht 2014

Abstract This paper reports on the challenge of evaluating students’ scientific literacy in a
writing-to-learn context, as illustrated by our experience with an online science-writing project.
In this mixed methods study, year 9 students in a case study class (13–14 year olds, n=26)
authored a series of two ‘hybridised’ short stories that merged scientific and narratives genres
about the socioscientific issue of biosecurity. In seeking to measure the efficacy of the
intervention, we sought evidence of students’ conceptual understanding communicated
through their stories. Finding a suitable instrument presented our first challenge. This led to
the development of scoring matrices to evaluate students’ derived sense of scientific literacy.
Student interviews were also conducted to explore their understanding of concepts related to
the biosecurity context. While the results of these analyses showed significant improvements
in students’ understanding arising from their participation in the writing tasks, the interviews
highlighted a second challenge in evaluating students’ scientific literacy: a disparity between
their written and vocalised understandings. The majority of students expressed a deeper level
of conceptual understanding during the interviews than they did in their stories. The interviews
also revealed alternative conceptions and instances of superficial understanding that were not
expressed in their writing. Aside from the methodological challenge of analysing stories
quantitatively, these findings suggest that in a writing-to-learn context, evaluating students’
scientific literacy can be difficult. An examination of these artefacts in combination with
interviews about students’ written work provided a more comprehensive evaluation of their
developing scientific literacy. The implications of this study for our understanding of the
derived sense of scientific literacy, as well as implications for classroom practice, are
discussed.

Keywords Scientific literacy . Assessment . Writing-to-learn . Narrative . Learning science .

Socioscientific issues

L. Tomas (*)
James Cook University, 1 James Cook Drive, Townsville, QLD 4811, Australia
e-mail: [email protected]

S. M. Ritchie
Murdoch University, 90 South Street, Murdoch, WA 6150, Australia
e-mail: [email protected]
 Res Sci Educ

Introduction

The development of scientific literacy remains a key priority of science education in schools
(Sadler 2004; Tytler 2007). At the same time, researchers continue to call for innovative
curricula and pedagogical approaches that address concerns about students’ negative attitudes
towards school science and declining participation in post-compulsory science studies
(Hackling et al. 2001; Lyons and Quinn 2010; Tytler 2007). As one way of addressing these
issues, we have been engaged in a program of research that has investigated the learning
potential of a writing-to-learn science strategy with the intention of developing students’
scientific literacy and engaging them positively in the learning of science. These research
projects have provided students at schools in different cities in Queensland, Australia, with the
opportunity to write and share ‘hybridised’ narratives (i.e. short stories that integrate scientific
information, which we call ‘BioStories’) about the socioscientific issue of biosecurity. While
detailed analyses of the cognitive and affective outcomes arising from this research have been
reported previously (see Ritchie et al. 2010; Tomas et al. 2011; Tomas 2012; Tomas and Ritchie
2012), in this paper we focus specifically on the challenges that we encountered in seeking to
evaluate students’ scientific literacy, as evidenced by the BioStories written by year 9 students.
In doing so, we present the findings of our evaluation as they pertain to the development of
students’ derived sense of scientific literacy through their participation in the study.

Writing for Scientific Literacy

There are many conceptions of scientific literacy presented within the literature (see Laugksch
2000). As well as the mastery of science content, a number of themes may be identified as
common to most definitions: nurturing and promoting students’ orientation and disposition
towards science; encouraging the ability to understand and apply scientific ideas; and educating
for future citizens, as opposed to disciplinary experts (Millar and Osborne 1998; Tytler 2007).
In his extensive review of scientific literacy and its role in science education, Roberts
(2007) highlights two potentially divergent visions of scientific literacy that have very different
implications for curriculum planning and assessment. Vision I is focused on the importance of
science subject matter, that is, scientific literacy as viewed from a scientists’ perspective.
Vision II acknowledges the ways in which science plays a role in human affairs—a
socioscientific view of scientific literacy. Roberts argues that over-emphasising either vision
I or II in any science curriculum is problematic. Vision I would have students view the world
through the eyes of a scientist, which risks narrowing ‘the student’s experience with the
breadth of science as a human endeavour’ (Roberts 2007, p. 767). At the same time, while
there may be an implicit assumption that teaching students scientific knowledge and methods
of inquiry will result in the socially responsible use of science, or a citizenry that will involve
themselves in scientific discussions and debates, vision II programs may not focus sufficiently
on scientific content (Eisenhart et al. 1996; Roberts 2007).
Notwithstanding the value of Roberts’ (2007) categories of scientific literacy, the role of
communication—specifically, reading, writing and, of course, language—remains fundamen-
tal to science and scientific literacy (Yore et al. 2003). While traditional views of the role of
writing in the classroom privilege the mastery of scientific discourse and the communication of
science knowledge (e.g. Langer and Applebee 1987; Wallace 1996), Glynn and Muth (1994)
assert that ‘… the importance of being able to understand and explain—in clear language—
the meaning of fundamental science concepts is central to scientific literacy’ (p. 1058,
emphasis added). This view aligns with broader notions of scientific literacy that require
Res Sci Educ

students to practise writing about science for non-expert readerships, a view of scientific
literacy that ‘… emphasises the centrality of communication skills and a commitment to
informed and accessible contributions to public debate of the uses of science’ (Hand et al.
1999, p. 1023).
In their exploration of scientific literacy, Norris and Phillips (2003) make a
distinction between a fundamental sense of scientific literacy (reading and writing
science content) and a derived sense (being knowledgeable, learned and educated in
science). A distinction also has been made between a simple sense of fundamental
scientific literacy, which refers to the ability to successfully decode texts, and an
expanded sense, which involves being able to infer meaning from texts (Norris and
Phillips 1994, 2003). They claim that while coming to know science requires com-
petency in both the derived and fundamental senses of scientific literacy, notions of
scientific literacy tend to privilege the former.
There is growing recognition amongst educators that there is value in writing to
learn science, beyond the traditional scientific genres taught in schools (Prain 2006;
see also Tomas 2012). In our program of research, we have explored the use of
BioStories as a diversified writing-to-learn science strategy that centralises the role of
communication through particular consideration of content, context, code and repre-
sentation (Kulgemeyer and Schecker 2013): students write hybridised scientific narra-
tives (representation) that communicate their conceptual understandings (content) re-
lated to the socioscientific issue of biosecurity (context) using clear, everyday lan-
guage and appropriate vocabulary (code). In a series of multi-method studies conduct-
ed with year 6, year 9 and year 12 students, it was reported that writing a sequence
of BioStories enhanced students’ familiarity with biosecurity issues and helped them
to develop a deeper understanding of related biological concepts (Ritchie et al. 2010);
significantly improved their attitudes towards science and science learning (Tomas et
al. 2011) and elicited positive emotional responses in science classes (Tomas and Ritchie
2012). In this paper, we report on the investigation into students’ developing scientific
literacy, and in doing so, explore the challenges that we encountered in this context.

The Challenges of Assessing Broader Notions of Scientific Literacy

Orpwood (2007) asserts that, for the past 35 years, notions of scientific literacy have
grown richer and more profound; however, such notions are under threat due to a lack
of creativity on the part of researchers to develop new approaches to assessing
scientific literacy. He argues that ‘politically high-profile assessments’ such as Trends
in International Mathematics and Science Study (TIMSS) and Programme for Interna-
tional Student Assessment (PISA) encourage ‘teaching to the test’ (p. 2). At the same
time, the development of a scientifically literate citizenry that is able to make
informed personal and social decisions about contemporary socioscientific issues that
‘… go well beyond the science that underpins TIMSS or PISA’ (Fensham 2013, p.
28) requires diverse, authentic and creative assessment techniques that meet the
broadening intentions of science education (Fensham and Rennie 2013).
Roberts (2007) claims that it is difficult to assess fairly notions of scientific literacy
according to vision II, as students’ own experiences and personal contexts are unique. This,
and the fact that vision II does not emphasise formal knowledge structures, is problematic for
international, national and school-based assessments that seek to capture a broader notion
of scientific literacy. The need for accountability by governments and schools often means that
 Res Sci Educ

assessment is driven and constrained by specific indicators and numeric scoring at the expense
of higher-order learning outcomes (Fensham and Bellocchi 2013; Tytler 2007).
A misalignment also exists between the goals of scientific literacy and the way in
which success in science learning has been traditionally defined and assessed at
school. Rather than examining how students use and produce science knowledge to
respond to a need or concern pertinent to their individual or community’s future,
‘success (in school science) takes the form of a predetermined response to a cooked-
up problem, an abstract set of ideals, predicated upon an imposed ideology’ (Roth and
Barton 2004, p. 8).
Unlike many pencil-and-paper tests, the PISA Science project overcame this problem by
adopting real-world contexts involving science and technology in their assessment of scientific
literacy (Fensham 2009, 2013). The PISA Science emphasises process- rather than content-
focused competencies by assessing ‘student practices in situations offered in the lived expe-
riences of everyday people’ (Sadler and Zeidler 2009, p. 911). In doing so, it clearly privileges
a vision II orientation. Although real-world contexts are rarely ‘purely scientific … their
inclusion of non-scientific aspects—social, aesthetic, economic and ethical’ (Fensham 2009,
p. 893) can offer a more balanced approach to the teaching of science while engaging students’
interest. While PISA Science is not a curriculum, it highlights the value of real-world science
and technology contexts in assessing broader notions of scientific literacy.

Research Problem

Implementing a diversified writing-to-learn strategy like BioStories with a view to develop

students’ scientific literacy raises the question of whether this goal has been met and calls for
the employment of appropriate evaluative techniques. In the context of this study, the
following questions arose: To what extent is students’ scientific literacy developed through
the writing of BioStories? and How can students’ scientific literacy be evaluated effectively in
this writing-to-learn science context?
This study explored students’ conceptual science understandings (a derived sense of
scientific literacy), framed within a view of scientific literacy as citizenship preparation that
draws upon visions I and II (Roberts 2007). Through their participation in the BioStories
project, students’ learning was contextualised within a socioscientific issue (cf. vision II);
however, the development of conceptual science understandings (cf. vision I) was also
prioritised. In keeping with this view of scientific literacy as citizen preparation, students
engaged with conceptual science understandings at a level that was appropriate in the context
of everyday conversations about science. That is, they were required to communicate their
understandings about biosecurity in clear language, as suggested by Glynn and Muth (1994)
through their hybridised narratives.

Setting and Procedures for Inquiry

The students in this case study, a single, coeducational year 9 class (13–14 year olds), wrote a
series of two BioStories over a 5-week period. A highly regarded and experienced science
teacher at the school, who was their regular classroom teacher, taught the class.
The writing tasks presented students with two unfinished stories that they were required to
complete (Appendix 1). The genre was ‘hybridised’ as students integrated information about a
Res Sci Educ

socioscientific issue within narrative storylines. The following excerpt from a student’s story
illustrates this merging of genres:

“Well,” Steve continued energetically, “fire ants, also known as Solenopsis invicta, are
native to South America. This nasty ant is an invasive pest in many parts of the world,
including Australia. It can affect our environment and lifestyle, and even our agricultural
production,” he explained.
“How on earth did fire ants enter Australia in the first place?” Jennifer asked.
“Good question, mate. They were discovered in Brisbane in 2001. It’s believed they
arrived here in contaminated soil carried by shipping containers. It reminds us why
quarantine is so important.”
Jennifer was very interested to learn this. “Wow! I didn’t realise quarantine had to watch
for something so small like foreign ants as well! How can they be identified?”
It’s important to keep an eye out for these little fellas. They range from 2-6 mm in length.
They are brown in colour, with their abdomen being slightly darker. “They are very
aggressive and inflict a nasty, burning sting, which can cause an allergic reaction —
hence their name, fire ants.”

Part A introduced students to two key characters, the late Australian environmen-
talist, Steve Irwin, and a young girl by the name of Jennifer, who meet at airport
customs. The story called for students to write about one of five biological incursions
(chytrid fungus, citrus canker, silverleaf whitefly, tilapia and fire ants) that had
affected Australian natural and/or agricultural ecosystems. In doing so, they were
required to write about the environmental, social and economic impacts of their
chosen incursion.
The part B scenario was about an incursion that had not been found in Australia for almost
20 years, avian influenza (or ‘bird flu’). In this case, students wrote about the potential
implications for Australia’s agricultural industry and the wider community if it were to break
through quarantine barriers.
Students wrote their stories with the support of a dedicated website and uploaded
them there so they could be read by their peers online. The website contained all the
necessary resources, including links to online resources about biosecurity, story tem-
plates and task instructions that guided students’ composition of stories.
The BioStories tasks were embedded in an existing 7-week ecology unit within the
year 9 science curriculum that examined concepts of food chains, food webs, adap-
tations and evolution. The writing tasks supplemented the work in the unit; that is, no
explicit teaching of related science concepts was afforded, with the exception of the
term ‘biosecurity’ when the tasks were introduced. The teacher’s role was limited to
introducing the writing tasks and explaining the requirements, assisting students with
their research, and reviewing and editing their written drafts. In this way, learning was
largely student-centred and student-driven. The teacher engaged in discussions about
particular science concepts with individuals or small groups of students when the need
arose, for example, when students asked a question about new information they had
researched.
This study adopted a mixed methods design (Erzberger and Kelle 2003) in which both
qualitative and quantitative data were generated to develop a deeper understanding of the
students’ developing scientific literacy. Given that this study extended our previous qualitative
treatment of students’ conceptual understanding of a writing-to-learn science project (see
 Res Sci Educ

Ritchie et al. 2010), we were interested in developing a data source that facilitated quantitative
analysis of students’ BioStories as a way of measuring the efficacy of the intervention. We
sought to triangulate these data through qualitative analysis of semi-structured interviews that
probed students’ understanding. In the following subsections, we describe these data sources
in more detail.

Exploring the Development of a Quantitative Data Source

Inferences about the development of the students’ scientific literacy were drawn from the
extent to which their written artefacts demonstrated their understanding of concepts related to
the biosecurity context. We wondered how students’ BioStories could be assessed for evidence
of their developing scientific literacy. We were interested in quantifying students’ conceptual
understandings so they could be analysed for any statistically significant changes. We focussed
on concepts related to biosecurity that were clearly communicated in the context of everyday
conversations about science through the characters in the stories. Therein lay our first key
challenge: finding an instrument that quantified students’ written understandings expressed in
their stories.
With the exception of standardised paper-and-pencil tests, a review of the literature revealed
few working examples of quantifying and making judgments about students’ developing
scientific literacy (see Laugksch 2000). In particular, there was a paucity of frameworks or
instruments specifically designed to scrutinise students’ written artefacts in writing-to-learn
science contexts. Initially, Bybee’s (1997) five dimensions of scientific literacy (i.e. scientific
illiteracy, nominal scientific literacy, functional scientific literacy, conceptual scientific literacy
and multi-dimensional scientific literacy) were explored and trialled as a framework for
making judgements about the students’ demonstrated scientific literacy.
The framework was applied to a sample of BioStories written by year 6 students in an
earlier case study (Ritchie et al. 2008). The stories were scrutinised qualitatively for instances
where scientific ideas or related concepts were communicated. Preliminary results found that
all 77 stories analysed generally presented correct scientific descriptions of ideas or concepts,
demonstrating an overall functional level of scientific literacy (Bybee 1997). This indicated
that the framework was not nuanced enough to identify variations in students’ developing
scientific literacy. Importantly, the framework did not enable us to assess how well the students
wrote about issues relating to biosecurity in the context of their stories. With this in mind,
matrices were developed that specifically assess how well students addressed the BioStories
task requirements. As the requirements for parts A and B differed, a separate matrix was
developed for each task (Appendix 2 and Appendix 3).
The preliminary trial analysis using Bybee’s (1997) framework revealed three general
trends in the ways in which the stories incorporated information about biosecurity. Firstly,
some stories did not include any of the required information specified in the task. Secondly, the
majority of the stories attempted to include most of the information required, with some
inaccuracies or alternative conceptions evident. Thirdly, some stories met the task requirements
fully, completely and accurately. For example, part A called for an explanation of how the
chosen biological incursion entered Australia. The students’ BioStories would either (a) not
explain how the incursion entered Australia at all, (b) incorrectly explain how the incursion
entered the country, or (c) correctly explain how the incursion entered Australia. For this
reason, the criteria within the matrices were assigned a score of 0, 1 or 2, depending on
whether the student addressed them and how well they did so (i.e. 0 for no attempt, 1 for an
incomplete or incorrect attempt, and 2 if the criteria were addressed completely and accurately).
It was not deemed necessary to differentiate between incomplete and incorrect responses as we
Res Sci Educ

adopted the position that a student’s demonstrated understanding of a particular concept was no
better or worse if they wrote an incomplete explanation, compared to one that was not fully
correct.
The matrices for parts A and B produce a derived scientific literacy score that reflects the
scope and accuracy of the conceptual information presented in the students’ stories. In order to
facilitate comparisons of students’ derived scientific literacy scores across the two tasks, they
were converted to a percentage of the highest possible score attainable for each part (i.e. a
maximum of 2 marks per criterion). An interpretation of the derived scientific literacy score
is presented in Table 1.
The reliability of the scoring matrices was established by moderating judgements between
two scorers: the first author and a secondary science teacher with 10 years of teaching
experience. Discussions between the scorers resolved slightly different interpretations of the
criteria and what constituted accurate responses to more open-ended criteria. These discussions
led to a refinement of the interpretations until the results from each scorer were in agreement.
These final scores were analysed using dependent-samples t tests in order to identify any
significant changes in students’ performance across parts A and B.

Exploring Students’ Understanding Qualitatively

Twenty-four students consented to participate in interviews conducted 2 to 6 weeks after

completion of the project (note that any students’ names referred to in the findings are
pseudonyms). Classroom observations and reviews of the students’ BioStories were used to
identify questions and issues for further exploration in the interviews.
The first author conducted the interviews with individual students in a computer lab. Each
student was asked to log on to the BioStories site, upon which time they were asked to tell the
researcher about the biological incursion that featured in their particular stories. These
responses were probed for understanding of related concepts, and further clarification was
sought where appropriate. For example, when students referred to some of the impacts of the
incursion in their part A stories, they were asked, ‘So, does it matter if some native lizards die
off as a result of the introduction of fire ants?’ For the part B stories, students were asked about
the impacts of a bird flu outbreak and what should happen if the disease was discovered by a
poultry farmer, in order to identify their understanding of the social and economic impacts, and
how the community should respond to an outbreak of the disease. Audio recordings of student
interviews were transcribed and analysed inductively for evidence of conceptual understanding
of biosecurity and related socioscientific concepts.

Table 1 An interpretation of the derived scientific literacy score as applied to parts A and B of students’
BioStories

Interpretation Derived scientific literacy score

0–3 Story includes little of the required information stipulated in the task, and/or the information that
has been included is incorrect and/or incomplete.
4–6 Story includes most or all of the required information stipulated in the task, some of which is
incorrect and/or incomplete.
7–8 Story includes all of the required information stipulated in the task, which is largely correct and
complete.

The highest possible derived scientific literacy score for parts A and B was 8—a maximum of 2 marks per
criterion in the scoring matrices
 Res Sci Educ

Evaluating Students’ Scientific Literacy Based on their Writing

In this section we discuss the second important challenge that we encountered in the assess-
ment of students’ scientific literacy, situated within our experiences of the BioStories project—
evaluating students’ learning based on their writing. In discussing this challenge, we present
the results of the quantitative analysis of the students’ BioStories and report on the major
findings of the student interviews.
While the students’ BioStories were analysed for instances in which students communicated
conceptual understandings related to the biosecurity context, the interviews provided an
opportunity to explore these understandings further. We initially sought to triangulate the
quantitative analysis of students’ stories with the interview findings; however, a comparison
of these analyses revealed two key observations, one of which presented a second key challenge
in seeking to evaluate their scientific literacy. Our first observation, as we expected, was that
many students recalled and explained the same concepts about which they wrote with compa-
rable detail and accuracy. In most cases, their explanations were accurate; however, a small
number of students recalled the same alternative conceptions evident in their stories. Sec-
ondly, and unexpectedly, the interviews revealed a significant disparity between students’
writing and the understandings that they articulated vocally. Specifically, most students
demonstrated a deeper level of conceptual understanding during the interview than what
had been expressed in their BioStories. These students elaborated on the concepts that they
had written about or, alternatively, introduced and explained new concepts that they had not
previously written about. Conversely, some could only offer superficial explanations of what
they had written or expressed alternative conceptions not evident in their writing.
In keeping with the tenets of mixed methodology, we present the results of the analysis of
students’ derived scientific literacy scores alongside the interview findings to illustrate each of
these observations, in turn. In doing so, we aim to develop a deeper understanding of students’
developing scientific literacy. A discussion of the challenges that we encountered, and their
implications, is presented in the final section.

Evidence of Comparable Understanding

Descriptive statistics for the students’ mean BioStories scores (i.e. a percentage of the highest
possible score attainable for each task) are presented in Table 2. The highest mean scores were
obtained for part B.
Table 3 presents a summary of the number of BioStories that fell within the various ranges
of scores identified earlier in Table 1. For parts A and B, the majority of derived scientific
literacy scores fell in the middle range, which indicates that students attempted to include most
or all of the desired scientific information in their BioStories, with some inaccuracies (Table 3).
There was also a considerable increase in the number of BioStories that fell in the middle
range, from part A to part B. A small number of BioStories fell in the high score range, which
indicates that all of the required information was presented and it was largely accurate.

Table 2 A summary of the descriptive statistics for each of the variables explored via dependent-samples t tests

Variable Mean (%) Number SD

Part A derived scientific literacy 58.65 26 21.44

Part B derived scientific literacy 74.04 26 9.30
Res Sci Educ

Table 3 A summary of the num-

ber of BioStories that fell within Derived scientific literacy score Number of BioStories in this range
each range of derived scientific lit-
eracy scores 0–3 Part A: 9
Part B: 0
4–6 Part A: 13
Part B: 23
7–8 Part A: 4
Part B: 3

Irrespective of the extent to which the students included the desired information in their
writing, 22 of the 24 students interviewed could accurately recall and explain the concepts in
their stories. This suggests that the students did learn science through their participation in the
BioStories project, and this learning had more than a short-term effect as they could recall
concepts 2 to 6 weeks after completing the final writing task.
An excerpt drawn from a part A story written by year 9 student Lizzy is presented in Fig. 1.
Her story is about an invasive fish, tilapia, and includes information about how the animal was
introduced into Queensland’s waterways, its reproductive habits and how it competes with
native fish for food. During the interview, Lizzy accurately recalled some of the concepts in her
story. Through drawing a comparison between tilapia and another invasive species in Australia
(‘Tilapia are like the rabbit of Queensland’s waterways’), she explained that tilapia breed
very quickly and present as a competition for native fish:

Researcher So, you wrote about tilapia in your part A story. Can you tell me a little more about tilapia?
Lizzy Tilapia are like the rabbit of Queensland’s waterways. They breed really quickly and they are not
really fussy eaters, so they basically take over the other fish’s habitat and outnumber them.
Researcher Okay. How do they do that?
Lizzy Because they breed so quickly and faster than all the normal fish … and they eat weed and plants
like that.
Researcher You said they breed really quickly. How does that affect native fish?
Lizzy Because it takes over their habitat and the tilapia eat the resources the other fish use, and the other
fish end up dying off because there are not enough resources for them.

In the context of avian influenza (part B), students could recall information about the
disease itself, and, to different extents, explain how to respond to an outbreak and what the
potential impacts were. For example, in the excerpt from a student’s part B story (see
Fig. 2), Tara writes that bird flu is a ‘highly contagious infection’ originating in wild
birds that can affect a variety of domestic poultry. She also writes that the disease can

“Tilapia invade and take over local waterways. They have a very effective reproductive
strategy which allows them to lay 1200 eggs at a time!” Steve replied enthusiastically.
“Wow, 1200 is a really big number!” Jennifer replied in amazement.
“Yeah, tilapia were brought into the country for the aquarium trade, but some were
accidentally let out in dams and rivers throughout Queensland,” Steve said.
“Oh no, that is terrible!” Jennifer said, sounding upset. “But how did they spread so
quickly?”
“Well Jennifer, tilapia aren’t very fussy with their food and will eat insects and weed, but
they mostly eat plants. They also take over the other fish’s habitat and eat the food the other fish
need to survive. They also breed very quickly,” Steve said.

Fig. 1 An excerpt from a part A story about tilapia written by Lizzy

 Res Sci Educ

be spread via contaminated cages, clothing and animal feed and that humans can
contract the disease.
During the interview, Tara recalled the same information, albeit in lesser detail. However,
the origin of the disease, the organisms it can infect and the ways in which it can be spread are
all mentioned. She also explained that humans could contract the disease by consuming
infected poultry, a concept that did not appear in her story:

Researcher Now you wrote about bird flu in your story as well. What can you tell me about that?
Tara That bird flu starts off in animals that are not domesticated and then it spreads through, and once it
gets to the domesticated animals, it gets worse and if one bird has it, it can, like, if it goes through
cages and another bird comes in contact with that cage, then it can get it and they all die off … It
would affect people as well because if people eat the bird that has bird flu, then they could get it
and pass it onto other humans.

Of the students who recalled the same information as was written in their stories, three
recalled, or confirmed, the same alternative conceptions or errors during the interview. For
example, in Lizzy’s part B story, the key characters discover a chicken exhibiting the
symptoms of avian influenza on their farm. The sick bird was dealt with by removing it from
the coop so as to avoid spreading the disease any further. It ultimately passed away and was
buried in a paddock. During the interview, Lizzy was asked how a farm should respond to bird
flu. She incorrectly described the response to an outbreak by recalling the approach outlined in
her story: ‘They would have to … well, in my part B story, I just said that they took the
affected bird out of the enclosure … Yeah, and check the other birds to make sure they do not
have any symptoms’. Her response demonstrates that she underestimated the seriousness of a
bird flu outbreak. Lizzy did not explain the importance of notifying appropriate authorities that
would enact quarantine measures.

Evidence of Deeper Conceptual Understanding

As shown in Table 4, dependent-samples t tests revealed a significant improvement in

students’ derived scientific literacy scores from part A (M=58.65 %, SD=21.44) to part B
(M=74.04 %, SD=9.30), t(26)=−4.33, p<.01. Effect size, as measured by Cohen’s d, was
0.85, which is indicative of a large standardised difference between the means (i.e. d≥0.8)
(Cohen 1988).

“Avian influenza or more commonly known as bird flu is a highly contagious infection which is
mostly found affecting domestic fowls like chickens, turkey, quail, pheasants, guinea fowl, and
other animals like our pigs!”
“But, that’s terrible, what problems does it cause on a farm like ours and to the wider
community?” Jennifer’s Dad replied anxiously.
“Well Dad, when domesticated birds catch the disease from wild birds the avian influenza
virus becomes extremely severe and kills the domestic fowl but usually not before it has time to
pass the disease on. So Dad potentially if an outbreak occurred on our farm that we weren’t unaware of
or didn’t act fast enough we could lose all of our animals! Now it gets worse. Not only can bird flu
affect domestic fowls but humans can be affected also and the disease can be carried between
humans,” Jennifer stated using her professor’s exact words.
“How are farmers controlling this disease?”
“The disease is very hard to control as it is past [sic] on from wild birds to domestic birds,
the case is then highly severe and is passed to animals through farm contaminated objects such as
cages, clothing and the animals feed and sometimes humans can also contract the disease …”

Fig. 2 An excerpt from a Tara’s part B story that outlines the origin of avian influenza, the organisms it can
infect and how it is spread
Res Sci Educ

Table 4 Significant results of the dependent-samples t test that examined changes in students’ derived scientific
literacy scores across the two BioStories tasks

Variable 1 Variable 2 t df p value d

Part A Part B −4.326 26 0.000a 0.85

Derived scientific literacy Derived scientific literacy
a
Significant at the 0.01 level (two-tailed)

As well as a significant improvement in students’ derived scientific literacy scores from part
A to part B, students’ responses at interview revealed instances in which they expressed a
deeper level of conceptual understanding than was evidenced in their writing by elaborating on
the concepts in their stories or introducing and explaining new concepts.
Fourteen students elaborated on the ecological, social and/or economic impacts of the
biological incursions in their part A BioStories. The following excerpt illustrates an instance in
which Lizzy explains some of the ecological impacts of tilapia by referring to ecological
relationships and food webs. Her part A story, however, did not make reference to such
concepts. Instead, her writing was dominated by descriptive information about tilapia (includ-
ing their habitat, food and reproductive habits, see Fig. 1), which she also recalled during the
interview.

Researcher If native fish die off because of the tilapia, should we be worried about that, is that a problem?
Lizzy Well, because eventually it would kill off some species of our native fish … because we eat fish and
it would just throw the whole cycle off because if one thing dies then other species would be
affected by it.
Researcher Oh yeah, tell me about that. Give me an example of how other species might be affected.
Lizzy Well, if the tilapia overtake all the native fish, they would die out, which will in the long run would
affect us because we eat fish, but it will not affect us that much because we have other resources
like meat and chicken.
Researcher So if the native fish did die out and we did not need them because we do not need to eat them, does
it really matter then?
Lizzy Probably, it still would because it would affect the food web, for animals lower than the fish.

Lizzy’s articulation of ecological relationships in food chains was not surprising as

the science unit in which the BioStories project was embedded had an ecology focus.
However, this topic was taught prior to the commencement of the writing tasks, and
the students’ science teacher made no explicit links between the concept of a food
web and the environmental impacts of the biological incursions in their stories. This
connection was one that Lizzy made herself and demonstrates a deeper understanding
of the impact of tilapia in Australian waterways than the descriptive information that
dominates her story.
Eight students could explain some of the social and economic impacts of an avian
influenza outbreak, while five students correctly explained how an outbreak should be
responded to, despite not writing about these concepts in their part B BioStories. Like
the previous example, most students’ stories included accurate descriptions of avian
influenza, including the organisms it affects and how it is spread (cf. Fig. 2).
However, the wider implications of the disease were not addressed, even though the
task asked students to write about ‘the problems that an outbreak of avian influenza
would cause on a farm and in the wider community (social and economic impacts)’.
 Res Sci Educ

Despite this, when Jackson was explicitly asked during the interview whether there
would be any impacts of an avian influenza outbreak, he explained:
It really impacts on the production of chickens and eggs, and it sends farmers out of their
jobs. They lose money and they go bankrupt, like a lot of companies due to epidemics of
bird flu and the outbreaks that occur. It’s good that it hasn’t happened in Australia yet.

Evidence of Superficial or Erroneous Understanding

Just as the interviews provided students with an opportunity to demonstrate a deeper level of
understanding than was evidenced by their stories, so too did they provide an opportunity to identify
alternative conceptions or superficial understandings omitted from students’ writing. It is important
to note, however, that no student demonstrated a superficial or erroneous level of understanding
overall, given that a significant improvement was found in students’ derived scientific literacy
scores from part A to part B. Instead, our findings of alternative conceptions or superficial
understandings pertained to their understanding of specific concepts. The majority of cases related
to avian influenza, most likely due to the complexities surrounding its biology, the strict quarantine
measures imposed when an outbreak is detected and the far-reaching implications of an outbreak.
Four students introduced alternative conceptions about avian influenza during the interviews.
For example, one student explained that her friend contracted bird flu when an infected bird
touched one of the school’s water bubblers and she drank from it (at the time that this study was
conducted, the last reported case of the disease in Australia was in 1997).
When questioned about how an avian influenza outbreak should be responded to or
managed, nine students (who did not address this aspect in their stories) could not offer an
explanation that referred to the role of quarantine measures. For example, one student
explained, ‘they (the farmers) would just have to kill all the birds’.
Six students offered superficial explanations of the social and economic impacts of an avian
influenza outbreak. For example, when questioned about this during the interview (a concept
that was not addressed in his part B story), one student explained that an outbreak would not
matter if it did not occur where he lived, as he would not be able to contract it. His response did
not demonstrate an understanding of the wider social and economic implications of the disease.
When explaining the impact of the biological incursions that featured in their part A stories,
four students intuitively identified the loss of native species as being unfavourable but could
not offer a scientific explanation. For example, when asked whether it was a problem that
tilapia were causing native fish to die off, one student explained that ‘it is important to have
fish that are native to Australia … it is nice to have Australian fish in Australian waters’.

Negotiating the Challenges: Discussion and Conclusions

We encountered two key challenges in seeking to evaluate the scientific literacy of year 9
students in a writing-to-learn science project. Firstly, after unsuccessfully trialling Bybee’s
(1997) five dimensions of scientific literacy as a framework for assessing students’ demon-
strated scientific literacy, a number of scoring matrices were designed to quantify written
conceptual understandings across the BioStories writing tasks. Unlike the other assessment
instruments, the matrices assessed conceptual understandings related to biosecurity at a level
that was appropriate in everyday conversations about science. In doing so, they aligned with
the notion of scientific literacy adopted in the BioStories project; one of citizen preparation that
draws upon both visions I and II (Roberts 2007).
Res Sci Educ

Secondly, evaluating students’ scientific literacy presented another important challenge.

Quantitative analysis of the BioStories revealed a significant improvement in students’ derived
scientific literacy scores from part A to part B. This indicates that they demonstrated greater
conceptual understanding in their part B BioStories, compared to part A, which suggests that
students’ participation in the writing tasks had a significant positive impact on their learning.
During the interview, however, students articulated some very different understandings that
were not evidenced in their stories. While every student was able to recall or explain some of
the concepts about which they wrote, some offered superficial or erroneous explanations of
particular concepts, or, conversely, demonstrated a deeper understanding by elaborating on the
concepts that they wrote about or by introducing and explaining entirely new ones. Most
significantly, 19 of the 24 students within the case study class demonstrated deeper levels of
conceptual understanding during the interview by correctly explaining some of the environ-
mental, social and economic impacts of the biological incursions that featured in their
BioStories; concepts that were not present in their stories or only superficially addressed.
These rich insights into the students’ developing scientific literacy could not have been
gleaned from analysis of their written artefacts alone.
These findings have important implications for the evaluation of scientific literacy in
writing-to-learn contexts. The difference between students’ written and vocalised understand-
ings might raise questions about the validity of analysing hybridised scientific narratives for
evidence of their scientific literacy. As shown by our findings, instances where students either
accurately explained new concepts or expressed superficial or erroneous understandings all
pertained to concepts that were called for in the task requirements, but, for one reason or
another, were not addressed in their stories. For example, while the part A and B writing tasks
called for an explanation of the environmental, social and/or economic impacts of the relevant
biological incursions, students like Lizzy and Jackson were able to explain some of these
impacts when directly asked, even though such explanations were absent from their stories. It
could be that the somewhat open nature of the writing tasks, where students composed their
own story, did not encourage them to be systematic about ensuring that all of the required
information was included. This issue could be addressed by engaging students in a reflective
review of their work through formative peer- or self-assessment. Such approaches can
encourage students ‘… to apply criteria to help them understand how their work might be
improved and to help deepen their understanding of the criteria by relating them to their own
specific examples’ (Black 2013, p. 214). This will help students to consider more carefully
whether they have addressed all aspects of the task through explicit engagement with the
criteria, so that their writing more accurately reflects their level of understanding.
The disparity between students’ written and vocalised understandings also reminds us of
the importance of employing multiple assessment strategies in order to gain a fuller picture of
students’ developing scientific literacy. Any single style of assessment, including those that
produce numerical scores, cannot offer a satisfactory representation of a student’s learning and
achievement (Champagne and Newell 1992; Fensham and Rennie 2013). An alternative is to
employ multiple probes and alternative assessment instruments, including qualitative assess-
ment tools such as interviews (Southerland et al. 2005; White and Gunstone 1992). As shown
by the findings in this study, interviews are useful tools for revealing alternative conceptions,
and can also provide students with opportunities to talk through and resolve opposing
conceptions (White and Gunstone 1992). In a classroom situation, structured interviews may
provide students with a useful forum through which to vocalise their science understandings in
a way that cannot be fully realised through writing (Southerland et al. 2005). From a research
perspective, mixed methods designs that complement quantitative analysis of written artefacts
with interviews about these artefacts provide a more comprehensive evaluation of students’
 Res Sci Educ

scientific literacy than either approach would individually. In a classroom situation, Fensham
and Rennie (2013) suggest that using several modes of authentic assessment provides a valid
way of capturing a broad range of science learning outcomes and is necessary to build a profile
of students’ achievement (see also Mintzes et al. 2005).
Evaluating students’ scientific literacy in a writing-to-learn context also led us to consider
the implications of this study for our understanding of the derived sense of scientific literacy.
Sadler (2007) presents two competing perspectives of the derived sense of scientific literacy:
cognitive and sociocultural perspectives. The cognitive perspective prioritises the development
of cognitive attributes through science education, such as conceptual understandings and
scientific processes. These attributes may be transmitted or constructed. In this context, he
argues, ‘the role and significance of language are minimised’ (Sadler 2007, p. 86). A cognitive
perspective therefore encourages the development of a simple fundamental sense of scientific
literacy, as language is simply a medium through which knowledge can be communicated. In
contrast to this position, a sociocultural perspective of the derived sense of scientific literacy
prioritises context, enculturation and practice (Sadler 2007). In other words, engaging students
in the practices of the scientific community is crucial, as the goal here is not the development
of cognitive attributes, but rather becoming members of the scientific community. From this
perspective, the role of language as scientific practice is consistent with an expanded funda-
mental sense of scientific literacy, as science is negotiated through written and spoken
language in a social context. Sadler argues that in this way, the boundaries between the
fundamental and derived senses of scientific literacy become blurred, as the ability to infer
meaning from written and spoken language and being knowledgeable in science are closely
intertwined.
Similarly, in the context of this study, it can be said that students’ expanded funda-
mental sense of scientific literacy also developed through their participation in the project,
as the students successfully read and interpreted scientific information from expository-
style websites and transformed and communicated it in their hybridised narratives. In this
way, the boundaries between the expanded fundamental and derived senses of scientific
literacy became blurred, as suggested by Sadler (2007), as it was through this process of
interpretation and transformation that the students developed conceptual understandings of
biosecurity. Thus, while important cognitive attributes were developed, in keeping with a
cognitive perspective of the derived sense of scientific literacy, language played a central
role in this context, as it was more than simply a medium for communicating knowledge;
it acted as a resource for meaning making, as students constructed and communicated
their science understandings in a new genre in the context of their BioStories. This
transformation contributed to their expanded fundamental sense of scientific literacy. This
finding therefore supports a hybridised perspective of the derived sense of scientific
literacy proposed by Sadler (2007). The BioStories project offers a way of developing
students’ cognitive attributes; however, written language is a medium through which
science is not only communicated but also negotiated and understood. The hybridised
writing tasks centralise the role of written language in a socioscientific context.
While the results of this study support the inclusion of diversified writing tasks such as
BioStories in science curricula as a way of developing broader notions of scientific literacy, at
the same time, they remind us of the methodological challenges of scrutinising students’ written
artefacts for evidence of their learning. Aside from the challenge associated with analysing
Res Sci Educ

students’ writing quantitatively in a way that aligned with our research context (i.e. the
expression of students’ derived scientific literacy in hybridised stories about a socioscientific
issue), the findings revealed that students do not always write what they know; evaluating their
writing alone would have provided a limited insight into the depth of their understanding of
biosecurity and related concepts, and thus the development of their scientific literacy. This
finding in itself is not remarkable, but what it does is highlight the importance of employing
multiple probes for research purposes and encouraging students to communicate their under-
standings more fully by engaging them explicitly with the task criteria. Employing formative
peer- or self-assessment strategies can strengthen the validity of evaluating students’ stories for
evidence of learning in writing-to-learn science contexts like this. Further studies that explore
ways of analysing students’ written artefacts quantitatively are also needed, particularly larger-
scale studies that call for more effective quantitative measures of scientific literacy.

Appendices

Appendix 1

An extract from the part A writing task

Part A: Crikey!

Since Steve Irwin’s fatal encounter with a stingray in 2006, September 4 is usually a sad day for Jennifer. On this
particular spring day, strolling between biology lectures at uni, Jennifer fondly remembered her first meeting
with the legendary environmentalist, affectionately known around the world as the Crocodile hunter …
Suddenly, there was a commotion at one of the checkpoints. A customs officer was trying to persuade a reluctant
passenger to part with some prohibited plants he had brought with him from the USA.
‘You know’, Steve started as he watched the passenger try to argue his way out of trouble, ‘biosecurity and
quarantine are so important to our country. We know how devastating it has been for our vulnerable
ecosystems when (species X, e.g. tilapia) got into the country somehow; it ruined (native ecosystem or
agricultural industry, e.g. local waterways)’, he explained.
‘How on Earth could something like that have such a terrible impact?’ Jennifer asked.
‘Well’, Steve continued energetically …
Your task: Write 200–250 words in order to complete the story. Your teacher will allocate you one of the
following scenarios, from which to insert into the storyline above. Be sure to research your biological
incursion by exploring the associated websites and reading the scientific information, before completing part
A. Your story must be informative and include scientific information. Remember, using the biological
incursion allocated to you, Steve is trying to help Jennifer understand the importance of quarantine. In the
conversation that you complete between the characters, aim to address the following information:
•State what the biological incursion is.
•Its country of origin.
•How it entered Australia.
•The problems it caused or continues to cause for native ecosystems or agricultural industries (i.e. its
environmental, social and economic impacts).
•The difficulties scientists and farmers face controlling the pest or how the pest was brought under control.
 Res Sci Educ

Appendix 2

Derived scientific literacy scoring matrix: part A

Criterion 1. The country of origin of the biological incursion.

0 The story does not include the biological incursion’s country of origin.
1 The country of origin is incorrect.
2 The story includes the biological incursion’s correct country of origin.
Criterion 2. How the biological incursion entered Australia.
0 The story does not explain how the biological incursion entered Australia.
1 The story incorrectly explains how the biological incursion entered Australia.
2 The story correctly explains how the biological incursion entered Australia.
Criterion 3. The problems the biological incursion has caused or continues to cause native and/or commercial
species or ecosystems (environmental, social and economic impacts).
0 The story does not address any environmental, social or economic impacts of the biological incursion.
1 The story incorrectly or incompletely addresses reasonable environmental, social and economic impacts
that pertain to the biological incursion.
2 The story accurately addresses reasonable environmental, social and economic impacts that pertain to the
biological incursion.
Criterion 4. The difficulties scientists and farmers face controlling the pest or how the pest was brought under
control.
0 The story does not explain any difficulties faced by scientists and/or farmers in controlling the biological
incursion or how the pest was brought under control.
1 The story incorrectly or incompletely explains the difficulties faced by scientists and/or farmers in
controlling the biological incursion or how the pest was brought under control.
2 The story accurately explains the difficulties faced by scientists and/or farmers in controlling the biological
incursion or how the pest was brought under control.
Total score: ……… out of 8

Appendix 3

Derived scientific literacy scoring matrix: part B

Criterion 1. What is avian influenza?

0 The story does not identify what avian influenza is.
1 The story incorrectly identifies what avian influenza is.
2 The story correctly identifies avian influenza as a contagious or viral disease/infection.
Criterion 2. The organisms affected by avian influenza or those at risk of infection.
0 The story does not state the organisms affected by avian influenza or those at risk of infection.
Res Sci Educ

Derived scientific literacy scoring matrix: part B

1 The story incorrectly and/or incompletely states the organisms affected by avian influenza or those at risk of
infection.
2 The story accurately states the organisms affected by avian influenza or those at risk of infection.
Criterion 3. The problems that an outbreak of avian influenza would cause on a farm and in the wider community
(social and economic impacts).
0 The story does not address any social or economic impacts of avian influenza.
1 The story incorrectly or incompletely addresses reasonable social and economic impacts that pertain to
avian influenza.
2 The story accurately addresses reasonable social and economic impacts that pertain to avian influenza.
Criterion 4. The difficulties scientists and farmers face controlling avian influenza.
0 The story does not explain any difficulties faced by scientists and/or farmers in controlling avian influenza.
1 The story incorrectly or incompletely explains the difficulties faced by scientists and/or farmers in
controlling avian influenza.
2 The story accurately explains the difficulties faced by scientists and/or farmers in controlling avian
influenza.
Total score: ……… out of 8

References

Black, P. (2013). Pedagogy in theory and in practice: formative and summative assessments in classrooms and in
systems. In D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing assessment in science education:
pedagogy, curriculum, policy (pp. 207–229). Netherlands: Springer.
Bybee, R. W. (1997). Achieving scientific literacy: from purposes to practices. Portsmouth: Heinemann.
Champagne, A. B., & Newell, S. T. (1992). Directions for research and development: alternative methods of
assessing scientific literacy. Journal of Research in Science Teaching, 29, 841–860.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahwah: Lawrence Erlbaum.
Eisenhart, M., Finkel, E., & Marion, S. F. (1996). Creating the conditions for scientific literacy: a re-examination.
American Educational Research Journal, 33, 261–295.
Erzberger, C., & Kelle, U. (2003). Making inferences in mixed methods: the rules of integration. In A.
Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 457–
488). Thousand Oaks: Sage Publications, Inc.
Fensham, P. J. (2009). Real world contexts in PISA Science: implications for context-based education. Journal of
Research in Science Teaching, 46, 884–896.
Fensham, P. J. (2013). International assessments of science learning: their positive and negative contributions to
science education. In D. Corrigan, R. Gunstone, & A. Jones (Eds.), Valuing assessment in science education:
pedagogy, Curriculum, Policy (pp. 11–31). Netherlands: Springer.
Fensham, P. J., & Bellocchi, A. (2013). Higher order thinking in chemistry curriculum and its assessment.
Thinking Skills and Creativity, 10, 250–264.
Fensham, P. J., & Rennie, L. J. (2013). Towards an authentically assessed science curriculum. In D. Corrigan, R.
Gunstone, & A. Jones (Eds.), Valuing assessment in science education: pedagogy, Curriculum, Policy (pp.
69–100). Netherlands: Springer.
Glynn, S. M., & Muth, K. D. (1994). Reading and writing to learn science: achieving scientific literacy. Journal
of Research in Science Teaching, 31, 1057–1073.
Hackling, M. W., Goodrum, D., & Rennie, L. J. (2001). The state of science in Australian secondary schools.
Australian Science Teachers Journal, 47(4), 6–17.
Hand, B., Prain, V., Lawrence, C., & Yore, L. (1999). A writing science framework designed to enhance science
literacy. International Journal of Science Education, 21, 1021–1035.
 Res Sci Educ

Kulgemeyer, C., & Schecker, H. (2013). Students explaining science—assessment of science communication
competence. Research in Science Education, 43(6), 2235–2256.
Langer, J. A., & Applebee, A. N. (1987). How writing shapes thinking: a study of teaching and learning.
Research report no. 22. Urbana: National Council of Teachers of English.
Laugksch, R. C. (2000). Scientific literacy: a conceptual overview. Science Education, 84, 71–94.
Lyons, T., & Quinn, F. (2010). Choosing science: understanding the declines in senior high school science
enrolments. Research report to the Australian Science Teachers Association. Retrieved from http://www.une.
edu.au/simerr/pages/projects/131choosingscience.php
Millar, R., & Osborne, J. (1998). Beyond 2000: science education for the future. London: King’s College
London, School of Education.
Mintzes, J. J., Wandersee, J. H., & Novack, J. D. (Eds.). (2005). Assessing science understanding: a human
constructivist view. Burlington: Elsevier.
Norris, S. P., & Phillips, L. M. (1994). The relevance of a reader’s knowledge within a perspectival view of
reading. Journal of Reading Behaviour, 26, 391–412.
Norris, S., & Phillips, L. (2003). How literacy in its fundamental sense is central to scientific literacy. Science
Education, 87, 224–240.
Orpwood, G. (2007). Assessing scientific literacy: threats and opportunities. Paper presented at the Linnaeus
Tercentenary 2007 Symposium, Uppsala University, Sweden.
Prain, V. (2006). Learning from writing in secondary science: some theoretical and practical implications.
International Journal of Science Education, 28, 179–201.
Roberts, D. A. (2007). Scientific literacy/science literacy. In S. K. Abell & N. G. Lederman (Eds.), Handbook of
research on science education (pp. 729–780). Mahwa: Lawrence Erlbaum Associates.
Ritchie, S. M., Rigano, D. L., & Duane, A. (2008). Writing an ecological mystery in class: Merging genres and
learning science. International Journal of Science Education, 30(2), 143–166.
Ritchie, S. M., Tomas, L., & Tones, M. (2010). Writing stories to enhance scientific literacy. International
Journal of Science Education, 33(5), 685–707.
Roth, W.-M., & Barton, A. (2004). Rethinking scientific literacy. New York: Routledge Falmer.
Sadler, T. D. (2004). Moral and ethical dimensions of socioscientific decision-making as integral components of
scientific literacy. Science Educator, 13, 39–48.
Sadler, T. D. (2007). The aims of science education: unifying the fundamental and derived senses of scientific
literacy. In C. Linder, L. Östman, & P. Wickman (Eds.), Promoting scientific literacy: science education
research in transaction. Uppsala: Proceedings of the Linnaeus Tercentenary Symposium at Uppsala
University.
Sadler, T. D., & Zeidler, D. L. (2009). Scientific literacy, PISA, and socioscientific discourse: assessment for
progressive aims of science education. Journal of Research in Science Teaching, 46, 909–921.
Southerland, S. A., Smith, M. U., & Cummins, C. L. (2005). “What do you mean by that?”: using structured
interviews to assess science understanding. In J. J. Mintzes, J. H. Wandersee, & J. D. Novack (Eds.),
Assessing science understanding: a human constructivist view (pp. 72–95). Burlington: Elsevier.
Tomas, L. (2012). Writing narratives about socioscientific issues: Engaging students and learning science.
Teaching Science, 58(4), 24-28.
Tomas, L., & Ritchie, S. M. (2012). Positive emotional responses to hybridised writing about a socio-scientific
issue. Research in Science Education, 42(1), 25–49.
Tomas, L., Ritchie, S. M., & Tones, M. (2011). Attitudinal impact of hybridised writing about a socioscientific
issue. Journal of Research in Science Teaching, 48(8), 878–900.
Tytler, R. (2007). Re-imagining science education: engaging students in science for Australia’s future. Australian
Council for Educational Research. Retrieved 20 July, 2007 from: http://www.acer.edu.au/documents/
AER51_ReimaginingSciEdu.pdf
Wallace, G. (1996). Engaging with learning. In J. Rudduck (Ed.), School improvement: what can pupils tell us?
(pp. 56–69). London: David Fulton.
White, R., & Gunstone, R. (1992). Probing understanding. London: Falmer.
Yore, L. D., Bisanz, G. L., & Hand, B. M. (2003). Examining the literacy component of science literacy: 25 years
of language arts and science research. International Journal of Science Education, 25, 689–25.