Teaching and Teacher Education 45 (2015) 83e93

Contents lists available at ScienceDirect

Teaching and Teacher Education

journal homepage: www.elsevier.com/locate/tate

Studying teacher noticing: Examining the relationship among

pre-service science teachers' ability to attend, analyze and respond
to student thinking*
Tara Barnhart*, Elizabeth van Es 1
School of Education, University of California, Irvine, 3200 Education, Irvine, CA 92697-5500, USA

h i g h l i g h t s

We investigated two cohorts of teacher candidates in a preparation program.

One cohort was enrolled in a course to develop reflective skills using video.

Course participants attended to and analyzed student ideas differently.

Sophisticated analyses and responses required high sophistication in attending.

Significant relationships existed between attending and analyzing.

a r t i c l e i n f o a b s t r a c t

Article history: This study investigates pre-service teachers' capacities to attend to, analyze, and respond to student
Received 12 February 2014 thinking. Using a performance assessment of teacher competence, we compare two cohorts of science
Received in revised form teacher candidates, one that participated in a video-based course designed to develop these skills and
19 September 2014
one that did not. Course participants demonstrate more sophisticated levels of attention to and analysis
Accepted 22 September 2014
Available online 11 October 2014
of student ideas. Analysis of the relationship among skills reveals that sophisticated analyses and re-
sponses to student ideas require high sophistication in attending to student ideas. However, high so-
phistication in attending to student ideas does not guarantee more sophisticated analyses or responses.
Keywords:
Reflective teaching
© 2014 Elsevier Ltd. All rights reserved.
Student teaching
Student teachers
Secondary school science
Teacher noticing
Pre-service teacher education

1. Introduction Braaten, 2011). Teachers who have opportunities to rigorously

reflect on their work and connect it to research and theory during
Few factors have a greater impact on learning than the quality of their professional preparation are better able to identify and
a student's teacher (Darling-Hammond, 2010; Strauss & Sawyer, respond to dilemmas of practice, more likely to take an analytic
1986; Whitehurst, 2002). While there is disagreement about stance toward their work, and demonstrate a willingness to take
what specific skills or dispositions make one teacher more effective risks and explore alternative pedagogical approaches (Darling-
than another (Goldhaber & Anthony, 2004), there is consensus Hammond & Bransford, 2005; Davis, Petish, & Smithey, 2006;
around the importance of teachers being able to critically analyze Zeichner & Liston, 1996). Moreover, research on teacher expertise
their practice (Little & Horn, 2007; Windschitl, Thompson, & shows that expert teachers can distinguish between important and
unimportant information in a complex situation, can reason about
what they observe and can use this analysis to make more informed
*
This research was supported in part by the Knowles Science Teaching Foun- teaching decisions (Berliner, 2001).
dation. The opinions expressed are those of the authors and do not necessarily Despite its value, building reflective and analytic skills can be
reflect the opinions of the supporting agency.
challenging, particularly in the context of pre-service teacher ed-
* Corresponding author. Tel.: þ1 562 708 9703.
E-mail addresses: [email protected] (T. Barnhart), [email protected] (E. van Es). ucation. The fieldwork sites may not promote systematic and
1
Tel.: þ1 949 824 7819. rigorous analysis; the preconceptions pre-service teachers bring

http://dx.doi.org/10.1016/j.tate.2014.09.005
0742-051X/© 2014 Elsevier Ltd. All rights reserved.
84 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93

into the profession can interfere with what they choose to reflect on teacher credentialing. To be clear, we do not focus on noticing and
and how they reason about the effectiveness of their teaching; and analysis during instruction. Though research points to the value of
pre-service teachers may lack the observation skills and pedagog- preparing beginning teachers to learn to notice and respond during
ical content knowledge required for sophisticated analyses of instruction (Kazemi, Lampert, & Franke, 2009; Windschitl,
teaching and learning (Borko & Livingston, 1989; Hammerness, Thompson, Braaten, & Stroupe, 2012), we focus on their written
Darling-Hammond, & Bransford, 2005; Hiebert, Morris, Berk, & analyses of teaching because research also suggests that teachers
Jansen, 2007; Schoenfeld, 2011; Star & Strickland, 2008). who systematically analyze teaching become more adept at
A further complication is that simply adding requirements to responding to student ideas (Windschitl et al., 2011). In addition,
teacher education programs to analyze practice, be it through we conjectured that the written responses would provide us with
coursework assignments or high-stakes portfolio assessments, access to pre-service teachers' ways of attending, analyzing and
without providing guidance in what should be analyzed, for what responding, not all of which may be possible to observe if we were
purpose, and how, results in superficial learning and may even be to analyze videos of instruction.
mis-educative (Dewey, 1933; Loughran, 2002; Zeichner & Liston, The questions we investigate are particularly relevant in sci-
1996). This is because, without structured support and appro- ence education internationally. In the US, proposals for the
priate framing, pre-service teachers' analyses tend to focus on the improvement of science teaching and learning emphasize teach-
actions and behaviors of the teacher rather than student thinking, ing students how to collect, interpret, and evaluate evidence to
learning and sense-making (Hammer, 2000; Levin, Hammer, & formulate scientific explanations (American Association for the
Coffey, 2009), and tend to be judgmental and lack evidential sup- Advancement of Science [AAAS], 2009; National Research
port and coherence (Davis, 2006; Sandoval, Deneroff, & Franke, Council [NRC], 2007, 2012; Sandoval et al., 2002; Windschitl,
2002). Thompson, & Braaten, 2008). This is also the case in other coun-
The purpose of this study is to investigate how a video-based tries where science curricula emphasize the importance of
course, Learning to Learn from Teaching (Santagata & van Es, developing scientifically literate citizens and is reflected in the
2010), supported secondary science pre-service teachers in highest proficiency levels on the Program for International Stu-
learning to analyze and reflect on teaching and learning in sys- dent Assessment [PISA] (OECD, 2013; Waddington, Nentwig, &
tematic ways. A central component of the course was learning to Schanze, 2007). Because the scientific body of knowledge is in
use evidence of student thinking as it unfolds in the lesson to draw constant flux with increasing amounts of information being added
inferences about the effectiveness of instruction and using this or modified (Gleick, 2011), students must be able to ask critical
analysis to make subsequent pedagogical decisions. This type of questions and possess appropriate skepticism about proposed
analysis requires that pre-service teachers: a) attend to student explanations and interpretations of scientific phenomena. More-
thinking and learning and the interactions that unfold among over, they need to successfully navigate the flood of information to
students and between teachers and students, b) interpret student participate knowledgeably in public discussions about science and
understanding from these interactions, and c) decide next steps technology and be sensible consumers of information about sci-
based on this analysis. Recent research refers to this collection of ence and related issues (NRC, 2012). Thus, learning science is not
skills as teacher noticing (Jacobs, Lamb, & Philipp, 2010). Of only about knowing science content, but also involves attending to
particular interest in this study is the relationship among these and reasoning about scientific ideas, generating and testing
skills. To date, several researchers have investigated these skills in models of scientific phenomena, and being effective problem
isolation, with some researchers focusing on pre-service teachers' solvers (AAAS, 2009; Levin et al., 2009; Ministry of Education-
abilities to articulate clear learning goals (Jansen, Bartell, & Berk, Singapore, 2013; NRC 2012). To achieve this vision of science ed-
2009; Morris, Hiebert, & Spitzer, 2009), others on their ability to ucation requires that pre-service teachers develop strategies for
attend to student thinking and learning in a lesson (Levin et al., systematically analyzing their ability to build students' scientific
2009; Nicol & Crespo, 2004; Ruiz-Primo & Furtak, 2007; Star & reasoning skills and for assessing students' progress in achieving
Strickland, 2008; van Es & Sherin, 2002; van Zee & Minstrell, these goals.
1997), and others on the use of evidence to support claims of
teaching effectiveness (Morris, 2006; Santagata & Yeh, 2013; 2. Theoretical framework
Stockero, 2008). Less attention has been paid to how develop-
ment of one skill influences development of the others and how 2.1. The importance of reflection and analysis for learning to teach
they coordinate to construct a coherent analysis of teaching prac-
tice. An empirical investigation into this relationship will advance This study is framed by research on reflection, teacher noticing
research on the constructs of noticing and analysis of teaching and and lesson analysis. Dewey (1933) and Scho € n (1983) each made
has implications for the design of teacher education. contributions to the field by defining the phases and mind-set
While research suggests that pre-service teachers can develop needed for critical reflection. Each believed reflective practi-
analytic skills in the context of a course where they analyze their tioners should be engaged in ongoing, systematic, rigorous, and
own and other's teaching (Pang, 2011; Santagata & Angelici, 2010), disciplined meaning-making with the aim of improving practice.
few studies have examined whether they draw on these skills when They identified effective reflection as an analytic approach to
they analyze their own teaching after the conclusion of the course. problem-solving distinct from informal ways of thinking about
Thus, the central research questions for this study include: a) Do teaching and instruction in which teachers direct their attention to
pre-service teachers who participated in a course designed to particular details of practice, make sense or give reason to these
scaffold systematic analysis of teaching through video analysis details, and use their analysis of these details to develop hypotheses
draw on the skills to analyze their own teaching compared to a about how to solve dilemmas of classroom practice. In this way,
cohort of teachers who did not participate in the course? and b) teachers become skilled professionals rather than mindless tech-
How are the skills of systematic analysis of teaching related to each nicians who consume and implement instruction designed by
other? To investigate these questions, we compare written analyses others.
of teaching of a cohort of secondary science teacher candidates that An important characteristic of these reflective models is the use
participated in the Learning to Learn from Teaching (LLfT) course to of evidence from teaching to inform practical theories to test in
one that did not in the context of a performance assessment for practice. In Dewey's (1933) perspective, all rigorous reflection is
 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93 85

grounded in experience. When an experience is perceived as a relationship between teaching and learning (Star et al., 2011; van Es
basis upon which to build future decisions, it becomes evidence. & Sherin, 2002). Other researchers emphasize the importance of
Similarly, Scho€ n (1983) writes that professionals bring past using analysis to make evidence-based decisions on how to
experience to bear on new situations and problems, allowing them respond to students' thinking, noting that it can improve with
to use experience as evidence to form new theories for future experience and practice (Hiebert et al., 2007; Jacobs et al., 2010;
action. Santagata & Angelici, 2010; Stürmer et al., 2012).
However, research on teacher noticing (Erickson, 2011) suggests
that what teachers attend to and use as evidence from their ex-
periences may not provide them with the kind of information they 2.3. Relationships among skills for analyzing teaching
need to draw meaningful inferences about student learning. This is
particularly true of inexperienced teachers. Novice teachers typi- Studies of teacher noticing, lesson analysis, and reflection often
cally focus on superficial features of classroom interactions or use conceptualize a connection among the elements on attending,
student behaviors, such as raising hands enthusiastically, staying analyzing, and responding (Franke & Kazemi, 2001; Levin et al.,
on-task, and following classroom routines in an orderly fashion to 2009; Santagata, Zannoni, & Stigler, 2007; Star & Strickland,
deduce that students understood the lesson (Carter, Cushing, 2008; van Es & Sherin, 2002, 2008). For example, Davis (2006)
Sabers, Stein, & Berliner, 1988; Copeland, Birmingham, DeMeulle, examined what aspects of teaching pre-service teachers consider,
D'Emidio-Caston, & Natal, 1994; So, 2012; Star & Strickland, emphasize, and integrate when they productively reflect on their
2008; Star, Lynch, & Perova, 2011). Erickson (2011) also found science teaching. A more integrated reflection was one that made
that novice teachers focused on the learning of the whole class, connections between features of instruction that pre-service
rather than attending to unique student ideas, making sense of teachers noticed. Research on lesson analysis implies a relation-
those ideas, and understanding how they fit into achieving the ship between attending and making sense of instruction by
broader learning goal. This results in overgeneralizations about examining how pre-service teachers learn to collect and use evi-
student learning, leaving novices with an inaccurate sense of the dence to draw inferences about teachers' instruction and student
effectiveness of the lesson (Lloyd & Mukherjee, 2012; Loughran, learning (Morris, 2006). Jacobs et al. (2010) “envision the existence
2002; van Es & Sherin, 2002; van Manen, 1977). of a nested relationship among the three component skills such
Simply encouraging pre-service teachers to pay attention to that deciding how to respond on the basis of children's un-
details of practice is not sufficient to improve practice. Zeichner derstandings can occur only if teachers interpret children's un-
and Liston (1987), and more recently Levin et al. (2009) contend derstanding, and these interpretations can be made only if teachers
that a key characteristic of effective reflection is how pre-service attend to the details of children's strategies” (p. 197).
teachers frame problems of practice because this influences Though research implies a relationship among the skills for
what details they identify as worthy of attention and how these systematically analyzing teaching, what is less clear is how skills
details will be analyzed and used to construct testable theories are connected and how the level of sophistication on one skill may
about practice. This suggests that pre-service teachers must be or may not relate to the level of sophistication on another. Thus, the
provided with tools and frameworks to help guide what they purpose of this study is twofold. First, we investigate if participa-
attend to in teaching, how they interpret these events, and how tion in a video-based course designed to facilitate systematic
they draw inferences from these experiences to make informed analysis of teaching supported secondary science teachers in
teaching decisions. Providing an organizational frame may permit analyzing their own practice in sophisticated ways as compared to a
more sophisticated attention to salient details and enable the cohort who did not enroll in the course. By using the term sys-
transformation of these noticed details into evidence that can be tematic analysis of teaching, we refer to demonstrating greater
used to inform future instructional decisions (Davis, 2006; Levin attention to the particulars of student thinking, using evidence of
et al., 2009; Santagata & Angelici, 2010; Stürmer, Ko €nings, & classroom interactions to draw inferences about student learning,
Seidel, 2012). and making instructional decisions based on their attention to and
analysis of student thinking. We conjectured that candidates in the
2.2. Learning to systematically analyze teaching course would show higher levels of sophistication on these three
skills. We compare the two cohorts' analyses of teaching from the
Recent research proposes frameworks to support pre-service Performance Assessment for California Teachers (PACT). The PACT
teachers in developing productive approaches for analyzing is one of three state-approved portfolio assessment models pre-
teaching (Gelfuso & Dennis, 2014; Hiebert et al., 2007; Santagata & service teachers must complete in order to earn state licensure.
Angelici, 2010; Windschitl et al., 2012). Consistent across these Because the PACT requires pre-service science candidates to plan,
models is a focus on defining a clear learning goal; using evidence enact, and reflect on an instructional sequence using video, it
from practice, such as student work or video of classroom in- provides a rich data source that captures the skills of interest in this
teractions, to inform analysis; and examining the cause-effect study.
relationship between particular teaching practices and student Second, we examine the relationships among the skills for
learning. Teaching is framed as an experiment with the teacher systematic analysis of teaching across the two cohorts to gain
engaged in a methodical approach to examine and learn from insight into how the skills interact with each other. We hypothe-
practice (Spitzer, Phelps, Beyers, Johnson, & Sieminski, 2011). sized that significant relationships existed between pairs of skills
We propose that to engage in this kind of systematic analysis and that these skills build on each other. Specifically, we conjec-
requires development of a variety of skills, including attending to tured that high sophistication on attending would support higher
what is noteworthy in classroom data, analyzing and interpreting levels of sophistication on analyzing, and higher sophistication on
that data with respect to defined goals, and deciding how to attending would in turn support higher levels of sophistication on
respond, what research refers to as teacher noticing (Jacobs et al., responding. Findings of this analysis will provide greater insight
2010; Star & Strickland, 2008; van Es & Sherin, 2002). While pre- into the construct of teacher noticing, as well as inform pre-service
service teachers tend to focus their attention on classroom man- teacher educators' efforts to design instruction to help pre-service
agement and tasks, research shows that with structured guidance, teachers develop their ability to systematically analyze and reflect
they can become better attuned to and inferential about the on teaching.
86 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93

3. Research design Table 1

Characteristics of LLfT and non-LLft cohorts.

3.1. Study context Non-LLfT LLfT

Gender
This study takes place in a nine-month, three quarter teacher Male 3 4
credential program at a large, public university in the western Female 5 12
United States. It is part of a larger research program that examines Mean Age in years 31.4 (30e37) 30.1 (27e38)
Mean UC G.P.A. 3.30 (2.23e3.98) 3.36 (2.97e3.97)
how pre-service teacher candidates learn to notice and analyze
Undergraduate Major
teaching using video case studies. All candidates complete super- Biology 3 6
vised fieldwork each quarter, primarily observing a classroom Chemistry 4 1
teacher in the first quarter and eventually taking on full re- Psychology 1 3
Ecology/Environmental Science 0 4
sponsibility as a student teacher by the beginning of the third
Physics 0 1
quarter. They also enroll in courses related to content area methods, Earth Science 0 1
general pedagogy, learning theory, language and literacy in the Mean Inquiry Lesson Score 2.35 (1e3.1.25) 2.21 (1e3)
content areas, and equity and diversity. All candidates seeking a
Note. ManneWhitney results for gender U(23) ¼ 56, Z ¼ .46, n.s. For mean age
secondary teaching credential enrolled in the Learning to Learn from U(23) ¼ 46.5, Z ¼ 1.04, n.s. For mean GPA U(23) ¼ 64, Z ¼ .03, n.s. For Inquiry
Teaching (LLfT) course in the first quarter of the program during the Lesson Score U(23) ¼ 39.5, Z ¼ 1.18, n.s. All tests were two-tailed. Range is in
year of this study. The course draws extensively on best practices parentheses.
p < .05.
for using video cases and uses structured frameworks to scaffold
and apprentice teacher candidates into reflective work in an
authentic practice-based context (Hiebert et al., 2007; Rodgers, after the portfolio assessments of the National Board for Profes-
2002; van Es & Sherin, 2002). In particular, the course intends to sional Teaching Standards (NBPTS, 2014), the Interstate New
develop pre-service teachers' skills in three domains: attention to Teacher Assessment and Support Consortium (http://www.ccsso.
student thinking; interpretation of student thinking; and planning org/Resources/Programs/Interstate_Teacher_Assessment_Consortiu
and enactment of strategies to make student thinking visible m_(InTASC).html) Council of Chief State School Officers, 2014, and
(Santagata & van Es, 2010). the Connecticut State Department of Education (Pecheone & Chung,
The course consists of three main phases (Santagata & van Es, 2006), secondary science teacher candidates complete a subject-
2010). The first phase introduces students to theories about ana- specific PACT Teaching Event at the beginning of the third quarter
lytic and responsive teaching practice. The second phase focuses on of study in the credential program. The Teaching Event consists of
providing structured practice using frameworks such as those five tasks centered on developing students' scientific concepts and
proposed by Hiebert et al. (2007) and Rodgers (2002) to analyze scientific inquiry: context for learning, planning, instruction,
student thinking and strategies that teachers in video case studies assessment, and reflection. Candidates submit written lesson plans
employed to promote classroom discourse to make student for a three to five day learning segment involving “critical teaching
thinking visible (Blythe, 1997; Hufferd-Ackles, Fuson, & Sherin, and learning tasks in the credential area,” an assessment tool,
2004; Lemke, 2002; van Zee & Minstrell, 1997). In the final phase, student work samples, a video of their teaching, and a written
students apply these skills to design, video record, and reflect on reflection (Pecheone & Chung, 2006). These materials are submit-
their own teaching practice. ted and scored using a published rubric to determine competency
to earn a teaching credential.
3.2. Participants We focused on the candidates' responses to three questions
related to analyzing their videos of teaching that specifically assess
Participants consisted of two cohorts of secondary pre-service the skills of interest in this study. These include: a) In the instruc-
science teachers: one cohort that participated in the LLfT course tion seen in the clips, how did you further the students' knowledge
(n ¼ 16) and one cohort that did not (n ¼ 8). The number of and skills and engage them intellectually while collecting,
credential program participants doubled between 2007 and 2009 analyzing, and interpreting data from a scientific inquiry? Provide
because of a concerted effort to increase program participation. All examples of both general strategies to address the needs of all of
candidates complete the PACT as part of the credentialing process. your students and strategies to address specific individual needs; b)
When considering age, undergraduate major and grade point av- Describe the strategies you used to monitor student learning during
erages, there were no noteworthy differences between the two the learning task shown on the video clips. Cite one or two ex-
cohorts except for the addition of the LLfT course to the credential amples of what students said and/or did in the video clips or in
program requirements. Using the “Essential Features of Classroom assessments related to the lesson(s) that indicated their progress
Inquiry and Their Variations” framework from the Inquiry and the toward accomplishing the lessons' learning objectives; c) Reflect on
National Science Education Standards: A Guide for Teaching and the learning that resulted from the experiences featured in the
Learning (NRC, 2000, p. 29), we also compared the lessons that were video clips. Explain how, in your subsequent planning and teaching,
the basis for the PACT videos and responses. Because the objects of successes were built upon and missed opportunities were
the analysis likely informed and influenced what participants had addressed. The authors did not participate in any activities related
available to attend to, analyze and respond to, we wanted to ensure to supporting candidates in completing the PACT or in the scoring of
that there was no qualitative difference between the two cohorts in the assessments. A member of the research team not related to this
terms of their lessons. Each lesson plan was given a score of 1e4, part of this study de-identified all science candidates' PACT
with four representing the most student-directed lesson and one Teaching Event responses of any personal and field-work infor-
the most teacher-directed (see Table 1). mation prior to analysis.

3.3. Data 3.4. Analysis

Data for the study consists of candidates' responses as part of the We adopted a mixed methods approach to our analysis for
Performance Assessment for California Teachers (PACT). Modeled several reasons (Tashakkori & Teddlie, 2010). Because candidates'
 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93 87

written responses were grounded in their experiences, we used analyses. To further refine the framework, members of the research
qualitative methods to honor the rich and nuanced nature of their team who were not affiliated with this particular study applied the
ways of viewing and analyzing teaching. This allowed us to framework to the six cases and discussed variations in the ratings.
construct a framework to characterize variations in the ways that Issues raised in the discussion were used to further clarify and
they attended to, analyzed, and responded to instructional in- refine the operational definitions of the framework in an iterative
teractions (Miles, Huberman, & Saldan ~ a, 2014). At the same time, fashion (Corbin & Strauss, 2008).
we used quantitative methods to provide us with analytic power In the third phase of analysis, we applied the framework to code
for exploring the relationship among these skills. The result is a the additional cases (n ¼ 24) to examine any differences that
valid, reliable, and grounded exploration and explanation of the existed between cohorts in the level of sophistication for all three
construct of pre-service teacher noticing and reflection in the skills (attending, analyzing, and responding). We reviewed the
context of a credential preparation program. candidates' responses to all three PACT questions and then assigned
Data analysis for the first research question took place in several an overall score for each category (Miles et al., 2014). Sophistication
phases. In the first phase, the first author selected a random sample scores were assigned a numerical value to reflect the ordinal nature
of PACT responses from six candidates, three from each cohort, and of the data as follows: low ¼ 1, medium ¼ 2, and high ¼ 3. To ensure
used a sentence-by-sentence coding technique to gain insight into inter-rater reliability, members of the research team independently
the similarities and differences between candidates' noticing and coded a randomly-selected subset of five cases. Inter-rater agree-
analysis of teaching in this context (Corbin & Strauss, 2008). This ment of 86% was achieved across the three skills (93% for attending,
analysis was informed by prior research on teacher noticing, lesson 73% for analyzing, and 93% for responding). We then conducted a
analysis, and teacher reflection (Davis, 2006; Hiebert et al., 2007; ManneWhitney U test to examine differences between LLfT and
Levin et al., 2009; Mason, 1998; Santagata, 2011; van Es, 2011; non-LLfT participants as compared to each other on each of the
van Es & Sherin, 2002), which highlights the importance of three three skills (attending, analyzing, and responding) and their com-
components: what teachers attend to, how they analyze instruc- bined rank score on the three skills. We used one-tailed tests
tion, and how they choose to respond to students. This literature because prior research suggests that teacher candidates learn to
also draws attention to the importance of specificity of language, systematically analyze teaching with structured opportunities to
linking elements of teaching and learning, and making evidence- learn, like that provided in the LLfT course (Chung & van Es, 2014;
based claims about the effectiveness of instruction. From this Santagata & Angelici, 2010; Star et al., 2011).
coding, the first author wrote analytic memos for each of the six Our second research question examined the relationship among
cases to capture the focus and content of candidates' attention and skills. For this analysis, we combined the two cohorts because we
analysis (e.g. student thinking, student behavior, teacher behavior, needed a wide range of responses to fully explore the construct of
classroom climate, science content), the level of detail used to noticing and analyzing teaching. The findings from our first analysis
describe and analyze events, and the relationship among what they revealed differences between the two cohorts in terms of sophis-
observed, their analyses, and their suggestions for improving tication of analyzing teaching, providing us with a range that would
teaching. When we reviewed these memos we identified three allow us to probe these relationships. Because the data are not
skills - the ways these candidates attended to, analyzed, and normally distributed, the sample size is small, and the data are
planned to respond to instruction e as well as variations in the ordinal rather than categorical, Kendall's Tau values were calcu-
sophistication of each skill. For the second phase of analysis, we lated to determine the strength of the relationship among the skills
developed a three-level framework to characterize the differences across all cases (attending  analyzing; attending  responding;
among the six cases for each skill. The resulting framework is dis- analyzing  responding). One-tailed tests were used because the
played in Table 2. literature suggests that these three skills would be positively
In this framework, attending is concerned with what candidates correlated.
attended to when they observed their teaching in their PACT
response. Analyzing is concerned with their interpretation and
sense-making of the events that they highlighted. Responding is 4. Results
concerned with how participants wrote about how they took up
and used student ideas to inform their teaching and proposed In the following sections, we present the results for each
appropriate next steps in a logical way. In categorizing responses in research question. We first present the comparisons between the
terms of sophistication, we intended to capture a range of two cohorts in terms of their attending, analyzing, and responding.
simplistic, generic observations to more robust, substantive Then, we report on the relationship among learning to attend to the

Table 2
Levels of sophistication for noticing skills.

Skill Low sophistication Medium sophistication High sophistication

Attending Highlights classroom events, teacher pedagogy, Highlights student thinking with Highlights student thinking with respect to the
student behavior, and/or classroom climate. No respect to the collection of data from collection, analysis, and interpretation of data
attention to student thinking. a scientific inquiry (science procedural from a scientific inquiry (science conceptual focus)
focus).
Analyzing Little or no sense-making of highlighted events; Begins to make sense of highlighted Consistently makes sense of highlighted events.
mostly descriptions. No elaboration or analysis events. Some use of evidence to Consistent use of evidence to support claims.
of interactions and classroom events; little or support claims.
no use of evidence to support claims.
Responding Does not identify or describe acting on specific Identifies and describes acting on a Identifies and describes acting on a specific student
student ideas as topics of discussion; offers specific student idea during the idea during the lesson and offers specific ideas of
disconnected or vague ideas of what to do lesson; offers ideas about what to what to do differently next time in response to
differently next time. do differently next time. evidence; makes logical connections between
teaching and learning.
88 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93

particulars of student thinking and candidates' abilities to analyze Table 4

and respond to student ideas. Comparison of mean attending, analyzing, responding, and total scores between two
cohorts.

4.1. Differences between cohorts' analyses of teaching Skill Non-LLfT candidates LLfT candidates

Attendingy 2 2.5
When comparing the two different groups, data analysis Analyzing 1.75 2
Respondingy 1.5 1.9
revealed that LLfT candidates were more sophisticated in attending
Total Score* 5.3 (4e7) 6.4 (3e9)
to specific instances of student thinking, analyzing this evidence, N 8 16
and commenting on adjustments to instruction in response to a
Note. ManneWhitney results for attending U(23) ¼ 88.0, Z ¼ 1.47, .07. For analyzing
student idea as compared to the cohort that did not enroll in the U(23) ¼ 77.0, Z ¼ .80, .21. For responding U(23) ¼ 86, Z ¼ 1.35, .10. For total score
course. Table 3 displays the differences between the two cohorts in U(23) ¼ 95.0, Z ¼ 1.90, .03. All tests were one-tailed. Range is in parentheses.
terms of level of sophistication in all three skills. We observed the *p < .05. yp  .10.
greatest difference between the cohorts in the sophistication of
attending to student thinking about the collection, analysis, and
interpretation of data from a scientific inquiry.
sophistication than those who did not take the course. When these
Looking at the distribution of the level of sophistication for the
small but appreciable differences in the sophistication of each skill
two cohorts in Table 3, although there are low scoring individuals in
are aggregated to generate a mean total sophistication score for
both groups, most of the candidates who participated in the course
both cohorts, the LLfT candidates' responses are statistically
scored in the medium to high range for the three skills, while a
significantly more sophisticated. The following excerpts from
greater number of the candidates who did not experience the
candidate responses typical for each cohort illustrate these
course scored in the low to medium range for all three skills. Also
differences.
noteworthy is that only two non-LLfT candidates scored in the high
Roger, a non-LLfT candidate, shared a segment from a middle
sophistication range for any skill, on the dimension of attending.
school general science lesson in which he asked students to work in
In terms of the framework, we observed high levels of sophis-
groups to develop a test to determine the salinity of a solution that
tication for over 60% of candidates in the based course compared to
would yield the “crispiest,” or most turgid, potato slice. They were
only 25% of the comparison group. More than half of both cohorts'
provided materials and equipment such as salt, water, a triple beam
analysis skills were coded as medium sophistication, suggesting
balance, beakers, and a graduated cylinder to develop their test and
they were using some evidence to support the claims they made in
collect data. Roger defined the goal for this lesson for students to
their PACT responses when prompted to do so, but only members of
design a controlled experiment, collect data, and identify sources of
the LLfT cohort showed high levels of analysis by consistently citing
error. When prompted to analyze how he promoted student inquiry
evidence in their responses. Finally, 75% of the course participants
through the collection and analysis of data, he wrote the following:
identified and described acting on or proposing a response to a
specific student idea, compared to 50% of the comparison group. Specifically, the inquiry lab seen in the first clip highly engaged
The distribution of scores across the three levels of sophisti- the students as they were allowed to explore a scientific prob-
cation shown in Table 3 suggests that the candidates who enrolled lem using methods they created themselves. They were able to
in the course demonstrated higher levels of sophistication in each have fun trying to solve a problem on their own, using their own
skill. To explore this further, an overall mean sophistication score creativity, rather than being told to follow a very structured lab
was calculated for both cohorts for each skill by summing each procedure.
candidate's scores on each skill (low ¼ 1, medium ¼ 2, high ¼ 3) and
obtaining the mean score for each cohort. Analysis reveals that the
Rather than focus on the substance of students' scientific
cohorts are marginally statistically significantly different in their
thinking, Roger mentioned the importance of students to “have
demonstration of attending and responding on the PACT (see
fun,” use their “creativity,” and solve a problem “on their own.” He
Table 4).
uses general language to describe students' participation. He also
Though the differences in sophistication for the skill of
uses general language in the way he frames his learning goal.
analyzing did not reach statistical significance, the difference be-
Although the goal was mentioned, it was procedural rather than
tween the two cohorts shows the same pattern as attending and
conceptual and not specific to the content of focus. Though con-
responding with LLfT candidates demonstrating higher
cepts of osmosis, diffusion, and the role of the cell wall in plant cells
could have been potential topics of investigation, they were not
Table 3
discussed in this lesson or in the subsequent lesson.
Comparison of the sophistication of attending, analyzing, and responding to
teaching between two cohorts.
Later in the analysis, Roger responded to a prompt to describe
what he did to monitor student progress. Roger explained that he
Skill Non-LLfT participants LLfT participants
conducted a demonstration of one method to determine the
Attending “crispiest potato” and charged his students with finding any errors
Low sophistication 25% (2) 13% (2) he made during his procedure. He wrote:
Medium sophistication 50% (4) 25% (4)
High sophistication 25% (2) 63% (10) During the demonstration, I asked the students to write down
Analyzing the errors that they spotted. At the end of the demonstration, I
Low sophistication 25% (2) 19% (3)
Medium sophistication 75% (6) 63% (10)
asked the students to tell me what they noticed. I then used
High sophistication 0% (0) 19% (3) these observations to further discuss the importance of the
Responding quality of data in an experiment. I directly questioned the stu-
Low sophistication 50% (4) 25% (4) dents so that they would make the discovery that all of the er-
Medium sophistication 50% (4) 56% (9)
rors made had to do with just one potato. I did this after all
High sophistication 0% (0) 19% (3)
errors had been reported by asking different students which
N 8 16 potato I had made the error with. Each student stated that the
Note. N values are in parenthesis. potato I had made the error with was potato C. I then related this
 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93 89

discovery back to the graph to show how great of an effect the confusion as to why their results were taking so long and suggested
errors had on my data. This questioning helped keep the stu- it might be because one of them was a tuba player and therefore
dents engaged and interested in the outcome of the experiment had greater lung capacity. Walter pursued this idea, pushing the
as well as obtain a greater understanding of the learning students to make the connection between what might be impor-
objective. tant about lung capacity in relation to the amount of carbon dioxide
he introduced into the solution, how that changed the pH of the
Several features stand out in this response. First, it is teacher- solution, and why that pH change required more sodium hydroxide
centered and focused almost exclusively on the teacher's own ac- to neutralize it.
tions. Roger makes one mention of an idea generated by the stu- Not only is Walter specific in his description of the student
dents e they observed “that the potato I had made the error with thinking in this interaction, he is also specific about the science
was potato C,” but he does not elaborate on this idea except to say concept he identifies for students e the relationship among carbon
that he linked this response to the bar graph he created to dioxide, blood pH, and aerobic capacity. It was not enough that
demonstrate the results. He closes his response with a claim that students recognize the relationship but that they understood the
his questioning kept the students “engaged and interested,” but biological mechanism that drives the relationship among the three
does not provide any specific evidence to support this claim. Sec- variables. This level of attention in the description and content of
ond, though Roger's lesson is designed to develop students' skills in student thinking is emblematic of a typical LLfT candidate response.
data collection and he claims that he provides freedom for students Another distinction in Walter's PACT response is how he built on
to develop their own procedures, it is not in the service of a broader students' science thinking during instruction. For example, he
conceptual goal of cellular transport or a “big idea” such as the wrote about how he pressed the class during their examination of
relationship of structure to function. Third, Roger's attempt to their pooled data displayed on the board. He questioned students
engage students in identifying his errors, in essence figuring out about why different groups' data may be different from each other.
what he already knows in a type of guessing game, is not in the The students responded that “individuals are different.” As in the
spirit of inquiry as advocated by the Next Generation Science previous example, he elaborated on his follow-up response to his
Standards (Achieve, Inc., 2014). In this context, his attention to students' answer to document how he monitored student progress
student ideas is different than attending to students' novel toward the learning goal:
thinking. He is not attempting to uncover and build on student
I then furthered the students thinking by asking, “what would
ideas, but rather to lead students to his own “correct” answer. We
be different about people?” The students then came up with
do not know what meaning Roger draws from his students' answer
responses about different people take in different levels of ox-
to know if or how well students understood why the procedures
ygen, some people are more fit, and different muscle mass.
used on potato C changed the results.
When students would suggest these differences, I would ask the
In contrast, Walter's PACT response is typical of the LLfT candi-
students to expand on their thinking and why it would make a
dates. His Biology students were collecting data to understand the
difference. Therefore, the students were building on their
factors that influence Adenosine Triphosphate (ATP) production
knowledge by connecting the content from lecture to the con-
and the role of oxygen and carbon dioxide in cellular respiration.
cepts involved in the lab.
Walter's students prepared a sample of water and Bromothymol
Blue and blew bubbles of carbon dioxide into it for a set amount of
time under different conditions (when at rest, after light exercise, As before, Walter makes a claim about what he is attempting to
and after vigorous exercise). The more acidic the water became do, in this case drawing students' attention to the connection be-
from the introduction of the carbon dioxide, the more sodium hy- tween physiological differences and the resulting changes in
droxide would be required to return it to its original color. A cellular respiration rate, provides a detailed account of what his
noteworthy aspect of his response is that he highlights two ex- students say and how he responds to their ideas, and then makes a
amples of how he helped students deepen their knowledge of the statement about the significance of the evidence he presented.
science concept while collecting data. In the first, he describes an Additionally, Walter is calling out specific interactions that involve
interaction with one of the lab groups that is confused as to why the students' making meaning of a specific science concept, not just
their solution requires so much more sodium hydroxide than their the skill of collecting data. The details he includes in his narrative
classmates' solutions. serve as evidence to support a claim he is making about what
students learned and how he assisted them in the process. Linking
The students discussed how it was because the person per-
student ideas and instructional actions was more common in the
forming the experiment was a tuba player. The dialogue about
LLfT candidates' responses than the non-LLfT candidates' responses.
the tuba player got the students building on their prior knowl-
These findings suggest that the LLfT candidates were demon-
edge that all individuals are different and that by being a tuba
strating higher levels of sophistication on multiple skills in their
player you would probably have a greater lung capacity. I then
responses. We then explored the differences between the two co-
asked the group that if the solution is taking a long time to turn
horts in terms of the co-occurrence of medium and high levels of
colors is the solution very acidic or basic and the students
sophistication.
answered acidic, which I responded by asking, meaning it has a
Table 5 shows that 60% of LLfT participants demonstrated me-
lot of and they answered CO2. Therefore, I am scaffolding the
dium or high levels of sophistication in all three skills, compared to
students into understanding the concepts and recognizing the
25% of the comparison group. Further, none of the participants in
connections of how one piece of information tells us some part
the comparison group demonstrated high levels of sophistication in
of the reaction that is occurring.
all three areas and a quarter of this group exhibited low levels of
sophistication in multiple skills. The majority of candidates from
Walter's response is different than Roger's in several ways. First, the course scored at medium and high levels of sophistication in all
he uses specific, student-centered examples to support his claim three skills. In contrast, the majority of comparison group scored at
that he is helping “further the students' knowledge and skills while medium and high levels of sophistication in only two skills com-
collecting data.” He provided a detailed example of students bined. This suggests a relationship between the three skills of
attempting to make sense of their lab experience. They expressed attending, analyzing and responding, the focus of our second
90 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93

Table 5 indication that the teacher is exhibiting more selective attention

Comparison of LLfT and Non-LLfT participants' sophistication of PACT responses in and focusing on significant classroom events rather than attending
combination.
to a wide range of events that take place in the classroom (van Es &
Skill Non-LLfT LLfT Sherin, 2002).
cohort cohort An example of this is in Walter's response detailed above. He
Low sophistication in all areas 0 6% (1) called out specific student ideas about how the amount of carbon
Medium or high sophistication in one area only 25% (2) 6% (1) dioxide is influenced by differences among people. Rather than
Medium or high sophistication in two areas only 50% (4) 25% (4)
accepting “people are different” as an acceptable answer to explain
Medium and high sophistication in all three areas 12.5% (1) 44% (7)
Medium in all areas 12.5% (1) 6% (1) the variation in carbon dioxide production and moving on, he
High sophistication in all areas 0 12.5% (2) probed students to uncover what his students meant when they
N 8 16
said that people were “different.” Walter is not just simply seeking
“correct” answers; rather he is seeking to more fully understand his
Note. N values are in parenthesis.
students' ideas and the depth of their understanding by asking
research question. We now turn to present our results in the next them to elaborate on their answers. Walter then interpreted what
section. these elaborations tell him about his students' developing under-
standing about the relationship between cellular respiration rate
and physiology.
4.2. Relationship among attending, analyzing, and responding
We now turn to the relationship between analyzing student
thinking and responding to student ideas. The Kendall's tau value
To answer the second research question, we explored the re-
supports the qualitative analysis that these skills appear to be
lationships among the three skills by examining if sophistication of
tightly coupled, ƬB ¼ .572 p ¼ .001. A positive and highly statistically
any one skill related to the sophistication of any other skill. In
significant pattern was observed between analyzing and respond-
particular, we hypothesized that high sophistication on attending
ing in this sample. High sophistication in analyzing tended to co-
would support higher levels of sophistication on analyzing, and
occur with high sophistication in responding, medium sophisti-
higher sophistication on attending would in turn support higher
cation in analyzing with medium sophistication of responding, and
levels of sophistication on responding.
low sophistication in analyzing with low sophistication in
To examine these relationships, we explored each pair of skills
responding. Participants were slightly more likely to demonstrate
(attending  analyzing, analyzing  responding, and
sophistication in analysis than in responding to student ideas.
attending  responding) in turn. Table 6 displays a relationship
Additionally, there was only one case of highly sophisticated
among sophistication in all three skill pairs.
analysis of student ideas without a highly sophisticated response
We anticipated that there would be statistically significant re-
and no cases of highly sophisticated responses with only medium
lationships among the levels of sophistication of all skill pairs, that
or low levels of sophistication in analyzing.
low levels of sophistication in some skills would inhibit candidates
To score at high levels of sophistication in responding required
from demonstrating high levels of sophistication in others, and that
participants to propose logical changes to future lessons and
candidates would be more likely to demonstrate high levels of
document an adjustment they made to instruction in real-time to a
sophistication in the skill of attending compared to analyzing and
student idea. A medium level of sophistication required that par-
responding. The results that follow largely support this conjecture.
ticipants propose logical changes to future lessons. To be considered
The first skill pair we analyzed was attending and analyzing. As
logical, the responses had to be informed by evidence they cited in
expected, candidates who demonstrated low sophistication in
their PACT narrative. An example of the typical logic for proposed
attending were unlikely to demonstrate high levels of sophisticat-
adjustments is Walter's response to the analysis of his lesson. He
ion in analyzing and vice versa. The Kendall's tau B coefficient in-
identified a learning goal for his biology students: that they should
dicates a positive and highly statistically significant relationship
understand the relationship among exercise, oxygen demand, pulse,
between these two skills, similar to the relationship found in prior
and ATP production. This example of homeostasis is an important
research, ƬB ¼ .387 p ¼ .003 (Santagata, 2011; van Es, 2011). This
concept in high school biology. Yet, when prompted to suggest a
pairing of attention to student ideas and analysis of what those
possible change he would make to this lesson in the future, he
ideas tell the teacher about student understanding may be an
mentioned that many students did not write extensive responses to
Table 6 the prompts on their written lab report. He proposed the following:
Observed relationship between attending, analyzing, and responding for all “A change I would make would be to place lines on the lab report
participants. pages, which would give the students an idea of how much writing
Level of Low Medium High would be required for each question.” This proposed adjustment,
sophistication while related to how students may experience writing a lab report,
Attending has little relevance to his stated learning goal, and therefore, scored
Analyzing Low 1 4 0 as a less sophisticated response for responding. Though it is
Medium 3 3 9 accompanied by a justification based on what he noticed about his
High 0 1 3
students' writing, it is not a response to students' ideas about ho-
ƬB ¼ .387 p ¼ .003
meostasis. Documentation of these types of management and
Responding behavioral adjustments were common among all participants in the
Analyzing Low 4 1 0
Medium 3 12 0
PACT. They serve as an important reminder of how difficult it can be
High 1 0 3 for pre-service teachers to act upon their analysis of student ideas in
ƬB ¼ .572 p ¼ .001 the moment by adjusting instruction. In fact, only three of 24 par-
Responding ticipants scored at high levels of responding.
Attending Low 1 3 0 The final skill pair to be analyzed was between attending and
Medium 4 3 1 responding. Participants were more likely to demonstrate higher
High 3 7 2 levels of sophistication in attending than responding. Moreover,
ƬB ¼ .140 p ¼ .397
participants who demonstrated low sophistication in attending
 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93 91

were unlikely to demonstrate high levels of sophistication in Our second research question concerned the relationship
responding. However, the qualitative examples combined with the among the three skills. While research on reflection (Dewey, 1933;
low and non-statistically significant Kendall Tau coefficient for this Scho€ n, 1983), awareness (Mason, 1998), and noticing (van Es, 2009)
skill pair suggests that these two skills are not meaningfully related suggest that these are recursive processes, our analysis reveals that
to each other, ƬB ¼ .140 p ¼ .397. That is to say, it is reasonable to systematic analyses of instruction may rely on skilled “noticing”
conclude that highly sophisticated analysis depends on highly so- (Sherin, Jacobs, & Philipp, 2011). The low occurrence of sophisti-
phisticated attention to student ideas. In turn, highly sophisticated cated analysis and responding without sophisticated attention
responses, whether proposed or enacted, depend on highly so- suggests that learning to see the important details of student
phisticated analyses of student ideas. Though teachers frequently thinking with respect to particular content is a cornerstone for
react to students during instructions, these reactions may be lack- more sophisticated analysis and instructional responses. Prior
ing in quality (Scribner, 2003). This suggests that attending without research finds that it is not a simple matter to support candidates in
analysis does not typically lead to sophisticated responses to stu- learning to see the details of student thinking e to learn to identify
dent thinking. what about student ideas are or are not important, how their sci-
entific thinking is evolving, and how students are reasoning and
making sense of scientific phenomena, rather than the correctness
5. Discussion or accuracy of their ideas (Hiebert et al., 2007; Kazemi & Stipek,
2001; Rodgers, 2002; Thompson, Braaten, & Windschitl, 2009).
We sought to investigate if a video-based course supported Thus, preparing pre-service teachers to learn how to attend to
secondary science teacher candidates in learning to systematically student ideas may be a critical prerequisite skill for more inte-
analyze teaching by developing their skills at attending, analyzing, grated, disciplined, and effective reflection proposed by research
and responding as compared to a cohort of candidates who did not (Davis, 2006; Hiebert et al., 2007; Rodgers, 2002).
enroll in the course and to explore the relationship among these We also found that a high level of attention to student ideas does
three skills as they arose in the candidates' reflections. We discuss not guarantee high levels of analyzing or responding to those ideas.
our findings below. That is, learning to see the details of student thinking and the de-
With respect to the first research question, this study reveals tails of interactions related to content as they unfold during in-
that the candidates who participated in the video-based course, struction does not ensure that candidates will take up those ideas
Learning to Learn from Teaching, demonstrated higher sophisticat- and use them as evidence to analyze teaching and learning or
ion overall in their attention, analysis, and responses to student inform instructional responses. The number of candidates scoring
thinking. This finding is consistent with other research that shows at high levels of sophistication was greatest in the skill of attending,
that pre-service teachers can learn to attend to student ideas when then analyzing, with responding having the lowest number of
instructed to do so (Davis, 2006; Davis & Smithey, 2009; Levin et al., candidates scoring at high levels of sophistication. The distribution
2009; Star et al., 2011). However, unlike previous studies, we of candidates' scores across the three skills of low, medium, and
examined candidates' analyses of teaching several months after the high sophistication indicate that attending, analyzing, and
conclusion of the course. Despite three months' time separating responding to student ideas may be successively difficult and
completion of the course and completion of the PACT assessment, complicated skills. It may be that analysis is the bridging skill be-
course participants demonstrated higher levels of sophistication tween attending and responding. Evidence does not speak for it-
with respect to the three skills, whereas the comparison group self; it only gains meaning and value when it is “shaped, organized,
demonstrated little to no attention to student thinking despite the and thought about.” (Earl & Timperley, 2008). Analysis is what
explicit prompting of the PACT assessment to do so. Such disci- gives reason to the other two skills and distinguishes them as part
plined awareness is more typical of expert practitioners (Berliner, of effective reflection rather than simply seeing and reacting
2001; Mason, 2002). This finding is encouraging, particularly (Rodgers, 2002). Engaging in this sort of interpretation and sense-
because prior research suggests that pre-service teachers ignore, making, and using what is observed to draw inferences, is chal-
discount, or discard, what they learn during their preparation lenging even for experienced teachers (Little & Curry, 2008).
programs (Korthagen, 2007; Richardson, 2003), and that their Further complicating the relationship among skills is how
teaching contexts do not often provide the structure and support to reflection and analysis are framed for teachers. Reflection must
promote the enactment of practices they develop in their prepa- have a purpose to be effective and that purpose must be framed as a
ration coursework (Thompson, Windschitl, Braaten, & Stroupe, way to problematize teaching (Loughran, 2002; Zeichner, 1987).
2013; Valencia, Martin, Place, & Grossman, 2009). Levin et al. (2009) found that the focus on covering the curriculum
In addition, though the participants in the course demonstrated and establishing routines in field sites likely influences whether or
greater sophistication overall, they did not consistently demon- not novice teachers attend to student thinking and how teachers
strate high levels of sophistication in all three skills in tandem. In respond to student ideas. They suggest that developing teacher
fact, many of them demonstrated what has been described as candidates' framing of instructional interactions is essential for
“mixed” noticing (van Es, 2011) with higher levels of sophistication helping them shift to adopt a more student-centered, responsive
on some dimensions and lower levels on others. One possible approach to instruction.
explanation is that the course only lasted ten weeks and the While the results of this study are encouraging, we acknowledge
reflective skills that were the center of the course were not that there are several limitations. First, although the findings
explicitly reinforced in other aspects of the program. Research regarding the differences between the course participants and non-
suggests that coherence in the learning goals across pre-service participants are statistically significant and echo similar findings in
teachers' experiences in a teacher credential program will result the literature, the small sample size curtails the broader implica-
in greater opportunities to learn core knowledge, skills and prac- tions that can be drawn from these results. Second, though the
tices for beginning teaching (Darling-Hammond, 2006; McDonald, same instructors taught both the 2007 and 2009 courses and no
Kazemi, & Kavanagh, 2013). From this perspective, the influence other substantial changes were made to the program between 2007
that participation in the course appears to have had on the candi- and 2009, it may be the case that other aspects of the candidates'
dates' abilities to systematically analyze teaching and learning experience influenced how they came to analyze instruction, such
despite the limited scope of the intervention is encouraging. as the extent to which their supervisors and mentor teachers
92 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93

prompted them to reflect in systematic ways or the content of Carter, K., Cushing, K., Sabers, D., Stein, P., & Berliner, D. (1988). Expert-novice dif-
ferences in perceiving and processing visual classroom information. Journal of
different coursework may have provided opportunities for more
Teacher Education, 39(3), 25e31.
substantive reflection. Understanding how candidates' experiences Chung, H. Q., & van Es, E. (2014). Pre-service teachers' use of tools to systematically
in the video-based course interact with their experiences in the analyze teaching and learning. Teachers and Teaching: Theory and Practice, 20(2),
credential program as a whole is an important subject for future 113e135.
Copeland, W. D., Birmingham, C., DeMeulle, L., D'Emidio-Caston, M., & Natal, D.
inquiry. Third, we only looked at candidates' proposed instructional (1994). Making meaning in classrooms: an investigation of cognitive processes
responses to student thinking as documented in the PACT, rather in aspiring teachers, experienced teachers, and their peers. American Educa-
than how actually responded during instruction as captured in the tional Research Journal, 31(1), 166e196.
Corbin, J., & Strauss, A. (2008). Basics of qualitative research: Techniques and pro-
videos of instruction. That said, by analyzing their written re- cedures for developing grounded theory (3rd ed.). Thousand Oaks, CA: Sage
sponses, we have insight into how they make sense of what they Publications.
observe, which is more difficult to capture by analyzing instruc- Council of Chief State School Officers (2014). Retrieved May, 2014 from http://www.
ccsso.org/Resources/Programs/Interstate_Teacher_Assessment_Consortium_
tional interactions. Analysis of their written responses, coupled (InTASC).html.
with the videos of their teaching, would likely provide a more Darling-Hammond, L. (2006). Constructing 21st-century teacher education. Journal
robust picture of what candidates attend to in teaching and how of Teacher Education, 57(3), 300e314.
Darling-Hammond, L. (2010). Recognizing and developing effective teaching: What
they respond. It also the case that we used only one data source, the policy makers should know and do. Partnership for Teacher Quality Policy Brief,
PACT assessment, to analyze pre-service teachers' reflective ca- May, 2010. Retrieved January, 2014 from http://www.nea.org/assets/docs/HE/
pacities. Because of the summative, high-stakes nature of the PACT Effective_Teaching_-_Linda_Darling-Hammond.pdf.
Darling-Hammond, L., & Bransford, J. (2005). Preparing teachers for a changing
assessment for teacher credentialing, the candidates may have
world: What teachers should know and be able to do. San Francisco, CA: Jossey-
been less inclined to critically analyze their instruction by focusing Bass.
on areas of improvement because they wanted to provide positive Davis, E. (2006). Characterizing productive reflection among preservice elemen-
depictions of their practice. A more robust study would provide tary teachers: seeing what matters. Teaching and Teacher Education, 22(3),
281e301.
multiple data sources that capture pre-service teachers' reflection Davis, E., Petish, D., & Smithey, J. (2006). Challenges new science teachers face.
on practice. Finally, we only looked at candidates' reflection and Review of Educational Research, 76(4), 607e651.
analysis of teaching at one point in time. Further analysis into pre- Davis, E., & Smithey, J. (2009). Beginning teachers moving toward effective
elementary science teaching. Science Education, 93(4), 745e770.
service teachers' learning to systematically analyze teaching over Dewey, J. (1933). How we think. Buffalo, NY: Prometheus Books.
the course of the credential program, as well as longitudinal studies Earl, L. M., & Timperley, H. (2008). Understanding how evidence and learning
following pre-service teachers into the early years of teaching, will conversations work. In L. M. Earl, & H. Timperley (Eds.), Professional learning
conversations: Challenges in using evidence for improvement (pp. 1e12). New
also yield valuable information about the influence that courses like York, NY: Springer.
Learning to Learn from Teaching have on teachers' professional Erickson, F. (2011). On noticing teacher noticing. In M. Sherin, V. Jacobs, & R. Philipp
trajectories. (Eds.), Mathematics teacher noticing: Seeing through teachers' eyes (pp. 17e34).
New York, NY: Routledge.
Feiman-Nemser, S. (2001). From preparation to practice: Designing a continuum to
6. Conclusion strengthen and sustain teaching. The Teachers College Record, 103(6), 1013e1055.
Franke, M., & Kazemi, E. (2001). Learning to teach mathematics: focus on student
thinking. Theory Into Practice, 40(2), 102e109.
To prepare future teachers to enact rigorous and responsive Gelfuso, A., & Dennis, D. (2014). Getting reflection off the page: the challenges of
instruction that is at the core of recent reforms in science education developing support structures for pre-service teacher reflection. Teaching and
Teacher Education, 38(1), 1e11.
(Achieve, Inc., 2014) requires that teacher candidates learn to
Goldhaber, D., & Anthony, E. (March 8, 2004). Can teacher quality be effectively
attend to the nuances of student ideas, how to interpret what they assessed? The Urban Institute. Retrieved February, 2014 from http://www.
see and hear, and how to best respond if they are to be effective, urban.org/UpLoadedPDF/410958_NBPTSOutcomes.pdf.
Gleick, J. (2011). The information: A history, a theory, a flood. New York, NY: Harper
ethical professionals (Ball & Cohen, 1999). Preparing candidates to
Collins.
learn to systematically attend, analyze and respond is also essential Hammer, D. (2000). Student resources for learning introductory physics. American
if teacher educators seek to develop teachers who treat teaching as Journal of Physics, Physics Education Research Supplement, 68(S1), S52eS59.
a learning profession, where they learn in and from their practice Hammerness, K., Darling-Hammond, L., & Bransford, J. (2005). How teachers learn
and develop. In L. Darling-Hammond, & J. Bransford (Eds.), Preparing teachers for
over time (Feiman-Nemser, 2001; Hiebert et al., 2007; Lampert, a changing world: What teachers should know and be able to do. San Francisco,
2010; Zeichner, 1987). Teacher preparation programs must CA: Jossey-Bass.
continue to develop these capacities while researchers track the Hiebert, J., Morris, A., Berk, D., & Jansen, A. (2007). Preparing teachers to learn from
teaching. Journal of Teacher Education, 58(1), 47e61.
development of these skills beyond teacher education programs Hufferd-Ackles, K., Fuson, K. C., & Sherin, M. G. (2004). Describing levels and
and into the induction years of teaching. Such research will provide components of a math-talk learning community. Journal for Research in Math-
important insights into how these capacities develop in relation to ematics Education, 35(2), 81e116.
Jacobs, V. R., Lamb, L. L., & Philipp, R. A. (2010). Professional noticing of children's
each other over time and the influence they have on teachers mathematical thinking. Journal for Research in Mathematics Education, 41(2),
learning in and from their practice. 169e202.
Jansen, A., Bartell, T., & Berk, D. (2009). The role of learning goals in building a
knowledge base for elementary mathematics teacher education. The Elementary
References School Journal, 109(5), 525e536.
Kazemi, E., Lampert, M., & Franke, M. (2009, July). Developing pedagogies in teacher
Achieve, Inc. (2014). The next generation science standards. Retrieved May, 2014 from education to support novice teacher's ability to enact ambitious instruction. In
http://www.nextgenscience.org/next-generation-science-standards. R. Hunter, B. Bicknell, & T. Burgess (Eds.), Crossing divides: Proceedings of the
American Association for the Advancement of Science. (2009). Benchmarks for sci- 32nd annual conference of the Mathematics Education Research Group of Aus-
entific literacy. New York, NY: Oxford University Press. tralasia (Vol. 1, pp. 12e30). Palmerston North, NZ: MERGA.
Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Kazemi, E., & Stipek, D. (2001). Promoting conceptual thinking in four upper-
toward a practice-based theory of professional education. In L. Darling-Ham- elementary mathematics classrooms. The Elementary School Journal, 102(1),
mond, & G. Sykes (Eds.), Teaching as the learning profession: Handbook of policy 59e80.
and practice (pp. 3e32). San Francisco, CA: Jossey-Bass. Korthagen, F. (2007). The gap between research and practice revisited. Educational
Borko, H., & Livingston, C. (1989). Cognition and improvisation: differences in Research and Evaluation, 13(3), 303e310.
mathematics instruction by expert and novice teachers. American Educational Lampert, M. (2010). Learning teaching in, from, and for practice: what do we mean?
Research Journal, 26(4), 473e498. Journal of Teacher Education, 61(1e2), 21e34.
Berliner, D. C. (2001). Learning about and learning from expert teachers. Interna- Lemke, J. (2002). Discursive technologies and the social organization of meaning.
tional Journal of Educational Research, 35(5), 463e482. Folia Linguistica, 35(1e2), 79e96.
Blythe, T. (1997). The teaching for understanding guide. San Francisco, CA: Jossey Levin, D. M., Hammer, D., & Coffey, J. E. (2009). Novice teachers' attention to student
Bass. thinking. Journal of Teacher Education, 60(2), 142e154.
 T. Barnhart, E. van Es / Teaching and Teacher Education 45 (2015) 83e93 93

Little, J. W., & Curry, M. W. (2008). Structuring talk about teaching and learning: the Santagata, R., & Yeh, C. (2013). Learning to teach mathematics and to analyze
use of evidence in protocol-based conversation. In L. M. Earl, & H. Timperley teaching effectiveness: evidence from a video-and practice-based approach.
(Eds.), Professional learning conversations: Challenges in using evidence for Journal of Mathematics Teacher Education, December, 1e24.
improvement (pp. 29e42). New York, NY: Springer. Santagata, R., Zannoni, C., & Stigler, J. W. (2007). The role of lesson analysis in pre-
Little, J. W., & Horn, I. S. (2007). ‘Normalizing’ problems of practice: Converting service teacher education: an empirical investigation of teacher learning from a
routine conversation into a resource for learning in professional communities. virtual video-based field experience. Journal of Mathematics Teacher Education,
In L. Stoll, & K. S. Louis (Eds.), Professional learning communities: Divergence, 10(2), 123e140.
detail and difficulties (pp. 79e92). Maidenhead, UK: Open University Press. Schoenfeld, A. H. (2011). Toward professional development for teachers grounded
Loughran, J. (2002). Effective reflective practice: In search of meaning in learning in a theory of decision making. ZDM, 43(4), 457e469.
about teaching. Journal of Teacher Education, 53(1), 33e43. Scho€n, D. A. (1983). The reflective practitioner. New York, NY: Basic Books.
Lloyd, M. M., & Mukherjee, M. (2012, October). Tell me what you see: Pre-service Scribner, J. P. (2003, April). Do-it-yourselfers or engineers? Bricolage as a metaphor for
teachers' recognition of exemplary digital pedagogy. Paper presented at Austra- teacher work and learning. Paper presented at the annual meeting of the
lian Computers in Education Conference, Perth, Australia. American Educational Research Association, Chicago, IL.
Mason, J. (1998). Enabling teachers to be real teachers: Necessary levels of aware- Sherin, M., Jacobs, V., & Philipp, R. (Eds.). (2011). Mathematics teacher noticing:
ness and structure of attention. Journal of Mathematics Teacher Education, 1, Seeing through teachers' eyes. New York, NY: Routledge.
243e267. So, W. W.-M. (2012). Quality of learning outcomes in an online video-based learning
Mason, J. (2002). Researching your own practice: The discipline of noticing. London: community: potential and challenges for student teachers. Asia-Pacific Journal
Routledge Falmer. of Teacher Education, 40(2), 143e158.
McDonald, M., Kazemi, E., & Kavanagh, S. S. (2013). Core practices and pedagogies of Spitzer, S. M., Phelps, C. M., Beyers, J. E., Johnson, D. Y., & Sieminski, E. M. (2011).
teacher education: a call for a common language and collective activity. Journal Developing prospective elementary teachers' abilities to identify evidence of
of Teacher Education, 64(5), 378e386. student mathematical achievement. Journal of Mathematics Teacher Education,
Miles, M. B., Huberman, A. M., & Saldan ~ a, J. (2014). Qualitative data analysis: A 14(1), 67e87.
methods sourcebook (3rd ed.). Thousand Oaks, CA: Sage Publications. Star, J., Lynch, K., & Perova, N. (2011). Using video to improve preservice mathe-
Ministry of Education-Singapore. (2013). Science Syllabus: Primary. Retrieved matics teachers' abilities to attend to classroom features. In M. Sherin, V. Jacobs,
February, 2014 from www.moe.gov.sg/education/syllabuses/sciences/files/ & R. Philipp (Eds.), Mathematics teacher noticing: Seeing through teachers' eyes
science-primary-2014.pdf. (pp. 117e133). New York, NY: Routledge.
Morris, A. K. (2006). Assessing pre-service teachers' skills for analyzing teaching. Star, J., & Strickland, S. (2008). Learning to observe: using video to improve pre-
Journal of Mathematics Teacher Education, 9(5), 471e505. service mathematics teachers' ability to notice. Journal of Mathematics Teacher
Morris, A. K., Hiebert, J., & Spitzer, S. M. (2009). Mathematical knowledge for Education, 11(2), 107e125.
teaching in planning and evaluating instruction: what can preservice teachers Stockero, S. (2008). Using a video-based curriculum to develop a reflective stance in
learn? Journal for Research in Mathematics Education, 40(5), 491e529. prospective mathematics teachers. Journal of Mathematics Teacher Education,
National Board for Professional Teaching Standards (2014) Retrieved May, 2014 11(5), 373e394.
from http://www.nbpts.org/mission-history. Strauss, R. P., & Sawyer, E. A. (1986). Some new evidence on teacher and student
National Research Council. (2000). Inquiry and the National Science Education competencies. Economics of Education Review, 5(1), 41e48.
Standards: A guide for Teaching and Learning. Committee on Development of an Stürmer, K., Ko €nings, K. D., & Seidel, T. (2012). Declarative knowledge and profes-
Addendum to the National Science Education Standards on Scientific Inquiry, sional vision in teacher education: effect of courses in teaching and learning.
Center for Science, Mathematics, and Engineering Education. The National Acad- British Journal of Educational Psychology, 83, 467e483.
emies Press. Retrieved February, 2014 from http://www.nap.edu/download. Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social
php?record_id¼9596. & behavioral research. Thousand Oaks, CA: Sage Publications.
National Research Council. (2007). Taking science to school: learning and teaching Thompson, J., Braaten, M., & Windschitl, M. (2009, June). Learning progression as
science in Grades K-8. Committee on Science Learning, Kindergarten through vision tools for advancing novice teachers' pedagogical performance. Paper pre-
Eighth Grade. In R. A. Duschl, H. A. Schweingruber, & A. W. Shouse (Eds.), Board sented at Learning Progressions in Science (LeaPS) conference, Iowa City, IA.
on Science Education, Center for Education, Division of Behavioral and Social Sci- Thompson, J., Windschitl, M., Braaten, M., & Stroupe, D. (2013). Developing a theory
ences and Education. The National Academies Press. Retrieved February, 2014 of ambitious early-career teacher practice. American Educational Research
from http://www.nap.edu/download.php?record_id¼11625. Journal, 50(3), 574e615.
National Research Council. (2012). A Framework for K-12 Science Education: Practices, van Manen, M. (1977). Linking ways of knowing with ways of being practical.
crosscutting concepts, and core ideas. Committee on a Conceptual Framework for Curriculum inquiry, 6(3), 205e228.
New K-12 Science Education Standards. Board on Science Education, Division of van Zee, E., & Minstrell, J. (1997). Using questioning to guide student thinking.
Behavioral and Social Sciences and Education. National Academies Press. Journal of the Learning Sciences, 6(2), 227e269.
Retrieved February, 2014 from http://www.nap.edu/download.php?record_ van Es, E., & Sherin, M. (2002). Learning to notice: Scaffolding new teachers’ in-
id¼13165. terpretations of classroom interactions. Journal of Technology and Teacher Edu-
Nicol, C., & Crespo, S. (2004). Learning to see in mathematics classrooms. In Pro- cation, 10(4), 571e596.
ceedings of the 28th Conference of the International (Vol. 3, pp. 417e424). van Es, E., & Sherin, M. (2008). Mathematics teachers' “learning to notice” in the
Retrieved February, 2014 from ftp://134.76.12.4/pub/EMIS/proceedings/PME28/ context of a video club. Teaching and Teacher Education: An International Journal
RR/RR263_Nicol.pdf. of Research and Studies, 24(2), 244e276.
OECD. (2013). PISA 2012 Results: What students know and can do e Student perfor- van Es, E. A. (2009). Participants’ roles in the context of a video club. Journal of the
mance in mathematics, reading, and science (Vol 1). Retrieved February, 2014 Learning Sciences, 18(1), 100e137.
from http://dx.doi.org/10.1787/9789264201118-en. van Es, E. (2011). A framework for learning to notice student thinking. In M. Sherin,
Pang, J. (2011). Case-based pedagogy for prospective teachers to learn how to teach V. Jacobs, & R. Philipp (Eds.), Mathematics teacher noticing: Seeing through
elementary mathematics in Korea. ZDM, 43(6e7), 777e789. teachers’ eyes (pp. 134e151). New York, NY: Routledge.
Pecheone, R. L., & Chung, R. R. (2006). Evidence in teacher education: the Perfor- Valencia, S. W., Martin, S. D., Place, N. A., & Grossman, P. (2009). Complex inter-
mance Assessment for California Teachers (PACT). Journal of Teacher Education, action in student teaching: Lost opportunities for learning. Journal of Teacher
57(1), 22e36. Education, 60, 304e322.
Richardson, V. (2003). Preservice teachers' beliefs. Advances in Teacher Education, 6, Waddington, D., Nentwig, P., & Schanze, S. (2007). Standards in science education:
1e22. Making it comparable. Munster, Germany: Waxman Publishing.
Rodgers, C. (2002). Seeing student learning: teacher change and the role of Whitehurst, G. (2002). Scientifically-based research on teacher quality: research on
reflection. Harvard Educational Review, 72(2), 230e253. teacher preparation and professional development. In Proceedings of the White
Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring teachers' informal formative House conference on preparing tomorrow's teachers. Retrieved February, 2014
assessment practices and students' understanding in the context of scientific from http://tepserver.ucsd.edu.
inquiry. Journal of Research in Science Teaching, 44(1), 57e84. Windschitl, M., Thompson, J., & Braaten, M. (2008). Beyond the scientific method:
Sandoval, W. A., Deneroff, V., & Franke, M. L. (2002, April). Teaching, as learning, as Model-based inquiry as a new paradigm of preference for school science in-
inquiry: Moving beyond activity in the analysis of teaching practice. Paper pre- vestigations. Science Education, 92(5), 941e967.
sented at the annual meeting of the American Educational Research Associa- Windschitl, M., Thompson, J., & Braaten, M. (2011). Ambitious pedagogy by novice
tion, New Orleans, LA. teachers: who benefits from tool-supported collaborative inquiry into practice
Santagata, R. (2011). From teacher noticing to a framework for analyzing and and why? Teachers College Record, 113(7), 1311e1360.
improving classroom lessons. In M. Sherin, V. Jacobs, & R. Philipp (Eds.), Windschitl, M., Thompson, J., Braaten, M., & Stroupe, D. (2012). Proposing a core set
Mathematics teacher noticing: Seeing through teachers' eyes (pp. 152e168). New of instructional practices and tools for teachers of science. Science Education,
York, NY: Routledge. 96(5), 878e903.
Santagata, R., & Angelici, G. (2010). Studying the impact of the lesson analysis Zeichner, K. (1987). Preparing reflective teachers: an overview of instructional
framework on preservice teachers' abilities to reflect on videos of classroom strategies which have been employed in preservice teacher education. Inter-
teaching. Journal of Teacher Education, 61(4), 339e349. national Journal of Educational Research, 11(5), 565e575.
Santagata, R., & van Es, E. A. (2010). Disciplined analysis of mathematics teaching as Zeichner, K., & Liston, D. (1987). Teaching student teachers to reflect. Harvard
a routine of practice. In J. Luebeck, & J. W. Lott (Eds.), Association for Mathematics Educational Review, 57(1), 23e49.
Teacher Education Monograph Series (Vol. 7, pp. 109e123). San Diego: Associa- Zeichner, K., & Liston, D. (1996). Reflective teaching: An introduction. Mahwah, NJ:
tion for Mathematics Teacher Educators. Lawrence Erlbaum Associates.