ECI 502: Continuous Improvement of School Technology Initiatives Evaluation Plan

Math 1 and Math 2 Block Schedule/Apex Learning Initiative Andrew Bachmann

Why did you do it? (Objective): 1. Improve student success on benchmark and standardized tests
What did you do?(Strategies): 1. Teach classes on a year long block schedule
2. Supplement lessons with Apex learning Tutorials
Activities:
Classes are scheduled such that they are 105 minutes long every day and last throughout the length of an entire
school year.
1. Students are able to spend more time practicing math skills while the teacher is present
With the extra time teachers are to use Apex learning tutorials to supplement instruction.
1. Tutorials have introduction and reteaching videos to teach content differently
2. Students may work at their own pace and retry concepts as much as they would like
3. Teacher can direct students to working on different things at different times to improve differentiation
Evaluation Questions Measures/Data Sources Results
What do you need to know? How will you find out? What were the results?
Strategy 1
Istheextraclasstimebeingused
effectively?
Howdoteachersfeelaboutthe
lengthenedperiod?
Lesson Plans/Observations
Teacher Survey
Not studied
Teachers generally felt that the lengthened period
was good for some students and bad for more. In
response the statement: I enjoy teaching on a
block schedule.
2 teachers said strongly disagree
5 teachers said disagree
1 teacher said neutral
0 teachers said agree or strongly agree
(see figure 1)
Strategy 2
HowoftenisApexusedbystudentsin
andoutofclass?
Lesson Plans/Computer
cart check out schedule
Not studied
Objective 1
Are student scores on benchmark tests Student benchmark test In one teachers classes, from 2013 to 2014 the
1
improving?
To what extent can the changes in
scores be attributed to block
scheduling or Apex?
Are students able to better learn what
they need when they miss a class?
scores
Student test scores
compared across class that
do and do not use each
method
Student survey/Attendance
and grade data
average test score on the midterm benchmark
went from a 77.4 to and 88.5. This difference was
found to be statistically significant. (p=.0045).
This was not studied, though it would be hard to
know what to attribute this growth to due to a
variety of changing factors and students not being
randomly chosen for the classes.
Not studied
Decisions (Guiding Questions: What do the results mean? What are you going to do now?)
Summary of Results (Interpretation of Data):
Based on the parts of the evaluation that were completed, there is a bit of a disconnect between how teachers feel
about the changes in the block schedule and apex tutorials, and the quantitative results when looking at a teachers
benchmark test data. Generally teachers had negative responses to the two initiatives, and yet, in at least one
classroom there seems to have been a great improvement in benchmark test scores which is exactly what the
initiative was designed to do. It is hard, though, to determine whether one, both, or neither of the initiatives is to be
given credit for the effect. There are a few other factors that may have also contributed to the changes including
slightly smaller class sizes this year and fewer students testing into math 1 in eighth grade.
Many teachers noted that the lengthened period was great for a few students and bad for many others.
It seems that behavior issues caused teachers much stress related to block scheduling and apex tutorials.
Many teachers suggested a split block where students would have the class once in the morning and once in the
afternoon instead of all at one time.
Next Steps (Action Steps):
Teachers, administrators, and councilors, should meet to discuss the type of student that does well with the
lengthened period, and the students who don't due to behavior concerns. Perhaps offering both a lengthened class
and a regular one as well as hand selecting students for each would be preferable.
Teachers and administrators should discuss the possibility of having the block sections split into a morning lesson
and an afternoon session instead of one solid block.
Before next year begins, teachers should be given high quality PD for teaching on a block schedule and be given
time to plan together and discuss what was effective in their own classrooms this past year.
2
Teachers should be shown assessment data comparisons to show the improvement for one year to the next in
hopes of increasing teacher buy in to the initiative.
Why did you do it? (Objective): 2. Reduce the number of students failing Math 1 and Math 2 and therefore number of
repeating students
What did you do?(Strategies): 1. Teach classes on a year long block schedule
2. Supplement lessons with Apex learning Tutorials
Activities:
Classes are scheduled such that they are 105 minutes long everyday and last throughout the length of an entire
school year.
1. There is more time to be able to catch up when missing school or gaps in content understanding
With the extra time teachers are to use Apex learning tutorials to supplement instruction.
1. Students work at their own pace and may watch reteaching videos or retry content areas as much as needed
2. Students will be motivated by tutorial star system and content mastery grading system
Evaluation Questions Measures/Data Sources Results
What do you need to know? How will you find out? What were the results?
Strategy 1
Howdostudentsfeelaboutthe
lengthenedperiod?
Student Survey (taken from teacher survey as no student survey
was administered) Generally teachers felt that the
majority of students do not enjoy the lengthened
period. The felt that 100 minutes is too long for
most students to be in a math classroom. When
teachers were asked to respond to the statement:
My students enjoy learning on a block schedule.
4 teachers said strongly disagree
3 teachers said disagree
1 teacher said neutral
0 teachers said agree or strongly agree
(see figure 1)
Strategy 2
DostudentslikeusingApex? Student Survey (taken from teacher survey as no student survey
was administered) Generally teachers felt that
3
Arestudentsactuallycompletingwhat
isassigned?
Teacher grade
data/Observations
students did not enjoy using apex tutorials due to
frustrations with navigation, the way some lessons
were taught, and some parts of the tutorial where
the curriculum did not match what they were
supposed to be learning. In response to the
statement: My students enjoy learning with Apex
tutorials.
2 teachers said strongly disagree
4 teachers said disagree
2 teacher said neutral
0 teachers said agree or strongly agree
(see figure 2)
Not studied
Objective 2
Are fewer students failing?
To what extent can the changes in
failing rates be attributed to block
scheduling or Apex?
Does the use of Apex tutorials mitigate
the impact poor attendance typically
has on student performance.
Passing/Failing in
gradebook
Are pass fail rates different
in the classes that use
different strategies.
Attendance data and test
scores.
Not studied
Not studied
Not studied
Decisions (Guiding Questions: What do the results mean? What are you going to do now?)
Summary of Results (Interpretation of Data):
Based on the parts of the evaluation that were completed. Teachers generally did not enjoy teaching on a block
schedule or using apex tutorials. Teachers also felt that their students did not enjoy either very much. Data was not
collected to see if fewer students were failing.
Generally teachers felt that the block schedule was too long for most students to be in a math classroom and that
their were many behavior issues that arose.
Generally teachers felt that Apex tutorials were not well enough matched to their curriculum and that this caused
4
many challenges in making use of them and caused much frustration among students.
Next Steps (Action Steps): (suggestions for the block schedule can be found under objective 1
Collect data to see if fewer students are failing.
Teachers, administrators, and perhaps employees of Apex tutorials should meet to reconcile the differences that
exist between the curriculum and the program. Perhaps the program can be changed. Perhaps curriculum
sequencing can be altered, and perhaps there is a different tutorial software that would work better.
Teachers should be provided high quality PD on and/or given time to meet with the technology facilitator and
discuss how to help students navigate Apex tutorials and keep students on task.
Why did you do it? (Objective): 3. Teachers will better differentiate instruction
What did you do?(Strategies): 2. Supplement lessons with Apex learning Tutorials
3. Provide PD on the use of Apex learning Tutorials
Activities:
Use Apex learning tutorials to supplement instruction.
1. Students may work at their own pace and retry concepts as much as they would like
2. Teacher can direct students to working on different things at different times
3. Tutorial content is taught in a variety of ways to address different learning styles
Evaluation Questions Measures/Data Sources Results
What do you need to know? How will you find out? What were the results?
Strategy 2
DostudentsfeelthatusingApexhas
beenhelpful?
Student Survey (taken from teacher survey as no student survey
was administered) There were mixed responses
as to how students did or did not benefit from apex
tutorials. Several teachers noted that students
would frequently become lost or frustrated by the
program and/or get distracted and begin doing
something else on the computer. Some teachers
would note, however, that seeing the material an
additional time in a different way was just what
some students needed. Specifically, students with
5
IEP or LEP modifications were noted as benefiting
from the program. In response to the statement: I
feel that Apex tutorials have been good for my
students.
0 teachers said strongly disagree
4 teachers said disagree
2 teacher said neutral
2 teachers said agree
0 teachers said strongly agree
(see figure 2)
Strategy 3
Do teachers feel like the training
equipped them with the adequate
knowledge to use the tool?
Do teachers implement what they
learning in the training?
Teacher Survey
Lesson plans/Observations
Generally teachers were mixed in their feelings to
how well prepared they were to teach using Apex
tutorials. Those who had experience with teaching
with computer tutorial programs in the past tended
to feel that the training was adequate. Those who
had not, tended to feel that they were under
prepared to use them. In response to the
statement: I feel that I was adequately prepared to
teach with Apex tutorials.
0 teachers said strongly disagree
2 teachers said disagree
3 teacher said neutral
2 teachers said agree
1 teachers said strongly agree
(see figure 3)
Not studied
Objective 3
To what extent do teachers feel that
Apex has helped them to differentiate
instruction?
Teacher Survey Teachers were mixed in their responses to how
helpful apex tutorials were in differentiation. Some
found them to be quite helpful while others found
the challenges of using apex tutorials to frequently
6
outweigh the benefits. In response to the
statement: I feel that Apex tutorials have helped
me to better differentiate instruction.
0 teachers said strongly disagree
4 teachers said disagree
2 teacher said neutral
2 teachers said agree
0 teachers said strongly agree
(see figure 4)
Decisions (Guiding Questions: What do the results mean? What are you going to do now?)
Summary of Results (Interpretation of Data):
There were mixed feelings regarding how helpful Apex tutorials were, how prepared they felt to use them, and how
Apex tutorials helped them in differentiation.
It seemed that the teachers who were the most experienced with using tutorial software in the classroom had the
best overall reaction.
Even the teachers that had positive reactions to Apex tutorials found that there were many places that the
curriculum did not match.
Next Steps (Action Steps):
Provide high quality and differentiated PD this summer to better prepare all teachers to use tutorial software.
Teachers newer to the concept will need more time and an in depth training on general use while teachers with
more experience will require less and can focus on specific techniques of differentiation. Perhaps more experienced
teachers can lead some of this PD.
Teachers, administrators, and the technology facilitator can meet to improve Apex tutorials usability or identify more
or different resources to aid in differentiation.
Have teachers meet throughout next year to discuss how they are using tutorial software to differentiate instruction
in their classrooms.
7
8
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0
1
2
3
4
5
6
Figure 1: Block schedule feelings
I enjoy teaching on a block
schedule.
My students enjoy learning on
a block schedule.
I feel that the block schedule
has been good for my
students.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Figure 2: Apex Tutorials feelings
I enjoy teaching with Apex
tutorials.
My students enjoy learning
with Apex tutorials.
I feel that Apex tutorials have
been good for my students.
9
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Figure 4: Differentiation
I feel that the block schedule
has helped me to better
differentiate instruction.
I feel that Apex tutorials have
helped me to better
differentiate instruction.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
0
0.5
1
1.5
2
2.5
3
3.5
Figure 3: Preparedness
I feel that I was adequately
prepared to teach on a block
schedule.
I feel that I was adequately
prepared to teach with Apex
tutorials.
10