See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/340515369

The Effect of Virtual Reality Technology on Welding Skill Performance

Article  in  Journal of Agricultural Education · April 2020

DOI: 10.5032/jae.2020.01152

CITATIONS READS
7 687

2 authors, including:

Trent Wells
Southern Arkansas University/Magnolia
17 PUBLICATIONS   66 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Technological Applications in Agricultural Education Settings View project

Agricultural Technology and Mechanical Systems (ATMS) in Agricultural Education Settings View project

All content following this page was uploaded by Trent Wells on 08 April 2020.

The user has requested enhancement of the downloaded file.

 Journal of Agricultural Education, 61(1), 152-171
https://doi.org/10.5032/jae.2020.01152

The Effect of Virtual Reality Technology on Welding Skill

Performance
Trent Wells1 and Greg Miller2

Abstract

Simulator technologies such as virtual reality (VR) can serve as practical tools in the educational
process. VR technology applications can be effectively used for weld process training. Weld process
training can often be found in university-level agricultural education settings. We sought to determine
if using a VR technology application within the context of a one-hour-long gas metal arc welding
(GMAW) process training impacted welding skill performance as determined by certified welding
inspectors (CWIs) who used a weld evaluation rubric based on American Welding Society (AWS)
standards. One-hundred-and-one students from Iowa State University participated in our study.
Participants were randomly placed into one of four protocol groups: (1) 100% live welding, (2) 100%
VR welding, (3) 50% live welding / 50% VR welding, or (4) 50% VR welding / 50% live welding. A
one-way analysis of variance (ANOVA) indicated there were no statistically significant differences (p
> .05) in total weld scores between participants in the four training protocol groups. We recommend
this study be replicated.

Keywords: agricultural mechanics; welding; skill development; virtual reality; simulation

Author Note: This paper is a product of the Iowa Agricultural and Home Economics Experiment
Station, Ames, Iowa. Project No. IOWO3813 and sponsored by Hatch Act and State of Iowa funds.

Introduction

As one of many educational technologies emerging over prior decades (Saettler, 2004),
simulator technologies for teaching and learning have been developed for and used in various contexts,
including: (1) medicine (Cope & Fenton-Lee, 2008; Gallagher et al., 2003; Gor, McCloy, Stone, &
Smith, 2003; Kilmon, Brown, Ghosh, & Mikitiuk, 2010; Kneebone, 2005; Seymour et al., 2002), (2)
mine safety training (Filigenzi, Orr, & Ruff, 2000), (3) welding (Abrams, Schow, & Riedel, 1974; Byrd,
2014; Byrd, Stone, Anderson, & Woltjer, 2015; Oz, Ayar, Serttas, Iyibilgin, Soy, & Cit, 2012; Stone,
McLaurin, Zhong, & Watts, 2013; Stone, Watts, Zhong, & Wei, 2011; White, Prachyabrued, Chambers,
Borst, & Reinders, 2011), (4) education (Agnew & Shinn, 1990; Nadolny, Woolfrey, Pierlott, & Kahn,
2013; Perritt, 1984), and (5) first responder training (Bliss, Tidwell, & Guest, 1997). Thiagarajan (1998)
described a simulation as “a representation of the features and behaviors of one system through the use
of another” (p. 35). As simulator technologies have become more widespread and will continue to
evolve over time to fulfill different roles (Thiagarajan, 1998), their beneficence and effectiveness are
expected to be quite high (Kneebone, 2005).

Using simulator technologies for teaching and learning purposes, such as training a welder how
to perform a specific weld joint configuration, can potentially positively impact skill development
(Nikolic, Radivojevic, Djordjevic, & Milutinovic, 2009; Scalese, Obeso, & Issenberg, 2008). Further,
using simulator technologies as part of the teaching and learning processes can help reduce anxiety
1
Trent Wells is an Assistant Professor of Agricultural Education in the Department of Agriculture at Southern
Arkansas University, 100 E. University, Magnolia, AR 71753, [email protected]
2
Greg Miller is a Professor of Agricultural Education in the Department of Agricultural Education and Studies
at Iowa State University, 217B Curtiss Hall, Ames, IA 50011, [email protected]

Journal of Agricultural Education 152 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

when learning within a new skill domain (Byrd, 2014). Various technologies such as augmented reality
(AR) (Lee, 2012; Yuen, Yaoyuneyong, & Johnson, 2011) and virtual reality (VR) (Youngblut, 1998)
can be used to provide for a diversity of simulator technologies (Scalese et al., 2008). However, as
Wenglinsky (1998) advised, the use of an educational technology alone does not necessarily result in
increased achievement. Rather, using educational technology to facilitate teaching and learning, such
as using a VR-based simulator technology to develop a skill set, should be done pragmatically and
deliberately (Wenglinksy, 1998). As such, the selection and appropriate use of educational technology
for a teaching and learning purpose should be conscientious.

The Virtual Reality Society (2017) described VR as “a three-dimensional, computer-generated

environment which can be explored and interacted with by a person” (¶ 5). Scholars (Bailenson, 2018;
Pantelidis, 1993) speculated VR technology could have much potential to assist with teaching and
learning procedures in the coming years. Helsel (1992) echoed this sentiment by stating “[v]irtual
reality holds much promise for education…[and] education has a tremendous wealth of information
and experience to bring to the VR curriculum” (p. 42). Since Pantelidis’s (1993) and Helsel’s (1992)
work, VR has continued to become further entrenched in educational settings. Bailenson (2018)
postulated that as a result of rapid technological changes, VR technology holds much potential to
continue impacting teaching and learning processes in the coming years.

Potkonjak et al. (2016) proposed integrating VR technology into science-, technology-, and
math-based content could serve as a practical educational intervention to help students acquire content-
specific knowledge and skills more efficiently and effectively. Moreover, Byrd (2014) and Stone et al.
(2011) indicated using VR technology applications could be pragmatic and effective for developing
welding-related psychomotor skills. As weld process training is often a component of agricultural
education programs at the university level (Burris, Robinson, & Terry, 2005), perhaps this approach
could be beneficial for developing psychomotor skills in the context of weld process training in
university-level agricultural education.

Psychomotor skills can be described as a linkage between various cognitive and physical
processes requiring physical motions and mental stimulation to successfully accomplish their objectives
(Lancelot, 1944; Venes, 2017). Psychomotor skills could include such tasks as operating power
machinery, performing open-heart surgery, or completing a hand-drawn sketch. Different career fields
require individuals to make certain hand and body motions to perform daily tasks. For example, in the
context of medical science, psychomotor skills are particularly important for surgical practitioners, as
fine movements must be made to ensure safe and effective surgical practice and the health of the patient
(Gallagher et al., 2003; Kaufman, Wiegand, & Tunick, 1987). In welding, a career area traditionally
within the scope of agricultural education (Burris et al., 2005; Pate, Warnick, & Meyers, 2012),
individuals use psychomotor skills to manipulate and maintain control over a molten weld puddle to
complete various tasks (Bowditch, Bowditch, & Bowditch, 2017; Byrd, 2014).

In the application of psychomotor skills to a given setting, human-related factors such as prior
experience and comfort, dexterity, and anxiety can influence performance (Byrd, 2014). Referencing
the development of psychomotor skills within university-level agricultural education settings, Osborne
(1986) noted the psychomotor skill development process itself could be complex and challenging yet
educationally rewarding. Wulf (2007) noted developing motor skills for use in a certain context, such
as the psychomotor skills used during welding activities, can take a considerable amount of time, effort,
feedback, and continued practice to refine the appropriate skills. As noted by Bowditch et al. (2017),
psychomotor skills used by welders can be quite varied but typically are related to hand-eye
coordination, hand and arm movements, and body positioning.

Journal of Agricultural Education 153 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Using effective strategies to enhance the acquisition of abilities in a subject area is of critical
importance for continued success in the teaching and learning of skills and knowledge (Goldsmith,
Stewart, & Ferguson, 2006). Phipps, Osborne, Dyer, and Ball (2008) noted technology usage is often
an important part of the teaching and learning processes in agricultural education settings. Moreover,
psychomotor skill development is often an objective of many aspects of agricultural education
(Lancelot, 1944; Osborne, 1986; Phipps et al., 2008). Considering these concepts, perhaps using a VR
technology application to develop psychomotor skills in the context of a university-level agricultural
education setting could yield practical results. Stone et al. (2011) indicated individuals who used VR
technology for welding skill development purposes performed comparably to, and in some cases
superior to, individuals who underwent traditional weld training procedures. Thus, the potential
advantages of using VR technology for weld training purposes become clearer.

Byrd (2014) suggested “VR gives participants the capability to hone task-related abilities” (p.
63). Like Stone et al. (2011), Byrd (2014) used weld process training protocols spanning several days,
thus immersing his study’s participants in a somewhat lengthy duration of training exposure. To date,
while several studies engaging participants over longer time frames have been conducted, limited data
exist describing the effects of short-duration training procedures on individuals’ welding skill
performance. What effects would a VR technology training approach conducted over a short time span
have on weld process skill performance?

Theoretical Framework

Skill acquisition theory (DeKeyser, 2015) underpinned our study. Skill acquisition theory
describes the development of skills through three stages: “declarative, procedural, and automatic”
(DeKeyser, 2015, p. 95). The first step in the skill acquisition process is the understanding of the skill
itself and the procedures to be performed, otherwise referred to as declarative knowledge. Declarative
knowledge can be obtained in several ways, including reading about a skill performance process,
observing someone else perform the skill, or watching a demonstration video. Transforming declarative
knowledge into procedural knowledge is the focus of the second step. During this step, which is often
rapid in nature, the application of the basic knowledge and understanding about a concept begins via
practice. Practice usually lasts for an extended time and is focused on improving time to completion,
accuracy, and gradually reducing the cognitive activity required to successfully complete the task.

Adequate practice is designed to help guide an individual to the third stage, automaticity
(DeKeyser, 2015). As an individual continues to move toward automaticity, he or she is expected to
continue improving until accuracy is high, task completion time is minimal in comparison to when the
task was first being practiced, and the cognition needed for successfully addressing the skilled task has
been minimized and is often inherent.

As focus is an important part of the skill acquisition process (Wulf, 2007), someone who is
practicing a skill can alter his or her focus as automaticity is reached. For example, a person learning
how to perform a 1G butt weld with the gas metal arc welding (GMAW) process while using a VR
technology application may use visual cues provided on a heads-up display screen to help him or her
understand how to properly apply travel angle theory to a virtual weld. While the skill is being
understood and applied early in the practice process (i.e., entering from the declarative stage to the
procedural stage), he or she may focus a great deal of time looking at the visual cue instead of the
virtual weld. Over time, his or her focus point may change over to the virtual weld bead being produced
instead of the visual cue, perhaps even removing the visual cue entirely once adequate practice with the
skill has been completed and he or she moves into the automaticity stage. As noted by several scholars
(DeKeyser, 2015; Lancelot, 1944; Osborne, 1986; Wulf, 2007), learning how to successfully acquire
and apply skills is a process requiring physical and mental exertion to achieve success.

Journal of Agricultural Education 154 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

When examining the application of skill acquisition theory into our study, our primary focus
was on welding skill application and performance in the context of a structured one-hour GMAW
process training session conducted by the lead researcher. We sought to apply the theoretical process
of skill acquisition (DeKeyser, 2015) into each of the structured training protocols we used. Four
different training protocols incorporating varying levels of VR technology application usage were
applied in our study. We conducted the training protocols with participants from different backgrounds
and varying levels of prior experiences with the GMAW process. Through this approach, we were able
to apply the different stages of skill acquisition (i.e., declarative, procedural, and automatic) into all
four training protocols and work directly with participants during each stage.

The declarative stage occurred through the lead researcher’s verbal instruction and visual
demonstration of the skill of focus, which was producing 2F tee joint welds using the GMAW process.
Each participant then entered the procedural stage and was allowed solo, semi-supervised practice for
a designated amount of time. The amount of time for practice varied based upon the training protocol
each participant was randomly assigned to. The objective of the practice session was to provide
adequate time for basic skill conceptualization and application to occur and allow for procession into
the automaticity stage. Afterward, a brief testing session occurred for us to collect physical weld data
for our study. As such, we used VR technology as an intervention technique within the skill acquisition
process. Our chief interest was determining if using differing applications of a VR technology
application for weld process training purposes had any impact on welding skill performance.

Purpose

The purpose of our study was to determine the effects of differing applications of VR
technology usage on welding skill performance during a one-hour-long GMAW process training
session. Our study aligned with the American Association for Agricultural Education National
Research Agenda Research Priority Area 3: Sufficient Scientific and Professional Workforce That
Addresses the Challenges of the 21st Century (Stripling & Ricketts, 2016). As the agricultural industry
continues to change (Doerfert, 2011), a workforce capable of addressing the challenges and needs of
the future will be vital (Stripling & Ricketts, 2016).

To help prepare future members of the agricultural industry workforce, such as school-based
agricultural education (SBAE) teachers, effective and appropriate instructional practices should be used
to ensure adequate and useful skills, such as GMAW process theory and welding equipment use, are
being developed and transferred. We postulated technology-based applications may be capable of
playing a pragmatic role in this process. Lindner, Rodriguez, Strong, Jones, and Layfield (2016) noted
technology is evolving and should be used to help solve problems facing agriculture currently and in
the future. As Stone et al. (2011) indicated, using a VR technology application could be a practical
method to help address workforce development needs.

Research Hypothesis

H1: Using VR technology to facilitate the development of welding skills to complete 2F tee
joints will result in a significant impact on total weld scores.

Methods and Procedures

Recruitment Procedures and Participant Information

Journal of Agricultural Education 155 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Our study was conducted during the Fall 2018 semester and consisted of undergraduate and
graduate students enrolled at Iowa State University (ISU). After the ISU Institutional Review Board
(IRB) approved our study, we invited all students majoring in Agricultural and Life Sciences Education
(n = 186), Agricultural Studies (n = 338), Agricultural Education and Studies (n = 70), Agricultural
Engineering (n = 229), Agricultural and Biosystems Engineering (n = 84), Agricultural Systems
Technology (n = 170), Industrial Technology (n = 290), and Mechanical Engineering (n = 2,424) to
participate via e-mail. We invited the students in these academic majors based on the lead researcher’s
experiences with students’ interests in a welding-focused agricultural mechanics course taught by him.
In addition to the e-mail sent to 3,791 students, the lead researcher used in-class announcements to
recruit the 38 students enrolled in the two agricultural mechanics courses he taught. The students in
these two courses were part of the group of 3,791 ISU students who had been sent the invitational e-
mail.

Within the invitational e-mail and the in-class announcements, students were asked to use a
Doodle poll to select one participation time slot best suited their personal schedules. To help incentivize
students to participate, we offered a chance to win one of four $40.00 gift cards. Reminder e-mails and
text messages were sent to students one day in advance of their scheduled participation time slot.
Students were also invited to share information about the study with anyone they believed may be
interested in participating regardless of their academic major. Students were notified via the invitational
e-mail and in-class announcements they needed to wear clothing appropriate for welding activities.

We developed a paper-based questionnaire to collect participants’ demographics data. The

questionnaire included eight questions related to participants’ gender, age, academic standing,
academic major, prior welding experiences and welding simulator use, and daily video game playing
time. To ensure wording clarity from the perspectives of individuals within our population of interest,
we had three students in an undergraduate-level agricultural mechanics course evaluate the
questionnaire. They suggested a couple minor question wording changes to improve clarity.

We added one question related to individuals’ dominant hand usage and three questions related
to participants’ prior welding experiences to better understand how participants’ welding experience
may impact their welding skill performance. We had three other students in an undergraduate-level
agricultural mechanics course evaluate the questionnaire to ensure each item was worded clearly. They
each reported all the items on the questionnaire were worded clearly.

Research Design

Our study used a randomized posttest-only experimental research design (Campbell & Stanley,
1963; Figure 1). The posttest in our design was participants’ test welds.

Producing Horizontal 2F Tee Welds Using the GMAW Process

R X1 O
R X2 O
R X3 O
R X4 O
Figure 1. Illustration of randomized posttest-only experimental design. R = random assignment; O
= observation of test weld produced; X1 = 100% live welding; X2 = 100% VR welding; X3 = 50%
live welding / 50% VR welding; X4 = 50% VR welding / 50% live welding

Posttest-only designs are suitable for rigorous educational research and help control for a wide
range of threats to internal and external validity (Campbell & Stanley, 1963). Our use of random
assignment, short duration training protocols, conducting one-on-one training with each participant,

Journal of Agricultural Education 156 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

detailed scripts for each training protocol group, a valid and reliable weld evaluation rubric, and
recruitment of all individuals from different majors controlled for this range of internal and external
threats to validity.

As noted by Campbell and Stanley (1963), using a pretest can confound research results. Had
we used a research design that incorporated a pretest of welding skill performance, pretest sensitization
as a threat to internal validity and testing as a threat to external validity could have resulted. Our
selection of a posttest-only design helped us to avoid these issues.

Instrumentation

Participants’ test welds were visually inspected and evaluated by American Welding Society
(AWS)-credentialed CWIs. The CWIs used a weld evaluation rubric co-developed by the lead
researcher and a CWI who did not evaluate welds for this study. The rubric was based on the AWS
D1.1 Table 6.1 visual inspection criteria for statically-loaded connections and consisted of visual
inspection criteria for weld cracks, the occurrence of porosity, completeness of fusion, both leg and
throat fillet sizes, the presence of undercut, crater cross-section, and the weld profile. A maximum
possible score of 100 points could be achieved using this rubric. The rubric is illustrated in Figure 2.

Journal of Agricultural Education 157 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Figure 2. Weld evaluation rubric.

After the rubric was developed, a panel of five experts was used to assess face validity and
content validity. The five experts consisted of two agricultural teacher education faculty members who
had taught school-based and university-level agricultural mechanics coursework including welding for
at least 10 years and three AWS-credentialed CWIs who had each been actively engaged in the welding
industry in various capacities over the last decade. The rubric and a panel of experts guidelines form
was submitted to each panel member for an initial review.

During the review process, each panel member provided corrective feedback on both the rubric
and the guidelines form. After all the panel members responded, the lead researcher made the

Journal of Agricultural Education 158 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

recommended changes and re-submitted the rubric and the panel of experts guidelines form back to
each panel member for a second review. During the second review, all five panel members agreed the
rubric was face valid and content valid and suitable for use in the study.

After the rubric was determined to be valid and suitable for use, we conducted a pilot study
during the Summer 2018 semester. The pilot study consisted of 20 undergraduate and graduate students
from ISU and was intended to help us identify and correct any issues associated with the study’s design
and provide test weld data useful for determining the reliability of the rubric. We used Statistical
Package for the Social Sciences (SPSS©) Version 24 software to help us randomly assign 40 different
participation slots. While we originally planned to have 40 participants in the pilot study, only 20
students participated. The random assignment generated by using SPSS resulted in unequally-sized
training protocol groups. Consequently, we opted to use another method during the formal study’s
implementation in the Fall 2018 semester. The four protocol groups described in Figure 1 were used
during the pilot study; participants in each group completed three test welds and chose one test weld to
be evaluated by five CWIs in Iowa who volunteered to evaluate all 20 test welds from the pilot study.

Two of the five CWIs were community college-level welding program instructors while the
other three CWIs were employed at a large commercial equipment manufacturing company. To
determine the internal consistency of the rubric during the pilot study, we used only first-round data,
which yielded a Cronbach’s alpha coefficient of .714 based on standardized items. Basing the
Cronbach’s alpha coefficient on the standardization of items was used due to the application of differing
rating levels to evaluate welds. Cohen et al. (2011) noted standardization allows for comparison
between items comprised of different scales. Using the Cronbach’s alpha interpretations offered by
George and Mallery (2003), we determined this rubric had an acceptable level of internal consistency.

We repeated these procedures with the data collected during the formal study conducted during
the Fall 2018 semester. Due to scheduling conflicts, only three CWIs who evaluated test welds during
the pilot study were able to evaluate test welds during the formal study. During this second
determination of internal consistency, the three CWIs evaluated all 101 test welds completed by the
formal study participants. Data collected from the first-round evaluation yielded a Cronbach’s alpha
coefficient of .860 based on standardized items and was also determined to be adequate using the
standards set forth by George and Mallery (2003).

Following the test-re-test reliability method noted by Cohen, Manion, and Morrison (2011),
the 20 test welds from the pilot study were twice evaluated independently by each of the five CWIs.
The first and second evaluations were conducted at least one week apart. We used intraclass correlation
coefficients (ICC) with 95% confidence intervals (CI) to determine both intra- and interrater reliability
of each CWI who evaluated the test welds from the pilot and formal studies. ICCs can be used to
determine the reliability of scales used in quantitative research designs (Fleiss & Cohen, 1973; Koo &
Li, 2016). As noted by LeBreton and Senter (2008), “[e]stimates of IRR [interrater reliability] are used
to address whether judges rank order targets in a manner that is relatively consistent with other judges”
(p. 816). In contrast, intrarater reliability “is estimated by having one rater score the same instrument
on two different occasions” (Scholtes, Terwee, & Poolman, 2011, p. 237).

Five CWIs independently evaluated the 20 test welds produced during the pilot study on two
separate occasions at least one week apart. Intrarater reliabilities were calculated using both the first
and second rounds of data while the interrater reliability was determined by using only the first-round
data. Intrarater reliabilities were as follows for CWIs one through five, respectively: .710, .951, .785,
.827, and .814. Interrater reliability between all five CWIs was determined to be .926. Regarding
interpretation, Koo and Li (2016) noted, “ICC values less than 0.5 are indicative of poor reliability,
values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good

Journal of Agricultural Education 159 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

reliability, and values greater than 0.90 indicate excellent reliability” (p. 158). These procedures were
repeated with the test welds produced during the formal study. Intrarater reliabilities were as follows
for CWIs one through three, respectively: .810, .670, and .819. Interrater reliability between all three
CWIs was determined to be .863.

Procedures

All research activities took place at the ISU Agricultural Mechanics Teaching Laboratory. Prior
to the implementation of the formal study, the lead researcher and another doctoral student not affiliated
with our study created a chart to determine random assignment sequencing. Due to the lead researcher’s
time constraints, we determined the creation of 140 different hour-long participation time slots would
be suitable for our research design. Four colored paper clips were drawn at random from a plastic cup
35 times to fill 140 participation time slots, thus ensuring each color was drawn 35 times to help create
four equally-sized protocol groups. The color of each paper clip corresponded to a designated group.
Random assignment was based on order of appearance, thus eliminating random assignment
sequencing errors if a student did not attend his or her designated participation time slot. For example,
if seven students were scheduled to participate over the course of a given day and only three students
participated, the random assignment designations were not impacted by the four students who did not
participate as the group number corresponded to the order of appearance.

The formal study was conducted on Tuesdays, Thursdays, and Fridays over several weeks
during the Fall 2018 semester. Each day, seven one-hour participation time slots were available with a
15-minute interval between each one. The 15-minute interval served as a time to reset welding
equipment, clean the workstations, and prepare for the next participant. Upon arrival, participants were
asked to complete a 12-item demographics questionnaire and were provided with welding personal
protective equipment (PPE), which included welding gloves, a welding helmet, a welding jacket, ear
plugs, and safety glasses. Individuals who did not wear clothing appropriate for welding activities were
not allowed to participate in the study activities and were asked to re-schedule their participation on the
Doodle poll.

Four different GMAW weld process training protocols were used. Each protocol lasted for
approximately one hour. Protocol one was a 100% live welding approach and included 25 participants.
Protocol two was a 100% VR welding approach and included 26 participants. Protocol three was a
blended 50% live welding and 50% VR welding approach and included 25 participants. Protocol four
was a blended 50% VR welding and 50% live welding approach and included 25 participants. Specific
information about each training protocol is depicted in Table 1.

Table 1
Weld Process Training Protocols
Protocol Number Protocol Descriptor Protocol Steps (Time Allowance)
1 100% live welding Informed consent letter reading
and signing, demographics
questionnaire completion
(10 minutes)

Researcher live weld

demonstration (Five minutes)

Participant live weld practice (30

minutes

Journal of Agricultural Education 160 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Live test weld production and

selection (Five minutes)

2 100% VR welding Informed consent letter reading

and signing, demographics
questionnaire completion
(10 minutes)

Researcher virtual weld

demonstration (Five minutes)
Participant virtual weld practice
(30 minutes)

Researcher live weld

demonstration (Five minutes)

Live test weld production and

selection (Five minutes)

3 50% live welding / 50% VR welding Informed consent letter reading

and signing, demographics
questionnaire completion
(10 minutes)

Researcher live weld

demonstration (Five minutes)

Participant live weld practice (15

minutes)

Researcher virtual weld

demonstration (Five minutes)

Participant virtual weld practice

(15 minutes)

Live test weld production and

selection (Five minutes)

4 50% VR welding / 50% live welding Informed consent letter reading

and signing, demographics
questionnaire completion
(10 minutes)

Researcher virtual weld

demonstration (Five minutes)

Participant virtual weld practice

(15 minutes)

Journal of Agricultural Education 161 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Researcher live weld

demonstration (Five minutes)

Participant live weld practice (15

minutes)

Live test weld production and

selection (Five minutes)

The lead researcher used a detailed written script to guide each training protocol. When going
through each protocol’s script with a participant, the lead researcher provided technical details about
the procedures he / she was expected to undergo and performed demonstrations of how to use the
welding equipment to perform the 2F tee joint welds. Participants were provided with an unlimited
supply of quarter-inch-thick mild steel plates to practice their welds and were allowed to ask the lead
researcher questions when needed.

Participants who underwent protocol groups one, three, and four used the same Miller® XMT®
350 CC / CV Multiprocess Welder to practice their welds. Participants in protocol groups two, three,
and four used the same Lincoln Electric® VRTEX® 360 to practice their virtual welds. Participants in
these training protocol groups were not allowed to use the VR welding system’s visual cues function
but were instead provided with system-scored visual post-weld feedback for different weld technique
parameters (e.g., contact-to-work distance [CTWD], travel speed, work angle, etc.). These feedback
data were displayed on a computer monitor attached to the system. Due to a technical issue with the
Lincoln Electric® VRTEX® 360 used in our study, participants in protocol groups two, three, and four
were not able to hear any sounds associated with the VR welding process. We used a secure digital
storage system to collect and save the virtual weld data produced by participants who underwent these
three training protocols. The virtual weld data were not reported in this manuscript.

At the end of each training protocol, each participant was provided with six, quarter-inch-thick
mild steel plates to produce three test welds using a Miller® XMT® 350 CC / CV Multiprocess Welder
and subsequently selected the single best weld for evaluation by the CWIs. Participants did not use the
same Miller® XMT® 350 CC / CV Multiprocess Welder to perform both their practice welds and their
three test welds. All participants did use the same Miller® XMT® 350 CC / CV Multiprocess Welder to
produce their three test welds. Each participant’s self-selected best weld was cooled, marked with an
identification code, and stored in a locked location accessible only by the lead researcher. The other
two welds were placed into a designated location in the ISU Agricultural Mechanics Teaching
Laboratory and were saved for metal recycling purposes.

Upon the formal study’s conclusion, a debriefing e-mail disclosing the study’s experimental
design was sent to all pilot and formal study participants. After the formal study concluded, all 101
formal study participants’ test welds were independently evaluated by three CWIs using the weld
evaluation rubric previously described. When one week had passed, all 101 test welds were re-evaluated
by the three CWIs. All evaluation data were compiled into an IBM SPSS Version 24.0 software data
set and analyzed.

Data Analysis

To test our research hypothesis, we used a one-way analysis of variance (ANOVA) to compare
the weld process training protocol groups’ mean total weld scores. Our dependent variable was total
weld score and our independent variable was the type of weld training protocol used. Prior to analyzing
our data, we averaged the total weld scores from all three CWIs. We used Omega squared (ω2) to

Journal of Agricultural Education 162 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

calculate effect size. ANOVA and Omega squared (ω2) are often used in tandem within quantitative
agricultural education research (Kotrlik, Williams, & Jabor, 2011).

Results

One-hundred-and-one participants provided the data reported in this manuscript. Most

participants were male (f = 87; 86.1%). The average age of the participants was 22.05 years (SD =
4.89). Most participants were right-hand dominant (f = 88; 87.1%). Nearly one-third of participants
were seniors (f = 32; 31.7%). Over half of the participants were majoring in Mechanical Engineering (f
= 54; 53.5%). On average, the participants spent 0.96 hours daily (Mdn = 0.5, Md = 0.0, SD = 1.39)
playing video games. Specific details about the participants’ demographics are provided in Table 2.

Table 2
Participant Demographics (n = 101)
Item f %
What is your gender?
Male 87 86.1
Female 14 13.9
What is your age?
18-21 64 63.4
22-25 25 24.8
26-29 5 5.0
30-33 2 2.0
34-37 2 2.0
38+ 3 3.0
Which hand is your dominant hand?
Left hand 13 12.9
Right hand 88 87.1
Please indicate your current academic standing.
Freshman 16 15.8
Sophomore 14 13.9
Junior 24 23.8
Senior 32 31.7
Graduate 15 14.9
What is your academic major?
Mechanical Engineering 54 53.5
Agricultural and Life Sciences Education 10 9.9
Industrial Technology 9 8.9
Agricultural Engineering 7 6.9
Agricultural Studies 7 6.9
Agricultural Education and Studies 4 4.0
Agricultural and Biosystems Engineering 4 4.0
Agricultural Systems Technology 3 3.0
Industrial and Agricultural Technology 1 1.0
Crop Production and Physiology 1 1.0
Aerospace Engineering 1 1.0

Journal of Agricultural Education 163 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Table 2
Participant Demographics (n = 101) Continued…
Estimate the average number of hours that you spend per day
playing video (e.g., computer, console, mobile, etc.) games.
0-1 hours 71 70.3
1-2 hours 17 16.8
2-3 hours 11 10.9
3+ hours 2 1.9

The participants’ welding experiences prior to engaging in our study are reported in Table 3.
Most participants had not used a welding simulation / simulator system before (f = 79; 78.2%) and
indicated they had welding experience prior to participating in our study (f = 70; 69.3%). Those who
had welding experience prior to participating indicated they had learned to weld, and had practiced
their welding skills, in a variety of settings. The participants most frequently reported prior experience
in the shielded metal arc welding (SMAW; f = 58; 57.4%) and the GMAW (f = 60; 59.4%) processes.
Regarding hours of experience with each process, on average participants reported they had the most
hours of experience with: the (1) flux-cored arc welding (FCAW) process (M = 79.29, SD = 210.31),
followed by the (2) GMAW process (M = 67.52, SD = 239.80), the (3) SMAW process (M = 62.54,
SD = 242.70), the (4) gas tungsten arc welding (GTAW) process (M = 35.18, SD = 83.98), the (5) oxy-
fuel welding (OFW) process (M = 10.19, SD = 9.98), and the (6) submerged arc welding (SAW) process
(M = 0.72, SD = 0.41).

Table 3
Formal Study Participants’ Prior Welding Experiences (n = 101)
Item f %
Have you ever used a welding simulation / simulator system
(e.g., virtual reality, augmented reality, etc.) before?
Yes 22 21.8
No 79 78.2
Have you ever welded before?
Yes 70 69.3
No 31 30.7
If you have welded before, where have you learned how to weld?
At my family’s farm or business 24 23.8
At a farm or business not owned by my family 14 13.9
In a facility at my house (e.g., garage, workshop, etc.) 19 18.8
In my high school’s Agricultural Education program 13 12.9
In my high school’s Industrial Technology program 27 26.7
Other location 28 27.7
If you have welded before, where have you gotten the opportunity to weld
or practice welding?
At my family’s farm or business 23 22.8
At a farm or business not owned by my family 18 17.8
In a facility at my house (e.g., garage, workshop, etc.) 21 20.8
In my high school’s Agricultural Education program 14 13.9
In my high school’s Industrial Technology program 26 25.7
Other location 23 22.8

Journal of Agricultural Education 164 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Table 3
Formal Study Participants’ Prior Welding Experiences (n = 101) Continued…
If you have welded before, which of the following weld processes have you
performed?
Shielded metal arc welding (SMAW; “stick welding”) 58 57.4
Gas metal arc welding (GMAW; “MIG”; “wire welding”) 60 59.4
Flux-cored arc welding (FCAW) 14 13.9
Submerged arc welding (SAW) 5 5.0
Oxy-fuel welding (OFW) 23 22.8
Gas tungsten arc welding (GTAW; “TIG”) 27 26.7

We assumed because participants were randomly placed into one of the four protocol groups
used in our study, any participants’ preexisting differences, such as prior welding experiences,
familiarity using welding simulation / simulator systems, and so forth, would fall within the range of
anticipated statistical variation and would not confound our results.

Table 4 reported descriptive statistics for each weld process training protocol group. The mean
total weld score across all four protocol groups was 74.69 with a standard deviation of 17.61. The
highest mean total weld score (M = 80.15, SD = 15.07) was from participants who underwent the 100%
VR welding training protocol (n = 26) while the lowest mean total weld score (M = 67.84, SD = 16.26)
was from participants who experienced the 50% live welding / 50% VR welding training protocol (n =
25).

Table 4
Descriptive Statistics for Participants’ Total Weld Scores by Group
95% Confidence
Interval for Mean
Group n M SD SE Min. Max. Lower Upper
100% live welding 25 74.40 17.09 3.42 36.00 98.00 67.35 81.45
100 % VR welding 26 80.15 15.07 2.96 39.00 98.00 74.07 86.24
50% live welding / 50% VR 25 67.84 16.26 3.25 28.00 100.00 61.13 74.55
welding
50% VR welding / 50% live 25 76.16 20.40 4.08 31.00 99.00 67.74 84.58
welding
Total 101 74.69 17.61 1.75 28.00 100.00 71.22 78.17

Research Hypothesis: Using VR Technology to Facilitate the Development of Welding Skills to

Complete 2F Tee Joints Will Result in a Significant Impact on Total Weld Scores

A Levene’s test for homogeneity of variances indicated the assumption of homogeneity was
met (p = .580). Our use of a one-way ANOVA revealed there were no statistically significant
differences (p > .05) in total weld scores between any of the four training protocol groups, F(3, 97) =
2.235, p = .089. Effect size was calculated using Omega squared (ω2 = 0.04), which was classified as
“very small” in accordance with the interpretations offered by Sawilowsky (2009, p. 599). Therefore,
we rejected our research hypothesis (Table 5).

Journal of Agricultural Education 165 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Table 5
Comparative Analysis of Total Weld Scores by Group Means
SS df MS F p
Between Groups 2005.381 3 668.460 2.235 .089
Within Groups 29006.105 97 299.032
Total 31011.485 100
Note. ω2 = 0.04

Conclusions, Discussion, Recommendations, Implications, & Limitations

Our results indicated using VR technology within a one-hour-long GMAW process training
session to facilitate development of welding skills to complete 2F tee joint welds did not result in a
statistically significant (p > .05) impact on welding skill performance as determined by CWIs who used
a weld evaluation rubric to determine total weld scores. As such, we concluded using VR technology
as part of this process neither improved upon nor detracted from participants’ total weld scores. The
individuals who participated in the 100% VR welding group (n = 26) had the highest mean total scores
for their test welds (M = 80.15, SD = 15.07). The individuals who participated in the 50% live welding
/ 50% VR welding group (n = 25) had the lowest mean scores for their test welds (M = 67.84, SD =
16.26). Perhaps the sequencing of the VR technology usage impacted performance. Researchers should
examine sequencing of VR technology use in future studies. Determining a suitable time frame and
method for introducing VR technology into the skill acquisition process could be impactful for skill
training purposes.

We recommend our study be replicated. We do wish to emphasize both total participant

quantities and group sizes should be increased to provide increased power for statistical testing. Perhaps
even reducing the number of protocol groups used in future research to increase the size of each group
could be beneficial. In terms of participant recruitment, the primary issue we experienced was our
research site was located off-campus and may not have been easily accessible for some prospective
participants, thus reducing our total number of participants. In several instances, prospective
participants contacted the lead researcher directly via e-mail or text message to notify him they were
unaware of the research site’s off-campus location and did not have suitable transportation there. Thus,
we recommend future studies be conducted at a central location easily accessible by all prospective
participants.

Considering the design of our study, we were left to question if our results would have been
different had the participants undergone training protocols spanning a longer time frame. Scholars
(DeKeyser, 2015; Lancelot, 1944; Osborne, 1986; Wulf, 2007) have previously noted the development
of skills can take time to fully materialize. Byrd (2014) used weld training protocols that were at least
one week in duration is his study. It is conceivable that had we extended the duration of our training
protocols to perhaps one eight-hour-long day, we may have found significant differences in total weld
scores between the protocol groups. However, it is also possible that increasing the length of time
individuals were asked to participate to our study may have resulted in increased subject mortality, thus
introducing an internal validity issue (Campbell & Stanley, 1963). We recommend future researchers
of welding skill performance carefully consider the durations of their studies and the effects of time on
participants’ skill acquisition processes.

We elected to focus our study on welding skill performance in the context of GMAW process
training. GMAW has fewer operator variables than other welding processes and can allow for quicker
skill acquisition for novice welders (Rose, Pate, Lawver, Warnick, & Dai, 2015). Further, we selected
2F tee welds due to their simplicity in comparison to other weld configurations (Stone et al., 2013). In

Journal of Agricultural Education 166 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

addition, participants who were assigned to the training protocols using VR welding were not allowed
to use visual cues as part of their practice sessions. Stone et al. (2013) noted using visual cues can be
useful in some instances and harmful in others. As such, we question if our results would have been
different had we elected to implement a different welding process such as SMAW, selected a different
weld configuration, or had allowed participants to use visual cues during VR welding practice. Further
research focusing on each of these variables could provide useful information relevant to welding
industry stakeholders.

Considering the majority of our study’s participants (n = 70; 69.3%) reported they had welding
experience prior to participating in our study, we speculate this may have been a factor contributing to
our lack of identifying statistically significant results. To help better control for this variable in future
studies, we recommend researchers consider conducting experimental or quasi-experimental research
with groups of novices. Large groups of novices could be found in numerous settings, including SBAE
programs or university-level agricultural mechanics coursework. Moreover, if conducted within the
scope of a secondary- or university-level course, future studies could examine the impacts of using VR
technology over a longer timeframe, such as an entire semester or academic year, while helping to
minimize participant attrition.

Regarding stakeholders, we recommend professionals involved in welding education, such as

agricultural education practitioners, continue to examine how educational technology-based practices
can assist in the teaching and learning of psychomotor skills. Stone et al. (2011) indicated using VR
technology for weld process training purposes shows promise for preparing skilled welders. While our
study did not indicate VR technology use made a statistically significant impact, follow-up research
should be conducted to help determine if such technology should be considered as viable in the
acquisition of welding-related psychomotor skills. Welding plays a considerable, traditional role in
agricultural education, particularly at the secondary and university levels (Burris et al., 2005) and is
included in many career areas in the agricultural industry. The agricultural industry is ever-changing
(Doerfert, 2011) and must continue to critically examine the roles and impacts new technologies and
products adoption can play (Lindner et al., 2016).

References

Abrams, M. L., Schow, H. B., & Riedel, J. A. (1974). Acquisition of a psychomotor skill using
simulated-task, augmented feedback (evaluation of a welding training simulator). Retrieved
from http://www.dtic.mil/dtic/tr/fulltext/u2/a000818.pdf

Agnew, D. M., & Shinn, G. C. (1990). Effects of simulation on cognitive achievement in agriculture
mechanics. Journal of Agricultural Education, 31(2), 12-16. doi:10.5032/jae.1990.02012

Bailenson, J. (2018). Experience on demand: What virtual reality is, how it works, and what it can
do. New York, NY: W. W. Norton & Company, Inc.

Bliss, J. P., Tidwell, P. D., & Guest, M. A. (1997). The effectiveness of virtual reality for
administering spatial navigation training to firefighters. Presence: Teleoperators and Virtual
Environments, 6(1), 73-86. doi:10.1162/pres.1997.6.1.73

Bowditch, W. A., Bowditch, K. E., & Bowditch, M. A. (2017). Welding fundamentals (5th ed.). Tinley
Park, IL: The Goodheart-Willcox Company, Inc.

Burris, S., Robinson, J. S., & Terry, R., Jr. (2005). Preparation of pre-service teachers in agricultural
mechanics. Journal of Agricultural Education, 46(3), 23-34. doi:10.5032/jae.2005.03023

Journal of Agricultural Education 167 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Byrd, A. P. (2014). Identifying the effects of human factors and training methods on a weld training
program. Retrieved from Iowa State University Digital Repository Graduate Theses and
Dissertations. (Paper 13991)

Byrd, A. P., Stone, R. T., Anderson, R. G., & Woltjer, K. (2015). The use of virtual welding
simulators to evaluate experienced welders. Welding Journal, 94(12), 389-395. Retrieved
from http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1114&context=imse_pubs

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research.
Chicago, IL: Rand McNally & Company.

Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education (7th ed.). New York,
NY: Routledge.

Cope, D. H., & Fenton-Lee, D. (2008). Assessment of laparoscopic psychomotor skills in interns
using the MIST virtual reality simulator: A prerequisite for those considering surgical
training? ANZ Journal of Surgery, 78(4), 291-296. doi:10.1111/j.1445-2197.2007.04440.x

DeKeyser, R. (2015). Skill acquisition theory. In B. VanPatten and J. Williams (Eds.), Theories in
second language acquisition: An introduction (2nd ed.) (pp. 94-112). New York, NY:
Routledge.

Doerfert, D. L. (Ed.) (2011). National research agenda: American Association for Agricultural
Education research priority areas for 2011–2015. Lubbock, TX: Texas Tech University,
Department of Agricultural Education and Communications.

Filigenzi, M. T., Orr, T. J., & Ruff, T. M. (2000). Virtual reality for mine safety training. Applied
Occupational and Environmental Hygiene, 15(6), 465-469. doi:10.1080/104732200301232

Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted Kappa and the intraclass correlation
coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613-
619. Retrieved from https://journals.sagepub.com/doi/pdf/10.1177/001316447303300309

Gallagher, A. G., Smith, C. D., Bowers, S. P., Seymour, N. E., Pearson, A., McNatt, S., Hananel, D.,
& Satava, R. M. (2003). Psychomotor skills assessment in practicing surgeons experienced in
performing advanced laparoscopic procedures. Journal of the American College of Surgeons,
197(3). 479-488. doi:10.1016/S1072-7515(03)00535-0

George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0
update (4th ed.). Boston, MA: Allyn & Bacon.

Goldsmith, M., Stewart, L., & Ferguson, L. (2006). Peer learning partnership: An innovative strategy
to enhance skill acquisition in nursing students. Nursing Education Today, 26(2), 123-130.
doi:10.1016/j.medt.2005.08.001

Gor, M., McCloy, R., Stone, R., & Smith, A. (2003). Virtual reality laparoscopic simulator for
assessment in gynaecology. BJOG: An International Journal of Obstetrics & Gynaecology,
110(2), 181-187. doi:10.1046/j.1471-0528.2003.02016.x

Journal of Agricultural Education 168 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

Helsel, S. (1992). Virtual reality and education. Educational Technology, 32(5), 38-42. Retrieved
from http://www.jstor.org/stable/44425644

Kaufman, H. H., Wiegand, R. L., & Tunick, R. H. (1987). Teaching surgeons to operate – Principles
of psychomotor skills training. Acta Neurochir, 87(1), 1-7. doi:10.1007/BF02076007

Kilmon, C. A., Brown, L., Ghosh, S., & Mikitiuk, A. (2010). Immersive virtual reality simulations in
nursing education. Nursing Education Perspectives, 31(5), 314-317. Retrieved from
http://journals.lww.com/neponline/Abstract/2010/09000/Immersive_Virtual_Reality_Simulat
ions_in_Nursing.11.aspx

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intracclass correlation
coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155-163. Retrieved
from https://www.sciencedirect.com/science/article/pii/S1556370716000158

Kotrlik, J. W., Williams, H. A., & Jabor, M. K. (2011). Reporting and interpreting effect size in
quantitative agricultural education research. Journal of Agricultural Education, 52(1), 132-
142. doi:10.5032/jae.2011.01132

Kneebone, R. (2005). Evaluating clinical simulations for learning procedural skills: A theory-based
approach. Academic Medicine, 80(6), 549-553. Retrieved from
https://journals.lww.com/academicmedicine/Fulltext/2005/06000/Evaluating_Clinical_Simul
ations_for_Learning.6.aspx

Lancelot, W. H. (1944). Permanent learning: A study in educational techniques. New York, NY:
John Wiley & Sons, Inc.

LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and
interrater agreement. Organizational Research Methods, 11(4), 815-852.
doi:10.1177/1094428106296642

Lee, K. (2012). Augmented reality in education and training. TechTrends, 56(2), 13-21. Retrieved
from https://link.springer.com/content/pdf/10.1007%2Fs11528-012-0559-3.pdf

Lindner, J. R., Rodriguez, M. T., Strong, R., Jones, D., & Layfield, D. (2016). Research priority 2:
New technologies, practices, and products adoption decisions. In T. G. Roberts, A. Harder, &
M. T. Brashears. (Eds.), American Association for Agricultural Education national research
agenda: 2016-2020. Gainesville, FL: Department of Agricultural Education and
Communication.

Nadolny, L., Woolfrey, J., Pierlott, M., & Kahn, S. (2013). SciEthics interactive: Science and ethics
learning in a virtual environment. Educational Technology Research and Development, 61(6),
979-999. doi:10.1007/s11423-013-9319-0

Nikolic, B., Radivojevic, Z., Djordjevic, J., & Milutinovic, V. (2009). A survey and evaluation of
simulators suitable for teaching courses in computer architecture and organization. IEEEE
Transactions on Education, 52(4), 449-458. Retrieved from
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4967893

Osborne, E. W. (1986). Teaching strategies for developing psychomotor skills. NACTA Journal, 30
(1), 54-57. Retrieved from

Journal of Agricultural Education 169 Volume 61, Issue 1, 2020

Wells and Miller The Effect of Virtual Reality…

http://www.nactateachers.org/attachments/article/1204/Osborne_NACTA_Journal_March_19
86-16.pdf

Oz, C., Ayar, K., Serttas, S., Iyibilgin, O., Soy, U., & Cit, G. (2012). A performance evaluation
application for welder candidate in virtual reality simulator. Procedia - Social and Behavioral
Sciences, 55, 492-501. doi:10.1016/j.sbspro.2012.09.529

Pantelidis, V. S. (1993). Virtual reality in the classroom. Educational Technology, 33(4), 23-27.
Retrieved from http://www.jstor.org/stable/44428033

Pate, M. L., Warnick, B. K., & Meyers, T. (2012). Determining the critical skills beginning
agriculture teachers need to successfully teach welding. Career and Technical Education
Research, 37(2), 171-184. doi:10.5328/cter37.2.171

Perritt, D. (1984). Effects of two instructional techniques used with the Ford power train simulator on
the performance of Mississippi vocational agriculture students. Journal of the American
Association of Teacher Educators in Agriculture, 25(1), 35-41.
doi:10.5032/jaatea.1984.01035

Phipps, L. J., Osborne, E. W., Dyer, J. E., & Ball, A. (2008). Handbook on agricultural education in
public schools (6th ed.). Clifton Park, NY: Thomson Delmar Learning.

Potkonjak, V., Gardner, M., Callaghan, V., Mattila, P., Guetl, C., Petrovic, V. M., & Jovanovic, K.
(2016). Virtual laboratories for education in science, technology, and engineering: A review.
Computers & Education, 95, 309-327. Retrieved from
https://doi.org/10.1016/j.compedu.2016.02.002

Rose, M., Pate, M. L., Lawver, R. G., Warnick, B. K., & Dai, X. (2015). Assessing the impact of
sequencing practicums for welding in agricultural mechanics. Journal of Agricultural
Education, 56(1), 92-102. doi:10.5032/jae.2015.01092

Saettler, P. (2004). The evolution of American educational technology. Greenwich, CT: Information
Age Publishing, Inc.

Sawilowsky, S. S. (2009). New effect size rules of thumb. Journal of Modern Applied Statistical
Methods, 8(2), 597-599. doi:10.22237/jmasm/1257035100

Scalese, R. J., Obeso, V. T., & Issenberg, S. B. (2008). Simulation technology for skills training and
competency assessment in medical education, Journal of General Internal Medicine, 23(1),
46-49. doi:https://doi.org/10.1007/s11606-007-0283-4

Scholtes, V. A., Terwee, C. B., & Poolman, R. W. (2011). What makes a measurement instrument
valid and reliable? Injury, 42(3), 236-240. Retrieved from
https://www.sciencedirect.com/science/article/pii/S0020138310007771

Seymour, N. E., Gallagher, A. G., Roman, S. A., O’Brien, M. K., Bansal, V. K., Andersen, D. K., &
Satava, R. M. (2002). Virtual reality training improves operating room performance. Annals
of Surgery, 236(4), 458-464. doi:10.1097/01.SLA.0000028969.51489.B4

Journal of Agricultural Education 170 Volume 61, Issue 1, 2020

 Wells and Miller The Effect of Virtual Reality…

Stone, R. T., McLaurin, E., Zhong, P., & Watts, K. (2013). Full virtual reality vs. integrated virtual
reality training in welding. Welding Journal, 92(6), 167-174. Retrieved from
http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1041&context=imse_pubs

Stone, R. T., Watts, K. P., Zhong, P., & Wei, C. (2011). Physical and cognitive effects of virtual
reality integrated training. Human Factors, 53(5), 558-572. doi:10.1177/0012720811413389

Stripling, C. T., & Ricketts, J. C. (2016). Research priority 3: Sufficient scientific and professional
workforce that addresses the challenges of the 21st century. In T. G. Roberts, A. Harder, & M.
T. Brashears. (Eds.), American Association for Agricultural Education national research
agenda: 2016-2020. Gainesville, FL: Department of Agricultural Education and
Communication.

Thiagarajan, S. (1998). The myths and realities of simulations in performance technology.

Educational Technology, 38(5), 35-41. Retrieved from http://www.jstor.org/stable/44428481

Venes, D. (2017). Taber’s cyclopedic medical dictionary (23rd ed.). Philadelphia, Pennsylvania: F. A.
Davis Company.

Virtual Reality Society. (2017). What is virtual reality? Retrieved from

https://www.vrs.org.uk/virtual-reality/what-is-virtual-reality.html

Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and
student achievement in mathematics. Retrieved from
https://files.eric.ed.gov/fulltext/ED425191.pdf

White, S. A., Prachyabrued, M., Chambers, T. L., Borst, C. W., & Reiners, D. (2011). Low-cost
simulated MIG welding for advancement in technical training. Virtual Reality, 15(1), 69-81.
doi:10.1007/s10055-010-0162-x

Wulf, G. (2007). Attention and motor skill learning. Champaign, IL: Human Kinetics.

Youngblut, C. (1998). Educational uses of virtual reality technology. Retrieved from

http://www.dtic.mil/dtic/tr/fulltext/u2/a339438.pdf

Yuen, S. C., Yaoyuneyong, G., & Johnson, E. (2011). Augmented reality: An overview and five
directions for AR in education. Journal of Educational Technology Development and
Exchange, 4(1), 119-140. Retrieved from
https://aquila.usm.edu/cgi/viewcontent.cgi?article=1022&context=jetde

Journal of Agricultural Education 171 Volume 61, Issue 1, 2020

View publication stats