Behav Analysis Practice (2018) 11:62–70

DOI 10.1007/s40617-017-0186-8

TECHNICAL AND TUTORIALS

Some Tools for Carrying Out a Proposed Process for Supervising

Experience Hours for Aspiring Board Certified
Behavior Analysts®
Katie Lynn Garza 1 & Heather M. McGee 1 & Yannick A. Schenk 1 &
Rebecca R. Wiskirchen 1

Published online: 14 April 2017

# Association for Behavior Analysis International 2017

Abstract While task clarification, goal setting, feedback, and equal importance is a supervisor’s ability to conduct supervi-
behavioral skills training (BST) are well-supported methods sion within a process that accounts for the contingencies act-
for performance improvement, there is no standardized ap- ing upon his environment and the environment of his
proach to supervising aspiring Board Certified Behavior supervisees. Insight from the work that has been done in or-
Analysts® (BCBAs®) that specifies how such practices ganizational behavior management (OBM) can help us design
should be used within a comprehensive supervision system, effective supervision systems.
namely for supervising those who are still accruing experience Behavior analysts should be accustomed to using a
hours for the purpose of becoming board certified. This article behavior-analytic technology to produce behavior change pro-
outlines a systematic approach to BCBA supervision and pro- grams for clients. However, many may not be as accustomed to
vides a set of tools that supervisors can use to ensure that they using the same technology to train and supervise individuals
are engaging in empirically based supervision practices. seeking certification, especially those who have limited practi-
cal experience. Organizational behavior management is a sub-
Keywords BCBA supervision . Supervision process . field of applied behavior analysis that focuses specifically on
Supervision materials organizational problems (Bucklin, Alvero, Dickinson, Austin,
& Jackson, 2000). Because the same empirical approach that is
followed in clinically focused behavior analysis is used in or-
The Behavior Analyst Certification Board® (BACB®)
ganizational behavior management, much of the research find-
Supervisor Training Curriculum Outline specifies that
ings from the OBM literature can be used to aid in the devel-
BCBAs trained in supervision techniques should be able to
opment of applied behavior analysis (ABA) practices, includ-
describe the purpose and important features of supervision,
ing the supervision of others.
use behavioral skills training, including the use of effective
For example, it is clear that ensuring that consequences are
performance feedback to teach targeted skills, evaluate the
contingent on the target performance improves that particular
effectiveness of their supervision, and proactively engage in
target performance. Komaki (1986) set out to, Bspecifically
professional development activities to improve supervisory
identify and empirically determine what constitutes effective
performance (BACB, 2012b). Each of these repertoires is es-
supervisory behavior^ (p. 270). The author found that effec-
sential for providing adequate supervision. We believe that of
tive managers spent significantly more time monitoring per-
Electronic supplementary material The online version of this article
formance and used work sampling as a specific form of per-
(doi:10.1007/s40617-017-0186-8) contains supplementary material, formance monitoring more often than marginally effective
which is available to authorized users. supervisors. Komaki also found that there was no difference
between the groups regarding the number of positive, nega-
* Katie Lynn Garza tive, or neutral consequences between effective and marginal-
[email protected] ly effective managers. In other words, Beffective managers
were no more likely than the marginal managers to provide
1
Department of Psychology, Western Michigan University, 1903 W. positive consequences, and they were no less likely to provide
Michigan Avenue, Kalamazoo, MI 49008-5439, USA negative consequences^ (pp. 275–6). The author concluded
Behav Analysis Practice (2018) 11:62–70 63

that because effective managers spent more time monitoring, In a survey to assess the needs of individuals supervising
their consequences were more likely to be contingent on, rath- experience hours for aspiring BCBAs, 18% of participants
er than independent of, employees’ performance. indicated that lack of available information or materials was
Task clarification, goal setting, and feedback are common a barrier to using assessment in supervision. Furthermore,
practices that have been used in a variety of settings to im- 30% of participants indicated that time or competing contin-
prove on-the-job performance. Tittelbach, DeAngelis, gencies were a barrier to completing assessments in supervi-
Sturmey, and Alvero (2007) used a task clarification, goal sion. When asked to indicate barriers to the use of behavioral
setting, and feedback intervention to improve punctuality, skills training in supervision practices, 54% of participants
greeting, and correct front-desk behavior of student advisors indicated that lack of time was a barrier, and 48% of partici-
in a university counseling center. Loewy and Bailey (2007) pants indicated that there were too many skills to teach. The
used graphic feedback, goal setting, and manager praise to majority of participants also indicated that additional materials
improve customer service (i.e., greeting, eye contact, and would be useful for teaching most items in sections I and II of
smiling) in two locations of a home improvement store. Gil the BACB Task List (Garza et al., manuscript in preparation).
and Carter (2016) used graphic feedback, later combined with In response to these expressed needs, the purpose of this paper
goal setting and performance feedback to improve data col- is to present a concise process for supervising aspiring
lection adherence of direct care staff in a large residential BCBAs and to provide materials to assist supervisors in pro-
treatment facility. It is reasonable to expect the application of viding high-quality supervision. Supervision for BCBA expe-
these practices to the supervision of aspiring BCBAs to result rience hours can be conceptualized as a process that occurs in
in supervisees’ performance improvement in key behavior- five phases: establishing a supervisory relationship, skills as-
analytic skills. sessment, training, ongoing performance monitoring, and
In 2016, Behavior Analysis in Practice released a special ending the supervisory relationship. The steps involved in
section on supervision. Prior to its publication, few articles each of these phases will be discussed below. Those phases
had been published on recommended practices for the super- that are already discussed at length in the existing literature on
vision of aspiring BCBAs. The special section includes arti- supervision will be covered in less detail than those for which
cles on ethical considerations in supervision (Sellers, Alai- the existing literature does not address how to complete the
Rosales, & MacDonald, 2016), recommendations for phase in the context of BCBA supervision. In addition, strat-
conducting group supervision (Valentino, LeBlanc, & egies for ongoing professional development will be discussed.
Sellers, 2016), recommendations for addressing barriers to
supervision (Sellers, LeBlanc, & Valentino, 2016), an exami-
nation of the relationship between supervision hours, supervi- Establishing a Supervisory Relationship
sor credentials, years of experience, and supervisor caseload
on ABA treatment outcomes (Dixon et al., 2016), and recom- Individuals may receive supervision as part of a university
mended practice guidelines and approaches to supervision practicum, from a supervisor within the organization at which
(Sellers, Valentino, & LeBlanc, 2016; Hartley, Courtney, he or she works, or from a BCBA who offers contracted su-
Rosswurm, & LaMarca, 2016; Turner, Fischer, & Luiselli, pervision services. Regardless of the arrangement of supervi-
2016). sion, no experience hours can be accrued before both the su-
Interestingly, though the supervision model presented pervisor and supervisee sign a contract outlining the respon-
by Turner et al. (2016) closely resembles the process that sibilities of each party. The BACB Experience Standards out-
will be presented in this paper, the two systems were line the specific items that must be included in the contract
established by two separate groups in two separate loca- (BACB, 2016), and the BACB® website offers several sam-
tions. The current paper further enhances the literature ple contracts to use as a template. We recommend using these
base on supervision by emphasizing that supervision is a templates as a guide and modifying the contract to describe
process with a beginning, middle, and end and by provid- the specific assessments and experience opportunities that the
ing templates to assist supervisors in carrying out that supervisee can expect to encounter within the organization in
process. which experience hours will be accrued. We also recommend
Knowing the general flow of a supervisory relationship using universities’ or other organizations’ legal or contract
from beginning to end helps supervisors to keep their docu- departments as a resource for reviewing contracts for proper
mentation organized and plan for supervisees’ training and legalese whenever such resources are available.
assessment needs. This is especially important given that su- In our professional experience, we have heard supervisors
pervision may often be a secondary job for BCBAs, resulting express concern that they could get locked into a supervisory
in time constraints and competing job responsibilities (Garza, relationship with a supervisee who does not make adequate
Peterson, McGee, Jackson, & Malott, manuscript in progress or who engages in unprofessional behavior. In their
preparation). Experience Standards, the BACB specifies that contracts
64 Behav Analysis Practice (2018) 11:62–70

should, Bdelineate the consequences should the parties not Job Model
adhere to their responsibilities (including proper termination
of the relationship)^ (2016, p. 3). If there are circumstances Job models can take different forms and be called by different
under which a supervisor would be unwilling to continue su- names (e.g., role/responsibility matrix) but typically include
pervising an individual, those circumstances should be clearly the accomplishments or performance responsibilities associat-
outlined in the supervision contract in order to protect both ed with the job. Additional information, such as tasks required
parties. to complete the performance, measures of performance, goals
During an initial meeting, the supervisor should describe or standards for performance, inputs to performance, recipi-
the purpose of supervision as described in the BACB’s ents of outputs of performance, and how feedback on perfor-
Supervisor Training Curriculum Outline and go over the su- mance will be provided, are sometimes also included in the
pervision contract with the supervisee. We recommend read- job model (Gilbert, 1996; Rummler & Brache, 2012).
ing over the terms of the contract aloud during the meeting, Because training (discussed in the next section) can be a
pausing after each section to allow for questions, and having time-consuming process, we recommend conducting an anal-
the supervisee sign his or her initials after each section of the ysis of the position supervisees will hold in the organization.
agreement. This protects both parties by ensuring that there is This analysis helps supervisors identify which skills the
a legitimate and understood agreement between the supervisor supervisee will actually practice throughout the supervisory
and supervisee. Once both parties sign the contract, the super- relationship. If the organization cannot provide specific train-
visory relationship has been officially established. ing on a particular skill, it is categorized as a low priority for
Sellers, Valentino, and LeBlanc (2016) provide excellent assessment and training. If the skill is practiced regularly with-
rationale and guidelines for effective establishment of the su- in the organization, it is a high priority for assessment and, if
pervisor–supervisee relationship. We recommend reviewing needed, training. This is not to say that the skills not practiced
their practice guidelines for more detailed information on cre- within the organization are not important to the field of behav-
ating supervision contracts, setting clear expectations for the ior analysis or would not be worthwhile for the supervisee to
supervisee, specifically setting expectations regarding feed- learn. However, it is best to match the training setting to the
back in the supervision process, and creating a Bcommitted environment in which the performance will occur naturally
and positive^ supervisory relationship (p. 276). Turner et al. (Kazbour, McGee, Mooney, Masica, & Brinkerhoff, 2013;
(2016) also discuss strategies for establishing a supervisory Mager, 1997). If there are no natural practice opportunities
relationship that results in collaborative and ethical practices. within the organization, it is best to leave the training to a site
In a manner similar to Sellers et al. (2016), Turner et al. (2016) in which those opportunities are available.
emphasize the importance of setting clear performance expec- Job models can be conceptualized as a sort of task clarifi-
tations and clearly defining the supervisory relationship. cation, which is a widely supported performance improve-
ment intervention in OBM (Cunningham & Austin, 2007;
Gravina, VanWagner, & Austin, 2008; Durgin, Mahoney,
Skills Assessment Cox, Weetjens, & Poling, 2014; Slowiak, 2014). Job models
can be organized in different ways depending on their specific
After establishing the supervisory relationship, a supervisor utility within the organization. In the context of supervision
must determine which performances and corresponding Task for aspiring BCBAs, it is appropriate to examine which Task
List (BACB, 2012a) items the supervisee currently possesses List (BACB, 2012a) items are practiced when a supervisee
and which will need to be developed over the course of super- engages in each given performance that is part of her position
vision. This determination is made through a performance/ within the organization.
skills assessment. In assessment, the supervisor is either ob- Appendix 1 is an example of a job model for a behavior
serving performance or reviewing products of performance consulting organization. On the top of the form, there is a
without providing prompts, coaching, or instructional feed- space to indicate the name of the position (e.g., graduate ther-
back. These components of training and performance manage- apist), the supervisor (e.g., KS), and the job purpose (e.g., to
ment could bias the results of the assessment. Assessment provide clinical services to adults and children with develop-
should be done before beginning the training process because mental disabilities and develop professional skills for provid-
it helps the supervisor pinpoint which skills the supervisee can ing effective and ethical treatment). Specifying the purpose of
successfully practice independently. This allows the supervi- the job helps establish the scope of the position and distin-
sor to focus training efforts on skills the supervisee has not yet guish the position from others within the organization. This
mastered. In our discussion of assessment, we will first present decreases the likelihood of ambiguity as to whose responsibil-
a method for supervisors to define the scope of their assess- ity it is to engage in a given responsibility. Job responsibilities
ment, followed by a method for conducting the assessment, or performances are listed in the left column of the table, and
and, finally, a method for using the results of the assessment. the corresponding BACB Task List items are listed in the right
Behav Analysis Practice (2018) 11:62–70 65

column. For example, a graduate therapist in the behavior Assessment of Skills

consulting organization might conduct a functional behavior
assessment, but they might not be responsible for conducting Assessing supervisees’ ability to perform the Task List items
discrete trial training. identified as relevant to the supervisee’s job position can also
We identified 11 different tasks from the Task List that decrease the time supervisees spend on training. Assessment
someone might engage in to complete a functional behavior helps determine which skills must be trained and which can be
assessment: G-01 Breview records and available data at the practiced without training, thereby preventing unnecessary
outset of the case,^ G-02 Bconsider biological/medical vari- training and premature independent practice. Furthermore,
ables that may be affecting the client,^ G-03 Bconduct a pre- initial assessment allows supervisors to obtain a baseline mea-
liminary assessment of the client in order to identify the referral sure of their supervisees’ performance, which allows them to
problem,^ G-04 Bexplain behavioral concepts using nontech- evaluate their own supervisory effectiveness (Kazdin, 1982).
nical language,^ G-06 Bprovide behavior-analytic services in There are different methods to assess supervisees’ ability to
collaboration with others who support and/or provide services perform job responsibilities. We recommend the following
to one’s clients,^ G-07 Bpractice within one’s limits of profes- sequence of assessment: self-assessment, oral and written
sional competence in applied behavior analysis, and obtain quizzes, and validation of self-assessment through review of
consultation, supervision, and training, or make referrals as permanent products and observation.
necessary,^ G-08 Bidentify and make environmental changes Turner et al. (2016) discuss the use of initial baseline as-
that reduce the need for behavior analysis services,^ I-01 sessment, including the use of direct observation; use of infor-
Bdefine behavior in observable and measurable terms,^ I-02 mal assessment such as interviews, conversation, and portfo-
Bdefine environmental variables in observable and measurable lio review; review of course syllabi; discussion of progress
terms,^ I-03 Bdesign and implement individualized behavioral with previous supervisors; and review of the Task List. Here,
assessment procedures,^ and I-07 Bdesign and conduct prefer- we present multiple steps of baseline assessment and discuss a
ence assessments to identify putative reinforcers^ (BACB, tool that can be used to organize assessment data for ongoing
2012a, pp. 6, 7). Note that job performances could be more use in the supervision process.
narrowly defined if it is likely that supervisees within the or- Self-assessment involves having the supervisee rate her
ganization will complete components of the larger perfor- skill level for each job responsibility and/or task list skill.
mance independently. For example, functional assessment in- While behavior analysts might be hesitant to use self-
terview, descriptive assessment, and functional analysis can be assessment methods in their supervision practices, asking a
listed as separate performances on the job model if they are supervisee to rate her ability to perform a skill has some ben-
likely to be practiced as discrete performances that involve the efits. Sellers et al. (2016) recommend having supervisees self-
use of unique sets of skills from the Task List. evaluate their ability to perform job responsibilities, noting
Once all of the job responsibilities and corresponding Task that such an evaluation might help supervisors target their
List items are identified for the job position, a list of all of the training efforts appropriately. If a supervisee indicates that
Task List items that can be practiced while the supervisee is in she has no or limited experience with a task or that she would
that position can be compiled. The Task List items for which need coaching to perform the task, she may actually not have
there are no natural practice opportunities can be better the skills to perform that particular task. In this situation, her
assessed and trained in another position, perhaps in another self-assessment would indicate a training need. However, it is
organization. While all of the skills on the Task List are rele- also possible that her rating does not match her actual skill
vant to behavior analysis as a field, not all of the skills are level (i.e., the supervisee might actually perform the task at a
relevant to every job position. For example, in organizations mastery level). Even in this situation, it would be worthwhile
that do not provide group instruction to clients, there may be to provide some training so that the supervisee can learn to tact
few or no opportunities to use Direct Instruction (item F-03 on her own correct performance and better recognize when she
the Task List). There also may not be an opportunity to use a has made an error. Turner et al. (2016) point out that the level
full range of experimental designs (Section I.B. on the Task of rapport between a supervisor and supervisee may affect a
List) within many organizations. Although these skills are supervisee’s tendency to seek her supervisor’s guidance over
valuable, eliminating them from a training program allows the guidance of others. Providing training when a supervisee
the supervisor to focus on teaching the skills that are immedi- indicates that her ability to perform a skill is below mastery
ately relevant for the supervisee’s role in the organization. levels could improve rapport and increase the frequency with
This, in turn, eliminates some competition between training which a supervisee asks for help when needed.
supervisees and the supervisor’s other job responsibilities. On the other hand, if a supervisee indicates that she could
Once all of Task List items that can be practiced within a perform the task independently or even coach others on the
supervisee’s job role have been identified, supervisors can performance, her performance could indicate otherwise (i.e.,
conduct a focused assessment to determine training needs. her performance could be well below mastery levels). Parsons,
66 Behav Analysis Practice (2018) 11:62–70

Rollyson, and Reid (2012) distinguish between performance introduce steps supervisors can take to set appropriate goals
skills and verbal skills in their discussion of the behavioral for their supervisees.
skills training (BST) model (discussed below). Just as it is The results of skills assessment can be used to set long-term
recommended that training differ depending on the nature of and short-term training goals for the supervisee. Long-term
the skills to be trained, we recommend different methods for goals can be reviewed each quarter or semester, and short-
verifying supervisee’s self-reports depending on the nature of term goals can be reviewed each supervisory period. Other
the skills being assessed. Verification of self-assessment can time frames can be used as well, depending on the nature of
be done via quizzes and interviews for knowledge-based or the supervisory relationship. Goal setting interventions often
verbally based skills (e.g., Section III: Foundational involve the use of baseline performance data to set interven-
Knowledge of the Task List; skills that involve explaining tion goals that are achievable and that, if met, would result in
concepts or treatment programs and skills that involve clinical meaningful performance improvement (e.g., Downing &
decision-making) and review of permanent products and di- Geller, 2012; Green et al., 2016). Additionally, there is some
rect observation for performance-based skills. If the evidence that setting goals based on an individual’s ability
supervisee has a portfolio of work she has done in the past, results in more pronounced performance improvements over
evaluating the items in the portfolio allows supervisors to goals that are assigned to an entire group, especially for low
assess skill levels without scheduling an observation. In fact, performers (Jeffrey, Schulz, & Webb, 2012).
we recommend encouraging supervisors to keep a record of To set appropriate long-term goals, supervisors should con-
work products for this reason. For items that cannot be sider the Task List items that are included in the job model, the
assessed via quizzes, interviews, and portfolio review, role supervisee’s overall progress on mastering those Task List
play or in vivo, direct observations can be used. items, and the anticipated needs of the organization during
The authors have developed a tool for organizing assess- the review period. For example, if the supervisee has observed
ment data. Readers may access this tool by emailing the first her supervisor collecting interobserver agreement (IOA) data,
author. The tool presents all items of the BACB Fourth but has not yet had the opportunity to practice independently,
Edition Task List and dropdown menus with self-assessment long-term goals for the quarter might include collecting integ-
options for each Task List item. The supervisee can self-assess rity data independently with 80% or better (IOA) with her
their ability to perform each skill by selecting an item from the supervisor. This goal could take one or several sessions to
dropdown menu. Once the supervisee completes the self-as- achieve. If the supervisee has never observed the process of
sessment, the assessment results populate other tabs in the writing a behavior support plan, one long-term goal for the
spreadsheet, which correspond to each section of the Task quarter might be to have the supervisee take part in the treat-
List. For each Task List item, there is a space for the self- ment planning process and read the resulting behavior support
assessment standing (which is automatically populated from plan.
the self-assessment), a broad training goal (train, generalize, Once goals are determined for the quarter or semester,
monitor, or lead/train others), validation method (portfolio re- short-term goals can be established with long-term goals in
view, natural observation, alternative training method, or mind. To set appropriate short-term goals, supervisors should
N/A), and status (validated, in training, or monitoring). consider the immediate needs of the organization as well as
There is also a space for the supervisor to make comments their supervisee’s ability to meet those needs. For example, if
regarding the supervisee’s progress on each Task List item. the organization is serving a client who needs a functional
behavior assessment, the supervisee’s demonstrated abilities
related to functional behavior assessment will determine how
much independence the supervisee should be allowed when
Goal Setting performing the task. If the supervisee indicated on her self-
assessment that she has no or limited experience in FBA,
A variety of studies have examined the effects of goal setting training goals for the supervisory period might include the
on performance. Goal setting has been used successfully in supervisee’s observation of the planning process. If the
interventions to improve the performance of soccer players supervisee has successfully completed a functional behavior
(Brobst & Ward, 2002), to increase safe methods of passing assessment, training goals for the supervisory period might
sharp instruments in a hospital operating room (Cunningham include the supervisee writing the protocol without guidance
& Austin, 2007), to increase cashiers’ checking of identifica- from the supervisor. The supervisor could then evaluate the
tion in a grocery store (Downing & Geller, 2012), and to protocol and provide positive and corrective feedback. A form
decrease office workers’ bouts of prolonged sitting (Green, for setting goals and delivering performance feedback is avail-
Sigurdsson, & Wilder, 2016). Additionally, Turner et al. able in Appendix 5. See the section on BOngoing Performance
(2016) discuss goal setting as one method for teaching and Monitoring^ below for a more detailed description of the
promoting skills within the supervisory relationship. Here, we Feedback and Goal Setting Form.
Behav Analysis Practice (2018) 11:62–70 67

Training After completing a task analysis, it is helpful to create a

training storyboard to facilitate the training process (Bunch,
The Supervisor Training Curriculum Outline from the BACB 1991). A storyboard is a training design document that
lists BBehavioral skills training of supervisee^ as an important prompts the trainer to identify the rationale for the perfor-
feature of supervision (2012, p. 2). In addition, the BACB mance, to develop instructions for the trainee, to determine
Experience Standards lists behavioral skills training as a com- how the performance will be modeled and what he or she will
ponent of Beffective behavior-analytic supervision^ (2016, p. say while modeling the performance, to determine what prac-
2). Therefore, our discussion of training is based on the be- tice opportunities will look like for the trainee, to determine
havioral skills training (BST) model. how feedback will be delivered to the trainee, and to deter-
Behavioral skills training is an approach to training that mine how the trainee’s performance will be evaluated to de-
includes five major components: rationale and instruction, termine whether mastery has been achieved. In other words,
modeling, opportunity to practice, feedback, and repetition the storyboard helps ensure that trainers are, in fact, designing
of practice and feedback until the learner achieves mastery BST with each of its components and not some other form of
(Parsons et al., 2012). Some models also include assessment training, which may or may not be evidence-based. Appendix
of generalization to other clients, settings, or behaviors 3 is a storyboard template that supervisors can use to develop
(BACB, 2012b, p. 3). their trainings.
To measure the correctness of a performance, the correct per- Once the training storyboard is developed, all materials to
formance must be operationally defined (Cooper, Heron, & be used in the training must also be developed or gathered.
Heward, 2007). Before implementing BST, one must define the Examples of training materials include written instructions,
target performance and create the necessary supporting materials. video-based models, practice activities, written tests, and ma-
We recommend beginning with a task analysis. A task analysis terials that are necessary for running protocols. Once the per-
breaks a performance down into its component steps. Behavior formance to be trained has been analyzed, the training has
analysts often use task analyses to break down target client per- been designed, and the training materials have been devel-
formances into smaller steps and then use chaining or shaping to oped, supervisors can implement BST according to the steps
improve client performance of the target. In training, task analy- outlined in the storyboard.
ses are used in the same way, but the focus is on supervisee
performance rather than client performance. See Appendix 2
for an example of a task analysis template. Our template includes
a place to indicate the performance to be taught, the triggering Ongoing Performance Monitoring
event (i.e., the stimulus that indicates it is time to begin the
performance), and the ending event (i.e., permanent products or Behavior analysis is not only a science of learning in the sense
changes in the environment that define the termination of the of skill acquisition. In fact, a significant portion of the field is
performance). Identifying the triggering event and the ending dedicated to understanding the variables that increase, de-
event helps to define the scope of the performance, which later crease, or maintain behavior after it is established (Skinner,
helps to define the scope of the corresponding training. The 1950). While the BACB Experience Standards state that, Bthe
template also has a place to indicate the tasks involved in the supervisee’s primary focus should be acquiring new behavior-
performance. For each task, there is a table to note the materials analytic skills related to the BACB Third Edition Task List or
that are needed to perform the task and the designated location the BACB Fourth Edition Task List as appropriate,^
for those materials. Additionally, the template has a place to supervisees also must maintain satisfactory performance after
indicate any steps, sub-steps, or decisions involved in the task. new skills are established (2016, p. 5).
Some performances may have only one task, while others might The Experience Standards require supervisors to observe
have multiple tasks, steps, sub-steps, and decisions. their supervisees in the natural environment at least once each
Task analyses make teaching job performances easier be- supervisory period (2016, p. 2). As Komaki, Zlotnick, and
cause the target skills are broken into smaller, more manage- Jensen (1986) point out, Bonly managers who systematically
able, sequential steps (Mager, 1997). Once the performance is monitor and collect data are likely to be knowledgeable about
broken into observable and measureable sub-steps, the task subordinates’ performance^ (p. 261). After training is com-
analysis can be used to create a data sheet. The data sheet plete for a given performance, ongoing monitoring serves to
can be used to guide an objective observation of the target continue the development of the supervisee’s skills and to
skill. Data sheets built from a task analysis are more likely assess for generalization to new clients, targets, and settings.
to capture the behaviors critical to successful performance. Ongoing performance monitoring involves regularly
As supervisors build a portfolio of task analyses and data conducting formal and informal assessments of supervisee’s
sheets related to the BACB Task List, they can be used to skills, updating goals according to assessment results, and
guide initial skills assessments as well. delivering performance feedback.
68 Behav Analysis Practice (2018) 11:62–70

Because we have already reviewed the process of Feedback and Goal Setting form can be used in conjunction
conducting assessments and establishing goals, this section with the tool for organizing assessment data that we described
will focus on the delivery of performance feedback. in the assessment section. Whereas assessment data may be
Feedback is information about performance that allows a per- collected only several times throughout the supervisory rela-
son to change that performance in the future (Daniels & tionship, the Supervision Feedback and Goal Setting form can
Bailey, 2014). Effective feedback contains specific informa- be used to document progress every supervisory period,
tion; pertains to performances under the control of the per- allowing for broad analyses of performance change as well
former; is individualized; is self-monitored when possible, if as more molecular analyses.
not self-monitored, is delivered by a person in charge; is fo-
cused on improvement; is easily understood; is graphed; is
used in conjunction with reinforcement; and is provided as
Ending the Supervisory Relationship
often as feasible (Alvero, Bucklin, & Austin, 2001;
Balcazar, Hopkins, & Suarez, 1985-86).
There are several steps supervisors should follow to appropri-
Delivering corrective feedback deserves additional discus-
ately conclude a supervisory relationship: evaluation of the
sion, as it can easily be done inappropriately or ineffectively.
supervisee’s overall skill development, evaluation of the super-
For example, anger is a motivating operation that might increase
visor’s supervision skills, and completion of documentation
the value of the reinforcing effects of yelling or other aggressive
required by the BACB. See Turner et al. (2016) for additional
behaviors. Therefore, feedback should not be delivered when
considerations for unplanned terminations of supervisory
angry. Delivering corrective feedback in public could be per-
relationships.
ceived by others in the organization as unfair, which might in-
crease the reinforcing effectiveness of behaviors that are incon-
sistent with the instructions given by the supervisor. Feedback
should therefore be given privately. Supervisors should also Evaluation of the Supervisee’s Overall Skill
avoid threatening consequences that they cannot deliver, so that Development
their feedback does not lose effectiveness in managing their
supervisees’ performance (Daniels & Bailey, 2014). To deliver As discussed earlier, evaluation of supervisees’ performance is,
corrective feedback, supervisors must describe the ineffective ideally, an ongoing process throughout supervision. However,
performance and the desired performance, provide rationale to remain consistent with the Banalytic^ dimension of applied
and instructions for the desired performance, model the correct behavior analysis, the supervisee’s performance should be eval-
performance, provide opportunities for the supervisee to practice uated before, during, and after supervision to provide a measure
the desired performance, provide feedback again, and follow-up of the supervisee’s overall progress and the impact of supervi-
as necessary (BACB, 2012b, p. 4). Note that this is the same sion (Baer, Wolf, & Risley, 1968; Turner et al., 2016). Quizzes
process that is followed for behavioral skills training. on the Foundational Knowledge section of the Task List, self-
In their Supervisor Training Curriculum Outline, the assessment measures, direct observation, and reviews of perma-
BACB states that supervisors should be able to describe sev- nent products—the same assessment tools and methods that are
eral different methods of delivering performance feedback: used to conduct an initial assessment of the supervisee’s
vocal, written, modeled, video, graphic, self-monitored, for- skills—can be used to obtain an overall measure of the
mal, and informal (2012c, pp. 4–5). Appendix 4 is a table that supervisee’s progress throughout the process of supervision
gives a description of each of these feedback methods and (see the BAssessment of Skills^ section above for a more de-
situations in which each method might be appropriate. tailed discussion of these tools). While the initial assessment
Appendix 5 is a Supervision Feedback and Goal Setting gives a baseline from which training and performance goals can
Form that supervisors can use to guide their written and vocal be established, the final assessment gives a post-supervision
feedback and establish goals for the next supervisory period. evaluation. Comparing the pre-supervision performance with
The form prompts the supervisor to identify the job perfor- post-supervision performance can provide a measure of the
mances that were practiced during the supervisory period, supervisee’s progress and growth over the course of supervi-
their supervisee’s current (validated) level of self-assessment, sion, which allows the supervisor to evaluate his or her own
and goals for the upcoming supervisory period (or the next effectiveness.
time the performance will be completed). The form also During the final supervision meeting, the supervisor should
prompts supervisors to comment on their supervisee’s profes- discuss the results of the supervision evaluation. Pre- and post-
sional behavior during the supervisory period and provides supervision graphs of supervisee performance, summary
space for the supervisor to write general comments. In addi- statements from pre- and post-assessments, the supervisee’s
tion, the form prompts supervisors to select several areas of updated portfolio, and written recommendations for which
focus for the next supervisory period. The Supervision skills to focus on improving through subsequent supervision
Behav Analysis Practice (2018) 11:62–70 69

experiences should guide discussion during the final supervi- general process to follow while supervising aspiring BCBAs
sion meeting. as well as materials to guide the implementation of that pro-
cess. We encourage supervisors to use these materials and to
share training protocols with others with the goal of maintain-
Evaluation of the Supervisor’s Effectiveness ing the integrity of behavior analysis and keeping an emphasis
on data-based decision-making.
Comparing pre- and post-assessments of supervisees’ self-
assessments and work products not only provides a picture of Acknowledgements Portions of this project were supported through
the supervisees’ progress; it also provides a rough measure of generous funding from the Michigan Department of Health and Human
Services (Grant No. 20152526; Principle Investigators Stephanie
the supervisor’s effectiveness. In addition to evaluating the
Peterson and Wayne Fuqua). The opinions expressed in this paper are
supervisee’s progress, supervisors can ask supervisees to com- those of the authors and do not necessarily reflect those of the funding
plete an evaluation of their supervisory effectiveness. Consistent agency. The authors would like to express appreciation to Dr. Stephanie
with the recommendations of Turner et al. (2016), we also rec- Peterson and Dr. Wayne Fuqua for making this project possible through
their efforts to obtain and maintain funding and their diligent management
ommend that supervisors self-evaluate their supervisory effec-
and oversight.
tiveness. Additionally, we recommend using the BACB (2012b)
Supervisor Training Curriculum Outline to guide such an eval- Compliance with Ethical Standards This project was funded by the
uation, as this document outlines what the BACB considers to Michigan Department of Health and Human Services (Grant no.
be essential features of supervision. Supervisors can self- 20152526).
evaluate their supervisory effectiveness using the same method
and compare their self-evaluations to their supervisees’ evalua- Conflict of Interest The authors declare that they have no conflict of
interest.
tions to help them set appropriate goals for professional devel-
opment. When possible, it is a good idea to have peers or men-
Ethical Approval This article does not contain any studies with human
tors observe and evaluate supervisory performance to obtain a participants or animals performed by any of the authors.
direct measure of supervisory behavior.

References
Professional Development Planning
Alvero, A.M., Bucklin, B.R., & Austin, J. (2001). An objective review of
Planning to improve supervision practices takes time and ef- the effectiveness and essential characteristics of performance feed-
fort. Appendix 6 is a template based on the literature on per- back in organizational settings (1985–1998). Journal of
formance management (e.g., Malott, 2003, 2004), task clari- Organizational Behavior Management, 21(1), 3-29. doi:10.1300/
J075v21n01_02 .
fication (e.g., Slowiak, 2014; Wilson, Boni, & Hogg, 1997; Anderson, D. C., Crowell, C. R., Doman, M., & Howard, G. S. (1988).
Slowiak, Madden, & Mathews, 2006), goal setting (e.g., Performance posting, goal setting, and activity-contingent praise as
Slowiak et al., 2006; Anderson, Crowell, Doman, & applied to a university hockey team. Journal of Applied Psychology,
Howard, 1988), and reinforcement (e.g., Skinner, 1999/ 73(1), 87–95.
1961; Skinner, 1999/1968) that supervisors can use to estab- Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimen-
sions of applied behavior analysis. Journal of Applied Behavior
lish goals and deadlines, determine the actions required to Analysis, 1, 91–97.
achieve the goals, and create a reinforcement plan to follow Balcazar, F. E., Hopkins, B. L., & Suarez, Y. (1985-1986). A critical, ob-
when those goals have been met. We hope that supervisors jective review of performance feedback. Journal of Organizational
will use this tool in their own practices to add to their imple- Behavior Management, 7(3–4), 65–89.
Behavior Analyst Certification Board®. (2012a). Behavior analyst certi-
mentation of effective supervision practices.
fication board: fourth-edition task list. Retrieved from http://www.
bacb.com/Downloadfiles/TaskList/BACB_Fourth_Edition_Task_
List.pdf
Conclusion Behavior Analyst Certification Board®. (2012b). Supervisor training
curriculum outline. Retrieved from http://bacb.com/wp-content/
uploads/2015/05/supervisor_curriculum.pdf.
BCBA supervision is a process concerned with teaching new
Behavior Analyst Certification Board®. (2016). Behavior analyst certifi-
behavior-analytic skills and maintaining those skills after they cation board: experience standards. Retrieved from http://www.
have been mastered. As a field that has developed effective bacb.com/Downloadfiles/ExamApplications/bcba/150706%
techniques for increasing, decreasing, and maintaining behav- 20experience%20standards.pdf.
ior, it makes sense that we would use the same technology to Brobst, B., & Ward, P. (2002). Effects of public posting, goal setting, and
oral feedback on the skills of female soccer players. Journal of
teach and maintain the skills of those in training to become Applied Behavior Analysis, 35, 247–257.
professional behavior analysts and to improve the effective- Bucklin, B. R., Alvero, A. M., Dickinson, A. M., Austin, J., & Jackson,
ness of supervision practices. This paper has proposed a A. K. (2000). Industrial-organizational psychology and
70 Behav Analysis Practice (2018) 11:62–70

organizational behavior management. Journal of Organizational Journal of Organizational Behavior Mangement, 27(3), 15–26.
Behavior Management, 20(2), 27–75. doi:10.1300/J075v20n02 03. doi:10.1300/J075v27n03 02.
Bunch, J. (1991). The storyboard strategy. Training and Development, Mager, R. F. (1997). Making instruction work or Skillbloomers: a step-
45(7), 69–71. by-step guide to designing and developing instruction that works
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior (Second ed.). Atlanta: CEP Press.
analysis (Second ed.). Columbus: Pearson. Malott, R. W. (2003). What OBM needs is more Jewish mothers. Journal
Cunningham, T. R., & Austin, J. (2007). Using goal setting, task clarifi- of Organizational Behavior Management, 22(2), 71–87. doi:10.
cation, and feedback to increase the use of the hands-free technique 1300/J075v22n02_07.
by hospital operating room staff. Journal of Applied Behavior Malott, R.W. (2004). Notes from an introspective behavior analyst.
Analysis, 40, 673–677. doi:10.1901/jaba.2007.673-677. Journal of Organizational Behavior Management, 24(1–2), 75–
Daniels, A. C., & Bailey, J. S. (2014). Performance management: chang- 112. doi: 10.1300/J075v24n01_06.
ing behavior that drives organizational performance (Fifth ed.). Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based
Atlanta: Performance Management Publications. staff training: a guide for practitioners. Behavior Analysis in
Dixon, D. R., Linstead, E., Granpeesheh, D., Novack, M. N., French, R., Practice, 5(2), 2–11.
Stevens, E., et al. (2016). An evaluation of the impact of supervision Rummler, G. A., & Brache, A. P. (2012). Improving Performance: How
intensity, supervisor qualifications, and caseload outcomes in the to Manage the White Space on the Organization Chart. Somerset:
treatment of autism spectrum disorder. Behavior Analysis in John Wiley & Sons, Incorporated. Retrieved from https://
Practice, 9, 339–348. doi:10.1007/s40617-016-0132-1. ebookcentral-proquest-com.libproxy.library.wmich.edu/lib/
Downing, C. O., & Geller, E. S. (2012). A goal-setting and feedback wmichlib-ebooks/detail.action?docID=821830.
intervention to increase ID-checking behavior: an assessment of Sellers, T. P., Alai-Rosales, S., & MacDonald, R. P. F. (2016a). Taking full
social validity and behavioral impact. Journal of Organizational responsibility: the ethics of supervision in behavior analytic practice.
Behavior Management, 32(4), 297–306. doi:10.1080/01608061. Behavior Analysis in Practice, 9, 299–308. doi:10.1007/s40617-
2012.729383. 016-0144-x.
Durgin, A., Mahoney, A., Cox, C., Weetjens, B. J., & Poling, A. (2014). Sellers, T. P., LeBlanc, L. A., & Valentino, A. L. (2016b).
Using task clarification and feedback training to improve staff per- Recommendations for detecting and addressing barriers to success-
formance in an East African nongovernmental organization. Journal ful supervision. Behavior Analysis in Practice, 9, 309–319. doi:10.
of Organizational Behavior Management, 34(2), 122–143. doi:10. 1007/s40617-016-0142-z.
1080/0160861.2014.914007. Sellers, T. P., Valentino, A. L., & LeBlanc, L. A. (2016c). Recommended
practices for individual supervision of aspiring behavior analysts.
Gil, P. J., & Carter, S. L. (2016). Graphic feedback, performance feed-
Behavior Analysis in Practice, 9, 274–286. doi:10.1007/s40617-
back, and goal setting increased staff compliance with a data collec-
016-0110-7.
tion task at a large residential facility. Journal of Organizational
Skinner, B. F. (1950). Are theories of learning necessary? Psychological
Behavior Management, 36(1), 56–70. doi:10.1080/01608061.
Review, 57(4), 193–216.
2016.1152207.
Skinner, B. F. (1999a). The design of cultures. In J. S. Vargas (Ed.),
Gilbert, T. F. (1996). Human competence: engineering worthy
Cumulative Record (Definitive ed.). Cambridge: B.F. Skinner
performance (Tribute ed.). Washington, DC: The International
Foundation (Original work published in 1961).
Society for Performance Improvement.
Skinner, B. F. (1999b). The design of experimental communities. In J. S.
Gravina, N., VanWagner, M., & Austin, J. (2008). Increasing physical
Vargas (Ed.), Cumulative record (Definitive ed.). Cambridge: B.F.
therapy equipment preparation using task clarification, feedback and
Skinner Foundation (Original work published in 1968).
environmental manipulations. Journal of Organizational Behavior
Slowiak, J. M. (2014). BHow may I help you?^ Improving telephone
Mangement, 28(2), 110–122. doi:10.1080/0160860802100931.
customer service in a medical clinic setting. Journal of
Green, N., Sigurdsson, S., & Wilder, D. A. (2016). Decreasing bouts of Organizational Behavior Management, 34, 39–51. doi:10.1080/
prolonged sitting among office workers. Journal of Applied 01608061.2013.873382.
Behavior Analysis, 49, 717–722. doi:10.1002/jaba.309. Slowiak, J. M., Madden, G. J., & Mathews, R. (2006). The effects of a
Hartley, B. K., Courtney, W. T., Rosswurm, M., & LaMarca, V. J. (2016). combined task clarification, goal setting, feedback, and performance
The apprentice: an innovative approach to meet the Behavior contingent consequence intervention prackage on telephone custom-
Analysis Certification Board’s supervision standards. Behavior er service in a medical clinic environment. Journal of
Analysis in Practice, 9, 329–338. doi:10.1007/s40617-016-0136-x. Organizational Behavior Management, 25(4), 15–35. doi:10.1300/
Jeffrey, S. A., Schulz, A., & Webb, A. (2012). The performance effects of J075v25n04_02.
an ability-based approach to goal assignment. Journal of Tittelbach, D., DeAngelis, M., Sturmey, P., & Alvero, A. M. (2007). The
Organizational Behavior Management, 32(3), 221–241. doi:10. effects of task clarification, feedback, and goal setting on student
1080/01608061.2012.698116. advisors’ office behaviors and customer service. Journal of
Kazbour, R. R., McGee, H. M., Mooney, T., Masica, L., & Brinkerhoff, Organizational Behavior Management, 27(3), 27–40. doi:10.1300/
R. O. (2013). Evaluating the impact of a performance-based meth- J075v27n03 03.
odology on transfer of training. Performance Improvement Turner, L. B., Fischer, A. J., & Luiselli, J. K. (2016). Towards a compe-
Quarterly, 26(1), 5–33. doi:10.1002/piq.21135. tency-based, ethical, and socially valid approach to the supervision
Kazdin, A. (1982). Single-case research designs: methods for clinical of applied behavior analytic trainees. Behavior Analysis in Practice,
and applied settings. New York: Oxford University Press. 9, 287–298. doi:10.1007/s40617-016-0121-4.
Komaki, J. L. (1986). Toward effective supervision: an operant analysis Valentino, A. L., LeBlanc, L. A., & Sellers, T. P. (2016). The benefits of
and comparison of managers at work. Journal of Applied group supervision and a recommended structure for implementation.
Psychology, 71(2), 270–279. Behavior Analysis in Practice, 9, 320–328. doi:10.1007/s40617-
Komaki, J. L., Zlotnick, S., & Jensen, M. (1986). Development of an 016-0138-8.
operant-based taxonomy and observational index of supervisory be- Wilson, C., Boni, N., & Hogg, A. (1997). The effectiveness of task clar-
havior. Journal of Applied Psychology, 71(2), 260–269. ification, positive reinforcement, and corrective feedback in chang-
Loewy, S., & Bailey, J. (2007). The effects of graphic feedback, goal ing courtesy among police staff. Journal of Organizational Behavior
setting, and manager praise on customer service behaviors. Management, 17(1), 65–96.