This article was downloaded by: [Colorado College]

On: 04 December 2014, At: 17:21

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Occupational Therapy in Mental Health

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/womh20

Assessment of Time Management Skills

(ATMS): A Practice-Based Outcome
Questionnaire
a b c
Suzanne M. White , Anne Riley & Peter Flom
a
SUNY Downstate Medical Center , Brooklyn , New York
b
Bloomberg School of Public Health , Johns Hopkins University ,
Baltimore , Maryland
c
Scientific Computing Center, SUNY Downstate Medical Center ,
Brooklyn , New York
Published online: 28 Aug 2013.

To cite this article: Suzanne M. White , Anne Riley & Peter Flom (2013) Assessment of Time
Management Skills (ATMS): A Practice-Based Outcome Questionnaire, Occupational Therapy in Mental
Health, 29:3, 215-231, DOI: 10.1080/0164212X.2013.819481

To link to this article: http://dx.doi.org/10.1080/0164212X.2013.819481

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 Occupational Therapy in Mental Health, 29:215–231, 2013
Copyright © Taylor & Francis Group, LLC
ISSN: 0164-212X print/1541-3101 online
DOI: 10.1080/0164212X.2013.819481

Assessment of Time Management Skills

(ATMS): A Practice-Based Outcome
Questionnaire

SUZANNE M. WHITE
SUNY Downstate Medical Center, Brooklyn, New York

ANNE RILEY
Downloaded by [Colorado College] at 17:22 04 December 2014

Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland

PETER FLOM
Scientific Computing Center, SUNY Downstate Medical Center, Brooklyn, New York

This study assessed test–retest reliability and internal consistency of

the Assessment of Time Management Skills (ATMS), a self-report
questionnaire designed to assess awareness and use time management
strategies to plan and manage daily life tasks. Participants included
18 to 65 year-old people (N = 241) from the general population.
The questionnaire’s content validity was established. The tool
demonstrated good internal consistency (α = 0.86). Test–retest
reliability for the score revealed a Pearson Coefficient of Correlation
(PCC) r = 0.89. The ATMS may be a useful tool for evaluating the
effectiveness of rehabilitation designed to improve time management
skills.

KEYWORDS time management, co-occurring disorders, severe

mental illness, substance-related disorders, reproducibility of results

INTRODUCTION

Today’s Western society is time dependent and demands high standards of

time management. For all people, time management is a fundamental skill of
independent living. For some, the challenge is to fulfill many responsibilities

Address correspondence to Suzanne M. White, MA, OTR/L, Occupational Therapy

Program, SUNY Downstate Medical Center, Box 81, 450 Clarkson Avenue, Brooklyn, NY 11203.
E-mail: [email protected]

215
 216 S. M. White et al.

in a limited amount of time. For others, the challenge is to establish routines

that bring meaning and structure to their lives. Regardless of their goals,
people commonly use calendars, appointment books, and a variety of strate-
gies to manage their time effectively.
Time management is defined as the awareness that time can be manip-
ulated; the active use of skills, strategies, and tools; creating and maintaining
a flexible routine; and the ability to evaluate one’s personal effectiveness.
Managing time effectively involves metacognitive processing: estimating
time, organizing, and following through with plans while exercising emo-
tional control for optimal goal achievement (Britton & Glynn, 1989; Lakein,
1991; Macan, 1994; Richards, 1987).
The activity and participation concepts of the International Classification
of Functioning, Disability and Health (ICF) highlight the central role of time
Downloaded by [Colorado College] at 17:22 04 December 2014

management. For example, budgeting time and making plans for separate
activities throughout the day is basic to the ICF category of Carrying out
daily routine, defined as carrying out simple or complex and coordinated
actions in order to plan, manage, and complete the requirements of day-to-
day procedures or duties. The management of energy and time is also fun-
damental to the ICF activity of Managing one’s own activity level (World
Health Organization, 2001).
The Occupational Therapy Practice Framework (AOTA, 2002, 2008)
defines performance patterns as the roles, habits, and routines used in the
process of engaging in occupations or activities. Performance patterns, which
all inherently involve the use of time, develop in response to environmental
demands and are shaped by a person’s abilities. Patterns that once were
adaptive may become unsuccessful as a person’s abilities and needs change.
Establishing more effective performance patterns requires active organiza-
tion, management, and adaptation to current environmental realities. When
practitioners understand a client’s patterns of performance and time-use,
they are better able to assess how effectively performance skills and occupations
are integrated into the client’s life. Skilled performance becomes productive
when it is embedded in patterns of behavior that meet social expectations
for timeliness and consistency. When a client is unable to embed those skills
in a productive set of performance patterns, health and participation may be
negatively affected (AOTA, 2008).
Occupational therapists have long understood that purposeful use of
time is central to the profession, as it can be both health maintaining and
health producing (Christiansen, 1996; Law, 2002; Law, Steinwender, & Leclair,
1998). Time-use research has been conducted by occupational therapists in a
range of populations (Bejerholm & Eklund, 2004; Huang & Zhang, 2001;
Janeslätt, Granlund, Kottorp & Almqvist, 2010). The tools used by occu-
pational therapists for measuring time-use include time-use diaries, experi-
ence sampling methods, time-use observations, time-use surveys, and time
geography (Eklund, Leufstadius, & Bejerholm, 2009). This research employs
 Assessment of Time Management Skills 217

time-use measurement tools to concretely depict how a person spends specific

hours engaged in daily activities. Time management, as distinguished from
time-use, identifies the active processes of manipulating time, creating flexible
schedules, using tools and strategies, and continually evaluating personal
effectiveness.
Serious Mental Illness (SMI) is a severe condition that often co-occurs
with substance-related disorders (SRD). Four million (12.2%) U.S. adults have
both SMI and SRD (SAMHSA, 2002). The lifetime prevalence rate of sub-
stance abuse in the SMI schizophrenic population is close to 50%, and esti-
mates of recent or current substance abuse range from 20% to 65%. (Mueser,
Wallace, & Liberman, 1995; Regier et al., 1990). People with both disorders
are referred to as having co-occurring disorders (SMI/SRD). Research has
shown that, among people with severe mental illness and a co-occurring
Downloaded by [Colorado College] at 17:22 04 December 2014

substance-use disorder, 50% get no care, 45% get poor treatment, and 5% get
evidence-based practice (Drake, Mueser, Clark, & Wallach, 1996). Co-occurring
severe mental illness and substance abuse is therefore a major health prob-
lem. This fact has been well established since the 1980s, yet there has been
relatively little research on treatment for people with co-occurring disorders
(McHugo et al., 2006).
Time management challenges affect many clients, especially those with
cognitive processing deficits. Cognitive processing difficulties are common
among people with SMI/SRD, as well as other disorders that contribute to
deficits in cognitive functioning, such as traumatic brain injury. The occupa-
tional therapy cognitive disability model provides an optimal foundation for
developing time management training for people with cognitive limitations
associated with SMI/SRD (Allen, Earhart, & Blue, 1992; White, Meade, &
Hadar, 2007). This is a particular treatment need for people with SMI/SRD,
many of whom have impairment in the cognitive processing skills which
lead to significant problems with time management. Even common disorders
such as mood and anxiety disorders are characterized by interference with
attention and decision-making (American Psychiatric Association, 2000).
Research indicates that 75–85% of people with SMI have cognitive impair-
ment, with various profiles involving deficits in attention, memory, speed of
information processing, and problem-solving (Nuttbrock, Rahav, Rivera,
Ng-Mak, & Struening, 1997; Revheim & Medalia, 2002). When SMI co-occurs
with SRD, the likelihood of impairment increases.
To our knowledge, there are few existing client-centered time manage-
ment tools: “On Time Management, Organization and Planning Scale” (ON-
TOP), “Weekly Calendar Planning Activity” (WCPA), and “Profiles of
Occupational Engagement for People With Schizophrenia” (POES). Solanto,
Marks, Mitchell, Wasserstein, and Kofman (2008) have developed the
24-item self-reported questionnaire “ (ON-TOP) for their psychosocial treat-
ment of adults with Attention Deficit Hyperactive Disorder (ADHD). The
data from this questionnaire are currently being standardized. Items from
 218 S. M. White et al.

this questionnaire were reviewed and contain such questions as complet-

ing a to-do list and maintaining an organized environment. The factor
analysis of this tool supports the functional interference of hallmark behav-
iors of ADHD, inattention and distractibility, and are the focus of the tool.
Presently, Toglia (2009), who specializes in tool design for Adults with
Traumatic Brian Injury, is in the process of collecting normative data on
the WCPA. This new performance-based assessment of executive function
measures organizational and planning strategies, accuracy in following
written instructions, and use of self-reflection. The WCPA has been piloted
and a baseline profile of youth-at-risk established. This assessment allows
detection of complex task performance, strategy use, and self-evaluation for
performance and error patterns, which can provide an expanded view of
the client’s capacities and specific intervention strategies (Weiner, Toglia, &
Downloaded by [Colorado College] at 17:22 04 December 2014

Berg, 2012). This activity-based tool could be used as an additional evalu-

ative measure to obtain a baseline of the client’s existing cognitive strate-
gies. At present, neither of these tools is specifically designed for the client
with SMI/SRD.
Eklund et al. (2009) state,

after searching and reviewing databases of CINAHL and Pub Med using
a combination of terms such as time use, time geography, occupational
balance, occupational pattern, assessment, intervention, occupational
therapy, lifestyle redesign, well-being, and mental health that the literature
search did not reveal one single publication of an intervention focusing
on time use. (p. 187)

Subsequent to that literature review, there has been one published

occupational therapy intervention focused on time use, Activity over Inertia
(AOL). Edgelow and Krupa (2011) published Randomized controlled pilot
study of an occupational time-use intervention for people with serious mental
illness using the AOL intervention. Their method of data collection included
time-use information using POES (Bejerholm & Eklund, 2006). The POES
uses data collected from a 24-hour time diary, which measures the amount
of actual time spent in an activity and personal experience or preference for
the activity, but does not identify the active processes of manipulating time
or creating flexible schedules.
Therefore, the existing time-use measurement tools do not capture self-
assessed behaviors or concepts of strategy building nor describe how clients
actively manage their time (e.g., prioritizing or using cognitive adaptors such
as appointment books). Many of these time-use tools also require significant
effort to complete. Thus, current tools are insufficient and impractical for
evaluating the behavioral changes expected to result from a well-designed
time management intervention.
 Assessment of Time Management Skills 219

The recovery process for people with SMI/SRD requires them to take
greater responsibility for managing ever-increasing activity levels and
daily routines. The Occupational Therapy: Let’s Get Organized time man-
agement program was designed and developed within the framework of
the cognitive disability model for use with these clients. The program
aims to foster development of effective time management habits and use
of cognitive adapters such as maintaining an appointment book, wearing
a watch, and using goal-directed, trial and error learning strategies of
cognitive rehabilitation. The program was developed over several years
of clinical interventions with individual clients and more recently with
small groups. Clients’ participation in 20 one-hour sessions over a 10-week
period allows them to develop the understanding and habits needed to
ensure that they can effectively implement time management practices in
Downloaded by [Colorado College] at 17:22 04 December 2014

their daily lives.

Preliminary outcomes, based on short, locally designed assessment
tools, demonstrated improvement in client behaviors, such as punctuality
and use of appointment books, that continued for most clients beyond the
10-week program (White et al., 2007). Anecdotally, clients typically reported
missing fewer appointments after they began using their appointment books.
In 2009, the Agency for Healthcare Research and Quality (AHRQ)
selected the Occupational Therapy: Let’s Get Organized intervention for
inclusion in the Health Care Innovation Exchange (White, 2009). The Health
Care Innovation Exchange is designed to support health care professionals
in sharing and adopting innovations that improve the delivery of care to
patients. AHRQ gave the intervention a moderate evidence rating stating,
“The evidence consists of a small, pre- and post-implementation compari-
son of knowledge and behaviors related to time management, along with
anecdotal, post-implementation reports from staff on changes in partici-
pants’ skills” (White, 2009).
Despite the early success of the Occupational Therapy: Let’s Get
Organized program, a standardized assessment tool was necessary to dem-
onstrate its effectiveness across multiple samples. The Assessment of Time
Management Skills (ATMS) was developed to measure the extent to which
people actively manage their own behavior to ensure effective management
of time. This article describes its development, content validity, and internal
consistency and test–retest reliability. Two phases of work are described. The
first phase consisted of establishing content validity using the SMI/SRD pop-
ulation. The second phase included the analysis of test–retest reliability and
internal consistency with the general population, a critical step in demon-
strating the psychometric strength of the ATMS to evaluate clients’ use of
time management strategies, tools, and skills. The ATMS is designed to assess
clients’ initial needs and evaluate their success following a time management
intervention.
 220 S. M. White et al.

METHOD

Content Validity in the Population of SMI/SRD (Phase 1)

The development of the ATMS proceeded naturally from our clinical
work, working with clients with SMI/SRD to enhance their use of time
management skills, using the Occupational Therapy: Let’s Get Organized
program. At all stages of the ATMS development, we have asked for
feedback from stakeholders, the general population, experts, and con-
sumers, consistent with recommendations from the Institute of Medicine
report, Crossing the Quality Chasm: “the shift toward patient-centered
care has meant that a broader range of outcomes from the patient’s
perspective needs to be measured in order to understand the true
benefits and risks of healthcare interventions” (Institute of Medicine,
Downloaded by [Colorado College] at 17:22 04 December 2014

2001, p. 11, Recommendation 9.2).

Multiple steps were involved. Initially, intervention outcomes were assessed
in a pre- and post-test design using the Time Management Knowledge Scale
(TKMS) from the Living Skills Recovery Workbook—six open-ended ques-
tions designed to assess time management knowledge (Precin, 1999) and
four Likert-scale items that assess behavioral changes expected from the
intervention. Paired-sample t-tests were used to compare pre- and post-test
scores for 16 clients who participated in the Occupational Therapy: Let’s Get
Organized clinical intervention. Scores on the TKMS showed significant
improvement after participation in the group intervention (t = 4.06, df = 15,
p = 0.001). Results support the efficacy of the intervention for improving
knowledge related to time management. Thus, the next steps in the design
of a scientifically based outcome measure were undertaken (White, 2007).
Subsequently, a review of the literature identified a large number of
articles, mainly about time management for healthy employed adults, typi-
cally those with rich, and sometimes pressured, performance patterns (Britton
& Glynn, 1989; Lakein, 1991; Macan, 1994; Richards, 1987). Key aspects of
the time management construct were identified, and items were developed
to operationalize each aspect. This was followed by three focus groups with
healthy adults and consultation with mental health professionals who are
familiar with SMI/SRD populations. The outcomes of the literature review,
focus groups, and experts allowed us to identify 10 core aspects of the time
management construct. These guided the development of items, ensuring
content validity (see Table 1).
Once these aspects of time management were identified, the previously
used questions and other instruments were reviewed for relevance. A 30-item
Likert style instrument was developed to operationalize each aspect of the
time management construct. A decision was made to use only items that
clearly describe specific behaviors, in order to increase the likelihood that
responses are accurate and consistent.
 Assessment of Time Management Skills 221

TABLE 1 Components of Effective Time Management Behavior

1. Awareness of the patterns of time within a day, week, etc., and awareness of how
much time is needed to accomplish routine tasks
2. Awareness of personal energy cycles and influences on attention and affect
3. Ability to organize the day to take advantage of personal energy patterns and
optimize capacities
4. Use of cognitive adaptors, external sources, to organize time (appointment books,
watches, notes, lists, etc.)
5. Expanding learning repertoire, building on “trial and error learning,” valuing
vicarious learning
6. Neutralizing anxiety about learning and managing time
7. Developing routines and habits, including time for recovery and sleep
8. Feeling a sense of competence
9. Willingness to accomplish tasks for social approval
10. Planning, ordering, following through in a sequence while self-monitoring the
results
Downloaded by [Colorado College] at 17:22 04 December 2014

In September 2007, a panel of experienced clinicians and researchers

involved with the SMI/SRD population met and reviewed the ATMS. They
made suggestions on the wording and inclusion of information on the demo-
graphic sheet. These suggestions were incorporated into the ATMS question-
naire. Following this, in March and June 2008, three workshops on time
management were held with mental health consumers, caregivers, and
clinicians, for whom an early version of the ATMS was given. Before each
session, attendees were asked to “Please help us understand better how
YOU manage your time by filling out this anonymous questionnaire.”
Included in the directions for the ATMS was a request: “write ‘Unclear’ next
to any item you feel was unclear.” We did this to ensure comprehension of
items by the general population of mental health consumers. The handwrit-
ten comments of those who chose to respond were reviewed. Frequency
distributions were computed for each item. The ATMS was modified based
on these findings. Specifically, some words and phrases were changed in
response to narrative suggestion, and some items were removed because the
frequency distribution data showed limited range of responses to these
items. The response format was changed from a 5-point to a 4-point Likert
scale to eliminate the midpoint and force a more definitive choice. To help
clients focus their attention on the concept of time, responses were relabeled
“None of the Time, Some of the Time, Most of the Time, All of the Time.”
All clients completed the ATMS in less than 10 minutes and were able to
do so in a group format. An informal, voluntary focus group was held fol-
lowing one of the training sessions. Here, participants expressed their opin-
ions that the ATMS was measuring relevant components of time management
and time-relevant organizational skills. In addition, they confirmed that the
experience of completing the questionnaire was emotionally neutral; no
question elicited a positive or negative affective response. This provided
 222 S. M. White et al.

initial support that the ATMS can be used with this client population in a
group format and assesses time management skills and organization, thus
helping to establish its feasibility, acceptability, and face and content validity.
The final version of the ATMS consists of 30 items, with a 4-point
response scale, with a range of possible scores from 30–120 (see Appendix).
Items assess three aspects of time management: awareness that you can
manage time, active behaviors and specific skills used to manage time, and
self-assessment of time management skills. Higher scores indicate better
performance of time management skills and practices; eight items are
reverse-scored. The Fleisch Reading Grade Level was estimated to be at the
4.8 grade level.

Reliability of the ATMS in a General Population (Phase 2)

Downloaded by [Colorado College] at 17:22 04 December 2014

A large general-population sample was then addressed to test the internal

consistency reliability and test–retest reliability of the ATMS. Internal consis-
tency estimates the extent to which items in a scale are inter-correlated.
Test–retest reliability provides a measure of the stability of the respondent’s
answers when no change has occurred. Using established criteria for reli-
ability ensures that the items have the same meaning across a range of
respondents; that the items are interrelated so that they effectively measure
a single concept; that the meaning is sufficiently clear that, on a second read-
ing, the items are interpreted the same way; and that the characteristic being
measured is sufficiently stable over a short period of time (McDowell &
Newell, 1996). Good reliability is necessary for an instrument to be valid.
The general population was involved in response to the local ethics
committee (SUNY Downstate Medical Center Institutional Review Board
[IRB]) recommendation that the ATMS be administered to a general popula-
tion sample prior to use with a vulnerable sample. This was based on a
concern about the ability of persons with psychiatric and SRD to provide
voluntary informed consent for research when the tool had not been tested
in the general population. The study was then designed as a computer-
administered test of the ATMS in a general population, which the IRB approved
as exempt research.
The sample comprised 241 adults, but there were only 109 complete data
sets from the general population recruited through Market Tools’ TrueSample,
a quality assured sampling service that is provided by Zoomerang, an online
survey tool. They screened all participants to ensure they were credible sub-
jects consistent with the study’s inclusion criteria of an age between 18 and 65
and an understanding of English, and exclusion criteria of no current involve-
ment in specialized training or education in time management.
Potential subjects read and agreed to terms and conditions pertaining
to privacy and security. Volunteers who met the inclusion criteria logged
onto the Zoomerang website to complete the survey at any time during the
 Assessment of Time Management Skills 223

designated day from any location they chose. They received $10 via Zoomerang
for completing the first administration. A week later, they received the ATMS
again with a reminder that, by completing it, they would again receive $10.
No identifiers were collected from participants. The data collected were sent
to the researchers in an Excel spreadsheet.

Analytic Plan
Two samples were analyzed, the 241 respondents who initially completed
the ATMS and the 109 who completed the ATMS two times, one week apart.
All data were inspected for expected values and outliers. Cleaned data were
analyzed using SAS 9.2TM (SAS Institute, 2008). Descriptive statistics were com-
puted for the two samples, including scale mean, standard deviation (SD)
Downloaded by [Colorado College] at 17:22 04 December 2014

and range of scores. The two standard measures of reliability were computed
to ensure that the ATMS has an acceptable level of error.
The internal consistency of the ATMS was estimated using Cronbach’s
alpha (α). Internal consistency is a measure of the extent to which the items
in a scale are interrelated, indicating how well the scale measures a single
concept. The minimum criterion for α was 0.70 (Nunnaly, 1978). This test
helps to identify items that do not contribute to the concept being measured.
Initially we planned to drop the items that were poorly correlated with the
others and re-compute the a if the internal consistency of the ATMS was not
above 0.7. The one-week test–retest reliability at the item and scale levels
was assessed using the Pearson product-moment correlation. This statistic is
a measure of the degree of consistency between two distributions of scores.
Floor and ceiling effects were assessed to determine the proportion of
respondents with the lowest and highest possible scores, respectively. Scales
with a large floor effect, many people having the worst score, are not able to
reflect decreased performance over time among those with poor perfor-
mance. Likewise, a scale with a high ceiling effect is not useful for assessing
improved performance among those who initially score high.

RESULTS

Descriptive statistics for age, sex, marital status, race, and level of education
are presented in Table 2. The initial sample of 241 adults was used to com-
pute internal consistency reliability and ceiling and floor effects.
Of these people, 109 completed the ATMS a second time, and they
comprise the sample on which retest reliability was computed. Their charac-
teristics were very similar to that of the initial sample, with the exception that
there were slightly more women than men in this sample (see Table 2).
The mean score of the 241 adults in the full sample was 83 (SD = 11).
The 109 adults who completed the ATMS twice had a mean score of 83
 224 S. M. White et al.

TABLE 2 Characteristics of Two Samples of Participants

N (%)

N = 241 N = 109

Age (years) 40 (SD = 12) 39 (SD = 12)

Gender
Male 112 (46) 48 (44)
Female 129 (53) 61 (56)
Race and ethnicity
Native American 1 (0) 0
African American 12 (5) 1 (1)
Asian American/Pacific 7 (3) 2 (2)
Islander
White (non-Hispanic) 207 (86) 100 (92)
Hispanic 9 (4) 6 (6)
Downloaded by [Colorado College] at 17:22 04 December 2014

Other (specify) 5 (2) 0

Marital Status
Single (never married) 76 (32) 38 (35)
Married 112 (46) 47 (43)
Living with significant 6 (2.5) 2 (2)
other
Separated 28 (12) 15 (14)
Divorced 1 (0.4) 0
Widowed 18 (7.5) 7 (6)
Education
Less than high school 10 (4) 3 (3)
High school 39 (16) 22 (20)
Vocational or technical 18 (7) 9 (9)
training
Some college 78 (32) 34 (31)
Graduated college 71 (29) 30 (28)
Post graduate education 25 (10) 11 (10)

(SD = 11) on the first administration, and 82 (SD = 11) on the post-test one
week later. There were no ceiling and floor effects, as no one had either the
highest or lowest possible scores (see Table 3).
The Cronbach’s alpha reliability of the 30-item scale was 0.86, indicat-
ing the scale has good internal consistency reliability. There was no need to
delete any of the items because doing so did not improve the level of
reliability.
The Pearson correlation between the pre-test scores and the post-test
scores of the 109 people who completed the ATMS was highly correlated
(r = 0.89), indicating very good stability of the tool over a one-week period.

DISCUSSION

The content validity and reliability of the ATMS were confirmed in this set of
studies designed to develop and provide preliminary validation. The psycho-
metric data obtained in this general population sample demonstrate that it is
 Assessment of Time Management Skills 225

TABLE 3 Descriptive Statistics

Variable N Mean SD Range

Full sample* 241 82.93 10.53 54.0–111.0

Retest sample**
Pre-score 109 83.31 11.02 57.0–111.0
Post-score 109 81.83 11.00 54.0–111.0

*Sample used to compute internal consistency reliability and ceiling and floor effects. **Sample used to
compute retest reliability.

understandable, relevant, and practical to administer, even using a computer.

The excellent internal consistency and test–retest reliability demonstrate that
the items have a consistent meaning across people and time, and that they
provide a coherent measure of time management.
Downloaded by [Colorado College] at 17:22 04 December 2014

These results indicate that the ATMS is reliable and practical for use in
the general population, many of whom struggle with time management
throughout their lives (Richards, 1987). This sample’s average score of 82 out
of a possible “best” score of 120 indicates that many people do not regularly
use the most efficient time management practices. The median and mode
were very close to this mean (83 and 83 respectively), indicating that these
scores were obtained by most people. Time management is likely to be
beneficial in a range of interventions in the populations served by occupational
therapists, including stress management and vocational and educational pro-
grams. The ATMS is a reliable measure for evaluating clients’ time management
skills and for use as an outcome tool for evaluating the effectiveness of the
intervention. All items were endorsed across the full range of responses,
indicating that they are relevant even for the general population. This
suggests that the three areas of focus—awareness that time can be managed,
use of skills and practices, and self-evaluation of the effectiveness of time
management—are meaningful to the general population. The psychometric
characteristics of the ATMS, when used with clients with cognitive limitations,
cannot be completely answered until the ATMS is fully tested in this popula-
tion. The strong reliability observed in this study, however, provides the
foundation for further study in clinical populations, even populations consid-
ered vulnerable, such as those in treatment for psychiatric and substance abuse
disorders.

Clinical Application
The content validity of the ATMS has been developed through several years
of clinical assessment with clients in time management interventions and
through input from clinical staff involved in our work as well as other stake-
holders involved in focus groups. We have also begun to demonstrate the
criterion validity of the ATMS in our clinical work. That is, the summary
score provides us with a useful evaluation of the extent of the needs of
 226 S. M. White et al.

individual clients, and a review of their responses provides a practical means

for identifying and communicating with clients about their strengths and
weaknesses.
The average score of 82 on the ATMS provides a benchmark for evalu-
ating clinical populations who are expected to score much lower on the
ATMS than the general population. It is also possible that the structure of the
ATMS will be different in clinical populations. In current work, a principal-
components analysis (factor analysis) was conducted to determine the
number of unique factors present. Three aspects of time management were
expected, but the analysis indicated only a single factor for the ATMS.
Although this is generally good, showing that all items were interrelated and
relevant, a single underlying factor could probably be assessed with a smaller
number of items. As the three aspects of time management included in the
Downloaded by [Colorado College] at 17:22 04 December 2014

ATMS (awareness that time can be managed, skills and practices, self-moni-
toring) may differ substantially for clients with different kinds of cognitive
deficits, and all items appear to contribute to the overall score, no effort was
made to reduce items or define the underlying factor structure at this time.
As an example of concern that clinical populations may differ from the
general population, the four items that were least well-correlated with the
total score focused on affect regulation, (feeling overwhelmed, mood affecting
time management behaviors, rushing, waiting to feel better). Affect instability
is a cardinal symptom in many clinical populations, so no attempt was made
to eliminate these items, which are likely to be valuable in treatment plan-
ning and outcome evaluation.

Limitations
The limitations inherent in any study of a single sample apply, especially in
terms of generalizability. This study cannot indicate how reliable the ATMS
will be in other samples of the general population nor in very different
populations, particularly in the vulnerable populations of clients with cogni-
tive limitations secondary to psychiatric, substance use, and other disorders,
for whom the Occupational Therapy: Let’s Get Organized program and the
ATMS were initially designed. Moreover, the construct validity of an out-
come questionnaire is an ongoing process, requiring multiple tests in rele-
vant populations and comparison of results with other established measures
(Anastasi, 1986).

Advances of the ATMS

Clinical interventions by occupational therapists can only become practice-
based evidence when they are shown to work using conceptually based,
reliable, and valid tools (D’Amico, Jaffe, & Gibson, 2010; Tsang, Sui, & Lloyd,
2011). Our original literature review failed to identify a single publication
 Assessment of Time Management Skills 227

using a self-report time management assessment instrument. ATMS provides

a practical and relevant scale for assessing understanding of time manage-
ment, use of skills, and self-evaluation in the general population and those
with cognitive limitations. To our knowledge, two other population-focused
time management instruments are also in the process of being validated.
ON-TOP, a 24-item self-reported questionnaire to evaluate psychosocial
treatment of adults with ADHD, is currently being standardized (Solanto,
et al., 2008). Some items are predictably similar to those in the ATMS. The
factor analysis, however, indicates that inattention and distractibility are a
focus of the ON-TOP tool, as these hallmark behaviors of ADHD are known
to interfere with time and self management. A second tool (WCPA), being
developed by Toglia and designed for adults with traumatic brain injury, may
be useful for the study population also served by occupational therapists. It
Downloaded by [Colorado College] at 17:22 04 December 2014

is an individual task assessment measuring organization, planning strate-

gies, accuracy in following written instructions, and use of self-reflection.
Normative data are currently being collected. This activity-based tool could
be used as an additional evaluative measure to obtain a baseline of the
client’s existing cognitive strategies. At present, neither of these tools is
standardized nor specifically designed for the client with SMI/SRD. The
POES, a time-use diary, is designed for the SMI population but is interview-
based and reflects the professional’s judgment (Bejerholm & Eklund, 2006).
The ATMS, however, allows the client to self-assess their own active processes
of managing time or creating flexible schedules.

Future Research
The demonstration of the reliability and preliminary validity of the ATMS in
the general population was the first step in a larger project designed to dem-
onstrate the validity of the ATMS as an outcome evaluation tool for occupa-
tional therapy time management interventions, such as the Occupational
Therapy: Let’s Get Organized program. Only by establishing the validity of
outcome tools will the field of occupational therapy be positioned to make
substantial contributions to the body of practice-based evidence. Since “dem-
onstrating instrument validity is an ongoing process” according to Anatasi, a
developer of the science of measurement (Anastasi, 1986, p. 12), future
research with the ATMS will involve clinical populations with severe mental
illness and substance use disorders, as well as other disorders associated
with cognitive impairments.

CONCLUSION

The ATMS has the potential to evaluate the effectiveness of occupational

therapy interventions designed to improve the time management skills and
 228 S. M. White et al.

organization and ultimately the social and occupational engagement of

patients with mental illness, substance disorders, and other disorders that
contribute to deficits in cognitive functioning such as traumatic brain injury.

ACKNOWLEDGMENTS

This project was possible because of the efforts of SUNY Downstate

Occupational Therapy Program Master’s Project, Dr. Joyce Sabari and the
Palladia, Inc., Starhill community. The following students generously contrib-
uted to various stages of this project: Thomas Trainor, Marc Freedman, Lena
Agyeman, Israel Cywiak, Yekaterina Kitachik, Mendel Klein, Shaquah West,
Nicole Ciezar, Shira Goldberg, Aliza Vishniavsky, Elyssa Yablow, and Morgan
Downloaded by [Colorado College] at 17:22 04 December 2014

Kalberer. We also appreciate Marge Hudson’s valuable editorial guidance.

Preliminary results of this study were presented at the World Federation of
Occupational Therapy Congress xxx, Santiago, Chile, May, 2010.

REFERENCES

Allen, C. K., Earhart, C. A., & Blue, T. (1992). Occupational therapy treatment goals
for the physically and cognitively disabled. Bethesda, MD: American Occupational
Therapy Association.
American Occupational Therapy Association [AOTA]. (2002). Occupational Therapy
Practice Framework: Domain and process. American Journal of Occupational
Therapy, 56, 609–639.
American Occupational Therapy Association [AOTA]. (2008). Occupational Therapy
Practice Framework: Domain and process (2nd ed.). American Journal of
Occupational Therapy, 62, 625–683.
American Psychiatric Association. (2000). Diagnostic and statistical manual of
mental disorders (4th ed., text rev.). Washington, DC: Author.
Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology,
37, 1–16.
Bejerholm, U., & Eklund, M. (2004). Time use and occupational performance among
persons with schizophrenia. Occupational Therapy in Mental Health, 20, 27–47.
Bejerholm, U., & Eklund, M. (2006). Construct validity of a newly developed instru-
ment: Profiles of occupational engagement for people with schizophrenia
(POES). Nordic Journal of Psychiatry, 60(3), 200–206.
Britton, B. K., & Glynn, S. M. (1989). Mental management and creativity: A cognitive
model of time management for intellectual productivity. In J. A. Glover, R. R.
Ronning, & C. Reynolds (Eds.), Handbook of creativity (pp. 429–440). New
York, NY: Plenum Press.
Christiansen, C. (1996). Nationally speaking-managed care: Opportunities and chal-
lenges for occupational therapy in the emerging systems of the 21st Century.
American Journal of Occupational Therapy, 50, 409–412.
 Assessment of Time Management Skills 229

D’Amico, M., Jaffe, L., & Gibson, R. W. (2010). Centennial vision: Mental health
evidence in the American Journal of Occupational Therapy. American Journal
of Occupational Therapy, 64, 660–669.
Drake, R. E., Mueser, K. T., Clark, R. E., & Wallach, M. A. (1996). The course, treat-
ment, and outcome of substance disorder in persons with severe mental illness.
American Journal of Orthopsychiatry, 66, 42–51.
Edgelow, M., & Krupa, T. (2011). Randomized controlled pilot study of an occupa-
tional time-use intervention for people with serious mental illness. American
Journal of Occupational Therapy, 65, 267–276.
Eklund, M., Leufstadius, C., & Bejerholm, U. (2009, Winter). Time use among people
with psychiatric disabilities: Implication for practice. Psychiatric Rehabilitation
Journal, 32, 177–191.
Huang, X. T., & Zhang, Z. J. (2001). Time management disposition and quality of life:
An exploratory study. Advances in Psychological Science, 3, 338–343.
Downloaded by [Colorado College] at 17:22 04 December 2014

Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the
21st Century. Washington, DC: Author.
Janeslätt, G., Granlund, M., Kottorp, A., & Almqvist, L. (2010, May). Patterns of time
processing ability in children with and without developmental disabilities.
Journal of Applied Research in Intellectual Disabilities, 23(3), 250–262.
Lakein, A. (1991). How to get control of your time and your life. New York, NY: New
American Library.
Law, M. (2002). Participation in occupations of everyday life. American Journal of
Occupational Therapy, 56, 640–649.
Law, M., Steinwender, S., & Leclair, L. (1998). Occupation, health, and well-being.
Canadian Journal of Occupational Therapy, 65, 81–91.
Macan, T. H. (1994). Time management: Test of a process model. Journal of Applied
Psychology, 79, 381–391.
McDowell, I., & Newell, C. (1996). Measuring health: A guide to rating scales and
questionnaires (2nd ed.). New York, NY: Oxford University Press.
McHugo, G. J., Drake, R. E., Brunette, M. F., Xie, H. I., Essock, S. M., & Green, A. I.
(2006). Enhancing validity in co-occurring disorders treatment research.
Schizophrenia Bulletin, 32, 655–665.
Mueser, K. T., Wallace, C. J., & Liberman, R. P. (1995). New developments in social
skills training. Behavior Change, 12, 31–40.
Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw Hill.
Nuttbrock, L., Rahav, R. M., Rivera, J., Ng-Mak, D., & Struening, E. (1997). Mentally
ill chemical abusers in residential treatment programs: Effects of psychopathol-
ogy on levels of functioning. Journal of Substance Abuse Treatment, 14(3),
269–274.
Precin, P. (1999). Living skills recovery workbook. Woburn, MA: Butterworth-Heinemann.
Regier, D. A., Farmer, M. E., Rae, D. S., Locke, B. Z., Keith, S. J., Judd, L. L., &
Goodwin, F. K. (1990). Co-morbidity of mental disorders with alcohol and other
drug abuse. Results from the Epidemiological Catchment Area (ECA) study.
Journal of American Medical Association, 19, 2511–2518.
Revheim, N., & Medalia, A. (2002). Verbal memory, problem-solving and community
status in schizophrenia. Schizophrenia Research, 68, 149–158.
Richards, J. H. (1987). Time management: A review. Work & Stress, 1, 73–78.
 230 S. M. White et al.

SAS Institute. (2008). SAS/ACCESSTM 9.2TM Interface to System 2000TM. Cary, NC: Author.
Solanto, M. V., Marks, D. J., Mitchell, K. J., Wasserstein, J., & Kofman, M. D. (2008,
May). Development of a new psychosocial treatment for adult ADHD. Journal
of Attention Disorders, 11(6), 728–736.
Substance Abuse and Mental Health Services Administration [SAMHSA]. (2002). For
the development of evidence-based treatments within this recovery model. Retrieved
May 10, 2009, from http://mentalhealth.samhsa.gov/cmhs/CommunitySupport/
toolkits/cooccurring/
Toglia, J. (2009). Weekly Calendar Planning Activity. Dobbs Ferry, NY: Mercy College.
Tsang, H. W. H., Sui, A. M. H., & Lloyd, C. (2011). Evidence-based practice in mental
health. In C. Brown & V. C. Stoffel (Eds.), Occupational therapy in mental health:
A vision for participation (pp. 57–69). Philadelphia, PA: FA Davis Company.
Weiner, N. W., Toglia, J., & Berg, C. (2012). Weekly Calendar Planning Activity (WCPA):
A performance-based assessment of executive function piloted with at-risk
Downloaded by [Colorado College] at 17:22 04 December 2014

adolescents. The American Journal of Occupational Therapy, 66(6), 699–708.

White, S. M. (2007). Let’s get organized: Cognitive intervention for persons with
co-occurring disorders. Psychiatric Services, 58, 713.
White, S. M. (2009, June 8). Time management program helps substance abuse facility
residents with co-occurring disorders improve skills required for community
reentry. Retrieved July 8, 2010, from http://www.innovations.ahrq.gov/content.
aspx?id=2310
White, S. M., Meade, S.-A., & Hadar, L. (2007, June 25). OT cognitive adaptation: An
intervention in time management for persons with co-occurring conditions. OT
Practice, 12, 9–14.
World Health Organization. (2001). International classification of functioning,
disability and health (ICF). Geneva, Switzerland: Author.

APPENDIX
ASSESSMENT OF TIME MANAGEMENT SKILLS (ATMS)

For each statement below, please circle the number that indicates how often
the statement applies to you. (None of the time = 1, Some of the time = 2, Most
of the time = 3, All of the time = 4)
None of Some of Most of All of the
the time the time the time time

1. I feel I manage my time well. 1 2 3 4

2. I use a calendar or an appointment 1 2 3 4
book as a way of remembering
my daily tasks.
3. I look at the calendar or appointment 1 2 3 4
book during the day to keep track
of my daily schedule.
4. I carry an appointment book. 1 2 3 4
5. I stop and plan out the steps before 1 2 3 4
I start something new.
6. I plan my daily activities. 1 2 3 4
7. (RS) I rush while completing my work. 1 2 3 4
(Continued)
 Assessment of Time Management Skills 231

None of Some of Most of All of the

the time the time the time time
8. I do my most difficult work at the 1 2 3 4
time of day when I have the most
energy.
9. (RS) I find that I am overwhelmed by my 1 2 3 4
daily routine.
10. I find that, even though I want to be 1 2
on time, I am often late.
11. Even if I do not like to do some- 1 2 3 4
thing, I still complete it on time.
12. (RS) I am not organized in my tasks. 1 2 3 4
13. I clear my work space before 1 2 3 4
beginning a task.
14. I complete the tasks on my schedule 1 2 3 4
or appointment book to my
Downloaded by [Colorado College] at 17:22 04 December 2014

satisfaction.
15. I make to-do lists. 1 2 3 4
16. I wait until I feel better before taking 1 2 3 4
on important tasks.
17. I reward myself for doing a good 1 2 3 4
job.
18. (RS) I put off things I do not like to do 1 2 3 4
until the very last minute.
19. I can correctly estimate the time I 1 2 3 4
need to complete my tasks.
20. I learn from my mistakes. 1 2 3 4
21. I make sure I have a good night’s 1 2 3 4
sleep.
22. (RS) I feel competent about managing my 1 2 3 4
time when I write down my
appointments.
23. My mood affects my ability to 1 2 3 4
manage my time.
24. I feel confident that I can complete 1 2 3 4
my daily routine.
25. I put in more effort to follow my 1 2 3 4
schedule when I see others
keeping up with their schedule.
26. (RS) I run out of time before I finish 1 2 3 4
important things.
27. I carry a pen or pencil daily. 1 2 3 4
28. I wear a watch or carry a cell phone 1 2 3 4
to keep track of the time.
29. I put my things back where they 1 2 3 4
belong or where I got them from.
30. (RS) I feel that I do not manage my time 1 2 3 4
well.

Note. RS indicates reverse score.