Acta Psychologica 235 (2023) 103900

Contents lists available at ScienceDirect

Acta Psychologica
journal homepage: www.elsevier.com/locate/actpsy

Validation study of the Roberts Apperception Test for Children (RATC) in

an adolescents' forensic sample
Pedro Armelim Almiro a, d, *, Pedro Rafael Fernandes Marques b, Mónica Costa Duarte b,
Isabel Marques Alberto b, Mário Rodrigues Simões c
a
Center for Research in Neuropsychology and Cognitive Behavioral Intervention (CINEICC, FPCEUC), Portugal
b
University of Coimbra (Portugal), Faculty of Psychology and Educational Sciences, Portugal
c
University of Coimbra (Portugal), Center for Research in Neuropsychology and Cognitive Behavioral Intervention (CINEICC), Faculty of Psychology and Educational
Sciences, Portugal
d
Autonomous University of Lisbon (Portugal), Psychology Research Centre (CIP/UAL), Portugal

A R T I C L E I N F O A B S T R A C T

Keywords: The Roberts Apperception Test for Children (RATC) is a projective method developed by McArthur and Roberts
Roberts Apperception Test for Children (RATC) (1982) to assess children and adolescents' behavioural, social and emotional functioning, concerns, conflicts and
Reliability analysis emotional management strategies through their perceptions of common interpersonal situations of everyday life.
Criterion validity
The aim of the present study is to contribute to the validation of the RATC in a forensic sample (N = 75)
Projective methods
Personality assessment
[constituted by a group of juvenile delinquents detained in educational centres (n = 40, 12–17 years old, 1–10
Juvenile delinquents years of schooling) and a group of maltreated adolescents integrated in residential care (n = 35, 11–16 years old,
Maltreated adolescents 5–10 years of schooling)], studying its psychometric properties, such as reliability and criterion validity
(convergent and discriminant validity), considering the results obtained in other instruments as external vali­
dation criteria: Wechsler Intelligence Scale for Children – Third Edition (WISC-III) and Youth Self-Report (YSR).
It also aims to search for some indicators based on means and standard deviations to interpret the scores obtained
in RATC for these forensic contexts, through the comparisons within forensic groups and the comparisons of the
forensic groups with a community sample. The RATC showed minimally acceptable reliability and adequate
validity indices, considering that this instrument is a projective method. This limitation is compensated by the
clinical value of the data obtained from the projection of individuals' thoughts, concerns, conflicts and problem-
solving styles, which are useful to assess their emotional and behavioural characteristics and psychological
functioning. The results also show statistically significant differences between the two forensic groups on RATC
scales, as well as between them and the community sample, as expected, underlining their different
characteristics.

1. Introduction (Ackerman, 2010; Heilbrun, 1992), and it is particularly critical in this

context given the implications it may have on personal life (e.g., re­
Psychological assessment is the discipline of scientific psychology striction of a person's freedom and well-being) and community (Heil­
which studies a given individual in a specific applied field (e.g., clinical, brun et al., 2009; Melton et al., 2007). The personality assessment, as a
forensic), using scientific tools (tests and other measurement in­ domain of psychological assessment, plays an important role to make
struments), with the purpose of answering individual's demands that informed decisions since it allows identifying what people are like and
require scientific operations such as describing, diagnosing, predicting, how they are likely to think, feel, and act (Cohen & Swerdlik, 2018;
and explaining (Cohen & Swerdlik, 2018). Weiner & Greene, 2017). Therefore, personality can be defined as the
Psychological assessment in the forensic settings allows the psy­ more or less stable and enduring organization of a person's character,
chologist to inform the court regarding the psychological functioning of temperament, intellect, and physique, which determines his unique
individuals, contributing to decision making in legal matters adjustment to the environment; character denotes a person's more or less

* Corresponding author at: Center for Research in Neuropsychology and Cognitive Behavioral Intervention (CINEICC, FPCEUC), Portugal.
E-mail address: [email protected] (P.A. Almiro).

https://doi.org/10.1016/j.actpsy.2023.103900
Received 14 March 2022; Received in revised form 13 February 2023; Accepted 27 March 2023
Available online 31 March 2023
0001-6918/© 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

stable and enduring system of conative behaviour (will); temperament, assumed that the individual projects his or her thoughts, concerns,
the system of affective behaviour (emotion); intellect, the system of conflicts and problem-solving styles onto these stories, therefore this
cognitive behaviour (intelligence); physique, the system of bodily instrument is a thematic approach to personality assessment.
configuration and neuro-endocrine endowment (Eysenck & Eysenck, The RATC consisting of 8 Adaptive Scales, 5 Clinical Scales and 3
1985). Or, in a simplest way, personality refers to psychological qualities Clinical Indicators, and this projective method was designed to allow for
that contribute to an individual's enduring and distinctive patterns of both quantitative and qualitative analyses (McArthur & Roberts, 1982).
feeling, thinking, and behaving, which distinguish one person from The Adaptive Scales are: Reliance on Others (REL), assesses individual's
another and are relatively stable in different situations and over time tendency to make up stories which characters reaches out to others for
(Cervone & Pervin, 2018). help in problem solving (it is an adaptative capacity to ask outside help);
Consequently, the personality is a fundamental component of the Support-Other (SUP-O), assesses the tendency to support others by
human being and it is characterised in a personal way and manifests giving help, emotional support, or material objects; Support-Child (SUP-
itself in all the attitudes, interests and behaviours of the individual. One C), assesses self-sufficiency and maturity as indicated by assertiveness or
of the means of access to the personality and its characteristics are the positive emotions; Limit Setting (LIM), assesses the extent to which
projective methods, based on the assumption that when faced with authority figures place reasonable limits on the child when he or she
ambiguous situations (a relatively unstructured materials), the indi­ break the rules; Problem Identification (PROB), assesses the ability to
vidual will respond according to his or her personality (Bellak, 1944; formulate concepts beyond the nature of the card (it is adaptive and
Cervone & Pervin, 2018; Frank, 1939, 1948). Therefore, projective requires lack of defensive behaviour to view others in dynamic in­
methods are the indirect methods to get access of individual's person­ teractions); Resolution-1 (RES-1), indicates a tendency to seek easy or
ality characteristics and they aim to apprehend the psychic dynamics of unrealistic solutions to problems (defence, denial, naivety); Resolution-
the individual as a whole; whereas the objective methods, through the 2 (RES-2), indicates a constructive resolution of a problem limited to the
self-report scales or questionnaires, are the direct methods to get access situation (of internal feelings, interpersonal relationship, external out­
of individual's personality characteristics (Anastasi & Urbina, 2000; comes); Resolution-3 (RES-3), indicates a constructive resolution of a
Cohen & Swerdlik, 2018). Nevertheless, self-report scales do not always problem with new insight beyond current problem.
are sufficiently discriminant and do not always show differences be­ The Clinical Scales are: Anxiety (ANX), assesses the manifest anxiety
tween the normative and clinical groups regarding emotional func­ or apprehension of the characters, and also worry, guilt, and remorse;
tioning, such as depression, anxiety and self-esteem. Thus, when Aggression (AGG), assesses the extent to which the characters express
children keep these perceptions to themselves to avoid them, the only anger and engage in physical or verbal aggression; Depression (DEP),
way to access them is through the use of projective methods (Leifer assesses sadness, despair, or physical symptoms of depression (fatigue,
et al., 1991). For example, according to Joiner (1996), the Roberts apathy, sleeplessness); Rejection (REJ), themes of separation, jealousy,
Apperception Test for Children (RATC) appears to suffer less from the discrimination, or feelings of being left out; Unresolved (UNR), when
effects of defensiveness than Children's Depression Inventory (CDI) to individual states a problem in the story which the characters are unable
evaluate depression, which may be an advantage; “self-report measures to or do not resolve.
may be more influenced by self-attributed than implicit motives, and And the Clinical Indicators are: Atypical Response (ATY), when in­
thus may be more affected by social desirability and defensiveness” dividual combines a number of responses which indicate extreme de­
(Joiner, 1996, p.804). Referring to self-report measures, social desir­ viation from the usual themes of the card, including distorted emotion,
ability refers to the bias or tendency of individuals to present themselves denial of obvious aspects of a picture, unrealistic content (primary
in a more favourable way toward others, giving answers that are in process thinking); Maladaptive Outcome (MAL), describes an outcome
accordance with social norms, and defensiveness refers to the tendency of characters acting in socially disapproved way, show inappropriate use
to be sensitive to criticism or comments about the person's limitations or of defences, resolving a problem by withdrawing or by taking over
difficulties, and to counter or deny them (as defined in VandenBos, autocratically, or acting out, deceiving, manipulating; Refusal (REF),
2015; see Wetzel et al., 2016). when individual refuses to give a response to a card (defensive behav­
In the domain of non-self-report measures to assess personality, the iour). In addition to the scales and indicators there are 3 supplementary
Roberts Apperception Test for Children (RATC) is a well-known projective measures: Ego Functioning Index, Aggression Index and Projection
method developed by Dorothea McArthur and Glenn Roberts to assess Levels (McArthur & Roberts, 1982). Defensive behaviour refers to the
children and adolescents' behavioural, social and emotional functioning, individuals' overuse of defence mechanisms operating at an unconscious
concerns, conflicts and emotional management strategies through their level (as defined in VandenBos, 2015).
perceptions of common interpersonal situations of everyday life (Anas­ The original RATC provides normative data from a sample of 200
tasi & Urbina, 2000; Cohen & Swerdlik, 2018; McArthur & Roberts, well-adjusted children (100 boys and 100 girls, aged between 6 and 15
1982). The RATC is based on the assumption that the child or adolescent years old) and the scores on each of the scales are converted into T-
participant responds according to his or her problems, characteristics scores (M = 50, SD = 10), with significant deviation operationalised as a
and coping strategies. Since the RATC uses novel scenarios through standard deviation in either direction (McArthur & Roberts, 1982).
cards, the participant responds with a minimum of distorting intention Regarding on psychometric studies of the RATC, McArthur and
or defensiveness, and he or she is less likely to engage in social desir­ Roberts (1982) obtained split-half reliability indices ranging from 0.86
ability bias. Aiken (1996, p.371) refers to the RATC as a “promising test to 0.44 using the Lord and Novick formula, and indices ranging from
which allows for the construction of stories on a wide range of topics, 0.86 to 0.48 using the Spearman-Brown correction (N = 400, 200
and which has a well-documented and easy to learn rating system”. normative group and 200 clinical group). And Alberto (1999), in a
The RATC is administered using 16 cards, 11 cards have different Portuguese sample, obtained internal consistency indices ranging from
versions for male and female participants, while the other 5 cards are 0.77 to 0.20 (N = 92).
gender neutral and are administered to all participants. These cards McArthur and Roberts (1982) examined the construct validity
were developed to be adjusted to children and adolescents, describing through principal component analysis with varimax rotation. They ob­
interpersonal situations of the various life contexts, involving children in tained a three-factor solution: Factor I (accounted for 29.60 % of the
their relationships with peers or with adults (e.g., parent-child re­ variance) defined by Unresolved and six Adaptive Scales, Factor II
lationships, sibling relationships, aggression situations). The child or (15.10 % of the variance) defined by four Clinical Scales, and Factor III
adolescent participant is asked to develop a story with a beginning, (10.80 % of the variance) defined by Limit Setting and Resolution-1.
middle and end about each card (McArthur & Roberts, 1982). Consid­ Comparisons between the normative group and a clinical group
ering the “projective hypothesis” (attributed to Frank, 1939, 1948), it is showed significant differences for almost all scales. Considering the

2
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Table 1
Studies with RATC based on a narrative review.
Authors Objectives Samples Results/conclusions

Alberto (1999) Assessing of symptomatology and PTSD in abused 92 children from the general population and 90 Group of abused children recorded higher values
and neglected children. abused children in residential care (both sexes, in Resolution 1 and in Support-Child and, on the
aged 10–15 years old). other hand, lower on the Support-Other scale.
Statistically significant results were identified
for the group × gender interaction on the scales
Limit Setting, Support-Child and Support-Other,
with the male group of the abuse group
registering higher values than the female gender,
while in the control group the opposite happens.
Statistically representative results were
identified for the Problem Identification scale,
with older children in the abuse group having
higher values than younger children.
No statistically significant differences were
identified between the abuse and control groups
on the Depression which corroborates to the lack
of discriminant validity of the RATC between the
control and clinical groups.
Bell and Nagle Validation study of the RATC in a sample of 86 children from the general population (59 The authors concluded that the sample used for
(1999) children, analysing the adequacy of the boys and 27 girls, mean age of 9 years and 11 standardization is inadequate and should not be
standardization norms with non-clinical samples. months). used for clinical diagnoses.
Burman et al. (1987) Analysing the relationship between parental 56 children (30 boys and 26 girls, aged 6–14 Fathers' overall marital satisfaction is positively
conflict and children's adjustment. years old) and their parents. associated with the sons' adaptability, as scored
on the RATC. Data from boys indicate that a
warm mother-child relationship is associated
with high adaptability, while a warm father-
child relationship is related to low RATC clinical
scale scores.
Canais (2012) Comparing the results between a group of 30 institutionalised youths (13 boys and 17 girls) The control sample has higher values in the
institutionalised and a group of non- and 30 youths from the general population (21 Support-Other, Identification Problems and
institutionalised youths in the RATC scales and the boys and 9 girls), aged 10–15 years old. Resolution-2 scales, these youths have a higher
Rosenberg Self-Esteem Scale (RSE). tendency to provide support to other people and
to analyse, identify and define problems because
they have had more enriching and reassuring
family and social experiences. Institutionalised
youths have difficulties in terms of awareness of
feelings, conflicts, or problems, and in
discriminating the resolution steps,
consequently having higher scores on the
Unresolved scale.
Duncan (1993) Studying differences between clinical and non- 70 mother-child dyads. Although no significant differences were found
clinical groups of children, and their respective on clinical scales between groups, children in the
mothers, and relations between children and their clinical group, as well as their mothers, were less
mothers in RATC scales. likely to project support-other, support-child,
problem identification, or problem solving
compared to with the control group. They were
also more likely to report ineffective or abusive
boundary-setting, maladaptive outcomes,
rejection and atypical themes.
Friedrich and Share Analysing the content analysis of stories developed 93 children (59 boys and 34 girls, aged 4–13 The sexual responses are clearly not specific only
(1998) from card 15 to identify sexual content. years old), considering three groups: 59 children to cases of possible or probable sexual abuse.
with no evidence of sexual abuse, 18 children Approximately 1 in 5 non-abused children
possible victims of sexual abuse, and 16 children referred for a psychological evaluation provided
victims of sexual abuse. responses that were scored as sexual.
The authors concluded that it is not possible to
identify only one measure to accurately identify
a child as sexually abused.
Gonçalves et al. Normative study of RATC data based on a sample of 80 children from the general population, The authors obtained statistically significant
(1999) well-adjusted children and comparison with a stratified by ages (40 boys and 40 girls, aged 6–9 differences between the two samples, with the
clinical sample. years old), and 44 children from a clinical clinical group having generally higher results on
sample (depression, anxiety, behavioural the clinical scales (e.g., Unresolved scale) and
problems). the normative sample registering higher results
on the adaptive scales (e.g., Support-Other and
Resolution-2 scales). There were no statistically
significant differences in the Anxiety,
Depression, Aggression and Rejection scales.
Adaptive Scales increase significantly with age.
Differences according to gender were recognized
on the Anxiety scale, where girls have higher
scores.
Headen (1986) Studying the discriminative capacity of the RATC 29 children with learning and/or behavioural Significant differences between the two samples
between a group of students with learning and/or problems (21 boys and 8 girls, aged 6–15 years in the type of resolution they gave to situations
behavioural problems and a group of children old), and 29 children from a control group (14 identified as problematic – children in the
without these problems. boys and 15 girls, aged 6–12 years old). clinical sample gave answers that corresponded
(continued on next page)

3
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Table 1 (continued )
Authors Objectives Samples Results/conclusions

to the lowest level of resolution. No differences

were found between the two samples in the
results obtained in the remaining adaptive and
clinical scales. In addition, he found a positive
correlation between cognitive performance
measures and the Adaptive Problem
Identification scale.
Joiner & Barnett Analysing the correlation between the RATC scales 53 male children/adolescents admitted to When checking the correlations between the
(1994) and the effect of age on the results of the various academic medical centres (aged 6–16 years old). results of the different RATC scales, they found
scales. that Aggression was strongly associated with
Rejection, while Support-Other emerged as a
moderating variable in the relationship between
Depression and Rejection. It was found that the
RATC Depression indexes interacted with the
measure of interpersonal style, in this case
Support-Other, to predict rejection rates.
Joiner (1996) Analysing the susceptibility of self-report measures 44 children and adolescents, psychiatry Defensiveness was more associated with self-
(CDI and RCMAS) and thematic measures (RATC) inpatients (20 boys and 24 girls, aged 6–16 years report measures than thematic measures of
to simulation in the assessment of depression in old). depression in both genders, but being
children and adolescents admitted to psychiatric particularly relevant in females.
clinics.
Lampel (1996) Studying the personality and parenting 20 families in dispute: one group of 10 children In the RATC results, was only found a
characteristics of litigious couples and their (6 boys and 4 girls) who have a preference for statistically significant difference, on the
children's patterns of preference for one parent in one of the parents, and another group of 10 Problem Identification scale, with the group of
custody dispute cases. children (6 boys and 4 girls) with no preference children who did not have a preference for either
for one of the parents. parent to narrate more situations as problematic.
Lavado (2008) Studying the attachment patterns and the presence 5 children (4 girls and 1 boy, aged 9–10 years In this study, the psychometric properties of the
of psychopathological symptoms in children with old) with atopic dermatitis. RATC were not analysed. However, high levels
atopic dermatitis. of anxiety were detected in all children in the
sample. Lower values were found on the
Support-Other scale in three children, revealing
difficulties in recognizing others as possible
allies in solving problems
Louw and Ramkisson Studying the adequacy of the RATC, the House- 23 sexually abused girls and 17 non-abused girls The two groups did not differ significantly with
(2002) Tree-Person (H-T-P) test, and the Draw-A-Person (aged 7–11 years old). regards to the variables on the Adaptive Scales
test as measures of assessing sexual abuse. and Clinical Scales and Indicators.
The sexually abused girls tended to reflect more
sexual responses on the RATC cards, and the
sexual content was defined as responses which
reflected an explicit sexual act or an implied
sexual act.
Mendes and Sani Studying children's representations of inter- 9 children (6 girls and 3 boys, aged 8–15 years The children projected in their narratives a
(2015) parental violence using RATC card 12 “parental old) exposed to inter-parental violence. parental relationship that was always
conflict” and a “Dinnertime Conflict” Story conflicting, with events of physical and
(MacArthur Story Stem Battery). emotional violence, without resolution.
The narratives reveal a diversity of emotional
and cognitive reactions, and the results confirm
the negative impact of violence on the child's
adjustment level. The elaborated stories are
disorganized and incoherent, presenting weak
verbal resources.
Palomares et al. Analysing the factor structure of the RATC in a 48 children with chronic diseases (aged 6–15 They obtained the three factors indicated in the
(1991) sample of children with chronic diseases years old). original study, although they point to some
(comparing with the factor structure of the original differences in the composition of the three
study). factors.
Wells et al. (2012) Analysing the effectiveness of neurocognitive 40 children (27 boys and 13 girls, aged 6–11 Children in the intervention group demonstrated
rehabilitation in children withdrawn from their years old) were removed from their biological significant improvements in executive and
parents due to substance exposure in the prenatal families, and 38 children in the control group emotional functioning when compared with the
period. (26 boys and 12 girls). control group.
There was a statistically significant decrease in
the results of the Resolution-1 scale in the
intervention group, because they adopted more
realistic, complex, and elaborated responses in
problem solving.
Worchel et al. (1992) Studying the influence of suggestion on the results 50 paediatric children/adolescents (28 boys and Self-report of depressive symptoms by paediatric
of various measures of depression in children. 22 girls, aged 6–17 years old) with chronic patients varied as a function of environmental
illness (19 with cancer and 31 with diabetes). cues, while scores on the RATC Depression scale
did not change, providing important validity
data.

construct validity, Palomares et al. (1991) carried out an principal the authors also performed a confirmatory factor analysis and yielded a
component analysis and also obtain a three-factor solution with factor satisfactory adjustment [normative group: χ 2(62) = 348.57, p < .001,
loadings >0.51 (comparing factor solutions for normative and clinical GFI = 0.89, RMS = 0.09; clinical group: χ 2(48) = 146.65, p < .001, GFI
groups), but only Factor I is equivalent to the original factorial structure; = 0.72, RMS = 0.12].

4
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

In Portuguese context, Gonçalves et al. (1999) carried out an forensic contexts, and (3) comparing indices obtained in the RATC
exploratory factor analysis (N = 123, 80 control group and 43 clinical among forensic groups and between the forensic groups and a commu­
group) and obtained a four-factor solution: Factor I (accounted for nity sample from Canais (2012)' study.
24.28 % of the variance) defined by most Adaptive Scales, Factor II The Canais (2012)' study was considered for the present study,
(14.56 % of the variance) defined by other Adaptive Scales (showing a instead of the Alberto (1999)'s and Gonçalves et al. (1999)', because it is
less adequate adjustment), Factor III (12.55 % of the variance) defined the most recent one and has similar age range; as far as is known, there
by Depression and Unresolved, Factor IV (12.40 % of the variance) are no studies with more recent normative data developed in Portugal.
defined by Aggression and Rejection. Comparisons between the control Considering these two different forensic groups, juvenile delinquents
group and a clinical group also showed significant differences for several and adolescent victims of maltreatment, the use of a projective method
scales. such as the RATC is very useful, because it assesses different areas of
And Alberto (1999) also carried out an exploratory factor analysis children and adolescents functioning/children and adolescents life
(varimax rotation) (N = 92, control group) and obtained a three-factor contexts, including relationships within the family, peers, and school.
solution: Factor I (accounted for 21 % of the variance) defined by five The use of RATC in these contexts and the objectives of the present study
Adaptive Scales and two Clinical Scales (positive social interaction), are based on the fact that both groups have a personal history charac­
Factor II (15 % of the variance) defined by other three Adaptive Scales terised by early adversity experiences, both in the family and in the
and three Clinical Scales (problem-solving), Factor III (13 % of the community, particularly of abuse and/or neglect, and it is also very
variance) defined by three Clinical Scales and two Clinical Indicators common for these juvenile delinquents detained in educational centres.
(depressive and maladaptive response). Comparisons between the con­ Thus, it seems relevant to identify the narratives about family, peers and
trol group (n = 92) and a clinical group (n = 90, victims of child abuse) school, and the individuals' scores on the Adaptive Scales, Clinical
also showed significant differences for several scales. Scales, and Clinical Indicators of the RATC to identify their emotional
Table 1 presents the studies using the RATC found in literature re­ and behavioural styles, to identify the similarities and differences be­
view (narrative review). tween these two groups, and to establish some preliminary indicators for
As Table 1 shows, there are not many published (validation) studies. the interpretation of RATC scores for clinical and forensic practice.
In fact, the lack of evidence for validity of the RATC is a problem that is The hypotheses for the present study are:
well documented in the literature (e.g., Dupree & Prevatt, 2003; Frick
H1. Roberts Apperception Test for Children (RATC) show adequate
et al., 2020). For this reason, RATC results cannot be used as the sole or
reliability indices.
main criterion in the diagnostic decision-making process. However, this
limitation and reservation regarding the use of the RATC is common to H2. Roberts Apperception Test for Children (RATC) show adequate
these thematic projective techniques and, strictly speaking, to the use of criterion validity indices, considering the results on Wechsler Intelli­
any other assessment instrument (Frick et al., 2020). Yet the RATC has gence Scale for Children – Third Edition (WISC-III) for convergent and
several advantages over other projective tests: unlike other thematic discriminant validity.
techniques, such as Children's Apperception Test (CAT) and Thematic
H3. Roberts Apperception Test for Children (RATC) show adequate
Apperception Test (TAT), the RATC has an explicit, structured and
criterion validity indices, considering the results on Youth Self-Report
standardized scoring system (although be rare the empirical research to
(YSR) for convergent and discriminant validity.
test and guide the interpretations), which makes it easier to administer,
quantify and compare results between individuals and allows the re­ H4. Considering the ages (11 to 17 years old) and years of schooling (1
searchers to examine its validity; is one of the few storytelling in­ to 10 years) range of whole forensic sample, it is expected to find an
struments, where the themes of the cards are not specific to effect of these variables on the RATC results.
psychodynamic theory; it is more culture-fair as it does not use cultural
H5. There are some significant mean differences on RATC between
references that may not be familiar to individuals from different back­
two forensic groups, juvenile delinquents and adolescent victims of
grounds (Frick et al., 2020; Roberts, 1994; Teglasi, 2010).
maltreatment (these groups have different characteristics and forensic
The RATC is also used in the assessment of children in forensic set­
issues, but both are forensic groups).
tings given its potentialities to analyse emotional and behavioural
functioning through the projective process, including victims of sexual H6. There are significant mean differences on RATC between the
abuse (see Louw & Ramkisson, 2002). However, as Joiner (1996) points forensic groups and the community sample [from Canais (2012)' study].
out, the validity of projective data is not yet clearly demonstrated, and
so they should be interpreted with caution, especially if they are not 2. Method
supported by other sources of information or assessment. In addition,
based on a sample of well-adjusted children, Bell and Nagle (1999) 2.1. Participants
found that the standardization of the RATC is inadequate, because of the
possible misclassification of children in some cases. Therefore, the au­ Participants are 75 youths of two groups were assessed in the
thors suggested not using the instrument for clinical diagnosis until a forensic context. The Group A, consisting of 40 juvenile delinquents
new standardization is completed. Therefore, more validation studies detained in educational centres of juvenile justice system under Portu­
are needed. guese tutelar education law (Law number 4/2015 of 15th January
In this sense, to meet this need, the present study aims to contribute 2015), all males, aged between 12 and 17 years old (mean age is 15.13
to the validation of the Roberts Apperception Test for Children (RATC) years old, SD = 1.18). These youths had between 1 and 10 years of
assessing two forensic samples [a group of juvenile delinquents and a schooling (M = 5.36, SD = 1.78).
group of adolescent victims of maltreatment, with little empirical The other group, the Group B, consisting of 35 adolescent victims of
research with RATC], by (1) enhancing the information about the psy­ maltreatment integrated in residential care under the protection of
chometric properties of the RATC [reliability (through the internal Portuguese social care system, 19 males (54.3 %) and 16 females (45.7
consistency and split-half methods) and criterion validity (convergent %), aged between 11 and 16 years old (mean age is 14.31 years old, SD
and discriminant validity), considering as external validation criteria = 1.37). These youths had between 5 and 10 years of schooling (M =
the results obtained in other instruments], (2) searching for some pre­ 7.86, SD = 1.46).
liminary indicators based on means and standard deviations for the Another group of children and adolescents (N = 30; 21 males and 9
interpretation of scores attained in the Adaptive Scales, Clinical Scales females, aged between 10 and 15 years old) from a community sample of
and Clinical Indicators of the RATC, for a more efficient use in these Canais (2012)' study, was considered for comparisons with the two

5
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

forensic groups of the present study. Declaration of Helsinki principles. All ethical principles were followed in
the present study. For Group A, RATC and WISC-III were administered,
2.2. Instruments and for Group B, RATC and YSR were administered. Data from Group A
and Group B were subsequently compared with a community sample
The Roberts Apperception Test for Children (RATC; McArthur & Rob­ from a previous study, the Canais (2012)' study.
erts, 1982; European Portuguese validations, Gonçalves et al., 1999, and Coding for RATC was carried out for two different psychologists. The
Alberto, 1999) was used to assess children and adolescents' behavioural, inter-rater concordance for RATC results was analysed by using as
social and emotional functioning. The RATC is a projective method agreement/disagreement criteria the absence or presence of one of
which is composed by 16 cards, including 11 specific cards for boys and RATC scales in the coding for each card administered. The percentage of
girls and 5 common cards, and can be administered to child or adoles­ agreement ranged from 50 % to 100 % (the lowest percentage was
cent participants aged between 6 and 15 years old, and may be used with registered in the absence of codes for problem-solving strategies in some
slightly older or younger children. The RATC consisting of several cards interpreted by another examiner as unresolved), with a mean of
Adaptive Scales, Clinical Scales, and Clinical Indicators: Reliance on 85 %, generally corresponding to a strong agreement between
Others (REL), Support-Other (SUP-O), Support-Child (SUP-C), Limit examiners.
Setting (LIM), Problem Identification (PROB), Resolution-1 (RES-1),
Resolution-2 (RES-2), Resolution-3 (RES-3), Anxiety (ANX), Aggression 2.4. Statistical analyses
(AGG), Depression (DEP), Rejection (REJ), Unresolved (UNR), Atypical
Response (ATY), Maladaptive Outcome (MAL), and Refusal (REF). To analyse the reliability of the Roberts Apperception Test for Chil­
The Wechsler Intelligence Scale for Children – Third Edition (WISC-III; dren (RATC), two alternative methods were performed: the Cronbach's
Wechsler, 2003; European Portuguese version, Simões et al., 2006) was alfa (α), corresponding to internal consistency method (Cronbach,
used to assess the cognitive abilities and functioning. The WISC-III is a 1951), and the Guttman's second lower bond for test reliability (λ2),
measure that can be administered to evaluate children and adolescents based on split-half method and considering the square root of the sums
aged between 6 and 16 years old and 11 months. It allows determining a of squares of the off diagonal elements of the scale (Guttman, 1945); the
general intelligence index (Full Scale Intellectual Quotient), two partial latter is a more robust reliability index when the instrument has multiple
quotients according to verbal abilities (Verbal Intellectual Quotient) and factors (see Callender & Osburn, 1979), which is the case of RATC.
non-verbal abilities (Performance Intellectual Quotient) and also three Criterion validity studies were performed through the Pearson's r cor­
indicators or factorial indexes: Verbal Comprehension Index, Perceptual relation coefficients. The study of the influence of age and years of
Organization Index and Processing Speed Index. It has 13 subtests: schooling on the RATC results was performed through ANOVA. Mean
Picture Completion, Information, Coding, Similarities, Picture differences and comparisons were performed using independent-
Arrangement, Arithmetic, Block Design, Vocabulary, Object Assembly, samples t-test and one-sample t-test, followed by the calculation of cor­
Comprehension, Symbol Search, Digit Span and Mazes (the latter three responding Cohen (1988)'s d indices (adapted to each method) for effect
subtests are optional). The WISC-III is an instrument with several vali­ size. Statistical analyses were performed using the IBM SPSS Statistics
dation studies for the Portuguese population and with well-documented (version 25.0; Armonk, New York, IBM Corp.) and JASP (version 0.16.1;
and established psychometric properties (Simões et al., 2006). University of Amsterdam) programs.
The Youth Self-Report (YSR; Achenbach, 1991; European Portuguese
version, Fonseca & Monteiro, 1999) was used to assess the behavioural 3. Results
and emotional functioning. The YSR is a self-report instrument that can
be administered to evaluate children and adolescents aged between 11 3.1. Reliability
and 18 years old, aiming to measure their social skills, activities, and
behavioural problems as perceived by themselves. The YSR is integrated Table 2 presents the reliability coefficients obtained in this study for
into Achenbach's System of Empirically Based Assessment (ASEBA) the whole sample (N = 75), which were estimated using the Cronbach's
which works as a structured process to collect information and assess the alfa (α) and Guttman's second lower bond for test reliability (λ2) (Hy­
perceptions of different stakeholders, such as parents, teachers and the pothesis 1). The reliability coefficients ranged for all scales from
children or adolescents themselves. The YSR is composed by six scales: “minimally acceptable” to “unacceptable”, according to the criterion
Behaviour Problems, Attention Deficit/Hyperactivity Problems, established by DeVellis (2017) to interpret the reliability indices (“un­
Anxious/Depressed, Withdrawn, Somatic Complaints, Thought Prob­ acceptable”, coefficients below 0.60; “undesirable”, between 0.60 and
lems. This instrument has adequate psychometric properties (Fonseca & 0.65; “minimally acceptable”, between 0.65 and 0.70; “respectable”
Monteiro, 1999). between 0.70 and 0.80; “very good”, between 0.80 and 0.90): Cron­
bach's alphas ranged from 0.65 to 0.23, and Guttman's λ2 coefficients,
2.3. Procedures from 0.68 to 0.30. All coefficients are less than acceptable, with co­
efficients below the minimum of 0.70 pointed out by several authors (e.
Participants were recruited from different forensic contexts, using g., Kline, 1993; Nunnally & Bernstein, 1994).
the nonprobability sampling method, with a convenience sampling. For
Group A, consisting of juvenile delinquents detained in educational 3.2. Criterion validity
centres of Portuguese juvenile justice system, the study was carried out
with the permission from the Directorate-General of Reintegration and Criterion validity can be examined through the degree of corre­
Prison Services (DGRSP) and from the Youth Detention Centres. For spondence, using the method of correlation, obtained between the
Group B, consisting of adolescent victims of maltreatment integrated in scores on the measuring instrument and the individuals' achievement in
residential care under the protection of Portuguese social care system, external criteria associated with or dependent on the psychological
the permission was obtained from the Technical Directors of the Chil­ dimension the instrument assesses (Nunnally & Bernstein, 1994). Thus,
dren and Youth Care Homes. Participants were asked for voluntary criterion validity should demonstrate that a test score is a predictor of
participation and the objectives and relevance of the present study were the criterion information obtained from other test scores, measuring
explained to them (no incentives were offered in exchange for partici­ similar or related constructs. Relationships between test scores and
pation). They were informed that their responses would remain anon­ other measures intended to assess the same or similar constructs provide
ymous, the confidentiality and data protection was guaranteed, and the convergent evidence (convergent validity), whereas relationships be­
informed consent information was gathered, according to the tween test scores and different constructs provide discriminant evidence

6
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

(discriminant validity) (American Educational Research Association (62.90) = 4.121, p < .001; d = 1.04], both with a very large effect size,
et al., 2014). and for Depression (DEP) [t(73) = 2.088, p = .040; d = 0.49], with a
To analyse the convergent and discriminant validity of the RATC, the medium effect size, showing that Group A tend to obtain higher scores in
scores obtained by participants in the Wechsler Intelligence Scale for these scales. On the other hand, the means were significantly and sta­
Children – Third Edition (WISC-III) and the Youth Self-Report (YSR) tistically different for Unresolved (UNR) [t(73) = − 4.118, p < .001; d =
were considered as external validation criteria, and Pearson's r corre­ − 0.96] and Atypical Response (ATY) [t(73) = − 3.371, p = .001; d =
lation coefficients were calculated. WISC-III was used for cognitive − 0.79], with a large effect size, showing that Group B tend to obtain
assessment of 40 juvenile delinquents from Group A, and its results were higher scores in these scales.
considered to analyse the convergent validity of RATC assessing in­ Considering the results, some preliminary indicators (based on
dividuals' problem-solving strategies [Adaptive Scales: Problem Identi­ means and standard deviations) were stablished for the interpretation of
fication (PROB), Resolution-1 (RES-1), Resolution-2 (RES-2), scores in the Adaptive Scales, Clinical Scales and Clinical Indicators of
Resolution-3 (RES-3)] and the discriminant validity of RATC for Clin­ the RATC in these forensic contexts (educational centres and residential
ical Indicators [Atypical Response (ATY), Maladaptive Outcome (MAL)] care) (see Table 5).
(Hypothesis 2). YSR was used for emotional and behavioural assessment
of the 35 adolescent victims of maltreatment from Group B, and its re­
sults were considered to analyse the discriminant validity of RATC for 3.5. Comparisons between forensic groups and community sample
the Adaptive Scales and the convergent validity for the Clinical Scales
and Clinical Indicators assessing individuals' behavioural, social and Table 6 presents the mean differences comparing the forensic sample
emotional functioning (Hypothesis 3). of the present study – Group A (n = 40) and Group B (n = 35) – with
Regarding problem-solving abilities, some significant and positive Canais (2012)' data from the community sample (N = 30) using the one-
correlations were obtained between RATC scales and WISC subscales sample t-test and calculating the corresponding Cohen (1988)'s d indices
and indices, including Full Scale IQ, Verbal IQ, and Performance IQ, (Hypothesis 6). The results indicate significant (p < .001, p < .01 and p
with a strength ranging from large (1 > r > 0.50) to medium (0.50 > r > < .05) and statistical differences for several scales, ranging from very
0.30), according to the criterion proposed by Cohen (1988) (see large to medium effect size, with the same tendencies considering Group
Table 3). The majority of correlations appear on RATC's Resolution-2 A and Group B, as expected. There are significant differences for the
(RES-2) and Resolution-3 (RES-3) scales. These correlation coefficients Support-Other (SUP-O) [Group A: t = 3.015, p = .006; d = 0.63; Group B:
ranged between 0.40 and 0.34 in Problem Identification (PROB) scale, t = 4.212, p < .001; d = 0.71, both with large effect], Unresolved (UNR)
between 0.52 and 0.36 in Resolution-2 (RES-2) scale, and between 0.75 [Group A: t = 2.107, p = .047; d = 0.44; medium effect; Group B: t =
and 0.43 in Resolution-3 (RES-3) scale. And a large and negative cor­ 7.944, p < .001; d = 1.34; very large effect], and Maladaptive Outcome
relation of − 0.54 was obtained between Maladaptive Outcome (MAL) (MAL) [Group A: t = 5.514, p < .001; d = 1.15; very large effect; Group
scale and Mazes. Other correlations were not significant and were low or B: t = 5.027, p < .001; d = 0.85; large effect] scales, showing that par­
null. ticipants from the forensic groups tend to obtain higher scores in these
Regarding emotional and behavioural problems, some significant scales. And there are significant differences for the Support-Child (SUP-
positive and negative correlations were obtained between RATC and C) [Group A: t = − 2.472, p = .022; d = − 0.52; with large effect; Group B:
YSR scales, with a strength ranging from large (1 > r > 0.50) to medium t = − 11.105, p < .001; d = − 1.87; very large effect], Problem Identifi­
(0.50 > r > 0.30) (see Table 4). The correlation coefficients were cation (PROB) [Group A: t = − 6.117, p < .001; d = − 1.27; Group B: t =
negative between RATC's Refusal (REF) and Maladaptive Outcome − 8.547, p < .001; d = − 1.45; both with very large effect], Resolution-2
(MAL) scales and YSR scales ranging between − 0.52 and − 0.31, and (RES-2) [Group A: t = − 17.603, p < .001; d = − 3.67; Group B: t =
were positive between Support-Other (SUP-O) and Somatic Complaints − 27.478, p < .001; d = − 4.63; both with very large effect], Anxiety
(0.34) and between Limit Setting (LIM) and Withdrawn (0.32). Other (ANX) [Group A: t = − 2.105, p = .047; d = − 0.44; Group B: t = − 2.852,
correlations were not significant and were low or null. p = .007; d = − 0.48; both with medium effect], Aggression (AGG)
[Group A: t = − 7.238, p < .001; d = − 1.51; very large effect; Group B: t
= − 3.123, p = .004; d = − 0.53; large effect], Depression (DEP)
3.3. Influence of age and years of schooling

For each forensic group, the study of the effect of age and years of Table 2
schooling on the RATC results was performed through ANOVA (Hy­ RATC: Reliability indices considering adaptative and clinical scales (total sam­
ple, N = 75).
pothesis 4). For age, there were no significant effects on RATC results for
Group A and Group B, except for the latter on Atypical Response (ATY) RATC scales Cronbach's α (internal Guttman's λ2 (split-
consistency method) half method)
[F(5) = 4.945, p = .002] and Rejection (REJ) [F(5) = 2.841, p = .033]
scales. For years of schooling, only one significant effect for Group A on Adaptive Scales
Unresolved (UNR) scale [F(8) = 2.472, p = .034], and for Group B on Reliance on Others 0.60 0.64
(REL)
Atypical Response (ATY) scale [F(5) = 2.627, p = .045].
Support-Other (SUP- 0.46 0.51
O)
3.4. Mean differences and some preliminary indicators for use in forensic Support-Child (SUP- 0.23 0.30
context C)
Limit Setting (LIM) 0.56 0.60
Problem 0.65 0.68
Table 5 presents the mean and standard deviations obtained for Identification (PROB)
RATC scales in the forensic sample (N = 75) of the present study: 40 Resolution-1 (RES-1) 0.39 0.46
juvenile delinquents detained in educational centres from Group A, and Resolution-2 (RES-2) 0.63 0.66
35 adolescent victims of maltreatment integrated in residential care Resolution-3 (RES-3) 0.47 0.59
Clinical Scales
from Group B. Anxiety (ANX) 0.27 0.35
The mean differences between Group A and Group B were examined Aggression (AGG) 0.50 0.54
using the independent-samples t-test and calculating the corresponding Depression (DEP) 0.58 0.61
Cohen (1988)'s d indices (Hypothesis 5). The results indicate that the Rejection (REJ) 0.47 0.50
Unresolved (UNR) 0.54 0.57
means were significantly and statistically different for Refusal (REF) [t
(44.13) = 4.281, p < .001; d = 1.29] and Support-Child (SUP-C) [t Note. RATC = Roberts Apperception Test for Children.

7
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

[significant differences only for Group B and Canais' comparison: t = On the reliability analyses of RATC in forensic context, considering
− 3.961, p < .001; d = − 0.69; large effect], and Atypical Response (ATY) the Cronbach's α and Guttman's λ2, all reliability coefficients ranged
[significant differences only for Group A and Canais' comparison: t = from “minimally acceptable” to “unacceptable” (ranging between 0.68
− 3.199, p = .004; d = − 0.67; large effect] scales, showing that partic­ and 0.23, in general, with several between 0.65 and 0.60 “undesirable”),
ipants from the community sample tend to obtain higher scores in these with coefficients below 0.70 (which are less than acceptable). These
scales. No significant differences were found for the remaining scales results are similar to Alberto (1999)'s study in the Portuguese context
(Reliance on Others, Limit Setting, Resolution-1, Resolution-3, and (ranging between 0.77 and 0.20) and McArthur and Roberts (1982)'
Rejection); the comparison for Refusal (REF) scale was not performed original study (ranging between 0.86 and 0.44), with normative and
due the score 0 obtained by Canais' community sample. larger samples. In addition, some authors (e.g., Anastasi, 1988; Nunnally
& Bernstein, 1994) refer that reliability indices can be influenced by the
4. Discussion scale length and shorter scales have some tendency to show lower co­
efficients than the longer ones, which is the case of RATC (only 16 items
The use of projective techniques in psychological assessment of per scale and low scores variability).
children and adolescents has been the subject of some controversy. As a Nevertheless, RATC is a projective method used to assess children
clinical tool, projective techniques allow for greater flexibility in and adolescents' behavioural, social and emotional functioning and it
administration and interpretation. However, with this flexibility, the has some psychometric limitations, namely for reliability, like some
interpretations that result from the assessment are much more suscep­ other important psychometric tests for personality assessment. At least
tible to be influenced by the examiner's idiosyncrasies, and the in­ these limitations can be studied, whereas in other thematic tests (such as
terpretations of the same material may vary between clinicians (see CAT) this is not possible, due to the absence of an explicit, structured
Frick et al., 2020). The RATC was developed to overcome the weak­ and standardized scoring system. Kline (1993) showed some examples of
nesses of projective tests, but also to correspond more than Children's these limitations in the psychometric analyses of objective personality
Apperception Test (CAT) and Thematic Apperception Test (TAT) to the assessment instruments in general – for example, the Minnesota Multi­
life contexts of children and adolescents (Roberts, 1994). Given the phasic Personality Inventory (MMPI), one of the most widely used per­
scarcity of studies aimed at analysing the psychometric qualities of the sonality tests. As Anastasi (1988) pointed out, when the projective
RATC and given its popularity (see Table 1), it is essential to deepen the methods are evaluated as psychometric instruments (analysing reli­
knowledge about this instrument in various contexts of application, ability and validity indices), the large majority make a poor showing,
which motivated this study. The main objective of this study was to however their popularity in clinical use still remains because of the
contribute to the validation of the RATC in forensic setting, considering richness of their analyses of psychological functioning and their
two different forensic groups (juvenile delinquents and adolescent vic­ contribution to treatment outcome. These aspects are also highlighted
tims of maltreatment) and searching for some preliminary indicators and discussed by Lilienfield et al. (2000), McGrath and Carroll (2012),
based on means and standard deviations for the interpretation of RATC and Piotrowski (2019), analysing the scientific status and the usefulness
results in these contexts. of projective methods, particularly the three major ones, which are the
Rorschach Inkblot Test, Thematic Apperception Test, and human figure
Hypothesis 1. Roberts Apperception Test for Children (RATC) show
drawings, being the Rorschach the most studied one.
adequate reliability indices.
Although previous studies have found similar reliability coefficients

Table 3
Convergent and discriminant validity between the results obtained in RATC problem solving indices (Adaptative Scales and Clinical Indicators) and WISC-III (n = 40,
subsample of Group A) (Pearson's r correlations).
WISC-III RATC

Adaptive Scales Clinical Indicators

Problem Identification Resolution-1 Resolution-2 Resolution-3 Atypical Response Maladaptive Outcome

(PROB) (RES-1) (RES-2) (RES-3) (ATY) (MAL)

Subscales
Information 0.23 0.19 0.40* 0.55** − 0.23 − 0.26
Similarities 0.23 0.28 0.37* 0.48** − 0.28 − 0.12
Arithmetic 0.14 − 0.25 0.29 0.43* 0.05 − 0.08
Vocabulary 0.39* 0.17 0.30 0.53** − 0.20 0.09
Comprehension 0.30 0.12 0.39* 0.75** − 0.17 − 0.13
Digit Span 0.31 − 0.03 0.45* 0.34 − 0.18 − 0.35
Picture Completion 0.29 0.04 0.33 0.25 − 0.18 − 0.08
Coding 0.13 0.03 0.31 0.20 − 0.03 − 0.07
Picture Arrangement 0.25 0.03 0.14 0.31 − 0.22 − 0.06
Block Design 0.27 0.10 0.26 0.64** − 0.29 − 0.15
Object Assembly − 0.01 − 0.02 0.01 0.47** − 0.17 − 0.16
Symbol Search 0.40* − 0.06 0.27 0.22 − 0.11 0.06
Mazes 0.11 0.34 0.27 0.24 − 0.21 ¡0.54**
Indices
Verbal IQ 0.34 0.07 0.45** 0.66** − 0.17 − 0.10
Performance IQ 0.28 − 0.02 0.30 0.48** − 0.24 − 0.08
Verbal Comprehension 0.25 0.17 0.36* 0.68** − 0.21 − 0.17
Index
Perceptual Organization 0.23 0.07 0.23 0.55** − 0.20 − 0.13
Index
Processing Speed Index 0.35 0.05 0.52* 0.36 − 0.01 − 0.23
Full Scale IQ 0.34* 0.07 0.43* 0.63** − 0.22 − 0.10

Note. RATC = Roberts Apperception Test for Children; WISC-III = Wechsler Intelligence Scale for Children – Third Edition; * p < .05; ** p < .01. Significant Pearson's r
correlations are presented in bold.

8
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Table 4
Convergent and discriminant validity between the results obtained in RATC scales and YSR (N = 35, Group B) (Pearson's r correlations).
RATC YSR

Behaviour Attention Deficit/Hyperactivity Anxious/ Withdrawn Somatic Thought

Problems Problems Depressed Complaints Problems

Adaptive Scales
Reliance on Others (REL) 0.07 − 0.26 − 0.11 − 0.21 0.02 − 0.21
Support-Other (SUP-O) 0.20 0.03 − 0.12 0.22 0.34* 0.16
Support-Child (SUP-C) − 0.03 0.04 0.12 0.06 − 0.06 − 0.00
Limit Setting (LIM) 0.03 0.14 0.18 0.32 − 0.02 − 0.07
Problem Identification − 0.22 − 0.24 − 0.11 − 0.22 0.01 0.10
(PROB)
Resolution-1 (RES-1) − 0.03 − 0.21 − 0.07 0.02 − 0.17 − 0.32
Resolution-2 (RES-2) − 0.13 0.02 − 0.08 0.07 0.16 − 0.11
Resolution-3 (RES-3) − 0.12 − 0.13 0.00 0.15 0.24 0.08
Clinical Scales
Anxiety (ANX) 0.07 0.00 0.06 − 0.09 0.02 0.16
Aggression (AGG) − 0.01 0.11 0.13 − 0.04 − 0.17 0.21
Depression (DEP) 0.04 − 0.02 0.10 − 0.03 0.04 0.08
Rejection (REJ) − 0.14 − 0.20 − 0.11 − 0.10 − 0.21 − 0.06
Unresolved (UNR) 0.07 − 0.12 0.15 − 0.23 0.11 0.29
Clinical Indicators
Atypical Response (ATY) − 0.17 0.19 0.08 0.07 − 0.04 0.07
Maladaptive Outcome (MAL) 0.03 0.10 − 0.27 − 0.10 ¡0.38* − 0.17
Refusal (REF) ¡0.45** ¡0.40* ¡0.36* ¡0.52** − 0.15 − 0.31

Note. RATC = Roberts Apperception Test for Children; YSR = Youth Self-Report; * p < .05; ** p < .01. Significant Pearson's r correlations are presented in bold.

Table 5
RATC: Preliminary indicators for interpretation based on mean differences comparing Group A (n = 40, juvenile delinquents) and Group B (n = 35, adolescent victims
of maltreatment).
RATC Group A Group B t df p Cohen's d

M (SD) M (SD)

Adaptive Scales
Reliance on Others (REL) 1.63 (1.90) 1.57 (1.52) 0.135 73 0.893 –
Support-Other (SUP-O) 3.58 (2.06) 3.94 (1.98) − 0.784 73 0.435 –
Support-Child (SUP-C) 1.25 (1.08) 0.43 (0.61) 4.121 62.90 <0.001* 1.04
Limit Setting (LIM) 1.85 (1.59) 1.83 (1.72) 0.056 73 0.956 –
Problem Identification (PROB) 10.90 (2.71) 9.94 (3.13) 1.420 73 0.160 –
Resolution-1 (RES-1) 1.55 (1.45) 1.26 (1.42) 0.881 73 0.381 –
Resolution-2 (RES-2) 2.45 (2.35) 2.00 (1.82) 0.917 73 0.362 –
Resolution-3 (RES-3) 0.23 (0.62) 0.03 (0.17) 1.925 45.54 0.061 –
Clinical Scales
Anxiety (ANX) 3.93 (1.70) 3.91 (1.84) 0.026 73 0.979 –
Aggression (AGG) 3.98 (1.64) 4.83 (2.16) − 1.939 73 0.056 –
Depression (DEP) 3.73 (2.05) 2.77 (1.88) 2.088 73 0.040* 0.49
Rejection (REJ) 2.50 (1.66) 2.54 (2.01) − 0.101 73 0.920 –
Unresolved (UNR) 5.18 (2.32) 7.54 (2.66) − 4.118 73 <0.001* − 0.96
Clinical Indicators
Atypical Response (ATY) 0.85 (1.15) 1.89 (1.51) − 3.371 73 0.001* − 0.79
Maladaptive Outcome (MAL) 1.58 (1.55) 1.06 (1.16) 1.649 71.39 0.104 –
Refusal (REF) 1.05 (1.34) 0.11 (0.32) 4.281 44.13 <0.001* 1.29

Note. RATC = Roberts Apperception Test for Children; Group A = juvenile delinquents, Group B = adolescent victims of maltreatment; M = Mean, SD = standard-
deviation, p = p-value, t = t-test, df = degrees of freedom, d = Cohen's d (effect size); * significant p-values for t-tests are presented in bold.

for RATC (and considering the limitations for other projective or which are the most adaptive way to solve problems], and the cognitive
objective instruments found in literature), the Hypothesis 1 of the pre­ abilities and functioning assessed by WISC-III. This constitutes evidence
sent study must be rejected. of the convergent validity. As expected, the Clinical Indicators, Mal­
adaptive Outcome (MAL) and Atypical Response (ATY) (the latter with
Hypothesis 2. Roberts Apperception Test for Children (RATC) show
no significant correlations), show a negative association with WISC-III
adequate criterion validity indices, considering the results on Wechsler
subscales and indices, reflecting the difficulties in problem solving in
Intelligence Scale for Children – Third Edition (WISC-III) for convergent
individuals who present more maladaptive ways of managing emotions
and discriminant validity.
and behaviours. And this constitutes an evidence of the discriminant
On the criterion validity studies of RATC, several significant corre­ validity. Moreover, these juvenile delinquents' development of adaptive
lations were obtained with WISC-III subscales and indices (convergent resolutions to the problems seems to be related to their cognitive func­
and discriminant validity), ranging from large to medium [coefficients tioning, particularly verbal skills. In this regard, Simões (2003) mentions
between 0.75 and 0.34, all positive, except − 0.54 between Maladaptive that verbal deficits limit response options in ambiguous or threatening
Outcome (MAL) and Mazes]. These results show the adequate associa­ social situations, since they reflect a lower ability to classify perceptions
tion between the problem-solving abilities and managing strategies to about the outside world and the emotions expressed by others.
deal with common interpersonal situations of everyday life assessed by Furthermore, the adaptation to unfamiliar situations, processing infor­
RATC [particularly with Resolution-2 (RES-2) and Resolution-3 (RES-3), mation speed, and reasoning, are all required skills for effective problem

9
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Table 6
RATC: mean differences comparing the forensic groups with Canais (2012)'s data from the community sample.
RATC Group A N = 40 Group B N = 35 Canais' N = 30 Group A × Canais' Community sample Group B × Canais' Community sample

M (SD) M (SD) M (SD) t p d t p d

Adaptive Scales
Reliance on Others (REL) 1.63 (1.90) 1.57 (1.52) 1.10 (1.21) 1.485 0.152 – 1.835 0.075 –
Support-Other (SUP-O) 3.58 (2.06) 3.94 (1.98) 2.53 (1.36) 3.015 0.006* 0.63 4.212 <0.001* 0.71
Support-Child (SUP-C) 1.25 (1.08) 0.43 (0.61) 1.57 (1.36) − 2.472 0.022* − 0.52 − 11.105 <0.001* − 1.87
Limit Setting (LIM) 1.85 (1.59) 1.83 (1.72) 2.33 (1.49) 0.684 0.501 – − 1.721 0.094 –
Problem Identification (PROB) 10.90 (2.71) 9.94 (3.13) 14.47 (1.94) − 6.117 <0.001* − 1.27 − 8.547 <0.001* − 1.45
Resolution-1 (RES-1) 1.55 (1.45) 1.26 (1.42) 1.50 (1.61) 0.355 0.726 – − 1.011 0.319 –
Resolution-2 (RES-2) 2.45 (2.35) 2.00 (1.82) 10.43 (3.48) − 17.603 <0.001* − 3.67 − 27.478 <0.001* − 4.63
Resolution-3 (RES-3) 0.23 (0.62) 0.03 (0.17) 0.07 (0.25) 1.015 0.321 – − 1.450 0.156 –
Clinical Scales
Anxiety (ANX) 3.93 (1.70) 3.91 (1.84) 4.80 (2.30) − 2.105 0.047* − 0.44 − 2.852 0.007* − 0.48
Aggression (AGG) 3.98 (1.64) 4.83 (2.16) 5.97 (2.42) − 7.238 <0.001* − 1.51 − 3.123 0.004* − 0.53
Depression (DEP) 3.73 (2.05) 2.77 (1.88) 4.03 (1.99) − 0.734 0.470 – − 3.961 <0.001* − 0.69
Rejection (REJ) 2.50 (1.66) 2.54 (2.01) 3.03 (1.63) − 2.029 0.055 – − 1.437 0.160 –
Unresolved (UNR) 5.18 (2.32) 7.54 (2.66) 3.97 (2.58) 2.107 0.047* 0.44 7.944 <0.001* 1.34
Clinical Indicators
Atypical Response (ATY) 0.85 (1.15) 1.89 (1.51) 1.73 (1.02) − 3.199 0.004* − 0.67 0.610 0.546 –
Maladaptive Outcome (MAL) 1.58 (1.55) 1.06 (1.16) 0.07 (0.25) 5.514 <0.001* 1.15 5.027 <0.001* 0.85
Refusal (REF) 1.05 (1.34) 0.11 (0.32) – 3.749 0.001* 0.78 2.095 0.044* 0.34

Note. RATC = Roberts Apperception Test for Children; Group A = juvenile delinquents, Group B = adolescent victims of maltreatment; M = Mean, SD = standard-
deviation, p = p-value, t = t-test, d = Cohen's d (effect size); * significant p-values for t-tests are presented in bold.

solving, and they are assessed by intelligence tests like WISC-III (Gott­ schooling (1 to 10 years) range of whole forensic sample, it is expected
fredson, 1997), particularly through the factorial indices (Perceptual to find an effect of these variables on the RATC results.
Organization, Processing Speed) (Kaufman & Lichtenberger, 2000). And
The study of the effect of age and years of schooling on RATC results
this is also true for projective tests, in a different way, like RATC, that
shows only a few effects for Group A and Group B. For age, significant
assesses problem-solving strategies (considering the children resources
effects were found only for Atypical Response (ATY) and Rejection (REJ)
for problem solving), identify cognitive deficits (e.g., poor solving skills)
scales in the Group B. And for years of schooling, one significant effect
and problem-solving components (e.g., identify problems, understand­
was found for Unresolved (UNR) scale in the Group A, and one signifi­
ing causal consequences) in unstructured social situation presented on
cant effect was found for Atypical Response (ATY) scale in the Group B.
the cards (Teglasi, 2010). Therefore, the Hypothesis 2 is confirmed.
These data show that the Hypothesis 4 must be rejected.
Hypothesis 3. Roberts Apperception Test for Children (RATC) show
Hypothesis 5. There are some significant mean differences on RATC
adequate criterion validity indices, considering the results on Youth Self-
between two forensic groups, juvenile delinquents and adolescent vic­
Report (YSR) for convergent and discriminant validity.
tims of maltreatment.
On the criterion validity studies of RATC, some significant correla­
To establish some preliminary indicators to interpret RATC data in
tions were also obtained with YSR scales (convergent and discriminant
these forensic contexts, considering our two groups [40 juvenile de­
validity), ranging from large to medium [negative coefficients between
linquents from Group A, and 35 adolescent victims of maltreatment from
− 0.52 and − 0.31, and positive coefficients of 0.34 and 0.32, considering
Group B] (and testing Hypothesis 5), means and standard deviations
the maladaptive (convergent validity) and adaptative (discriminant
were calculated and comparisons within forensic groups were carried
validity) aspects of behaviour, respectively], showing the association
out (followed by the comparisons between the forensic groups and the
between the behavioural, social and emotional functioning dealing with
community sample from Canais' study; see Table 6). Because of these
interpersonal situations assessed by RATC and the emotional and
groups' different characteristics and their forensic issues, some signifi­
behavioural problems examined by YSR. In this case, most correlations
cant and statistical differences, with very large to medium effect size,
appear on the Refusal (REF) scale, which is a Clinical Indicator. There­
were found between Group A and Group B, as expected (see Table 5). As
fore, the greater the tendency toward Refusal (REF), reflecting higher
these results showed, Group A (juvenile delinquents) tends to obtain
levels of defensive behaviour, the lower the tendency to identify in
higher scores in the Refusal (REF) (Clinical Indicator), Depression (DEP)
oneself the emotional and behavioural problems. Consequently, lower
(Clinical Scale), and Support-Child (SUP-C) (Adaptive Scale), whereas
occurrence of Refusal (REF) is associated with the manifestation of these
Group B (adolescent victims of maltreatment) tends to obtain higher
problems in this group of adolescent victims of maltreatment, such as
scores in the Unresolved (UNR) (Clinical Scale) and Atypical Response
attention deficit or hyperactivity problems, withdrawn, or thought
(ATY) (Clinical Indicator). McArthur and Roberts (1982) refer that
problems (YSR).
higher scores in Clinical Scales may suggest individuals' psychological
Overall, no significant correlations were found between the Clinical
difficulties and conflicts, and higher scores in Clinical Indicators point
Scales of RATC and the scales of YSR, but significant correlations were
toward significant clinical information to be analysed. Therefore, these
found for Clinical Indicators [Refusal (REF) and Maladaptive Outcome
findings were expected considering the individuals' behavioural and
(MAL)], which partly constitutes evidence of the convergent validity. In
emotional functioning in these forensic settings. The least expected
addition, no significant correlations were found between the Adaptive
result is the differences with Support-Child (SUP-C) scale, used when the
Scales of RATC and the scales of YSR [except between Support-Other
individual shows self-sufficiency and maturity. Compared to Group B,
(SUP-O) and Somatic Complaints (0.34) and between Limit Setting
Group A shows higher scores on this scale [but their scores are signifi­
(LIM) and Withdrawn (0.32)], which mainly constitutes evidence of the
cantly lower when compared with community sample (t = − 2.472, p =
discriminant validity. Therefore, Hypothesis 3 is only partially
.022; d = − 0.52), see Table 6], which may reflect the low self-esteem,
confirmed.
low self-efficacy, and emotional problems felt by adolescents who
Hypothesis 4. Considering the ages (11 to 17 years old) and years of were victims of maltreatment and abuse. The Hypothesis 5 is confirmed.

10
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Hypothesis 6. There are significant mean differences on RATC be­ (75 % of responses, same for the original data, see Roberts, 1994), and
tween the forensic groups and the community sample [from Canais this is coded with Anxiety (ANX) scale (among others), hence it accounts
(2012)' study]. for Anxiety (ANX) (among other examples). Aggression (AGG) is used
when the character expresses anger (well managed or not) or when is
The study of mean differences between the forensic groups (Group A
involved in verbal or physical aggression, destruction of objects, etc. In
and Group B) of the present study and Canais' data from the community
Gonçalves et al. (1999)' study, for example, in card 13 “Aggression
sample also showed several significant and statistical differences, with
Release” the most frequent theme was “Child expressing anger; resolved
very large to medium effect size, as expected (see Table 6). As these
by another person” (30 % of responses, and a further 50 % for other
results showed, the forensic groups tend to obtain higher scores in the
types of resolution; corresponding to a total of 80 % of the most expect
Support-Other (SUP-O), Unresolved (UNR), and Maladaptive Outcome
responses for this card; see Roberts, 1994), and this is coded with
(MAL) scales, and the community sample tends to obtain higher scores
Aggression (AGG) scale (among others). Therefore, these results can also
in the Support-Child (SUP-C), Problem Identification (PROB),
be evidenced in individuals from a normative sample.
Resolution-2 (RES-2), Anxiety (ANX), Aggression (AGG), Depression
Depression (DEP) mean scores are indeed higher in the community
(DEP), and Atypical Response (ATY) scales. As expected, the forensic
sample (Table 6), and these are the least expected results of the present
groups generally score higher in Clinical Scales and Clinical Indicators,
study. Comparing among the forensic groups, Depression (DEP) mean
and score lower in Adaptative Scales.
scores are higher in Group A than Group B [t(73) = 2.088, p = .040; d =
Unresolved (UNR) is a Clinical Scale which scores when the indi­
0.49] (Table 5). Depression (DEP) is used when the character expresses
vidual states a problem in the story and the characters do not resolve it,
sadness, disappointment, fatigue, nightmares, boredom, suffering,
and Maladaptive Outcome (MAL) is a Clinical Indicator which is used
apathy, etc. In Gonçalves et al. (1999)' study, for example, in card 12
when the outcome of characters are inappropriate (socially disapproved
“Parental Conflict/Depression” the second most frequent theme was
behaviour, inadequate use of defences, withdrawing or taking over
“Family sadness by event or problem” (20 % of responses; see Roberts,
autocratically, deceiving, manipulating), and all those characteristics
1994), and this is coded with Depression (DEP) scale (among others).
are coherent with the individual's behavioural, social and emotional
Therefore, these results are less frequent, but can also be evidenced in a
functioning assessed in a forensic setting. Support-Other (SUP-O) is an
normative sample. Atypical Response (ATY) was also not expected for
Adaptive Scale which is used when the story entails giving help and
this group, because it is a Clinical Indicator which reflects an extreme
emotional support; several cards (such as cards 2 “Maternal Support”, 5
deviation from the usual themes of the card (distorted emotion, denial of
“Parental Affection”, or 10 “Sibling Rivalry”) show clearly supportive
obvious aspects of a picture, unrealistic content). This is a significant
interaction, and high scores are frequent in both well-adjusted and
clinical information and may reflect some psychological difficulties or,
clinical individuals (see Roberts, 1994). However, Refusal (REF) is a
more likely, a defensive behaviour in some participants of this com­
Clinical Indicator which is used when the individual rejects the card, and
munity sample. Even individuals from a normative sample may also use
this is a significant clinical information, because it never happens in
denial (which is different from refusal) to manage strong negative
well-adjusted individuals, only in maladjusted ones (individuals with
emotions, for example, in situations of explicit aggression, such as cards
insufficient cognitive development, individuals with emotional block­
9 “Physical Aggression” and 13 “Aggression Release”. Or may be a
ing, or individuals with oppositional behaviours) (see Roberts, 1994). In
regular situation, if they tell stories about “ghosts” or “monsters” from a
the forensic sample of the present study, Group A show a mean score of
nightmare in cards 7 “Dependency/Anxiety” and 11 “Fear”, which are
1.05 (SD = 1.34) for this scale, and Group B a mean score of 0.11 (SD =
also coded with Atypical Response (ATY) scale (among others). There­
0.32), whereas the Canais' community sample did not score for this
fore, the Hypothesis 6 is confirmed.
scale. All these results were expected. In Group A, consisting of juvenile
The fact that some significant differences between Group A and
delinquents, the scores on Refusal may be related with emotional
Group B, and several significant differences between the forensic groups
problems and/or oppositional behaviours; whereas in Group B, con­
and the Canais' community sample were found, supports the establish­
sisting of adolescent victims of maltreatment, the scores on Refusal may
ment of some preliminary indicators for the interpretation of the RATC
be related with emotional problems (such as emotional blocking).
data to enhance the assessment of the adolescents in these forensic
Support-Child (SUP-C), Problem Identification (PROB), and
contexts, particularly in educational centres and residential care.
Resolution-2 (RES-2) are Adaptive Scales. Support-Child (SUP-C) is used
In general, the RATC showed minimally acceptable reliability indices
when the individual shows self-sufficiency and maturity, Problem
and adequate criterion validity indices, considering that this instrument
Identification (PROB), when the individual formulates concepts beyond
is a projective method and a thematic approach, despite lower psycho­
the nature of the card, and Resolution-2 (RES-2) when the individual
metric robustness when compared with the psychometric tests, as ex­
indicates a constructive resolution for a problem (limited to the situa­
pected, including objective personality assessment instruments. This
tion). The higher scores of the community sample on these scales were
limitation is compensated by the clinical value of the data that this type
expected (Table 6), since Adaptive Scales indicate greater adaptive
of instrument allows to get access through the projection of individual's
functioning (emotional, behavioural and socially) and they are frequent
thoughts, concerns, conflicts and problem-solving styles dealing with
in well-adjusted individuals (see McArthur & Roberts, 1982). Yet,
stimuli presentation. In spite of these limitations, RATC showed to be
Anxiety (ANX), Aggression (AGG), and Depression (DEP) are Clinical
useful to assess children and adolescents' behavioural, social and
Scales which are used when the characters show this kind of feelings and
emotional functioning in different forensic contexts, as Alberto (1999)
behaviours. According to McArthur and Roberts (1982), higher scores in
and Gonçalves et al. (1999) evidenced in previous validation studies of
Clinical Scales may reflect individuals' psychological difficulties and
this instrument with well-adjusted children and clinical data. However,
conflicts, but they do not entail necessarily a psychological maladjust­
as with other tests, data from RATC need to be complemented with in­
ment (further information is needed). Several cards can be interpreted
puts from other assessment instruments [e.g., Child Behavior Checklist
with these themes (some are more explicit than others) and they should
(CBCL), Teacher's Report Form (TRF), Youth Self-Report (YSR), and
be analysed from case to case. Anxiety (ANX) is used when the character
other projective and objective tests] and techniques (e.g., interview,
expresses anxiety or reveals apprehension, surprise, guilt, embarrass­
observation), and other sources of information (e.g., parents, teachers).
ment, doubts about his or her own competence, illness, etc. Gonçalves
Two limitations of the present study are the relatively small sample
et al. (1999) studied the most frequent themes for each of the sixteen
size (in total), and the large portion of male participants. A larger
cards with clinical and well-adjusted Portuguese children; for example,
forensic sample is needed. And a more representative sample of delin­
in card 4 “Support/Aggression” the most frequent theme was “The girl
quent female adolescents from the educational centres and from resi­
lying down is hurt or sick, and the other girl will help her or seek help”
dential protection is also needed (all participants in Group A are male,

11
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

and 54.3 % of the participants in Group B are also males). This should be Duncan, L. A. (1993). The Roberts apperception test for children: A study of characteristics of
non-clinical children and their mothers and clinical children and their mothers. Master’s
improved in future validation studies to analyse a more representative
thesis (unpublished). Los Angeles, US: California School of Professional Psychology
forensic sample to establish indicators of interpretation and carry out https://www.proquest.com/openview/3ba8c4b82f506ce795bdeb229928de68/1?
factor analysis studies. Another limitation is the use of a reference pq-origsite=gscholar&cbl=18750&diss=y.
sample for comparison (Canais' study). Therefore, it is desirable to Dupree, J. L., & Prevatt, F. (2003). Projective storytelling techniques. In C. R. Reynolds,
& R. W. Kamphaus (Eds.), Handbook of psychological and educational assessment of
extent the validation studies with a larger, representative, and norma­ children: Personality, behavior, and context (2nd ed., pp. 66–90). The Guilford Press.
tive sample for comparisons. WISC-III and YSR were used in different Eysenck, H. J., & Eysenck, M. W. (1985). Personality and individual differences: A natural
subsamples and this also constitutes a limitation; for future studies, the science approach. Plenum Press.
Fonseca, A. C., & Monteiro, M. C. (1999). Um inventário de problemas de
evaluation protocols should include more instruments to become more comportamento Para crianças e adolescentes: O youth self-report de achenbach [An
wide-ranging and more effective for criterion validity studies and ana­ inventory of behavioural problems for children and adolescents: Achenbach's youth
lysing the RATC potentialities. It is also desirable to develop further self-Report]. Psychologica, 21, 79–96.
Frank, L. K. (1939). Projective methods for the study of personality. The Journal of
criterion validity studies of the RATC using other personality assessment Psychology: Interdisciplinary and Applied, 8(2), 389–413. https://doi.org/10.1080/
instruments, including projective and objective ones [e.g., projective 00223980.1939.9917671
methods, such as the Children's Apperception Test (CAT), Thematic Frank, L. K. (1948). Projective methods. Charles C. Thomas.
Frick, P. J., Barry, C. T., & Kamphaus, R. W. (2020). Clinical assessment of child and
Apperception Test (TAT), House-Tree-Person (HTP); objective tests: the adolescent personality and behavior (4th ed.). Springer.
Minnesota Multiphasic Personality Inventory – Adolescent (MMPI-A), Friedrich, W. N., & Share, M. C. (1998). The Roberts apperception test for children: An
Piers-Harris Children's Self-Concept Scale (PHCSCS)], as well as some exploratory study of its use with sexually abused children. Journal of Child Sexual
Abuse, 6(4), 83–91. https://doi.org/10.1300/J070v06n04_06
behavioural and emotional functioning measures, already mentioned,
Gonçalves, M., Morais, A. P., Pinto, H., & Machado, C. (1999). Teste Aperceptivo de
including symptoms checklists [e.g., Children's Depression Inventory Roberts para Crianças [Roberts apperception test for children]. Master’s thesis
(CDI)]. (unpublished). In M. R. Simões, M. M. Gonçalves, & L. S. Almeida (Eds.), Vol.2. Testes
e provas psicológicas em Portugal (pp. 185–198). APPORT/SHO.
Gottfredson, L. S. (1997). Editorial: Mainstream science on intelligence: An editorial with
Declaration of competing interest 52 signatories, history, and bibliography. Intelligence, 24(1), 13–23. https://doi.org/
10.1016/S0160-2896(97)90011-8
We declare that we have no conflicts of interest. Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10,
255–282. https://doi.org/10.1007/BF02288892.
Headen, E. H. (1986). Roberts’ apperception test for children: Referred and nonreferred
Data availability student profiles. Master’s thesis (unpublished). US: University of Nebraska at Omaha
https://digitalcommons.unomaha.edu/cgi/viewcontent.cgi?article=1259&context
The data that has been used is confidential.
=studentwork.
Heilbrun, K. (1992). The role of psychological testing in forensic assessment. Law and
Human Behavior, 10(3), 256–272. https://doi.org/10.1007/BF01044769
References Heilbrun, K., Grisso, T., & Goldstein, A. M. (2009). Foundations of forensic mental health
assessment. Oxford University Press.
Joiner, T. E. (1996). The relations of thematic and nonthematic childhood depression
Achenbach, T. M. (1991). Manual for the Youth Self-Report and 1991 Profile. University of
measures to defensiveness and gender. Journal of Abnormal Child Psychology, 24(6),
Vermont.
803–813. https://doi.org/10.1007/BF01664741
Ackerman, M. J. (2010). Essentials of forensic psychological assessment (2nd ed.). John
Joiner, T. E., & Barnett, J. (1994). A test of interpersonal theory of depression in children
Wiley & Sons.
and adolescents using a projective technique. Journal of Abnormal Child Psychology,
Aiken, L. R. (1996). Personality assessment: Methods and practices (2nd ed.). Hogrefe and
22, 595–609. https://doi.org/10.1007/BF02168940.
Huber Publishers.
Kaufman, A. S., & Lichtenberger, E. O. (2000). Essentials of WISC-III and WPPSI-R
Alberto, I. (1999). Avaliação da Perturbação Pós-Stress Traumático (PTSD) e suas
assessment. John Wiley & Sons.
consequências psicológicas em crianças e adolescentes vítimas de maltrato. [Assessment of
Kline, P. (1993). Personality: The psychometric view. Routledge.
Post-Traumatic Stress Disorder (PTSD) and its psychological consequences in children and
Lampel, A. K. (1996). In , 34. Children’s alignments with parents in highly conflicted custody
adolescents victims of abuse]. Doctoral dissertation (unpublished). Portugal: Faculty of
cases Family and Conciliation Courts Review (pp. 229–239). https://doi.org/10.1111/
Psychology and Educational Sciences, University of Coimbra https://estudogeral.sib.
j.174-1617.1996.tb00416.x
uc.pt/handle/10316/949?locale=pt.
Lavado, A. M. (2008). Relações de vinculação em crianças com dermatite atópica
American Educational Research Association, American Psychological Association,
[Attachment relationships in children with atopic dermatitis]. Lisboa, Portugal: Instituto
National Council on Measurement in Education. (2014). Standards for educational
Superior de Psicologia Aplicada. https://repositorio.ispa.pt/bitstream/10400.12/
and psychological testing. https://www.apa.org/science/programs/testing/standa
598/1/DM%20LAVA-A1.pdf.
rds.
Leifer, M., Shapiro, J., Martone, M. W., & Kassem, L. (1991). Rorschach assessment of
Anastasi, A. (1988). Psychological testing (6th ed.). MacMillan.
psychological functioning in sexually abused girls. Journal of Personality Assessment,
Anastasi, A., & Urbina, S. (2000). Testagem psicológica. In Psychological testing. Artmed.
56, 14–28. https://doi.org/10.1207/s15327752jpa5601_2
Bell, N. L., & Nagle, R. J. (1999). Interpretive issues with the Roberts apperception test
Lilienfield, S. O., Wood, J. M., & Garb, H. N. (2000). The scientific status of projective
for children: Limitations of the standardization group. Psychology in the Schools, 36
techniques. Psychological Science in the Public Interest, 1(2), 27–66. https://doi.org/
(4), 277–283. https://doi.org/10.1002/(SICI)1520-6807(199907)36:4<277::AID-
10.1111/1529-1006.002
PITS1>3.0.CO;2-Q
Louw, A. E., & Ramkisson, S. (2002). The suitability of the Roberts apperception test for
Bellak, L. (1944). The concept of projection. Psychiatry, 7(4), 353–370. https://doi.org/
children (RATC), the house-tree-person (H-T-P) and draw-A-person (D-A-P) scales in
10.1080/00332747.1944.11022519
the identification of child sexual abuse in the indian community: An exploratory
Burman, B., John, R. S., & Margolin, G. (1987). Effects of marital and parent- child
study. Southern African Journal of Child and Adolescent Mental Health, 14, 91–106.
relations on children’s adjustment. Journal of Family Psychology, 1, 91–108. https://
https://doi.org/10.1080/16826108.2002.9632431
doi.org/10.1037/h0080439
McArthur, S., & Roberts, G. (1982). Roberts apperception test for children manual. Western
Callender, J. C., & Osburn, H. G. (1979). An empirical comparison of coefficient alpha,
Psychological Services.
Guttman's lambda-2, and MSPLIT maximized split-half reliability estimates. Journal
McGrath, R. E., & Carroll, E. J. (2012). The current status of “projective tests”. In
of Educational Measurement, 16(2), 89–99. https://doi.org/10.1111/j.1745-
H. Cooper (Ed.), Vol.1. APA handbook of research methods in psychology (pp.
3984.1979.tb00090.x
329–348). American Psychological Association.
Canais, J. C. P. (2012). Diferenças no funcionamento emocional, comportamental e social
Melton, G. B., Petrila, J., Poythress, N. G., & Slobogin, C. (2007). Psychological evaluations
(Roberts Apperception Test for Children) e na autoestima (Rosenberg Self-Esteem Scale)
for the courts: Handbook for mental health professionals and lawyers (3rd ed.). Guilford
entre jovens institucionalizados e não institucionalizados [Differences in emotional,
Press.
behavioural and social functioning (Roberts Apperception Test for Children) and self-
Mendes, T. C., & Sani, A. (2015). Representações de crianças expostas a violência
esteem (Rosenberg Self-Esteem Scale) between institutionalised and non-institutionalised
interparental através de provas projetivas [Representations of children exposed to
youths]. Master’s thesis (unpublished). Coimbra, Portugal: Instituto Superior Miguel
interparental violence through projective tests]. Revista de Psicologia da Criança e do
Torga https://repositorio.ismt.pt/handle/123456789/108.
Adolescente, 6, 171–192.
Cervone, D., & Pervin, L. A. (2018). Personality: Theory and research (14th ed.). Wiley.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence
Palomares, R. S., Crowley, S. L., Worchel, F. F., Olson, T. K., & Rae, W. A. (1991). The
Erlbaum Associates.
factor analytic structure of the Roberts apperception test for children: A comparison
Cohen, R. J., & Swerdlik, M. E. (2018). Psychological testing and assessment: An
of the standardization sample with a sample of chronically ill children. Journal of
introduction to tests and measurement (9th ed.). McGraw-Hill.
Personality Assessment, 56(3), 414–425. https://doi.org/10.1207/
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests.
s15327752jpa5603_4
Psychometrika, 16, 297–334. https://doi.org/10.1007/BF02310555.
DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE.

12
P.A. Almiro et al. Acta Psychologica 235 (2023) 103900

Piotrowski, C. (2019). Projective techniques are not moribund [editorial]. Journal of Teglasi, H. (2010). Essentials of TAT and other storytelling instruments assessment (2nd ed.).
Projective Psychology & Mental Health, 26, 73–76. Wiley.
Roberts, G. (1994). Interpretive handbook for the Roberts Apperception Test for Children. VandenBos, G. R. (2015). APA dictionary of psychology (2nd ed.). American Psychological
Western Psychological Services. Association.
Simões, M. R. (2003). Avaliação neuropsicológica e comportamentos anti-sociais: Notas Wechsler, D. (2003). Escala de Inteligência de Wechsler para Crianças – Terceira Edição
breves em torno do papel das competências verbais e das funções executivas (WISC-III): Manual [Wechsler Intelligence Scale for Children – Third Edition (WISC-
[Neuropsychological assessment and antisocial behaviour: Brief notes around the III): Manual]. CEGOC-TEA.
role of verbal skills and executive functions]. In I. Alberto, A. C. Fonseca, Weiner, I. B., & Greene, R. L. (2017). Handbook of personality assessment (2nd ed.). Wiley.
C. P. Albuquerque, A. G. Ferreira, & J. Rebelo (Eds.), Comportamento anti-social: Wells, A. M., Chasnoff, I. J., Schmidt, C. A., Telford, E., & Schwartz, L. D. (2012).
Escola e família [Antisocial behaviour: School and family] (pp. 129–143). Centro de Neurocognitive habilitation therapy for children with fetal alcohol spectrum
Psicopedagogia da Universidade de Coimbra. disorders: An adaptation of the alert Program®. American Journal of Occupational
Simões, M. R., Seabra-Santos, M. J., Albuquerque, C. P., Pereira, M. A., Almeida, L. S., Therapy, 66, 24–34. https://doi.org/10.5014/ajot.2012.002691
Rocha, A. M., Ferreira, C., Lopes, A. F., Gomes, A. A., Xavier, R. E., Rodrigues, F., Wetzel, E., Böhnke, J. R., & Brown, A. (2016). Response biases. In F. Leong, D. Bartram,
Lança, C., Matoso, J., Nunes, J., Filipe, M., & Eusébio, C. (2006). Escala de F. Cheung, K. F. Geisinger, & D. Iliescu (Eds.), The ITC international handbook of
Inteligência de Wechsler para Crianças – Terceira Edição (WISC-III) [Wechsler testing and assessment (pp. 349–364). Oxford University Press.
Intelligence Scale for Children – Third Edition]. In M. M. Gonçalves, M. R. Simões, Worchel, F. F., Rae, W. A., Olson, T. K., & Crowley, S. L. (1992). Selective responsiveness
L. S. Almeida, & C. Machado (Eds.) (2ª ed.,, Vol.1. Avaliação psicológica: Instrumentos of chronically ill children to assessments of depression. Journal of Personality
validados para a população portuguesa [Psychological assessment: Instruments validated Assessment, 59(3), 605–615. https://doi.org/10.1207/s15327752jpa5903_13
for the Portuguese population] (pp. 199–232). Quarteto.

13