Test Development A good item on a nrt is an item from which high

scorers on the test respond correctly, low

Process of Developing a test:
scorers respond to that same item incorrectly;
1. Conceptualization same as crt.
2. Construction
Pilot Work
3. Tryout
4. Analysis - Pilot study or research
5. Revision - To evaluate whether they
should be included in the final
form.
I. Test Conceptualization - Developer typically attempts to
- Emerging social phenomenon determine how best measure a
or pattern of behavior might targeted construct. Creation
serve as the stimulus for the revision and deletion of many
development of a new test. items are part of the process.
- May be in response to a need to II. Test Construction
assess mastery in an emerging - Generally concerned with
occupation or profession. turning performance into
numbers.
The difference of norm and criterion is based on
the scores. Some tests can provide both results. Scaling

- May be defined as the process

of setting rules for assigning
Criterion Referenced Tests numbers in measurement.
- Compare a person’s knowledge - The process by which a
or skills against a measuring device is designed
predetermined standard, cut and calibrated.
score or other criterion. - Which scale values are assigned
- The performance of other to different amounts of the
students does not affect a trait, attribute, or characteristic
student’s score. being measured.

Norm Referenced Tests Types of Scales

- Compare a person’s knowledge Age-based scale – if the test taker’s test

or skills to the knowledge or performance as a function of age is of critical
skills of the norm group. interest.
- Norm groups – age, race, etc. Grade-based scale – As a function of
- Individual percentile rank grade.
describes their performance in
comparison to the norm, but Stanine Scale – Transformed into scores
does not indicate whether or that can range from 1 to 9.
not they met or exceed a
specific standard.
Writing Items - Binary Choice Item – Two
possible responses only.
- What range of content should
the items cover?
- Which of the many different
Constructed-response format
types of item formats should
be employed? - Completion Item – Provide a
- How many items should be word or phrase to complete a
written in total and for each sentence.
content area covered? - Short Answer – A word, a term,
a sentence, or a paragraph.
When devising a standardized test using a
- Essay – Respond to a question
multiple-choice format – twice the number of
by writing a composition,
items.
typically one that demonstrates
Item pool recall of facts, etc.
o Depth of Knowledge.
- Reservoir
- Will or will not be drawn for the
final version
PARAMETIC TEST
Clinical setting – clinicians, patients, patients’
- Makes assumptions about the
family members, clinical staff.
population
Personnel psychologists – members of a - Uses a normal probabilistic
targeted industry or organization. distribution
- Uses a mean value for the
School Psychologists – teachers, administrative
central tendency
staff, educational psychologists.
- Requires previous knowledge
Item Format about the population
- Interval and ratio
- Variables such as the form, plan
structure, arrangement, and NONPARAMETIC TEST
layout of individual test items.
- Doesn’t make any assumptions.
Item Format - The distribution is arbitrary.
- Uses the median value for the
1. Selected-response format
central tendency.
2. Constructed-response format
- Doesn’t require previous
Selected-response format knowledge about the
population.
- Multiple choice – Choose one - Nominal and Ordinal
or more items
- True or False Norms
- Matching Type – Determine
- Refers to the performances by
which response is best
defined groups on particular
associated with which premise
tests.
 - Norms are established by - Norm Referenced – To derive
determining what persons in a meaning from a test score is to
representative group actually evaluate the test score in
do on the test relation to other scores on the
same test.
Types of Norms
Criterion Referenced Test – Test that describes
- Percentile – raw data from a
the specific types of skills, tasks or knowledge of
test’s standardization sample
an individual relative to a well-defined criterion.
converted to percentile form
- Age norms – age-equivalent TYPES OF PSYCHOLOGICAL TESTS
scores, indicate the average
- According to the number of
performance of different
test takers:
samples of test takers who were
o Individual Testing- one
at various ages the time the test
person
was administered.
o Group testing – one or
- Grade norms – Indicate the
average test performance of more person.
test takers in a given school - To the Variable being
grade. measured:
- National norms – Derived from o Ability tests
a normative sample that was  Achievement –
nationally representative of the degree of
population at the time the learning
norming study was conducted.  Aptitude –
- National Anchor norms – potential for
provide some stability to test learning or
scores by anchoring them to acquiring a
other test scores. specific skill.
- Subgroup norms – Can be  Intelligence –
segmented by any of the person’s
criteria initially used in selecting general
subjects for the sample. potential to
- Local norms – Provide solve problem
normative information with and profit from
respect to local’s population’s experience.
performance on some test.  Personality
- Tracking - tendency to stay at tests
about the same level relative to Objective – usually in a self-report format the
one’s peers. require the subject to choose between two or
- Fixed Reference Group Scoring more response such as true or false.
Systems Norms – provide a
context for interpreting the Projective – tests’ stimulus subject’s response
meaning of a test score. ambiguous.
The Sixteen Personality Factor Questionnaire between examinees who are
(16PF) is a self-report personality test knowledgeable and those who
developed over several decades of empirical are not.
research by Raymond B. Cattell, Maurice
Types of discrimination index
Tatsuoka and Herbert Eber.
- Zero discrimination - equal
- Positive “– upper group answers
ITEM ANALYSIS the item correctly than the
lower.
- A process which examines
- Negative “- lower group
examinee responses to
answers the item correctly than
individual test items in order to
the upper.
assess the quality of those
items.

Tool tests:

- Item-difficulty index
- Item-reliability index
- Item-validity index
- Item-discrimination index

Item Difficulty index D = (U-L)/ n

- Indicates the proportion of test- U – correct answer in high
takers who got the item right.
- High percentage – easy item. L – “low
- Low percentage – difficult item. N – number of examinees
P = R/N x 100 Other Considerations in Item Analysis
R – number of test takers •Guessing – consider the following
N – total number of test-taker three criteria that any correction for
guessing must meet as well as the
P – percent of people passing an item interacting problems that must be
addressed.

• Item fairness – biased item –

favors one particular group of
examinees in relation to another.
ICC can be used.

• Speed test – under speed

conditions yield misleading or
The Item-Discrimination Index uninterpretable result.

- Is a measure of how well an Qualitative Item Analysis

item is able to distinguish
 - The examiner carefully - Examiners may be involved to
proofreads the test before it is greater or lesser degrees in final
administered. scoring of protocols.
- Anchor protocol – Scored by a
highly authoritative scorer that
is designed as a model.
- Scoring Drift – Discrepancy
Expert Panels between anchor and scoring of
- Find the information provided another protocol.
by the experts. 3 of many applications of IRT:
Sensitivity Review • Evaluating existing tests for the
- Study of test items, typically purpose of mapping test revisions
conducting during the test •Determining measurement
development process. equivalence across test takers
- Fairness to all prospective test population
takers.
•Developing item banks
- Presence of offensive language,
stereotypes.

Test Revision OBJECTIVE TEST

- Characterize each item to its - Can be objectively scored and

strength and weaknesses whose meaning or purpose is
- Highly reliable but lack criterion hidden from subjects.
validity. - A psychological test that
measures an individual’s
Cross-Validation or Co-Validation characteristics in a way that
isn’t influenced by the
- Cross validation - Refers to the examiner’s own beliefs.
revalidation of a test on a - EXAMPLES:
sample of test takers other than - •Myers-Briggs Type Indicator
those on whom test (MBTI)
performance originally found to •Neo Pi-R
be valid predictor. •Minnesota Multiphasic
- Validity Shrinkage – Decrease in Personality Inventory (MMPI)
item validities. •16 PF
- Co-validation – conducted on PROJECTIVE TECHNIQUES
two or more test using the
same sample of test takers. - Application of vague,
- Co-norming – Conjunction with unstructured stimulus objects
or situations in order to
the criterion of norms or the
manifest the individual’s
revision of existing norms.
characteristics modes of
Quality assurance perceiving his world.
 - Theoretical concept - Projective Construction Technique - Creating of An
techniques originated from Imaginable Production for which The Test
psychoanalytic theory Materials Provide a Frame Work Ex: Thematic
Apperception Test-TAT, Fairy Tale Test.
Francis Galton (1879): Utilize word association
method for measuring intelligence. Expressive technique does not depend on test
stimuli, but rather ask the subject to perform an
Kraepelin (1892): Used Association method for artistic or creative action
studying effects of fatigue, hunger etc.
Carl Jung: The first to formulate a standardized
word association projective test.
Murray and Morgan (1935): Developed
Thematic Apperception Test in 1935 in USA.
Sentence completion technique: flourished in
1920s, brought into concern in 1950s. A stem of
sentence was given and the second part was
free to fill in.
Goodenough and Harris (1930s): ‘Expressed
drawings’ as a supplementary measure of
intelligence.
Buck (1949): Developed House-Tree-Person
Test, which consist of drawings of a house, a
tree and some persons.
Karen Machover (1949): Utilize Draw-A-Person
Test for the purpose of personality assessment.
Projective test – assumed that an individual’s
responses to an unstructured stimulus are
influenced by his needs, motives, fears,
expectations, and concerns.

Intelligence – The ability to learn from

experience, solve problems, and use of
knowledge to adapt to new experiences.
Elements:
Association Technique: It Inquires the Subject - Abstract thinking and reasoning
to state What Is Suggested by A Verbal, Visual, - Problem solving ability
Or Auditory Stimuli. Ex: Word Association, - Capacity to acquire knowledge
Rorschach Ink-blot Test - Memory
 - Adaptation to one’s Intelligence Quotient Terman coined the term
environment “Intelligence Quotient” (IQ) in 1916 Developed
- Mental Speed Intelligence Quotient IQ = (MA/CA)*100
- Linguistic Competence
MA= Mental Age ; CA = Chronological Age
Factor Analysis – is a statistical analysis method
used to describe variability among observed,
correlated variables in terms of a potentially
lower number of unobserved variables called
factor.
Spearman’s two factor theory ENVIRONMENTAL INFLUENCES ON IQ

General Factor – “mental energy” - Factors that enhance IQ:

▸ Stimulating environment
- Abstract reasoning, problem ▸ Good medical care/nutrition
solving ▸ Parental involvement in
Specific Factor – Less complex tasks learning
▸ Rich language environment
- Motor speed, attention, v-m
coordination, memory Factors that negatively impact
IQ:
High positive correlations – highly saturated - Persistent poverty
with G. - Perinatal complications,
Low or moderate viewed as measures of S such - Inadequate stimulation in
as visual or motor ability. environment
- lead exposure
Thurstone’s theory of primary mental abilities - Large family size
- Spatial Ability - Nutrition during gestation and
- Verbal Comprehension early childhood
- Word fluency
- Perceptual speed
- Numerical ability
- Inductive reasoning
- Memory
Cattell and Horn’s fluid and crystallized
intelligence
Fluid – inborn abilities
Crystallized – acquired skills
Piaget’s theory of development
- Intelligence may be conceived
of as a kind of evolving
biological adaptation to the
outside world.
- Two basic mental operations:
assimilation and accomodation.