Limitation of Test

► 1) Varying Interpretation and uses

► 2) Uncertainty of Measurements

► 3) Changing Circumstances

► 4) Cultural Biases

Characteristics of a Good Test

► 1. Objectivity - It should be free from subjective-judgement regarding the ability, skills,

knowledge, trait or potentiality to be measured and evaluated.

► 2. Reliability – This refers to the extent to which the obtained results are consistent and
reliable.

► 3. Validity – It refers to the extent to which the test measures what it intends to measure.
It explains us whether the test fulfills the objective of its development.

► 4. Norms – This refer to the average performance of a representative sample on a given

test. It gives a picture of average standard of a particular sample. Norms are standard
scores, developed by the person who develops test, so that the future test users can
compare their scores with norms to know the level of their sample.

► 5. Practicability – Test must be practicable in-time required for completion, the length,
number of items or questions, scoring, etc. It should not be lengthy & difficult to answer
as well as scoring.

CORRELATION COEFFICIENT

Correlation

► Correlation means association - more precisely it is a measure of the extent to which two
variables are related. There are three possible results of a correlational study: a positive
correlation, a negative correlation, and no correlation.

► A positive correlation is a relationship between two variables in which both

variables move in the same direction. Therefore, when one variable increases as
the other variable increases, or one variable decreases while the other decreases.
An example of positive correlation would be height and weight. Taller people
tend to be heavier.

Correlation
 ► A negative correlation is a relationship between two variables in which an
increase in one variable is associated with a decrease in the other. An example of
negative correlation would be the % correct in a test and the number of your
mistakes. As you have a high % correct, (increase in score) it gets lesser mistakes
(decrease in wrong answers).

► A zero correlation exists when there is no relationship between two variables.

For example there is no relationship between Religion and Grade Point Average

Uses of Correlation

Prediction

● If there is a relationship between two variables, we can make predictions about one from
another.

Validity

● Concurrent validity (correlation between a new measure and an established measure).

Reliability

● Test-retest reliability (are measures consistent)-it refers to the extent that a test produces
similar results over time.

Generally a test-retest reliability correlation of at least 0.80 or higher indicates good reliability.

We calculate the test-retest reliability by using the Pearson Correlation Coefficient, which takes
on a value between -1 and 1 where:

 -1 indicates a perfectly negative linear correlation between two scores

 0 indicates no linear correlation between two scores
 1 indicates a perfectly positive linear correlation between two scores

● Inter-rater reliability (are observers consistent). (IRR) ensures that there is a level of
agreement between judges in a given competition.
Theory verification

● Predictive validity.- refers to the ability of a test or other measurement to predict a future
outcome.

Scattergram (scatter plot, scatter graph, scatter diagram)

► A scattergram is a graphical display that shows the relationships or associations between

two numerical variables (or co-variables), which are represented as points (or dots) for
each pair of score.

► A scatter graph indicates the strength and direction of the correlation between the co-
variables.

Correlation: Measuring the linear association between 2 variables

To compute, use the use the Pearson r formula:

r = _______12(724)-(76)(119)____

12(678)-(76)2 12(1,561)-(119)2

r = -356__ = r = -0.1084 or -0.11

 3,284.44

very low correlation between the results of the scores obtained by the 12 students in the
reading comprehension and vocabulary test.

Correlation coefficient
• Correlation coefficients are used to measure the strength of the relationship between two
variables.

• Pearson correlation is the one most commonly used in statistics. This measures the
strength and direction of a linear relationship between two variables.

• Values always range between -1 (strong negative relationship) and +1 (strong positive
relationship). Values at or close to zero imply weak or no linear relationship.

• Correlation coefficient values less than +0.8 or greater than -0.8 are not considered
significant.

CORRELATION TABLE INTERPRETATION

Reliability

► Reliability – is the extent to which a score or measure is free of measurement error. It is

the consistency of a measuring instrument or a test.

► If results are replicated consistently they are reliable. A correlation coefficient can be
used to assess the degree of reliability. If a test is reliable it should show a high positive
correlation.

Reliability:
(2 Types)

► There are two types of reliability:

► Internal reliability assesses the consistency of results across items within a test.

► Ex. Satisfaction Survey of Customers

► External reliability refers to the extent to which a measure varies from one use to
another.
 Assessing Reliability

Split-Half Method (Internal Consistency)

► The split-half method assesses the internal consistency of a test, such as psychometric tests
and questionnaires. There, it measures the extent to which all parts of the test contribute
equally to what is being measured.

► KR20 = Kuder and Richardson – another method for estimating internal consistency.

► Coefficient Alpha – Cronbach Alpha

Alternate or Parallel form Reliability

► Alternate or parallel form reliability indicates how consistent test scores are likely to be
if a person takes two or more forms of a test.

► A high parallel form reliability coefficient indicates that the different forms of the test are
very similar which means that it makes virtually no difference which version of the test a
person takes. On the other hand, a low parallel form reliability coefficient suggests that
the different forms are probably not comparable; they may be measuring different things
and therefore cannot be used interchangeably.

► Ex. SRA-Verbal Test - Form A ex. 1. Begin means the same as opposite of:
 ► a) morning b) work c) start d)
journey

SRA-Verbal Test - Form B ex. 1. Severe means the same as or opposite of:

❖ a) cloudy b) lax c) flat d) rustic

Test-Retest Method

► The test-retest method assesses the external consistency of a test. Examples of

appropriate tests include questionnaires and psychometric tests. It measures the stability
of a test over time.

Inter-Rater Reliability

► The test-retest method assesses the external consistency of a test. This refers to the degree
to which different raters give consistent estimates of the same behavior. Inter-rater
reliability can be used for interviews.

How Reliable is Reliable?

► Evaluating the reliability of ones test, we must first specify the source of measurement
error – the difference between the true score and the observed score results from
measurement error-(Standard Error of Measurement – SEM)

► X – T = E (X-observed score – T - true score = E error of measurement )

► The SEM is a useful measure of the accuracy of individual test scores. The
smaller the SEM, the more accurate the measurements.

► It gives the margin of error that you should expect in an individual test score
because of imperfect reliability of the test.

► The wider the interval, the lower the reliability of the score. Using SEM, we can
say that we are confident that a person’s true score falls between two values.

How reliable is reliable?

What to do about Low Reliability

► A Test constructor might wish to boost the reliability by increasing the test length or by
using factor analysis to divide the test into homogenous subgroups of items.

► In research – if the reliability is really unacceptable, we can estimate what the correlation
between tests would have been if there had been no measurement of error. This
 procedure is called correction for attenuation. One needs to know the reliabilities of two
tests and the correlation between them.

Principles of Assessment

► Use only reliable assessment instruments and procedures. (in other words – use only
assessment tools that provide dependable and consistent information)

► Use only assessment procedures and instruments that have been demonstrated to be valid
for the specific purpose for which they are being used.

► Use assessment tools that are appropriate for the target population.

Points for Reflections (Assignment):

► Why are longer tests more reliable than short quizzes?

► What should be considered when writing test items to help ensure reliability?

Validity

By; Ms. N.

What is Validity?

► Validity is the most important issue in selecting a test.

► Validity refers to what characteristic the test measures and how well the test

measures that characteristic.

► tells you if the characteristic being measured by a test is related (e.g. - to job
qualifications and requirements.

► Gives meaning to the test scores.

► describes the degree to which you can make specific conclusions or

predictions about people based on their test scores. In other words, it
indicates the usefulness of the test.

Validity …..

∙ Validity tells you if the characteristic being measured by a test is related to job
qualifications and requirements.
 ∙ Validity also describes the degree to which you can make specific conclusions or
predictions about people based on their test scores. In other words, it indicates the
usefulness of the test.

Types of Validity

Content Related Validity

Content validity assesses whether a test is representative of all aspects of the construct.

To produce valid results, the content of a test, survey or measurement method must cover all
relevant parts of the subject it aims to measure. If some aspects are missing from the
measurement (or if irrelevant aspects are included), the validity is threatened.

Face Validity

Face validity

Face validity considers how suitable the content of a test seems to be on the surface. It’s similar
to content validity, but face validity is a more informal and subjective assessment.

Construct Validity

Construct validity

Construct validity evaluates whether a measurement tool really represents the thing we are
interested in measuring. It’s central to establishing the overall validity of a method.

What is a construct?

A construct refers to a concept or characteristic that can’t be directly observed, but can be
measured by observing other indicators that are associated with it.

Criterion Related Validty

CRITERION RELATED VALIDITY = is said to have criterion-related validity when the test
has demonstrated its effectiveness in predicting criterion or indicators of a construct, such as
when an employer hires new employees based on normal hiring procedures like interviews,
education, and experience.4

2 TYPES

1. CONCURRENT

2. PREDICTIVE VALIDTY

CONCURRENT VALIDITY
This is the degree to which a test corresponds to an external criterion that is known concurrently
(i.e. occurring at the same time).

If the new test is validated by a comparison with a currently existing criterion, we have
concurrent validity.

Very often, a new IQ or personality test might be compared with an older but similar test known
to have good validity already.

PREDICTIVE VALIDITY

This is the degree to which a test accurately predicts a criterion that will occur in the future.

To Sum up for Validity

● Construct validity: Does the test measure the concept that it’s intended to measure?

● Content validity: Is the test fully representative of what it aims to measure?

● Face validity: Does the content of the test appear to be suitable to its aims?

● Criterion validity: Do the results correspond to a different test of the same thing?

Reliability vs Validity

Reliability is about method’s consistency while Validity is about accuracy.

Culture Fair IQ Test (CFIT)

Back then American and European tend to have lower IQ scores. This has led some people to
believe that
non-white people or the blacks are less intelligent than the white people, however thus
conclusion is
incorrect.

Raymond Cattell

 After seeing the damage caused by World War I he became interested in applying science
to resolve issues affecting people.
 He developed the Culture Fair Intelligence Test with the intention of creating a test that
was not influenced by these cultural elements.

Culture Fair IQ Test (CFIT)

 An intelligence test in which performance is not based on experience with or knowledge

of a specific culture.
  The Culture Fair Intelligence Test (CFIT) was constructed by Raymond B. Cattell, PhD.
in 1949
 An attempt to produce a measure of cognitive abilities that accurately estimated
intelligence devoid of sociocultural and environmental influences.
 Cattell proposed that general intelligence comprises both Fluid Intelligence and
Crystallized Intelligence.

Purpose

 This nonverbal intelligence test is culture fair because it avoids cultural and language
biases and focuses on logical reasoning only.
 It assess a person's IQ in a way that minimize the impact of linguistic fluency, cultural
context, and educational attainment.
 The test, which may be given to an individual or a group, is a non-verbal and just requires
that the subject be able to recognize relationships between different shapes and figures.
 A culture-fair test is test designed to be free of cultural bias, as far as possible, so that no
one culture has an advantage over another.
 The test is designed to not be influenced by verbal ability, cultural climate, or educational
level.
 This IQ test uses one type of question with which your intelligence is assessed. Each
question consists of a 3 by 3 matrix, with a question mark in one of the cells. Your job is
to to identify the missing element that completes a pattern of shapes by picking the
correct element out of 6 options.
 This IQ test is designed to test non-verbal intelligence while minimizing cultural or
educational biases.
 It's a culture fair IQ test that will measure your intellectual ability, entertain you
infinitely, and tie your brain in a thousand knots!
 It is based on image patterns, thus avoiding the cultural biases inherent in other
intelligence tests. The questions are ordered in terms of difficulty.

A non-verbal paper test designed as a culturally unbiased test to measure one’s intelligence.

 30 questions, 30 minutes

 Test your cognitive ability

 Test used for admission to the International IQ Society

 Scientifically validated

 Official IQ score certificate included Sample IQ Report

Administration

 Both groups & individuals may take the test.

 Exam booklets can be reused by making answers either in the test booklet or on
separate response sheets.

Basic Personality Inventory (BPI)

Author:  Dr. Douglas N. Jackson

Published by : Sigma Assessment Systems Inc.

Purposes

 To meet the challenge of response bias in personality assessment by suppressing the role
of the general evaluative dimension in scale construction.
 To make available a questionnaire measure that would be capable of yielding reliable
discrimination among persons showing different patterns of dysfunction within both the
normal and pathological ranges.
 To employ dimensions that refer to relatively homogeneous behavioral characteristics
that vary systematically in the general population and within deviant groups.

Administration

 Adolescents and adults

 Can be self-administered or computer-administered 240 questions.
 There are different ways of checking based on the administration.
 It is important to explain to the respondents the aims and purposes of the testing and the
subsequent use to be made of the results.
 Group testing situations should be supervised and respondents arranged so that they
cannot observe one another's responses.
 Interaction between respondents should be discouraged. In an individual testing situation,
a respondent should be placed in a space free from distraction.
 Respondents should be provided with a pencil and an eraser.

For both group and individual administration, it is often helpful to familiarize respondents with
what is required by reading aloud the instructions on the BPI cover. The instructions to be read
are as follows:
In your BPI booklet, you will find a series of statements that people might use to describe
themselves. Read each statement and decide whether or not it describes you. Then indicate your
answer on the separate answer sheets. If you agree with a statement or decide that it does
describe you, answer TRUE. If you disagree with a statement or feel that it is not descriptive of
you, answer FALSE. In marking your
answers on the answer sheet, be sure that the number of the statement you have just read is the
same as the number on the answer sheet. Answer every statement either true or false, even if you
are not completely sure of your answer. The examiner should ask the respondents whether or not
they understand the directions before instructing them to proceed.

The use of answer sheets should be explained. Before permitting a respondent to

proceed, the examiner should establish that the respondent understands all
instructions, can purposefully answer questions and can record responses on the
answer sheet.

If a respondent has a question about a particular item, it should be taken up

individually with the testing supervisor It is permissible for the examiner to give the
definition of a word but care should be taken to avoid suggesting how to respond.

There is no time limit for completing the BPI, In group sessions, it is often useful for the
examiner to have a second task available so that individuals completing the BPI before others do
not disturb the group by leaving the room or otherwise distracting them.

In the case of respondents who demonstrate low-level reading skills, it may be advisable to
administer the BPI orally in an individual testing session.

To ensure that the test and the identifying information have been completed properly, answer
sheets should be scanned upon return.

Brief Description

The Basic Personality Inventory (BPI) is a 240 item questionnaire that measures personality and
psychopathology. It was designed to yield information that will be helpful for psychologists and
other professionals in a wide variety of contexts. It was developed to achieve the following aims:

1.yield a relative short profile measure of major constructs of psychopathology that would be
useful in psychological screening, assessment and research;
2.meet the challenge of response bias in personality assessment by suppressing the role of
general evaluative dimension in scale construction;

3.make available a questionnaire measure that would be capable of yielding reliable

discrimination among persons showing different patterns of dysfunction within both the normal
and pathological ranges and

4.employ dimensions that refer to relatively homogenous behavioral characteristics that vary
symmetrically in the general population and within deviant groups

III. Scoring 

Scoring the BPI hand-scored answer sheet is an easy clerical task requiring a single template.
Because the BPI is organized so that items keyed on the same scale are arranged into vertical
columns on the answer sheet, all that it is necessary after aligning the template to the orientation
marks in the corners of the answer sheet is tocount in the appropriate columns the number of
items that a respondent has answered in the keyed direction.

1. For example, for the hypochondriasis scale the examiner should count the number of items the
respondent has answered in the False direction in the 1st column and the number of items
answered in the True direction in the 13th column.

2. Similarly, for the Depression scale, the items answered True in the 2nd column and those
answered false in the 14th column should be counted.3.The total is then recorded in the space at
the bottom of the answer sheet labeled with the abbreviation of the scale.4.After scoring all 12
scales, the scores from the answer sheet should be transferred to the corresponding boxes on the
profile sheet.5.Separate norms have been developed for male and female adults and adolescents
which are printed on the alternate sides of the profile sheet.

6. Adolescent norms and profiles should be used between 12 to 18 years of age while adult
norms and profiles should be employed for persons 19 years of age and older. It is important to
use the side of the profile sheet that matches the sex of the respondent.

7.After recording the raw numerical scores on the profile sheet, the scores are plotted in such a
way that it is possible to convert them graphically to standard score form.8.The profiles are so
organized in such a way that the raw scores are transformed into the metric of T scores with a
mean of 50 and a standard deviation of 10.9.It is possible to convert BPI scores to standard
scores without the use of profile by consulting Tables 2-5 and 2-6 for adult males and females,
respectively, and Tables 2-7 and 2-8 for adolescent males and females respectively.

IV. The BPI Profile

There are two sets of BPI profiles, one set for adolescents and the second set for adults.
Different norms are employed for the two sets of profiles, one based on separate male and female
adolescent norms, respectively, printed on alternative sides of the sheet and the second based on
separate adult male and female norms printed on alternate sides of another sheet.

Results for respondents 18 years of age or younger should be plotted on adolescent profiles.

Clinicians and other users should exercise particular care on choosing the correct profile for
plotting a respondent’s results.

The mechanics of plotting scores is relatively simple.

After scoring results and recording total scores on the answer sheet, transfer these totals to the
corresponding spaces on the correct side of the appropriate profile sheet.

Next, locate the raw score level for each scale in the appropriate vertical column corresponding
to the score recorded for that scale.

Place a small “x” in the vertical column at the correct raw score level. When this has been done
for every scale, the “x’s” may be connected by straight lines.

It is then possible to observe which BPI scales are elevated with respect to the general
population and in comparison with the other scales profiled for the respondent.

It is also possible to estimate T scores for the respondent. When BPI scoring and profiling are
completed by computer, the above steps are accomplished automatically.

Even though a particular computer program may undertake interpretation based on certain
statistical rules, the examiner still has the opportunity to review and evaluate the computer-
generated profile in relation to background information, presenting symptoms, and any
situational influences that might have affected assessment.

An alternative to the use of the BPI profile is the use of tables that contain equivalent statistical
information. Tables 2-5 and 2-6 permit conversion of raw scores to standard (T) scores for adult
males and females respectively, and Tables 2-7 and 2-8permit conversion for adolescent males
and females respectively.

V. Pattern of Evaluation

 When interpreting BPI results, it is important to look not only at the results of individual
scales but also at the pattern of elevations.
 There are several common patterns of scale elevation. For example, when the scales that
are related to an individual’s affective state, such as Depression, Social Introversion, and
Self Depreciation, occur together, a more pronounced and well defined problem area is
indicated.
  Individuals whose behavior causes conflict with the law and social convention tend to
have elevated scores on Interpersonal Problems, Alienation, and Impulse Expression.
 When individuals obtain high scores on these three scales, without any elevation on
Anxiety, Depression, or Self Depreciation, it suggests that the individual is fairly
comfortable with the deviant lifestyle and is not likely to request intervention into this
pattern of behavior.
 Adolescents who have high scores on both the mental health scales and the “delinquent”
scales tend to be difficult to treat because of the combination of their personal difficulties
and their unmanageable behavior.
 Results from other populations tested with the BPI have also followed particular patterns
of elevation. These “typical” patterns of elevation are discussed at greater length in the
“ProfileTypes” section.
 A low score on many BPI scales can usually be considered a positive sign. For example,
an individual who has a relatively low score on Impulse Expression may plan activities
carefully and behave under the control of higher cognitive processes.
 An individual with a low score on Denial may be very open about his or her thoughts and
feelings.
 These more positive interpretations should not be overlooked by the focus of the BPI or
the manner in which clients are referred to an agency (e.g., after finding of delinquency).

VI. Interpretation of Scales Individual Scale Considerations

 One of the first steps in interpreting BPI results is to consult the BPI scale descriptions
for the respondent’s high and low scale scores
 While a high score on a particular scale may indicate the presence of certain
characteristics represented by that scale, it is also important to note scales with low scores
because it is possible to interpret these as areas of personal strength and stability.
 Table 2-1 provides descriptions of both high and low scorers for each of the 12 BPI
scales. The following individual scale considerations may be used to supplement the
scale descriptions provided in Table 2-1.
 These “scale considerations” are most appropriately consulted when interpreting BPI
profiles for deviant populations, such as young offenders.
 These considerations are based on the professional opinion of psychologists who have
worked with such populations.
 They also contain practical information that may not be included in the scale descriptions.
 Such details may seem obvious, yet they can be easily overlooked.

Myers-Briggs Type Indicator (MBTI) assessment

Who created the Myers-Briggs Type Indicator® (MBTI®) assessment?

  The MBTI® assessment was developed by Katharine Cook Briggs and Isabel Briggs
Myers. Both were highly educated college graduates who employed the scientific method
in creating the assessment.
 Myers worked with the Educational Testing Service in Princeton, New Jersey, a major
assessment publisher, who helped develop the MBTI assessment and publish it in 1962.
 Since then, the MBTI assessment has been updated regularly based on continuing
research by trained psychologists.

Were Briggs and Myers qualified psychologists?

While neither Myers nor Briggs were psychologists, they based the MBTI assessment on the
work of Carl Jung, Swiss psychiatrist, psychoanalyst, and founder of analytical psychology.

Both Briggs and Myers spent many years studying Jung’s theory of psychological types in order
to create the assessment.

The history of the MBTI assessment spans many years, from its inception by Katharine Briggs,
based on Carl Jung’s theory, development by Isabel Myers until her death, and to the ongoing
development today by teams of psychologists including the research team at The Myers-Briggs
Company (formerly CPP, Inc.).

 1943: The first version of the MBTI assessment is developed

 1962: Educational Testing Service (ETS) publishes an updated form of the MBTI
assessment
 1977: CPP releases Form G, the original commercial version of the MBTI assessment
 1997: OPP Ltd (UK distributor of the MBTI assessment) releases the European English
Step I™ assessment after extensive national data collection¹
 1998: CPP releases Form M of the MBTI Step I™ assessment after extensive national
data collection
 2001: CPP releases Form Q of the MBTI Step II™ assessment
 2003–2007: OPP and CPP research and release new version of the MBTI Step II™
assessment in European English and 8 other European languages
 2018: The Myers-Briggs Company (formerly CPP, Inc.) releases an international
revision, the Global Step I™ and Global Step II™ assessments

The MBTI global assessments more accurately measure personality type across different
countries and cultures and provide a consistent assessment and reporting experience for all
respondents, with no reduction in the accuracy of the resulting type preferences.

3. Who uses the MBTI® assessment?

The MBTI® assessment is most often used by organizational development professionals,

coaches, and consultants, as well as by career counselors and educators.

A fundamental step in any change process is to develop and improve self-awareness.

For the MBTI assessment, this information can then be used to improve interpersonal skills,
manage conflict, improve relationships, and inform career choices.

Researchers in a variety of domains make use of the MBTI assessment and type concepts when
examining normal personality and related attributes.

The MBTI assessment is not used to a great extent by clinical psychologists because it assesses
normal personality, not mental health and disorders.

The MBTI assessment is also used by human resource professionals for a variety of purposes.

However, the MBTI assessment is not intended for use as part of a hiring process, nor to assign
people to specific teams, roles, or functions within an organization.

More about who uses the MBTI assessment

The MBTI® assessment was designed to help people understand personality differences in the
general population.

While there are no “better” or “worse” personality preferences, the MBTI assessment can help
people understand their strengths and blind spots and how they might differ from others.

Organizational experts have drawn on these insights for many years to help individuals and
teams be more effective at work.

It is most often used by organizations to help individuals develop and build self-awareness and to
help teams work better together.

For example, the MBTI assessment can help in conflict resolution, leadership development,
career coaching, team development, managing change, improving communication, and decision
making.

Similarly, other professionals use insights from the MBTI assessment to advise students about
educational decisions, to counsel couples, and to help people in various non-work settings.

Businesses, government agencies, colleges, universities, schools, charities, and sports teams use
the MBTI assessment.

The MBTI assessment should not be used to identify personality “disorders” or mental illness.

Therefore, it is not used in clinical psychology settings or to diagnose conditions such as

depression, narcissism, or anxiety.