Reliability analysis based on the Salford gait tool

Jesus David Franco Gomez Lizeth Fernanda Correa Acevedo Santiago Victoria Marin
Engineering department Engineering department Engineering department
Universidad Autónoma de occidente Universidad Autónoma de occidente Universidad Autónoma de occidente
Cali, Colombia Cali, Colombia Cali, Colombia
[email protected] [email protected] [email protected]

ABSTRACT- - Select the test patient.

ONE OF THE MOST IMPORTANT ASPECTS TO CONSIDER IN A GAIT - Place fluorescent markers anatomically on the
ANALYSIS IS THE RELIABILITY, NOT ONLY OF THE EVALUATOR BUT ALSO
patient's greater trochanter, lateral femoral condyle,
OF THE TOOL THAT WILL BE USED FOR THE ANALYSIS. BASED ON THIS,
lateral aspect of the calcaneus, and at the junction
IN THIS REPORT A STATISTICAL ANALYSIS WAS CARRIED OUT ACCORDING
TO THE DATA OBTAINED FROM THE GAIT OF THE SAME SUBJECT IN 3 between the fifth metatarsal and the first phalanx of
DIFFERENT VIDEOS. THIS WAS DONE WITH THE AIM OF OBTAINING A the fifth toe.
CORRELATION VALUE AND A STANDARD ERROR OF MEASUREMENT - Place the tripod at a minimum distance of 5 meters
BETWEEN THE DATA COLLECTED. IN THIS SENSE, AN ADEQUATE from the patient.
RELIABILITY VALUE AND A MINIMUM STANDARD ERROR OF - Record the video using a camera with a minimum
MEASUREMENT ALLOW FOR A CORRECT DIAGNOSIS FOR THE PATIENT
definition of 30 fps.
BASED ON THE SALFORD CLINICAL GAIT VALIDATION TOOL.
- Calibrate the Kinovea analysis software with the
KEYWORDS- STANDARD ERROR OF MEASUREMENT, CORRELATION,
camera definition and the patient's reference (height)
RELIABILITY, ACCURACY RATE, SALFORD TOOL. for the corresponding analysis.

For this practice, three video captures were made to perform

I. INTRODUCTION multiple measurements. In the first part, the intra-observer
test-retest reliability was determined, in which each member
There are numerous analyses that can be performed in of the team measured the same three videos of a patient at
studies of human gait, such as obtaining diagnoses based on two different times, with a waiting period of one day
gait phases, trends that the patient may exhibit, among between measurements. It should be noted that these
others. However, to perform a correct analysis, it is of evaluations were carried out focusing only on the initial
utmost importance to know the reliability of the data support phase of gait. This intra-observer test was conducted
collected in the laboratory. Therefore, prior to making to estimate the reliability of the data obtained by each
diagnoses based on the results obtained from gait analysis, a member of the team. This test determines how much error in
statistical study of the data collected, both from the observer the test results is due to systematic errors such as incorrect
and the tool used, must be conducted. This requires placement of virtual markers in the software.
estimating the correlation coefficient and standard error of
measurement based on a study of the gait data in several After the two measurements were taken with the
videos of the same patient, as well as in a specific video corresponding time interval, the statistical calculation of
with several measurements, to determine how much error reliability was performed using the Pearson correlation
there is in the data obtained due to systematic errors of the coefficient (denoted by the letter R) with the average of the
equipment used [1]. Subsequently, an intra-observer data obtained from the angles of the hip, knee, and ankle.
reliability analysis was performed to determine the
correlation of the data obtained according to the six gait 𝑛
∑ (𝑋𝑖−𝑋)(𝑌𝑖−𝑌)
phases suggested by the Salford clinical gait validation tool 𝑅= 𝑖=1
(1)
[2]. 𝑛 2 𝑛 2
∑ (𝑋𝑖−𝑋) ∑ (𝑌𝑖−𝑌)
𝑖=1 𝑖=1

The intra-observer and inter-observer tests will allow

determining the reliability of the data obtained by each This coefficient is a measure of linear association between
evaluator, as well as the validity of the tool used for the two variables [3], in this case each variable corresponds to
analysis. All of this is necessary to make proper use of the the mean of each measurement taken. The value of this
tool and to be able to make a relevant diagnosis according to coefficient will be a value between -1 and 1; where -1
the analyzed gait phases. indicates that there is a perfectly negative linear correlation
between the variables, if the value is close to or equal to 0 it
would indicate that there is no linear correlation between the
two variables, so the data is very scattered and has a weak
II. METHODOLOGY
relationship between them; the opposite case would be if the
coefficient has a value close to or equal to 1, since this value
To carry out the statistical analysis to determine the would indicate that the data is strongly related to each other
reliability and absolute error of measurement, it is and will have low dispersion, so the variables will have a
necessary to prepare and record the patient's normal gait. perfectly positive linear correlation [3]. In Figure 1, the
To do this, the following steps must be taken: behavior of the data can be observed according to a scatter
plot in each case of the value of the Pearson correlation It should be noted that to determine the angles properly and
coefficient. use Salford's rating system, according to Hamill et al.
(2015), the absolute angles of the trunk, thigh, leg, and foot
from the patient's reference must be determined, and
subsequently the corresponding angles of the hip, knee, and
ankle must be determined [4]. Next, the formulas used to
determine the angles of the hip (3), knee (4), and ankle (5)
from the absolute angles indicated by Hamill et al. (2015)
Figure 1. Scatter plots in relation to Pearson correlation values. Taken
from: Máxima Formación. will be shown.

Based on the correlation coefficient found, the standard

error of measurement is determined through 3
measurements on the same video of the same test subject. It
should be noted that the angle measurements were taken
during the initial contact phase. Once the measurements
were taken, the standard error of measurement was
calculated based on the standard deviation (s) of the three
measurements, taking into account the angles of the hip,
knee, and ankle. Additionally, the correlation coefficient (R)
previously found by each member of the team was used.
The formula for calculating the standard error of
measurement is as follows.

𝑆𝐸𝑚 = 𝑠 1 − 𝑅 (2)

Once the intra-observer test-retest reliability was completed,

the reliability of the Salford rating tool [2] was determined
through inter-observer tests. For this purpose, a new video
was taken where all the measurements were performed in
the six phases of gait indicated by Salford. The phases used
were: Initial contact, end of double support, midstance,
initial swing, initial swing, and midswing. These phases can
be observed in Figure 2.

Figure 4. Subphases of the gait cycle for analysis (A) Initial contact
(B) End of double stance (C) Mid stance (D) Start of double
Figure 2. Subphases of gait as seen from the sagittal plane. Taken from stance (E) Toe-off of the left leg (initial swing) (F) Mid swing.
Toro et al. (2007).

It should be noted that to determine the angles properly and Based on the determined angles of the hip, knee, and ankle,
use Salford's rating system, according to Hamill et al. each group member assigned a rating based on the results
(2015), the absolute angles of the trunk, thigh, leg, and foot obtained, using the Salford rating system [2] shown in table
from the patient's reference must be determined, and 1.
subsequently the corresponding angles of the hip, knee, and
ankle must be determined [4]. Next, the formulas used to Table 1. Equivalency of grades earned in the Salford grading system. Taken
determine the angles of the hip (3), knee (4), and ankle (5) from Toro et al. (2007
from the absolute angles indicated by Hamill et al. (2015)
will be shown.

Øℎ𝑖𝑝 = Ø𝑡ℎ𝑖𝑔ℎ − Ø𝑡𝑟𝑢𝑛𝑘 𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 (3)

Ø𝑘𝑛𝑒𝑒 = Ø𝑡ℎ𝑖𝑔ℎ 𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 − Ø 𝑙𝑒𝑔 𝑎𝑏𝑠𝑜𝑙𝑢𝑡𝑒 (4)

Ø𝑎𝑛𝑘𝑙𝑒 𝑗𝑜𝑖𝑛𝑡 𝑎𝑛𝑔𝑙𝑒 = Ø𝑙𝑒𝑔 − Ø 𝑓𝑜𝑜𝑡 + 90° (5)

 Table 4. Results of intra-observer test-retest reliability for Santiago.

Based on the averages obtained by each member, the

Pearson correlation coefficient was determined using
formula (1), the results of which can be found in Table 5.

Table 5. Results of the Pearson correlation coefficient of all members.

Once the reliability of the measurements taken by each

Figure 5. Assignment of labels after the sum of categories. Taken from Toro member was determined, three measurements were taken
et al. (2007) again in the same patient video, and the average, standard
deviation, and standard error of measurement were
Based on the assigned rating, the reliability of category calculated for each measurement. Tables 6, 7, and 8 show
assignment in the six evaluated stages was determined by the results obtained by each participant.
calculating the percentage of correct assignments for each
joint category assigned by the three team members. Table 6. Results of the intra-observer test-retest of the three measurements
in the same video of Jesus.
Following this, a pair of tables were constructed indicating
the sum of the obtained results and the assigned labels
(based on Figure 5) by each team member using the Salford
tool. Based on the percentage of label accuracy, the
reliability of the use of the Salford tool by the group was
also determined.

III. RESULTS
As mentioned in the methodology, the first part of the
Table 7. Results of the intra-observer test-retest of the three measurements
practice involved the analysis of intra-observer test-retest in the same video of Lizeth.
reliability to determine reliability according to each
evaluator. Once the initial contact phase of gait was
analyzed in the three videos at two different times, a table
was obtained with the integrated results of the angles of the
hip, knee, and ankle, along with their average, to
subsequently determine the correlation coefficient of the
two measurements made.

Table 2.Results of intra-observer test-retest reliability for Jesus.

Table 8. Results of the intra-observer test-retest of the three measurements

in the same video of Santiago.

Table 3.Results of intra-observer test-retest reliability for Lizeth.

Based on the results obtained by each member, the standard Table 11. Results of the assigned knee categories in the six moments of
walking with their percentage of success.
error of measurement was calculated using formula (2), and
Table 9 shows the values obtained by each person
integrated.

Table 9. Results of the standard error of measurement of all members.

After conducting the intra-observer tests, each team member

carried out measurements of the angles according to the six
gait phases indicated by the Salford rating tool. Table 11. Results of the assigned ankle categories in the six moments of
walking with their percentage of success.
Subsequently, the percentage of accuracy was determined
based on the ratings obtained and the labels suggested by
the tool. The integrated results of each participant to obtain
the percentage of accuracy based on the rating obtained in
each phase can be seen in Table 10.

Table 10. Integrated results of each member according to the qualification

system of the Salford tool.

After this, the sum of all categories was obtained based on

the ratings made by each member. Tables 14 and 15 show
Based on the rating obtained in each of the six phases the results of the sum of categories and the labels assigned
analyzed by each participant, the reliability of each category based on figure 5, respectively.
corresponding to the hip, knee, and ankle was calculated
using the percentage of accuracy according to the category Table 14. Sum of the assigned categories corresponding to the hip, knee
given by each participant. Table 11, 12, and 13 show the and ankle.
assigned rating according to the corresponding joint.

Table 11. Results of the assigned hip categories in the six walking moments
with their percentage of success.

Table 15. Labels assigned based on the sum of the hip, knee and ankle
categories.
 IV. DISCUSSION Finally, the categories obtained by each evaluator were
organized in a better way in table 14 according to the total
As requested in the guide, each evaluator made three sum of the qualification made, where it is observed that for
measurements on the first phase of gait with three different hip and knee an equal value of "-1" was obtained for hip and
videos, and after two days these videos were measured "7" for knee except for ankle where Lizeth and Jesús
again, i.e., in total each evaluator made six measurements, obtained a qualification of "5" and Santiago of "6". These
obtaining the results of the tables (2, 3 and 4) in which values were related according to the labels indicated by the
appear the angles obtained for each joint based on each Salford tool (figure 5) in table (15), so the possible gait
analyzed video and if this refers to measurement 1 or disturbances would be obtained following the Salford
measurement 2, with this the Pearson correlation coefficient instructions and followed by the previously described
is obtained (table 5), this value is important to know the qualification, which gives a percentage of success of 66%,
reliability of the data obtained by each evaluator, where it is where the three evaluators have the same result for a slight
obtained that the evaluator Lizeth has the highest reliability hip extension and a slight knee flexion by the test subject;
in their measurements with a value of 0. 9997, followed by the mentioned value is the percentage of the reliability of
Santiago 0.9986 and finally Jesus with a value of 0.9978. the use of the Salford tool by the working group.
Subsequently, 3 measurements were made to the same gait
video of the patient, where from the data obtained the V. CONCLUSIONS
standard deviation was obtained for each measurement
made by the three evaluators (whose values can be seen in Based on the results obtained in the analysis of Saldorf's
tables 6, 7 and 8) and based on this it was obtained that the gait, it can be concluded that the evaluation carried out by
evaluator with the least dispersion of the data has Lizeth the three evaluators showed high reliability in their sensors,
with a standard deviation of 0.8238 followed by Santiago indicating that the technique used is reliable and can be used
with 1.00 and then Jesus with 1.88. From the reliability and to evaluate gait in different subjects. It is noteworthy that
the standard deviations obtained by each evaluator, the evaluator Lizeth obtained the highest reliability in her
standard error of measurement was determined (Table 9), measurements and was also the participant with the least
where again Lizeth with a value of 0.0046 has the smallest data dispersion, suggesting that her measurements were
error in contrast to Jesus who obtained a value of 1.8895 more precise and consistent compared to the other
and Santiago with 0.0377. evaluators. This result is important as it suggests that
evaluators should be especially careful when taking
Subsequently, the six stages of gait were analyzed in order measurements to minimize data scarcity.
to use the Salford tool and give a score to all the angles
obtained (table 10) to carry out the inter-observer reliability Regarding the results obtained in the analysis of the six gait
tests, as can be seen among the three evaluators, the phases, it was demonstrated that the variation in results
variation of the assigned scores is low and this is verified in between evaluators was low. Overall, a high rating was
the subsequent results (seen in tables 11, 12 and 13) since obtained in the hip and knee joints, except for the ankle
almost the same values were obtained according to the where there was a discrepancy in the rating obtained by
angles of each joint in the analyzed stages. Based on this, Santiago compared to the other evaluators. This suggests
the percentage of acceptance of each joint in the six stages that there may be a lack of agreement in how evaluators are
of gait was determined, whose value represents the evaluating the ankle joint and further analysis may be
reliability of the assignment of the categories in the necessary.
moments of gait evaluated in the patient, where a percentage
of 100% was obtained for the hip and knee, except for the Finally, the categorization obtained by each evaluator
ankle where a value of 50% was obtained and where showed similar values in the rating of hip and knee joints,
Santiago's qualification had a different value from that of except for the ankle. Regarding possible gait disturbances, it
the other evaluators in the stages middle position, start of was found that evaluators obtained the same result for a
support and toe of, this most probably due to errors slight extension of the hip and a slight flexion of the knee by
associated with the poor location of the markers since it is the test subject. Overall, it can be concluded that the Salford
very difficult to visualize and place the angles in the gait analysis technique is a useful and reliable tool for
Kinovea software. evaluating gait in different subjects and detecting possible
disturbances.

In summary, the results of this study suggest that the Salford

gait evaluation technique is a useful and reliable tool for
evaluating gait in different subjects and detecting possible
disturbances. However, the need for further training and
care when taking measurements should be considered to
minimize data scarcity and improve measurement precision.

VI. REFERENCES
[1] ¿Qué es el análisis de confiabilidad? (Definición y
ejemplo). (s.f.). Statologos: El sitio web para que aprendas
estadística en Stata, R y Phyton.
https://statologos.com/analisis-de-fiabilidad/

[2] Toro B., Nester C. J. & Farren P. C. (2007). The

Development and Validity of the Salford Gait Tool: An
Observation-Based Clinical Gait Assessment Tool.
Archives of Physical Medicine and Rehabilitation, 88(3),
321-327. https://doi.org/10.1016/j.apmr.2006.12.028

[3] Coeficiente de correlación de Pearson. (s.f.).

Statologos: El sitio web para que aprendas estadística en
Stata, R y Phyton.
https://statologos.com/coeficiente-de-correlacion-de-pears
on/

[4] J. Hamill, K.M. KNutzen, T. R. Derrick.

Biomechanical basis of human movement.
Fourth Edition. Wolters Kluwer Health. Philadelphia, US.
2015.