Getting to Know Your Data

 Data Objects and Attribute Types

 Basic Statistical Descriptions of Data

 Data Visualization

 Measuring Data Similarity and Dissimilarity

 Summary

1
 Types of Data Sets
 Record
 Relational records
 Data matrix, e.g., numerical matrix,
crosstabs
 Document data: text documents: term-
frequency vector
 Transaction data
 Graph and network
 World Wide Web
 Social or information networks
 Molecular Structures
 Ordered
 Video data: sequence of images
 Temporal data: time-series
 Sequential Data: transaction sequences
 Genetic sequence data
 Spatial, image and multimedia:
 Spatial data: maps
 Image data:
 Video data:
2
 Types of dataset Visualization

Record data is usually stored either in flat files or in relational databases.

Application : Student records, Employee details, etc..
 Transaction data
TID ITEMS
1 Apple , Banana , Milk , Rice
2 Guava , Curd , Rice
3 Apple , Guava , Curd , Rice
4 Banana , Milk
5 Apple , Rice
Application : Online/offline Shopping dataset, patient drug dataset

TID Apple Bana Curd Guava Milk Rice

na
1 True True False False True True
2 False False True True Fals True
e
3 True False True True Fals True
e
4 False True False False True False
 Data Matrix

Example
 Document Term Matrix

Documents :

D1: Text mining is to find useful information from text.

D2: Useful information is mined from the text.
D3: Dark came.
Linked web pages
8
 Benzene Molecule

Example : Movie database

10
Ordered data
Sequential Transaction data
Genomic sequence data
Temperature Time series data
 Data Objects

 Data sets are made up of data objects.

 A data object represents an entity.
 Examples:
 sales database: customers, store items, sales
 medical database: patients, treatments
 university database: students, professors, courses
 Also called samples , examples, instances, data points,
objects, tuples.
 Data objects are described by attributes.
 Database rows -> data objects; columns ->attributes.
14
 Attributes
 Attribute (or dimensions, features, variables): a
data field, representing a characteristic or feature of
a data object.
 E.g., customer _ID, name, address
 Types:
 Nominal

 Binary

 Ordinal

 Numeric: quantitative

 Interval-scaled

 Ratio-scaled

15
 Attribute Types
 Nominal: categories, states, or “names of things”
 Hair_color = {auburn, black, blond, brown, grey, red, white}
 marital status, occupation, ID numbers, zip codes
 Binary
 Nominal attribute with only 2 states (0 and 1)
 Symmetric binary: both outcomes equally important
 e.g., gender
 Asymmetric binary: outcomes not equally important.
 e.g., medical test (positive vs. negative)
 Convention: assign 1 to most important outcome (e.g., HIV
positive)
 Ordinal
 Values have a meaningful order (ranking) but magnitude between
successive values is not known.
 Size = {small, medium, large}, grades, army rankings

16
 Numeric Attribute Types
 Quantity (integer or real-valued)
 Interval
 Measured on a scale of equal-sized units
 Values have order
 E.g., temperature in C˚or F˚, calendar dates
 No true zero-point
 Ratio
 Inherent zero-point
 We can speak of values as being an order of
magnitude larger than the unit of measurement
(10 K˚ is twice as high as 5 K˚).
 e.g., temperature in Kelvin, length, counts,
monetary quantities
17
Features Interval scale Ratio scale
Ratio scale has all the characteristics
All variables measured in an interval
of an interval scale, in addition, to be
Variable scale can be added, subtracted, and
able to calculate ratios. That is, you
property multiplied. You cannot calculate a
can leverage numbers on the scale
ratio between them.
against 0.
Zero-point in an interval scale is
arbitrary. For example, the The ratio scale has an absolute zero
Absolute Point
temperature can be below 0 degrees or character of origin. Height and
Zero
Celsius and into negative weight cannot be zero or below zero.
temperatures.
Statistically, in a ratio scale, the
Statistically, in an interval scale, the
Calculation geometric or harmonic mean is
arithmetic mean is calculated.
calculated.
Interval scale can measure size and Ratio scale can measure size and
Measurement magnitude as multiple factors of a magnitude as a factor of one defined
defined unit. unit in terms of another.

A classic example of an interval scale

is the temperature in Celsius. The Classic examples of a ratio scale are
difference in temperature between any variable that possesses an
Example
50 degrees and 60 degrees is 10 absolute zero characteristic, like age,
degrees; this is the same difference weight, height, or sales figures.
between 70 degrees and 80 degrees. 18
 Discrete vs. Continuous Attributes
 Discrete Attribute
 Has only a finite or countably infinite set of values

 E.g., zip codes, profession, or the set of words in a

collection of documents
 Sometimes, represented as integer variables

 Note: Binary attributes are a special case of discrete

attributes
 Continuous Attribute
 Has real numbers as attribute values

 E.g., temperature, height, or weight

 Practically, real values can only be measured and

represented using a finite number of digits

 Continuous attributes are typically represented as

floating-point variables
19
 Basic Statistical Descriptions of Data
 Motivation
 To better understand the data: central tendency,
variation and spread
 Data dispersion characteristics
 median, max, min, quantiles, outliers, variance, etc.
 Numerical dimensions correspond to sorted intervals
 Data dispersion: analyzed with multiple granularities
of precision
 Boxplot or quantile analysis on sorted intervals
 Dispersion analysis on computed measures
 Folding measures into numerical dimensions
 Boxplot or quantile analysis on the transformed cube
20
 Measuring the Central Tendency
 Mean (algebraic measure) (sample vs. population): 1 n
x   xi   x
Note: n is sample size and N is population size. n i 1 N
n
 Weighted arithmetic mean:
w x i i
 Trimmed mean: chopping extreme values x i 1
n
 Median: w
i 1
i

 Middle value if odd number of values, or average of

the middle two values otherwise
 Mode
 Value that occurs most frequently in the data
 Unimodal, bimodal, trimodal
 Empirical formula:

mean  mode  3  (mean  median)

21
Mean, Median and Mode for Interval Data

So 2 runners took between 51 and

55 seconds, 7 took between 56 and
60 seconds, etc

22
Estimating the Mean from Grouped Data

23
24
Estimating the Median from Grouped Data
The median is the middle value, which in
our case is the 11th one, which is in the 61
- 65 group:
We can say "the median group is 61 - 65"

•L = 60.5
•n = 21

•B = 2 + 7
•G = 8

•w = 5

25
26
 Estimating the Mode from Grouped Data
We can easily find the modal group (the group with the highest frequency),
which is 61 - 65
We can say "the modal group is 61 - 65"
But the actual Mode may not even be in that group! Or there may be more than
one mode. Without the raw data we don't really know.
But, we can estimate the Mode using the following formula:

•L = 60.5
•f
m-1 = 7
•f
m = 8
•f
m+1 = 4
•w = 5

27
28
 Baby Carrots Example
 Example: You grew fifty baby carrots using special soil. You dig them up and measure
their lengths (to the nearest mm) and group the results:

29
Mean

30
Median

31
Mode

32
Symmetric vs. Skewed Data

December 8, 2023 Data Mining: Concepts and Techniques 33

Datset : 4; 5; 6; 6; 6; 7; 7; 7; 7; 8

Answer : skewed to the left

 Dispersion
 Measures of variability describe the spread or
dispersion of a set of data
 Reliability of measure of central tendency
 To compare dispersion of various samples
 Measures of Variability or Dispersion
 Range
 Quantiles
 Quartiles
 Percentiles
 Inter-quartile range
 Variance
 Standard deviation
Quartiles
Quartiles : Example
• Range: the difference between the highest and lowest
values
• Interquartile range: the range of the middle half of a
distribution
• Standard deviation: average distance from the mean
• Variance: average of squared distances from the mean

39
 Range

 The spread of your data from the lowest to the highest

value in the distribution. It’s the easiest measure of
variability to calculate.
 To find the range, simply subtract the lowest value from
the highest value in the data set.

40
 Inter-Quartile Range
 The interquartile range gives you the spread of the middle of your
distribution.
 For any distribution that’s ordered from low to high, the interquartile range
contains half of the values. While the first quartile (Q1) contains the first
25% of values, the fourth quartile (Q4) contains the last 25% of values.

41
42
Standard Deviation

43
Variance

44
 Measuring the Dispersion of Data
 Quartiles, outliers and boxplots
 Quartiles: Q1 (25th percentile), Q3 (75th percentile)
 Inter-quartile range: IQR = Q3 – Q1
 Five number summary: min, Q1, median, Q3, max
 Boxplot: ends of the box are the quartiles; median is marked; add
whiskers, and plot outliers individually
 Outlier: usually, a value higher/lower than 1.5 x IQR
 Variance and standard deviation (sample: s, population: σ)
 Variance: (algebraic, scalable computation)
1 n 1 n 2 1 n 1 n
1 n
s 
2

n  1 i 1
( xi  x ) 
2
[ xi  ( xi ) 2 ]
n  1 i 1 n i 1
 
2

N

i 1
( xi  
2
) 
N
 xi   2
i 1
2

 Standard deviation s (or σ) is the square root of variance s2 (or σ2)

45
 Boxplot Analysis
 Five-number summary of a distribution
 Minimum, Q1, Median, Q3, Maximum
 Boxplot
 Data is represented with a box
 The ends of the box are at the first and third
quartiles, i.e., the height of the box is IQR
 The median is marked by a line within the
box
 Whiskers: two lines outside the box extended
to Minimum and Maximum
 Outliers: points beyond a specified outlier
threshold, plotted individually

46
 Inclassexercise#

a) Find the mean price in each city and then state which
city has the lower mean
b) Find the standard deviation of each city's prices.
c) Which city has the more consistently priced petrol? Give
reasons for your answer
Solution
 Example 1: If a die is rolled, then find the variance and
standard deviation of the possibilities.

 When a die is rolled, the possible number of outcomes is 6.

So the sample space, n = 6 and the data set = { 1;2;3;4;5;6}.
 To find the variance, first, we need to calculate the mean of
the data set.
 Mean, x̅ = (1+2+3+4+5+6)/6 = 3.5
 We can put the value of data and mean in the formula to get;
 σ2 = Σ (xi – x̅ )2/n
 σ2 = ⅙ (6.25+2.25+0.25+0.25+2.25+6.25)
 σ2 = 2.917
 Answer: Therefore the variance is σ2 = 2.917,
and standard deviation,σ = √2.917 = 1.708
Example 1: If a die is rolled, then find the variance and
standard deviation of the possibilities.

Example 2: Find the standard deviation of the average

temperatures recorded over a five-day period last winter:
18, 22, 19, 25, 12 (The mean = 19.2)

50
 Example 2: Find the standard deviation of the average
temperatures recorded over a five-day period last winter:
18, 22, 19, 25, 12 (The mean = 19.2)

 To find the variance, we divide 5-1 = 4

 94.8/4 = 23.7
 Finally, we find the square root of this variance. √23.7 = 4.9
 So the standard deviation for the temperatures recorded is
4.9; the variance is 23.7
 Finally, we find the square root of this variance. √23.7 ≈ 4.9
 Answer: So the standard deviation for the temperatures
recorded is 4.9; the variance is 23.7.
 Visualization of Data Dispersion: 3-D Boxplots

December 8, 2023 Data Mining: Concepts and Techniques 52

Graphic Displays of Basic Statistical Descriptions

 Boxplot: graphic display of five-number summary

 Histogram: x-axis are values, y-axis repres. frequencies
 Quantile plot: each value xi is paired with fi indicating
that approximately 100 fi % of data are  xi
 Quantile-quantile (q-q) plot: graphs the quantiles of
one univariant distribution against the corresponding
quantiles of another
 Scatter plot: each pair of values is a pair of coordinates
and plotted as points in the plane
53
 Examples Boxplot
 Data 1:
10,12,11,15,11,14,13,17,12,22,14,11
 Data 2:

22,25,17,19,33,64,23,17,20,18

54
 Histogram Analysis
 Histogram: Graph display of
tabulated frequencies, shown as 40
bars 35
 It shows what proportion of cases 30
fall into each of several categories
25
 Differs from a bar chart in that it is
20
the area of the bar that denotes the
value, not the height as in bar 15
charts, a crucial distinction when the 10
categories are not of uniform width
5
 The categories are usually specified
0
as non-overlapping intervals of 10000 30000 50000 70000 90000
some variable. The categories (bars)
must be adjacent

55
Histograms Often Tell More than Boxplots

 The two histograms

shown in the left may
have the same boxplot
representation
 The same values
for: min, Q1,
median, Q3, max
 But they have rather
different data
distributions

56
 Quantile Plot
 Displays all of the data (allowing the user to assess both
the overall behavior and unusual occurrences)
 Plots quantile information
 For a data x data sorted in increasing order, f
i i
indicates that approximately 100 fi% of the data are
below or equal to the value xi

Data Mining: Concepts and Techniques 57

 Quantile-Quantile (Q-Q) Plot
 Graphs the quantiles of one univariate distribution against the
corresponding quantiles of another
 View: Is there is a shift in going from one distribution to another?
 Example shows unit price of items sold at Branch 1 vs. Branch 2 for
each quantile. Unit prices of items sold at Branch 1 tend to be lower
than those at Branch 2.

58
 Scatter plot
 Provides a first look at bivariate data to see clusters of
points, outliers, etc
 Each pair of values is treated as a pair of coordinates and
plotted as points in the plane

59
Positively and Negatively Correlated Data

 The left half fragment is positively

correlated
 The right half is negative correlated

60
Uncorrelated Data

61
 Data Visualization
 Why data visualization?
 Gain insight into an information space by mapping data onto graphical
primitives
 Provide qualitative overview of large data sets
 Search for patterns, trends, structure, irregularities, relationships among
data
 Help find interesting regions and suitable parameters for further
quantitative analysis
 Provide a visual proof of computer representations derived
 Categorization of visualization methods:
 Pixel-oriented visualization techniques
 Geometric projection visualization techniques
 Icon-based visualization techniques
 Hierarchical visualization techniques
 Visualizing complex data and relations
62
 Similarity and Dissimilarity
 Similarity
 Numerical measure of how alike two data objects are

 Value is higher when objects are more alike

 Often falls in the range [0,1]

 Dissimilarity (e.g., distance)

 Numerical measure of how different two data objects

are
 Lower when objects are more alike

 Minimum dissimilarity is often 0

 Upper limit varies

 Proximity refers to a similarity or dissimilarity

63
 Data Matrix and Dissimilarity Matrix
 Data matrix
 n data points with p  x11 ... x1f ... x1p 
 
dimensions  ... ... ... ... ... 
 Two modes x ... xif ... xip 
 i1 
 ... ... ... ... ... 
x ... xnf ... xnp 
 n1 
 Dissimilarity matrix
 n data points, but
 0 
 d(2,1) 0 
registers only the  
 d(3,1) d ( 3,2) 0 
distance  
 A triangular matrix  : : : 
d ( n,1) d ( n,2) ... ... 0
 Single mode

64
 Suppose that we have n objects (e.g., persons,
items, or courses) described by p attributes (also
called measurements or features, such as age,
height, weight, or gender).

 The objects are x1 =(x11,x12, : : : ,x1p),

x2 = (x21,x22, : : : ,x2p) , and so on,
where xij is the value for object xi of the jth
attribute.

65
 Data Matrix and Dissimilarity Matrix
 Data matrix (object by attribute structure or n by p
matrix)
 n data points with p dimensions

 Two-mode

 x11 ... x1f ... x1p 

 
 ... ... ... ... ... 
x ... xif ... xip 
 i1 
 ... ... ... ... ... 
x ... xnf ... xnp 
 n1 

66
 Dissimilarity matrix (object by object structure)
 n data points, but registers only the distance

 A triangular matrix

 Single-mode

 0 
 d(2,1) 0 
 
 d(3,1) d ( 3,2) 0 
 
 : : : 
d ( n,1) d ( n,2) ... ... 0

67
 Proximity Measure for Binary Attributes
Object j
 A contingency table for binary data
Object i

 Distance measure for symmetric

binary variables:
 Distance measure for asymmetric
binary variables:
 Jaccard coefficient (similarity
measure for asymmetric binary
variables): 1 – d (i, j)
 Note: Jaccard coefficient is the same as “coherence”:

68
 Dissimilarity between Binary Variables
 Example
Name Gender Fever Cough Test-1 Test-2 Test-3 Test-4
Jack M Y N P N N N
Mary F Y N P N P N
Jim M Y P N N N N

 Gender is a symmetric attribute

 The remaining attributes are asymmetric binary
 Let the values Y and P be 1, and the value N 0

69
 Distance measure for asymmetric attributes

Fever Cough Test Test Test Test 1 0

I II III IV 1 2 0 2
Jack 1 0 1 0 0 0 0 1 3 4
Merry 1 0 1 0 1 0 3 3 6

Contingency table

01
d ( jack , mary )   0.33
2 01
11
d ( jack , jim )   0.67
111
1 2
d ( jim , mary )   0.75
11 2
70
 Proximity Measure for Nominal Attributes

 Can take 2 or more states, e.g., red, yellow, blue,

green (generalization of a binary attribute)
 Method 1: Simple matching
 m: # of matches, p: total # of variables
d (i, j)  p 
p
m

 Method 2: Use a large number of binary attributes

 creating a new binary attribute for each of the
M nominal states

71
Example I

1-0/1 = 1

72
 Example II
RollNo Marks Grade

1 90 A

2 80 B

3 82 B

4 90 A

d(RollNo1,RollNo1) d(RollNo1,RollNo2) d(RollNo1,RollNo3) d(RollNo1,RollNo4)

d(RollNo2,RollNo1) d(RollNo2,RollNo2) d(RollNo2,RollNo3) d(RollNo2,RollNo4)

d(RollNo3,RollNo1) d(RollNo3,RollNo2) d(RollNo3,RollNo3) d(RollNo3,RollNo4)

d(RollNo4,RollNo1) d(RollNo4,RollNo2) d(RollNo4,RollNo3) d(RollNo4,RollNo4)

distance(object1, Object2) = P – M / P
P is total number of attributes
M is total number of matches
73
 Distance measure

d(1,1)
=P–M/P
d(RollNo1,RollNo2) d(RollNo1,RollNo3) d(RollNo1,RollNo4)
=2–2/2
=0

d(2,1)
d(2,2) = P – M / P
=P–M/P
= (2 – 2) / 2 d(RollNo2,RollNo3) d(RollNo2,RollNo4)
= (2 – 0) / 2
=0
=1
d(3,1)
d(3,2) = P – M / P d(3,3) = P – M / P
=P–M/P
= (2 – 1 )/ 2 = (2 – 2 )/ 2 d(RollNo3,RollNo4)
= (2 – 0) / 2
= 0.5 =0
=1
d(4,1)
d(4,2) = P – M / P d(4,3) = P – M / P d(4,4) = P – M / P
=P–M/P
= (2 – 0) / 2 =( 2 – 0 )/ 2 =( 2 – 2) / 2
= (2 – 2) / 2
=1 =1 =0
=0

74
 Standardizing Numeric Data
x
z   
 Z-score:
 X: raw score to be standardized, μ: mean of the population, σ:
standard deviation
 the distance between the raw score and the population mean in
units of the standard deviation
 negative when the raw score is below the mean, “+” when above
 An alternative way: Calculate the mean absolute deviation
s f  1n (| x1 f  m f |  | x2 f  m f | ... | xnf  m f |)
where m  1 (x  x  ...  x )
n 1f 2 f xif  m f
.
f nf

zif  sf
 standardized measure (z-score):
 Using mean absolute deviation is more robust than using standard
deviation

75
 Example:
Data Matrix and Dissimilarity Matrix
Data Matrix
x2 x4
point attribute1 attribute2
4 x1 1 2
x2 3 5
x3 2 0
x4 4 5
2 x1
Dissimilarity Matrix
(with Euclidean Distance)
x3
0 4 x1 x2 x3 x4
2
x1 0
x2 3.61 0
x3 5.1 5.1 0
x4 4.24 1 5.39 0

76
Distance on Numeric Data: Minkowski Distance
 Minkowski distance: A popular distance measure

where i = (xi1, xi2, …, xip) and j = (xj1, xj2, …, xjp) are two p-
dimensional data objects, and h is the order (the
distance so defined is also called L- h norm)
 Properties
 d(i, j) > 0 if i ≠ j, and d(i, i) = 0 (Positive definiteness)
 d(i, j) = d(j, i) (Symmetry)
 d(i, j)  d(i, k) + d(k, j) (Triangle Inequality)
 A distance that satisfies these properties is a metric
77
 Special Cases of Minkowski Distance
 h = 1: Manhattan (city block, L1 norm) distance
 E.g., the Hamming distance: the number of bits that are
different between two binary vectors
d (i, j) | x  x |  | x  x | ... | x  x |
i1 j1 i2 j 2 ip jp

 h = 2: (L2 norm) Euclidean distance

d (i, j)  (| x  x |2  | x  x |2 ... | x  x |2 )
i1 j1 i2 j 2 ip jp

 h  . “supremum” (Lmax norm, L norm) distance.

 Attribute f that gives the maximum difference in values
between any component (attribute) of the vectors (objects)

78
Example

f1 difference |1-3| = 2
f2 difference |2-5| = 3
Select maximum difference i.e. 3
79
 Example: Minkowski Distance
Dissimilarity Matrices
point attribute 1 attribute 2
x1 1 2
x2 3 5
x3 2 0
x4 4 5 Manhattan (L1)

Euclidean (L2)
x2 x4

4 Supremum

2 x1

x3
0 2 4
80
 Example: Minkowski Distance
Dissimilarity Matrices
point attribute 1 attribute 2 Manhattan (L1)
x1 1 2
L x1 x2 x3 x4
x2 3 5 x1 0
x3 2 0 x2 5 0
x4 4 5 x3 3 6 0
x4 6 1 7 0
Euclidean (L2)
x2 x4
L2 x1 x2 x3 x4
4 x1 0
x2 3.61 0
x3 2.24 5.1 0
x4 4.24 1 5.39 0

2 x1
Supremum
L x1 x2 x3 x4
x1 0
x2 3 0
x3 x3 2 5 0
0 2 4 x4 3 1 5 0
81
 Example

Given two objects represented by the tuples (22, 1,

42, 10) and (20, 0, 36, 8):
1. Compute the Euclidean distance between the
two objects.
2. Compute the Manhattan distance between the
two objects.
3. Compute the Minkowski distance between the
two objects, using h = 3.
4. Compute the supremum distance between the
two objects.

82
83
 Ordinal Variables

 An ordinal variable can be discrete or continuous

 Order is important, e.g., rank
 Can be treated like interval-scaled
 replace x by their rank rif {1,..., M f }
if

 map the range of each variable onto [0, 1] by replacing

i-th object in the f-th variable by
rif 1
zif 
M f 1

 compute the dissimilarity using methods for interval-

scaled variables

84
Example

85
 Dissimilarity measure for ordinal data
 There are three states for test-2: fair, good, and
excellent, that is, Mf = 3.
 step 1 - Replace each value for test-2 by its rank,
four objects are assigned the ranks 3, 1, 2, and
3, respectively.
 Step 2 - Normalizes the ranking by mapping rank
1 to 0.0, rank 2 to 0.5, and rank 3 to 1.0.
 step 3 – Use the Euclidean distance

86
Example

87
 Cosine Similarity
 A document can be represented by thousands of attributes, each
recording the frequency of a particular word (such as keywords) or
phrase in the document.

 Other vector objects: gene features in micro-arrays, …

 Applications: information retrieval, biologic taxonomy, gene feature
mapping, ...
 Cosine measure: If d1 and d2 are two vectors (e.g., term-frequency
vectors), then
cos(d1, d2) = (d1  d2) /||d1|| ||d2|| ,
where  indicates vector dot product, ||d||: the length of vector d
88
 Example: Cosine Similarity
 cos(d1, d2) = (d1  d2) /||d1|| ||d2|| ,
where  indicates vector dot product, ||d|: the length of vector d

 Ex: Find the similarity between documents 1 and 2.

d1 = (5, 0, 3, 0, 2, 0, 0, 2, 0, 0)
d2 = (3, 0, 2, 0, 1, 1, 0, 1, 0, 1)

d1d2 = 5*3+0*0+3*2+0*0+2*1+0*1+0*1+2*1+0*0+0*1 = 25
||d1||= (5*5+0*0+3*3+0*0+2*2+0*0+0*0+2*2+0*0+0*0)0.5=(42)0.5
= 6.481
||d2||= (3*3+0*0+2*2+0*0+1*1+1*1+0*0+1*1+0*0+1*1)0.5=(17)0.5
= 4.12
cos(d1, d2 ) = 0.94
89
 Attributes of Mixed Type
 A database may contain all attribute types
 Nominal, symmetric binary, asymmetric binary,

numeric, ordinal
 One may use a weighted formula to combine their effects
 pf  1 ij( f ) dij( f )
d (i, j) 
 pf  1 ij( f )
 if either
 xif or xjf missing (i.e., there is no measurement of attribute f for
object i or object j), or
 xif = xjf = 0 and attribute f is asymmetric binary;
 otherwise,
90
 Attributes of Mixed Type
 f is binary or nominal:
dij(f) = 0 if xif = xjf
dij(f) = 1 otherwise
 f is numeric: use the normalized distance

Where h runs over all non missing objects for the attribute f.
 f is ordinal
 Compute ranks rif and r 1
zif 
if
 Treat zif as interval-scaled M 1 f

91
Example

92
 Dissimilarity Matrix for test I
 di,j = p – m / p ( simple matching)
 P=1

93
 Dissimilarity Matrix for test II
zif  r  1
if

M 1 f

94
 Dissimilarity Matrix for test III

95
 Solution
 Dissimilarity Matrix

96
Example

97
Solution (a)

98
Solution (b)

99
 References
 W. Cleveland, Visualizing Data, Hobart Press, 1993
 T. Dasu and T. Johnson. Exploratory Data Mining and Data Cleaning. John Wiley, 2003
 U. Fayyad, G. Grinstein, and A. Wierse. Information Visualization in Data Mining and
Knowledge Discovery, Morgan Kaufmann, 2001
 L. Kaufman and P. J. Rousseeuw. Finding Groups in Data: an Introduction to Cluster
Analysis. John Wiley & Sons, 1990.
 H. V. Jagadish, et al., Special Issue on Data Reduction Techniques. Bulletin of the Tech.
Committee on Data Eng., 20(4), Dec. 1997
 D. A. Keim. Information visualization and visual data mining, IEEE trans. on Visualization
and Computer Graphics, 8(1), 2002
 D. Pyle. Data Preparation for Data Mining. Morgan Kaufmann, 1999
 S. Santini and R. Jain,” Similarity measures”, IEEE Trans. on Pattern Analysis and
Machine Intelligence, 21(9), 1999
 E. R. Tufte. The Visual Display of Quantitative Information, 2nd ed., Graphics Press,
2001
 C. Yu , et al., Visual data mining of multimedia data for social and behavioral studies,
Information Visualization, 8(1), 2009
100