Chapter 14 – Association

Rules and Collaborative

Filtering
What are Association Rules?
Study of “what goes with what”
 “Customers who bought X also bought Y”
 What symptoms go with what diagnosis
Transaction-based or event-based
Also called “market basket analysis” and
“affinity analysis”
Originated with study of customer
transactions databases to determine
associations among items purchased
Used in many recommender
systems
Generating Rules
Terms

“IF” part = antecedent

“THEN” part = consequent

“Item set” = the items (e.g., products)

comprising the antecedent or consequent

Antecedent and consequent are disjoint

(i.e., have no items in common)
Tiny Example: Phone
Faceplates
Many Rules are Possible

For example: Transaction 1 supports several

rules, such as
“If red, then white” (“If a red faceplate is
purchased, then so is a white one”)
“If white, then red”
“If red and white, then green”
+ several more
Frequent Item Sets
Ideally, we want to create all possible
combinations of items

Problem: computation time grows

exponentially as # items increases

Solution: consider only “frequent item sets”

Criterion for frequent: support

Support

Support for an itemset = # (or percent) of

transactions that include an itemset
Example: support for the item set {red,
white} is 4 out of 10 transactions, or 40%

Support for a rule = # (or percent) of

transactions that include both the
antecedent and the consequent
Apriori Algorithm
Generating Frequent Item
Sets
For k products…
1. User sets a minimum support criterion
2. Next, generate list of one-item sets that
meet the support criterion
3. Use the list of one-item sets to generate
list of two-item sets that meet the support
criterion
4. Use list of two-item sets to generate list of
three-item sets
5. Continue up through k-item sets
Measures of Rule
Performance
Confidence: the % of antecedent
transactions that also have the consequent
item set

Confidence is conditional probability.

Measures of Rule
Performance
Lift = confidence/(benchmark confidence)

Lift > 1 indicates a rule that is useful in

finding consequent items sets (i.e., more
useful than just selecting transactions
randomly)
Alternate Data Format: Binary
Matrix
Process of Rule Selection
Generate all rules that meet specified
support & confidence
Find frequent item sets (those with sufficient
support – see above)
From these item sets, generate rules with
sufficient confidence
Example: Rules from {red, white,
green}

{red, white} > {green} with confidence = 2/4 = 50%

[(support {red, white, green})/(support {red, white})]

{red, green} > {white} with confidence = 2/2 = 100%

[(support {red, white, green})/(support {red, green})]

Plus 4 more with confidence of 100%, 33%, 29% & 100%

If confidence criterion is 70%, report only rules 2, 3 and

6
Interpretation
Lift ratio shows how effective the rule is in
finding consequents (useful if finding
particular consequents is important)

Confidence shows the rate at which

consequents will be found (useful in
learning costs of promotion)

Support measures overall impact

Caution: The Role of Chance
Random data can generate apparently
interesting association rules

The more rules you produce, the greater this

danger

Rules based on large numbers of records are

less subject to this danger
Example
Transactio Items
n ID
1 Milk, Candy
2 Milk, Candy
3 Milk, Cheese
4 Milk, Candy
5 Butter, Cheese, Candy
6 Butter, Candy
7 Butter, Cheese
Please calculate the support, confidence and lift:
If Milk then Candy:
If Butter then Cheese:
 Example: Charles Book Club
ChildBks YouthBks CookBks DoItYBks RefBks ArtBks GeogBks ItalCook ItalAtlas ItalArt Florence
0 1 0 1 0 0 1 0 0 0 0
1 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
1 1 1 0 1 0 1 0 0 0 0
0 0 1 0 0 0 1 0 0 0 0
1 0 0 0 0 1 0 0 0 0 1
0 1 0 0 0 0 0 0 0 0 0
0 1 0 0 1 0 0 0 0 0 0
1 0 0 1 0 0 0 0 0 0 0
1 1 1 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 1 0 0 0 1 0 0 0 0

Row 1, e.g., is a transaction in which books were bought in

the following categories: Youth, Do it Yourself, Geography
Summary – Association Rules

Association rules (or affinity analysis, or market

basket analysis) produce rules on associations
between items from a database of transactions
Widely used in recommender systems
Most popular method is Apriori algorithm
To reduce computation, we consider only
“frequent” item sets (=support)
Performance of rules is measured by confidence
and lift
Can produce a profusion of rules; review is
required to identify useful rules and to reduce
redundancy
Collaborative Filtering
User based methods
Item based methods
Item-user matrix
Cells are user preferences, r ij, for items
Preferences can be ratings, or binary (buy,
click, like)
More efficient to store as
rows of triplets
Each row has the user ID, the item ID, and
the user’s rating of that item

(Uu, Ii, rui)

User-based Collaborative
Filtering
Start with a single user who will be the
target of the recommendations
Find other users who are most similar,
based on comparing preference vectors
Measuring Proximity
Like nearest-neighbor algorithm
But Euclidean distance does not do well
Correlation proximity does better (Pearson)
For each user pair, find the co-rated items,
calculate correlation between the vectors of
their ratings for those items
 Note that the average ratings for each user are across all
products, not just the co-rated ones
Cosine Similarity
Like correlation coefficient, except do not
subtract the means
“Cold start” problem: For users with just
one item, or items with just one neighbor,
neither cosine similarity nor correlation
produces useful metric
Binary matrix? Must use all the data, not
just the co-rated items.
This can add useful info – in the Netflix
contest, information about which movies
users chose to rate was informative.
Example – Tiny Netflix subset

Consider users 30878 and 823519

Correlation between users 30878 and
823519

First find average ratings for each user:

Find correlation using departure from avg.

ratings for the co-rated movies (movies 1, 28
and 30):
Find cosine similarity for same
users
Use raw ratings instead of departures from
averages:

Ranges from 0 (no similarity) to 1 (perfect

match)
Using the similarity info to make
recommendations
Given a new user, identify k-nearest users
Consider all the items they
rated/purchased, except for the co-rated
ones
Among these other items, what is the best
one? “Best” could be
Most purchased
Highest rated
Most rated
That “best” item is the recommendation for
the new user
Item-based collaborative
filtering
When the number of users is huge, user-
based calculations pose an obstacle
(similarity measures cannot be calculated
until user shows up)
Alternative – when a user purchases an
item, focus on similar items
1.Find co-rated (co-purchased) items (by any
user)
2.Recommend the most popular or most
correlated item
Summary – Collaborative
Filtering
User-based – for a new user, find other users
who share his/her preferences, recommend
the highest-rated item that new user does
not have.
User-user correlations cannot be calculated
until new user appears on the scene… so it is
slow if lots of users
Item-based – for a new user considering an
item, find other item that is most similar in
terms of user preferences.
Ability to calculate item-item correlations in
advance greatly speeds up the algorithm
Association Rules vs. Collaborative
Filtering
AR: focus entirely on frequent (popular)
item combinations. Data rows are single
transactions. Ignores user dimension.
Often used in displays (what goes with
what).
CF: focus is on user preferences. Data
rows are user purchases or ratings over
time. Can capture “long tail” of user
preferences – useful for recommendations
involving unusual items