CUSTOMER RELATIONSHIP MANAGEMENT: FROM CONVERSION TO

CHURN TO WINBACK

A Dissertation
Submitted
to the Temple University Graduate Board

In Partial Fulfillment
of the Requirements for the Degree of
Doctor of Philosophy

By
Ke Li
May, 2013

Examining Committee Members:

Anthony Di Benedetto, Advisory Co-Chair, Department of Marketing and Supply Chain
Management
Eric M. Eisenstein, Advisory Co-Chair, TU Department of Marketing and Supply Chain
Management
Eric T. Bradlow, University of Pennsylvania
Ji Zhu, University of Michigan
Sanat K. Sarkar, External Member, TU Department of Statistics

ii
 UMI Number: 3564826

All rights reserved

INFORMATION TO ALL USERS

The quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.

UMI 3564826
Published by ProQuest LLC (2013). Copyright in the Dissertation held by the Author.
Microform Edition © ProQuest LLC.
All rights reserved. This work is protected against
unauthorized copying under Title 17, United States Code

ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
 ©
Copyright
2005

by

KeLi
All Rights Reserved

iii
 ABSTRACT

With the grant of a big CRM dataset from a large media company, this

dissertation examines four different categories of factors that could impact three stages of

customer relationship management, namely customer acquisition, retention, and winback

of lost customers. Specifically, with the aid of machine learning method of random

forests and text mining technique, this study identify among the factors of customer

heterogeneity (e.g. in usage of self-care service channels, duration of service,

responsiveness to marketing actions), firm’s marketing initiatives (e.g. the volume of the

marketing communications, the depth of the promotion, the different communication

channels they use, and the marketing penetration in different geographical areas),

customer self-reported deactivation reasons, as well as the call centers notes in text form,

which factors play bigger roles than others during each of the three stages of CRM.

Furthermore, the authors also examine how these factors evolve throughout these three

stages of CRM in terms of their effects on shaping customers’ decision making of

whether to convert to paid customer, to churn, or to reactivate their service with the

company. The findings help managers better allocate their resources in the processes of

acquiring, retaining and winning back customers.

iv
 ACKNOWLEDGMENTS

I would like to express my sincere gratitude to my “Fantanstic Four” dissertation

committee members who have guided me and assisted me throughout the completion of

this dissertation. Dr. Anthony Di Benedetto, it is your book New Product Management

that convinced me that Marketing is the most interesting research area that I should spend
the rest of my life time with. Dr. Eric Eisentein, it is your consistent belief in me and

persistent pushing me into this quantitative world that enables me to handle the analysis

of the big data from SiriusXM. Dr. Ji Zhu, we only met once when you came to Temple

University to give presentation but you quickly impressed me by already coming up with

the machine learning methods to deal with the classification problem in Sirius data after

we only chatted for 5 minutes. Afterwards, both of your expertise and spiritual support

carry me through this process of completing the dissertation. Finally Dr. Eric Bradlow,

even though I was deeply impressed by your presentation at O-Bayes conference at

Wharton in June 2009 but it was not until several years later that you finally become my

dissertation committee member and you have absolutely become the game-changer in my
career ever since. Your guidance to my dissertation direction, your prompt replies to my

every weekly report with your insights and encouragement, as well as your word-by-

word review and revision of my dissertation not only turns the dissertation into an

extremely interesting project to me but also paved way for me to become a serious

researcher. With all of your deep intelligence and great patience this experience of

working with all of you will surely benefit my whole career.

v
 Dr. Sanat Sarkar, you are not only my external reader of my committee but also

one of my mentors in this school who supported me in all kinds of ways.

Dr. Aaronson, when you interviewed me for the PhD program at Fox School of

Business you told me Fox School needs students like me. For this very line of words I

have strived not to let you down. In addition, throughout all these years at Temple, you

have always given me a hand whenever it is the most dire situation for me.

I would also thank CIBER Temple, particularly Ms. Kim Cahill and Mr. Arvind

Phatak, without your supporting me financially I won’t be able to accomplish this task.
When other people doubted me, it is you, Kim, who always hold strong confidence in me

which means everything to me.

I would acknowledge WCAI and SiriusXM, this dissertation is based on your

grant of data. Without it, it is a mission impossible.

I would also acknowledge Dr. Bei Kang, who assisted me in many ways to bring

this effort into conclusion.

Finally I would like to thank my family. My mother Xi Liu, my father Faliang Li,

and my sister Hao Li, without their unconditional love and support I would never be able

to make it. Particularly my mom, even though I did not become a scientist as she wished

me to be, this is at least the first step to become a scholar if she could compromise her

dream. However, her training and education of me to be an independent woman has

become an invaluable asset to me that will benefit me throughout my whole life time.

Thank you, mom.

vi
This dissertation is dedicated to my grandparents Cuihua Zeng, Mengqiu Zhang,
Yingquan Li, Yingzhou Liu, and my parents Xi Liu and Faliang Li.

vii
 TABLE OF CONTENTS

PAGE

ABSTRACT....................................................................................................................... iv

ACKNOWLEDGMENTS ...................................................................................................v

DEDICATION .................................................................................................................. vii

LIST OF TABLES ............................................................................................................. xi

LIST OF FIGURES .......................................................................................................... xii

CHAPTER

1. INTRODUCTION ...................................................................................................1

2. LITERATURE REVIEW ........................................................................................8

2.1 CRM (CUSTOMER RELATIONSHIP MANAGEMENT) ............................8

2.1.1 ACQUISITION AND CONVERSION…...……...............……………8

2.1.2 RETENTION AND CHURN………………...………………………11

2.1.3WINBACK…………..……………………..…………………………16

2.1.4 ACQUISITION, RETENTION AND WINBACK……….…….……18

2.2 TEXT MINING ..............................................................................................19

2.2.1 TRANSFORMATION OF TEXT ………………...............................20

2.2.2 TEXT CLASSIFICATION ……………………..……………….…..26

3. HYPOTHESES ........................................................................................................30

3.1 SELF-CARE ...................................................................................................30

3.2 MARKETING INITIATIVES ........................................................................33

viii
 3.2.1 MARKETING COMMUNICATION CHANNELS AND
CUSTOMER RESPONSIVENESS ………………………………...34

3.2.2 RADIO DENSITY…………………………………………………..39

3.2.3 PROMOTIONAL DEPTH…………………………………………..39

3.2.4 PRICING AND CHURN AND WINBACK………………………...40

3.3 DEACTIVATION REASONS .......................................................................42

3.4 DURATION ON WINBACK.........................................................................43

4. METHODOLOGY ..................................................................................................44

4.1 CLASSIFICATION METHODS....................................................................44

4.1.1 BAYESIAN LOGIT/PROBIT MODEL…………………………......44

4.1.2 ADABOOST…….……………………………………….…………..47

4.1.3 RANDOM FOREST……….………………………………………...50

4.2 CLASSIFIER PERFORMANCE MEASURE ...............................................55

5. DATA AND VARIABLES .....................................................................................59

5.1 DATA .............................................................................................................59

5.2 VARIABLES ..................................................................................................61

5.3 TIME FRAME OF VARIABLES ..................................................................63

6. RESULTS AND DISCUSSION ..............................................................................69

6.1 VARIABLE IMPORTANCE IN FOUR EVENTS ........................................69

6.2 HYPOTHESIS TESTING ..............................................................................76

3. LIMITATIONS, FUTURE DIRECTIONS AND CONCLUSIONS ...........................85

REFERENCES ..................................................................................................................89

APPENDICES

A. LIWC2007 OUTPUT VARIABLE INFORMATION ........................................101

B. SUMMARY STATISTICS OF CONVERSION TABLE... ................................103

C. SUMMARY STATISTICS OF CHURN TABLE...............................................107

ix
D. SUMMARY STATISTICS OF NONCONVERTED WINBACK TABLE ........112

E. SUMMARY STATISTICS OF CHURNED WINBACK TABLE .....................116

F. CONVERSION VARIABLE IMPORTANCE TABLE ......................................121

G. CHURN VARIABLE IMPORTANCE TABLE .................................................126

H. NONCONVERTED WINBACK VARIABLE IMPORTANCE TABLE ..........131

I. CHURNED WINBACK VARIABLE IMPORTANCE TABLE ........................136

x
 LIST OF TABLES

Table Page

1. Factors Affecting Aacquisition and Conversion .......................................................50

2. Factors Affecting Retention and Churn ..................................................................57

3. Factors Affecting Winback. ......................................................................................67

4. Data for 4 Events ......................................................................................................75

5. LIWC Output Snapshot ..........................................................................................100

6. Selected Variables and Their Definitions ............................................................100

7. Self-care Channel Activity in 4 Events .................................................................67

8. Marketing Variable Estimates ...............................................................................67

9. Variables Predicting all 4 Events ............................................................................67

Appendix A LIWC2007 output variable information ......................................................101

Appendix B Summary Statistics of Conversion table......................................................103

Appendix C Summary Statistics of Churn table ..............................................................107

Appendix D Summary Statistics of NonConverted Winback table .................................112

Appendix E Summary Statistics of Churned winback table ............................................116

Appendix F Conversion Variable Importance table ........................................................121

Appendix G Churn Variable Importance table ................................................................126

Appendix H NonConverted Winback Variable Importance table ...................................131

Appendix I Churned Winback Variable Importance table ..............................................136

xi
 LIST OF FIGURES

Figure Page

1. Conversion Variable Importance Plot.......................................................................50

2. Churn Variable Importance Plot ...............................................................................57

3. Unconverted Winback Variable Importance Plot. ....................................................67

4. Churned Winback Variable Importance Plot ............................................................75

xii
 1

CHAPTER 1

INTRODUCTION

Customer Relationship Management (CRM) has become increasingly prominent

in marketing since firms realized that, in order to maximize their profit, they should

carefully manage their relationship with their customers, often attempting to anticipate

consumer behavior. This is particularly the case in the service industry. Even though

CRM is a relatively recent term, the emphasis of marketing strategies on customers has

had a long history. From the very beginning of marketing campaigns, entrepreneurs and

firms have relied on all kinds of means to attract potential customers and to keep them

loyal for repeated purchases while providing them with the “right” products. Practitioners

have always emphasized both products and customers simultaneously, but the shift of

focus of marketing research from product to customers became practicable only after the

birth of database marketing in the 1980s.

Regarded as the predecessor of CRM, early database marketing merely consisted

of customers’ demographics and their transactions with firms, but these early efforts held

the promise of enabling targeted direct marketing to individual customers. Over time,

increased processing power and advances in methodology raised the competitive bar so

that, in order to be competitive, firms must have some form of CRM strategy in place.

Today, many companies apply RFM (Recency, Frequency, and Monetary Value)

methods as their CRM models of choice (Reinartz & Kumar, 2003; Fader, Hardie, & Lee,

2005; Cui et al., 2006). These models use secondary data as an input, and segment

customers so that they can target the right customers for promotion, and to profit from the
 2

relationship (Berger & Magliozzi 1992). The most valuable customers identified by RFM

segmentation are the ones with the highest recency, frequency, and monetary value.

Criticism for this segmentation approach has pointed out that it assumes that customers’

behavior patterns are static throughout their whole life cycle. If incorrect, this assumption

results in companies overmarketing to the most attractive RFM segments while

neglecting other segments that could be profitable if properly developed. Another

application that database marketing has facilitated is the customer life cycle or customer

lifetime value (CLV) analysis. As early as 1940s, companies had already begun to

approximate the average value of their customers (Petrison et al., 1997), but academic

research on CLV flourished only after the introduction of database marketing. Most of

the CLV literature has been focused on constructing different probabilistic models of the

underlying consumer behavior. A smaller set of papers explore the relationship between

firm profitability, marketing actions and CLV (Hogan et al., 2002; Reinartz & Kumar,

2003; Rust, Zeithaml, & Lemon, 2004). Studies show that CLV is an effective metric to

segment customers and allocate marketing resources (Haenlein, Kaplan, & Schoder,

2006). A recent study by Fader, Hardie and Lee (2005) links both RFM and CLV by iso-

value curves in customer base analysis showing statistical sufficiency of RFM under a

restricted class of buy-till-you-die models.

Database marketing evolved into relationship marketing during the 1990s due to

four different drivers according to Fletcher, Wheeler and Wright (1991): the changing

role of direct marketing, changing cost structures, technology and economic conditions.

With increasing marketing costs, companies realize that to gain competitive advantage
 3

and higher profitability, it is important to develop individualized relationships with

customers based on their specific needs so as to increase the length of customer life cycle

as well as the value of the transactions between the firm and the customer during each

stage of that life cycle. As a result, marketing activities are increasingly being organized

around relationships with customers rather than products and services. One example of

this has been the increased use of loyalty programs (Tellis & Zufryden 1995; Kim, et al.

2001; Kivetz & Simonson 2002; Roehm, et al. 2002; Kivetz & Simonson 2003; Lewis

2004). In addition, the change of database constructs with the advances in technology

from buyer database to the customer database integrated with marketing activities (Shaw

and Stone 1988) at inexpensive costs, as well as the separation of consumer and business

markets also smoothed the progress of the evolution from database marketing to

relationship marketing.

Nowadays, a fully-developed CRM system provides a 360 degree view of

customer data. It involves using technology to organize, automate, and synchronize sales,

marketing, customer service, and technical support so that firms can track not only the

behavior and spending history of their customers but also the marketing activities,

customer services as well as the communications that companies have had with each

individual customer. Furthermore, the development of methods for differentiating

customer communications brings about the analysis of communication data from social

media sites such as Twitter, LinkedIn, and Facebook. Thus companies can track and

communicate with customers who share opinions and experiences about their company,

products, or services, even when the sharing has taken place in a non-company forum.
 4

The challenge facing both marketing practitioners and academics is no longer the scarcity

of data but the tremendous abundance and variety of data that CRM systems generate.

The explosion of data has forced market researchers to seek new techniques, which have

been borrowed from machine learning, computer science, and database management.

The goal of CRM remains to achieve customer loyalty and to maximize customer

lifetime value. Modern CRM systems attempt to realize these goals by approaching each

stage of the customer’s relationship with the company as separate “stage”: customer

acquisition, customer retention, and winning back lost customers. All activities

performed during each of these three stages are of great importance since customer

lifetime value can be maximized only when the allocation of resources are balanced in

these activities (Reinartz, Thomas, & Kumar, 2005). Although researchers have

recognized the importance of each stage, the limited availability of CRM data in the past,

particularly the focus of most company databases solely on existing customers, has led

the majority of academic research in marketing to address customer retention being

disproportionately focused on. Customer winback (or recapturing of lost customers) is

extremely under-researched but can provide huge return on investment to companies

(Thomas, Blattberg, & Fox, 2004). Even though there is some existing research done on

examining customer acquisition and retention simultaneously, none of the existing

research has investigated all three stages of CRM activities together. With gratitude to

The Wharton Customer Analytics Initiatives (WCAI), this dissertation is endowed with

the access to the CRM database of a large North American media company that covers all

three stages of CRM activities, as well as the text data from the call center notes.
 5

Equipped with this rich, multi-stage subscriber interactions data, I apply classification

methods from machine learning to predict whether customers will convert, churn or

reactivate their service based on their past usage patterns, their text communications with

customer services, the firm’s marketing actions, as well as customers’ self-reported

deactivation reasons. With the aid of the variable importance from random forests and

gradient boosting, I am able to identify the most important factors that impact each of the

three stages of customer relationship management, namely customer acquisition,

retention, and winback of lost customers, and I am also able to explore how each factor

evolves throughout these stages with regard to their effects on shaping customers’

decision making processes of whether to convert to paid customer, to churn, or to

reactivate their service with the company.

Achieving these research goals makes both theoretical and managerial

contributions. Theoretically, this is by far the most comprehensive analysis of customer

relationship management in the academic realm in terms of data range and variety, as

well as the possible factors that impact customer acquisition, retention and winback. For

the first time in marketing literature, different factors such as customer duration,

marketing communication channels, customer responsiveness to the marketing efforts,

customer satisfaction, customer self-reported deactivation reasons, pricing, service

quality, customers’ usage of self-service, as well as text mining of call center notes are

empirically examined together throughout the three stages of CRM. Moreover, given that

current research on winback is scant, this study enriches the literature by exploring how

factors affecting customer conversion and retention might also impact customer winback.
 6

Managerially, understanding how different factors affect different stages of

customer relationship management, as well as which factors are more influential than

others, is critical to firms because companies are frequently faced with the problem of

limited resources and how to optimally allocate them. If it is marketing activities that

play a bigger role in acquiring, retaining and winning back lost customers, firms should

maintain or increase their budget to marketing initiatives. If customer heterogeneity in

their responsiveness to marketing communications, in their usage of self service channels,

or in their sensitivity to price can better forecast customers’ future behavior, this

dissertation can identify those customer characteristics and thus facilitate companies to

target the right customers at the right time. If the service dissatisfaction, product

dissatisfaction, or competition revealed from the lost customers’ self-reported

deactivation reasons turn out to be the major factors that drive customers to discontinue

their services with the company, managers may find it profitable to improve their service

or product quality. If the text mining of the customer call center notes is more predictive

of customers’ decision making in terms of whether to continue or terminate their

relationship with the company, firms should invest more in understanding their customers’

direct feedback. Furthermore, this dissertation assists managers to find answers to the

following relevant questions: 1) among all the marketing communication channels that

they apply to reach out customers (e.g. email, direct mail, or phone calls), which ones are

the most effective? 2) Which customers’ own activities such as contacting the customer

service center or employing online self service are more indicative of their future

behavior? 3) Should they apply the same strategy to recapture the customers who never
 7

converted to a paid customer and the customers who churned after being a paid one? This

dissertation sheds light on all of these important issues, and it should aid firms in

deciding whether they should attach more importance to their own marketing initiatives,

service quality, or customers’ activities and characteristics.

The subsequent chapters of this dissertation are organized in the following

manner: chapter 2 reviews the literature in customer relationship management and text

mining; chapter 3 lays out the conceptual hypothesis; chapter 4 discusses the

methodology that will be applied; chapter 5 describes the data and preprocessing steps;

chapter 6 discusses the results and implications; and finally chapter 7 points out the

limitations of the research and provides future direction and conclusions.

 8

CHAPTER 2

LITERATURE REVIEW

2.1 CRM (Customer Relationship Management)

Drawing from the CRM literature and based on my research objectives, this

review focuses on the contributing factors that extant research has revealed in influencing

the three stages of a company’s CRM activities: 1) customer acquisition, 2) retention and

3) winback of lost customers. I also discuss how prior researchers treat the relationship

among customer acquisition, retention and winback from an analytical point of view.

2.1.1 Customer Acquisition and Conversion

The customer acquisition literature is closely related to that of new product

adoption and diffusion of innovation (Bass 1969, Mahajan, Muller, & Kerin, 1984;

Norton & Bass 1987; Davis 1989; Davis et al., 1989; Rogers, 1995; Zhu et al., 2003; Zhu

& Kraemer, 2005; Campbell & Frei, 2010). Early research investigates what factors are

related to aggregate product adoption; for example, in the Bass model, Bass (1969) uses

the product characteristics and the number of previous adopters to predict product

adoption. Mahajan et al. (1982) also discover that the pricing package for new products

has a major impact on their adoption and retention. Subsequently, using individual data,

researchers find that individual characteristics such as demographics play as important a

role in product adoption as product characteristics and marketing efforts (Chatterjee &

Eliashberg, 1990; Sinha & Chandrashekaran, 1992; Goolsbee & Klenow, 2002). Lewis

(2006) in particular reveals that acquisition discount depth is negatively related to repeat-
 9

buying rates and customer asset value using customer-level data from a newspaper and an

online grocery store.

In investigating what factors influence customer service adoption, Villanueva, et

al. (2008) study the impact of marketing-induced versus word-of-mouth customer

acquisition on customer equity growth and find out that customers acquired by marketing

actions are of less value to the firm compared to customers persuaded by word-of-mouth;

this is because the former only adds more short-term value, but the latter adds more long-

term value to the firm. Musalem and Joshi (2009) identify the relationship between

customer responsiveness to a firm’s CRM efforts and customer acquisition and retention.

They recommend that firms invest more in moderately responsive customers instead of

the high responsive customers since the latter are also pursued by competing companies

which would lead to the erosion of the effects of an individual firm’s CRM efforts. Using

a nine-year period of U.S. airlines industry data, Grewal et al. (2010) discover that

heterogenity in customer satisfaction impacts both acquisition and retention sales. Nam et

al. (2010), on the other hand, explore the role of word-of-mouth resulting from service

signal quality as well as advertising and the retail environment on service adoption.

Finally Xue et al. (2011) relate customer demand for banking services, the availability of

alternative channels, customers' efficiency in service coproduction, and local Internet

banking penetration to Internet banking adoption.

The study of free trial and subsequent conversion can be dated back to 1974 when

Scott (1974) examines whether offering a two-week free newspaper trial can induce a

higher probability of conversion to a six-month newspaper subscription compared with

 10

the control group. Research on conversion explores how free samples can help induce

paid purchases (Gedenk and Neslin 1999; Bawa and Shoemaker 2004). Contrary to

Scott’s negative results, Gedenk and Neslin show positive short-term and long-term

effects of free samples on acquiring customers. In exploring what factors have

contributed to customer conversion from free to fee, Pauwels and Weiss (2008) find that

adjusting price promotion, email and search-engine referrals can all facilitate customer

conversion.

Table 1 Factors affecting Acquisition and Conversion

Factors Articles
Acquisition
product characteristics, pricing Bass (1969), Mahajan et al. (1982)
individual characteristics Chatterjee and Eliashberg 1990, Sinha and
Chandrashekaran 1992, Goolsbee and Klenow
2002, Xue et al. 2011
marketing efforts Lewis (2006)
word-of-mouth Villanueva, et al. (2008), Nam et al. (2010)
customer responsiveness to a Musalem and Joshi (2009)
firm’s CRM
customer satisfaction heterogenity Grewal et al. (2010)
customers' efficiency in using Xue et al. 2011
self-service channels, local
Internet banking penetration
Conversion
free trial, free samples Scott (1974), Grewal et al. (2010)
price promotion, email and Pauwels and Weiss (2008)
search-engine referrals
 11

In summary, prior research has uncovered a wide range of factors that could affect

customer acquisition or conversion: from product characteristics, marketing efforts (such

as pricing strategy, advertising and promotion), customer heterogeneity in demographics,

in responsiveness to CRM efforts, in satisfaction level, in usage of self-service channels,

to word-of-mouth as signal of service quality (See Table 1 above). In this dissertation, I

examine these factors all together and rank their importance so that I know, at least which

customer acquisition factors are more important than others.

2.1.2 Retention and Churn

Customer retention and churn has long been a concern for the service industry,

and marketers and marketing researchers have tried to understand the underlying

mechanisms. Two research streams can be identified in the customer retention literature

and they are closely related to one another: 1) research exploring how some factors can

increase or decrease the customer retention rates and 2) research studying why customers

churn, or discontinue their relationship/service with the company. In essence, these two

streams aim for the same goal: to retain customers as long as possible so that companies

can maximize customer lifetime value. Identifying the churn factors and finding out

strategies to offset it enables better retention.

In examining the potential factors and their relationship with customer retention,

McGahan and Ghemawat (1994) analyze life insurance data and find that in very

competitive environments, large firms achieve greater customer retention than their

smaller rivals. Ganesh, et al. ’s (2000) paper also discloses that customer heterogenity can
 12

result in different retention rates. Rust and Zahorik (1993), Bolton (1998) and Gustafsson

et al. (2005) discover that customer satisfaction levels are positively related to the

duration of customer retention. In the meantime, Gustafsson et al’s (2005) examination of

the impact of affective commitment (a emotional factor that indicates the degree of

reciprocity or personal involvement in the relationship between a customer and a

company), calculative commitment (which signifies that the customer-firm relationship is

based on economic dependence on the product benefits), as well as prior churn on

retention reveal consistent effects of calculative commitment, and prior churn on

retention. While both studying the effects of dual pricing, namely access and usage

prices for subscription services on customer retention, Danaha (2002) finds that access

rates have much stronger effect on retention than usage price. Lambrecht (2006)

discovers that flat rate does not increase churn rate, but pay-per-use increases churn rate.

Some researchers have examined the role of the channel of acquisition in CRM.

For example, online versus traditional channels could have different impacts on customer

retention, and Hitt and Frei (2002) show that retention is marginally higher for customers

using online channels. In exploring how service quality can affect retention, Boulding et

al. (1993) link perceptions of the dimensions of service quality to a person's overall

quality perception to predict customer retention; the researchers find that service quality

perceptions positively affect intended behaviors. Zeithaml, Berry, and Parasuraman (1996)

show that service quality affects subscriber behavioral intentions with respect to the

service, which in turn affects their retention decisions.

 13

Using a sample of retail banking customers observed over a 30-month period at a

large U.S. bank, Campbell and Frei (2010) discover that customers’ usage of self-service

channels are positively associated with customer retention. With the subscription

databases of seven services offered by a telecommunications provider, Schweidel, Fader,

and Bradlow (2008) apply five factors to forecast customer retention behavior: (1)

duration dependence, (2) promotional effects, (3) subscriber heterogeneity, (4) cross-

cohort effects, and (5) calendar-time effects (e.g., seasonality). The results show

promotional effects significantly improve the forecast accuracy of retention behavior

across all seven services. Customer heterogeneity, calendar-time effects, and duration

dependence only improve in five services, while cross-cohort effects are insignificant in

all services.

In exploring what factors have contributed to customer churn or switching service,

Zeithaml, Berry, and Parasuraman (1996) look at whether service quality on particular

behaviors signals customer switching and they find that customers’ behavioral intentions

are strongly influenced by service quality. Bolton (1998), Verhoef (2003), and

Gustafsson et al. (2005) examine whether customers’ dissatisfaction with service and

commitment was related to customer churn.

Keaveney (1995) conducts a field study which identifies more than 800 critical

behaviors of service companies that could cause customers to churn. The author groups

these 800 critical behaviors into 8 general categories: 1) pricing (which includes the

subcategories of high prices, price increases, unfair pricing practices, and deceptive

pricing practices); 2) inconvenience (such as inconvenient service provider location,

 14

hours of operation, waiting time for service or to for an appointment, etc.); 3) core service

failures caused by mistakes, billing errors or technical problems; 4) service encounter

failures (which is the interaction failure between customers and customer service

personnel at company caused by employee attitude or inability to provide service that

customers expect); 5) employee responses to service failures; 6) competition; 7) ethical

problems (such as dishonest or intimidating behavior, unsafe or unhealthy practices, or

conflicts of interest) ; and 8) involuntary switching. Most of these factors were echoed or

further studied by other researchers (e.g., Shaffer and Zhang (2002) also find that

customers churn due to incentives promoted by competitors).

Braun and Schweidel (2011) classified the causes of churn into three categories: 1)

controllable factors such as price, dissatisfaction with service, or with the product; 2)

uncontrollable reasons which include moving from the service area and

death/divorce/family issues; and 3) firm-initiated churn which is due to customer

nonpayment or abuse of service. The researchers use the data from a land-based

telecommunications services organization to test their competing-risk model. They

conclude that the ability of a firm to reduce customer churn is diminished by the

pervasiveness of uncontrollable influences which have a "dampening effect" on the return

of a firm's retention marketing efforts.

Other factors that are found to cause churn include customer heterogeneity

(Morrison and Schmittlein 1980), payment equity (Bolton, Kannan, and Bramlett 2000;

Bolton and Lemon 1999), loyalty programs (Bolton, Kannan, and Bramlett 2000;
 15

Verhoef 2003), marketing activity (Lewis 2004, 2005), retention duration (Hughes 2006;

Reichheld, 1996), and multichannel communication (Godfrey, et al. 2011), to name a few.

In summary, prior research shows that the factors that influence a consumer’s

decision to remain a customer with a company or to churn are similar to the ones that

have impact on customer acquisition. Customer heterogeneity, marketing initiatives, as

well as customer satisfaction and perceived service quality all impact both acquisition

and retention. However, no prior studies have examined all of these factors concurrently

in both of these stages or in the winback stage. This study fills this gap by exploring how

these same factors behave in different stages of customer relationship management.

Table 2 Factors affecting Retention and Churn

Factors Articles
Retention
Firm size in competitive environments: Mcgahan and Ghemawat (1994)
large vs. small
Customer heterogenity Ganesh, et al. (2000), Schweidel, Fader,
Bradlow (2008)
Customer satisfaction levels Rust and Zahorik (1993), Bolton (1998) and
Gustafsson et al. (2005)
Affective commitment, calculative Gustafsson et al. (2005)
commiment, prior churn on retention
Access rate vs. usage price Danaha (2002)
Online vs. traditional channels Hitt and Frei (2002)
Service quality Boulding et al (1993); Zeithaml, Berry, and
Parasuraman (1996)
Self-service channels Campbell and Frei (2010)
Duration dependence, promotional Schweidel, Fader, and Bradlow (2008)
effects, cross-cohort effects, calendar-
time effects (e.g., seasonality)
Churn
Flat rate vs. pay-per-use Lambrecht (2006)
 16
(Table 2, continued)
Pricing , inconvenience, core service Keaveney (1995)
failures caused by mistakes, billing
errors or technical problems, service
encounter failures, employee responses
to service failures, ethical problems,
involuntary switching
Competition Keaveney (1995), Shaffer and Zhang (2002)
Controllable factors, uncontrollable Braun and Schweidel (2011)
reasons, firm-initiated churn
Customer heterogeneity Morrison and Schmittlein 1980
Payment equity Bolton, Kannan, and Bramlett 2000; Bolton
and Lemon 1999
Loyalty programs Bolton, Kannan, and Bramlett 2000; Verhoef
2003
Marketing activity Lewis 2004, 2005
Retention duration Hughes 2006; Reichheld 1996
Multichannel communication Godfrey, et al 2011

2.1.3 Winback

Even though I believe that each of the CRM stages from acquisition to retention

to winback are all important to any firm that intends to maximize their profitability with

their customers, winback or the reacquisition of lost customers is by far the least

researched area among the three. As Griffin and Lowenstein (2001) and Thomas, et al.

(2004) point out, the importance of allocating more resources to winning back customers

is apparent: the probability of a firm winning back a churned customer and the net return

on investment can be eight times as high as winning a new customer. However, very

limited research has explored which factors can facilitate firms winning back lost

customer except Stauss and Friege (1999) and Thomas, et al. (2004). Stauss and Friege

(1999) develop the second lifetime value (SLTV) metric to facilitate evaluating the NPV
 17

generated after a customer has been won back and point out that not every customer

should be won back. Thomas, et al. (2004) focus on the reacquisition pricing strategy and

find out the optimal pricing strategy for a firm to recapture their churned customers

involves a low reacquisition price and higher prices when customers have been

reacquired.

Even though both customer heterogeneity and pricing strategy are important

factors in winning back lost customers, I assert that other factors that have be influential

in shaping customer acquisition and retention should also be examined for their effect on

winback. Therefore, in this dissertation, I explore how prior marketing activities, prior

customer usage of self service channels, prior deactivation reasons, customer

responsiveness to marketing initiatives as well as the prior call center notes by customers

affect the odds of winning back customers. Particularly in this study, through the rich

data, I classify the lost customer into two categories: those who tried the product for some

trial period but decided not to convert being a paid customer, and those who have become

a paid customer for some time and then decided to churn. Making this distinction in the

lost customer base is of great importance to both academics and managers because it

allows us to assume that there are different underlying factors that influence recapturing

these two different kinds of lost customers. Hence this research can assist companies in

allocating their resources so that they can achieve a greater return on investment.

Table 3 Factors Affecting Winback

Factors Articles
Winback
 18
(Table 3, continued)
Customer heterogeneity Stauss and Friege (1999)
Optimal pricing strategy for a firm to Thomas, et al. (2004)
recapture their churned customers
involves a low reacquisition price and
higher prices when customers have
been reacquired

2.1.4 Acquisition, Retention and Winback

In addition to exploring possible factors and their relationship with customer

acquisition, retention and winback, prior researchers also have had extensive discussions

about how these three events are related, especially between customer acquisition and

retention. Two absolutely opposite assumptions about their relationships have been made.

One assumption is that customer acquisition and retention are two independent events

(Blattberg and Deighton 1996; Gupta, Lehmann et al. 2004). The other assumes that the

customer acquisition process affects the customer retention process (Hansotia and Wang

1997; Thomas 2001; Reinartz, Thomas et al. 2005; Schweidel, Fader, Bradlow 2008).

Blattberg and Deighton (1996) find that independently developed acquisition

response and retention curves can appropriately relate marketing expenditures to

customer acquisition and retention rates. Thomas (2001), on the other hand, develops a

model for estimating the length of a customer's lifetime linking customer acquisition to

customer retention and shows the financial impact of not accounting for the effect of

acquisition on customer retention. Gupta, et al. (2004) also study the customer value by

treating customer acquisition and retention as two independent events, but Schweidel,
 19

Fader, and Bradlow (2008) explore the duration dependency between acquisition and

retention.

No matter whether we believe these events are independent or not, the consensus

is that we should truncate and censor the data (Bolton's 1998, Schmittlein and Helsen

1993) in order to separate acquisition data from retention data for further statistical

analysis. In this study, I hold that all CRM events are related, particular acquisition

activities affect customer retention, both acquisition and retention efforts have an impact

on winning back lost customers.

In the following section, I briefly review the text mining and text classification

methods that will be used.

2.2 Text Mining

Today about 80% of the data held within an organization is in the form of text

documents —for example, reports, Web forms, open-ended survey responses, news feeds,

e-mails, and call center notes, etc. Texts are essential for an organization to gain a better

understanding of their customers’ behavior and the abundance of the text data forced

organizations to seek ways to explore and leverage this information. Text mining is a

technology that aims to capture key concepts and themes and to uncover hidden

relationships and trends from collections of textual materials without requiring

knowledge of the precise words or terms that authors have used to express those concepts.

Text mining is one branch of data mining, or the analysis steps of Knowledge Discovery

in Databases (KDD) process that results in the discovery of new patterns in large data

sets. The data that text mining deals with are usually referred to as unstructured data
 20

since the data cannot be stored in a relational database and thus cannot be structured by

categorical, ordinal, and continuous variables. Data consisting of both structured and

unstructured data are called semistructured data.

Text mining usually involves two steps: the first step transforms the textual data

into structured data with specialized text analytic techniques; and the second step applies

learning algorithms such as clustering, classification or predictive modeling to explore

relationships within the data. Sometimes the transformed data can be combined with

other structured data to make predictions of the future behavior. In this dissertation, I use

semi-structured data to predict customer acquisition, retention and winback.

2.2.1 Transformation of Text

The primary problem with the management of all of the unstructured text data is

that there are no standard rules for writing text so that a computer can understand it. The

language, and consequently the meaning, varies for every document and every piece of

text. The only way to accurately retrieve and organize such unstructured data is to

analyze the language and thus uncover its meaning. There are several different automated

approaches to the extraction of concepts from unstructured information. These

approaches can be broken down into two kinds, linguistic and nonlinguistic.

2.2.1.1 Nonlinguistic Approach

Nonlinguistic solutions are based on statistics and computer technology. They

treat each document d j as an array of words and apply computer technology to quickly

scan and categorize key concepts within the text. These key concepts are called
 21

vocabulary and they are used as the feature set to train the classifier. After key concepts

are identified in each document, one counts the number of times each feature word occurs

and calculates their statistical proximity to related concepts. From raw text documents to

the data matrix that is needed for further analysis, there are several preprocessing steps

that need to be taken:

1) The removal of function words: since not all words can be used to train the

classifier, some words or phrases have to be removed. The first of these words are

function words. Function words are also called stop words, which include: auxiliary verbs,

conjunctions, articles, prepositions, etc. These words appear frequently in most text

documents but they do not contribute to the training of the classifier, therefore they need

to be removed before the next preprocessing step.

2) Stemming: the process of grouping the words which share the same

morphological root and misspelled words. For example, words “precise”, “precision”,

“preciseness” should be grouped as one word “precise”. Even though Baker and

McCallum (1998) argue that stemming can sometimes hurt effectiveness, the common

practice is still to adopt it since it can reduce both the dimensionality of term space and

the stochastic dependence between terms (Sebastiani 2002).

3) Representation of feature value: After feature words or phrases are determined,

I need to decide how to represent the feature values. One way is to use a binary vector,

assigning 1 to the feature value if the document contains the feature word and 0 otherwise.

Another way is to count the frequency of the feature words that appear in each document.

This, however, leads to problem of how to weight the feature words.

 22

According to Salton and McGill (1983), there are four kinds of automatic

weighting systems to assign weights to feature terms extracted from the documents.

a) tf (term frequency) weighting system, which sets the weight of term k in

document i to be the frequency of occurrence of word construct k in document i.

Therefore, terms occurring in every document of a collection are treated equally with

those that occur in only a few documents.

WEIGHTik = FREQik

b) idf (inverse document frequency) weighting system, which takes into

account the number of documents that each term is assigned and assumes that the content

representation by the frequency of the term in the document is decreased by the

frequency of the number of documents DOCFREQk that each term is assigned to.

FREQik
WEIGHTik 
DOCFREQk

c) Term discrimination weighting system, which depends on the degree to

which the assignment of a term to the documents is capable of decreasing the average

distance between the documents. DISCVALUEk, the discrimination value of term k, is

calculated as the difference between two measurements of document space density,

corresponding to the densities before and after assigning term k.

WEIGHTik = FREQik .DISCVALUEk

d) Term relevance weighting system, which is based on probability indexing

theory stating that the best index terms tend to occur in the relevant documents with

respect to some query. If we define TERMRELk as the ratio of the proportion of relevant
 23

items in which term k occurs to the proportion of nonrelevant items in which the term

occurs, then the term relevance weighting system can be expressed as:

WEIGHTik = FREQik .TERMRELk

The most commonly used weighting systems are actually the product of the

term frequency and the inverse document frequency ( tf  idf , or tfidf) and normalized

tf  idf , which incorporates the length of document vectors into the weighting system

(Salton and Buckley 1988).

4) Feature selection: since the number of extracted feature words for most

documents of a collection is several thousands, it can induce high dimensional term space

and overfitting problems. Feature selection is the name given to a broad set of methods

used to reduce dimensionality. It is a necessity in text classification. Feature selection is

an attempt to select, from the original set T , the set T ' terms (with | T ' | | T | ) that yields

the highest effectiveness for document indexing.

The most commonly used feature selection method applies an evaluation function

that to a single word (Soucy and Mineau, 2003). First, all words are evaluated and sorted

independently according to the assigned criterion. Then a predefined number of best

features are used to form the best feature subset. Individual words are scored using

measures such as mutual information, information gain, odds ratio,  2 statistics, term

strength and so on (Brank et al. 2002; Torkkola 2002; Forman 2003; Sousa et al. 2003).

These metrics have their origin in machine learning or text retrieval but there are also

new related stastistical regularization and variable selection methods such as LASSO

(least absolute shrinkage and selection operator, Tibshirani 1996) and elastic net (Zou &
 24

Hastie, 2005). These techniques are similar to ridge regression. The lasso minimizes the

residual sum of squares, subject to the sum of the absolute values of the coefficients

being less than a constant. Thus it restricts some coefficients to be 0, which has the effect

of reducing the dimensionality of the data. An elastic net is the combination of the ridge

and lasso since it applies both L2 and L1 penalties for regularization. Elastic net is

particulary applicable for large datasets when the number of predictor variables is much

greater than that of the obervations.

However, even after all these procedures are performed, the text classification

completed by a nonlinguistic approach still suffers from limited accuracy. Due to the

large presence of polysemy (e.g. a word or phrase that has multiple related meanings),

homonymy (e.g. a group of words that share the same spelling and pronunciation but

different meanings), and synonymy (e.g. different words with same or similar meanings),

the original terms may not be the optimal dimensions for document content

representation. To compensate for this, linguistic approaches have been applied to help

reduce the dimensionality even further and hence substantially enhance the prediction

accuracy.

2.2.1.2 Linguistic Approach

Linguistics-based text mining, applies the principles of natural language

processing (NLP)—the computer-assisted analysis of human languages—to the analysis

of words, phrases, and syntax, or structure, of text. A system that incorporates NLP can

intelligently extract concepts, including compound phrases. Moreover, knowledge of the

underlying language allows classification of concepts into related groups, such as

 25

products, organizations, or people, using meaning and context. This approach offers the

speed and cost-effectiveness of statistics-based systems, but it offers a far higher degree

of accuracy while requiring far less human intervention.

Two commonly used linguistic approaches to further reduce the dimension of the

text transformed data matrix are term clustering and latent semantic indexing. Term

clustering groups words with a high degree of pairwise semantic relatedness, so that the

groups of their centroids (or representative of them) may be used instead of the terms as

dimensions of the vector space. Term clustering addresses synonymy and it can be done

either by unsupervised clustering or supervised clustering. Unsupervised clustering

means clustering is not affected by the category labels attached to the documents (Lewis

1992; Li and Jain 1998). Supervised clustering gathers those terms that tend to indicate

the presence of the same category, or group of categories (Baker and McCallum 1998;

Tishby 2001).

Latent Semantic Indexing (LSI, Deerwester et al. 1990) is a technique developed

in Information Retrieval to address the dimensional problem deriving from the usage of

synonyms and polysemes. It infers the dependence among the original terms from a

corpus and transfers this dependence into the newly obtained, independent dimensions.

LSI applies singular value decomposition to the matrix formed by the original document

vectors to map it to the new vectors. Therefore, unlike term clustering, the newly

obtained dimensions by LSI are not intuitively interpretable. However, they work well in

bringing out the “latent” semantic structure of the vocabulary used in the corpus. The

drawback is that, if some original terms are particularly good for discriminating a
 26

category, that discrimination power may be lost in the new vector space (Sebastiani,

2002).

After both non-linguistic and linguistic approaches are performed to transform the

original text data, the newly generated data matrix is ready for further text classification.

2.2.2 Text Classification

Text classification is also called text categorization, document classification, or

document categorization. According to Sebastiani (2002), there are two approaches to

text classification: knowledge engineering techniques and machine learning approaches.

Knowledge engineering techniques relies on expert system which typically consists of a

set of manually defined logical rules, one per category of type

If < DNF formula > then <category>.

A DNF (“disjunctive normal form”) formula is a disjunction of conjunctive clauses; the

document is classified under <category> iif (if and only if) it satisfies at least one of the

clauses. The most famous example of this approach is the CONSTRUE system (Hayes et

al. 1990) constructed by Carnegie Group for the Reuters news agency. DNF systems

suffer from the same problems as other expert systems, such as the well-studied

“knowledge acquisition bottleneck”, which refers to that knowledge acquisition is the

major impediment and most time-consuming part of development of an expert system.

Machine learning methods can be used to overcome the bottleneck problem, because

machine learning methods do not require antecedent expert knowledge, but rather rely

solely on the data itself to build the classifiers. For this reason, machine learning methods

have progressively increased in popularity and dominance in most text classification

 27

applications. In Chapter 4 I will give a detailed description of the two machine learning

methods used in this study. Below I talk about two other issues in text classification:

binary verses multiclass text classification and single-label vs. multi-label text

classification.

2.2.2.1 Binary vs. Multiclass

Most text classification research today is about binary coding of content since the

most important text classification applications are binary classifications. For example,

filtering is used to decide whether document I is about one particular category or not

(Sebastiani 2002). In addition, binary classification algorithms serve as the basis for

multiclass classification. Most multi-class classifiers are built as extensions of their

binary counterparts. They decompose the multi-class classifier into binary classification

using either a one-against-all or all-against-all strategies. One-against-all method uses the

training set to build one classifier per class and to distinguish the samples in a single class

from the samples in all remaining classes. All-against-all method builds classifier for

each pair of classes. There are also direct approaches to build multi-class classifications,

such as nearest neighbor, the generative approach and naïve Bayes.

2.2.2.2 Single-label vs. Multilabel Text Classification

Another important dimension of text classification is the distinction between

single-label and multilabel classification. In single-label cases, one document is assigned

to only one category, therefore it is also called nonoverlaping category classification. In

multi-labeling cases, any number of categories from 0 to C can assigned to the same

document hence it is also known as overlapping category classification. An example of

 28

multi-label classification is the categorization of movies. For example, the best motion

picture award winner at the year 2012 Academy Awards, the Artist, can be classified into

three categories simultaneously: comedy, romance and drama. Multi-label classification

methods have been increasingly called for by the modern applications such as music

categorization (Li & Ogihara, 2003), semantic scene classification (Boutell et al., 2004)

and protein function classification (Luo & Zincir-Hyewood, 2005).

There are several approaches to multi-label classification (Tsoumakas 2007): 1)

ranking, to order a set of labels L, so that the topmost labels are more related to the new

instance; 2) hierarchical classification, to label a data set which has a hierarchical

structure; if each document is labeled with more than one node of the hierarchical

structure, then it is called hierarchical multilabel classification; 3) multiple-instance

learning, to learn a concept given a set of positive and negative of instances

simultaneously (Maron & Lozano-Perez, 1997).

The extant multi-label classification methods can be grouped into two categories:

problem transformation methods and algorithm adaptation methods. Problem

transformation methods transform the multi-label classification problem either into one or

more single-label classification (Boutell et al., 2004) or regression problems. Algorithm

adaptation methods adapt machine learning algorithm to multi-label cases. For example,

AdaBoost.MH and AdaBoost.MR (Schapire & Singer, 2000) are two extensions of

AdaBoost (Freund & Schapire, 1997) for multi-label classification. Other algorithm

adaptation methods include ML-kNN (Zhang & Zhou, 2005) which is an adaptation of

the kNN lazy learning algorithm for multi-label data, and improved SVMs using ranking
 29

(Elisseeff & Weston, 2002) and stacking (Godbole & Sarawagi, 2004). In this

dissertation, I apply text mining techniques to examine the text notes from the call center

of a large media and entertainment company to explore if the text data can help predict

whether customers will convert from free to fee, to churn or be won back.
 30

CHAPTER 3

HYPOTHESES

3.1 Self-care

Self-care, or self-service is one of the customer contact modes that customers use

in order to interface with service providers. Kellogg and Chase (1995) define customer

contact as the function of the interaction between a customer and a service provider.

Based on whether customer contact is technologically involved, Froehle and Roth (2004)

classify the context under which customers and their service providers interact with each

other into five categories: 1) technology-free customer contact, in which a customer has

face-to-face service interactions (Chase 1978) with a human service provider or customer

representative. 2) technology assisted customer contact, where the customer

representative employs technology as an aid to improve the face-to-face contact, but the

customer does not have access to the technology. 3) technology-facilitated customer

contact, in which both customer representative and customers have access to the

technology during their face-to-face service encounter. 4) technology mediated customer

contact, where the customer and customer representative are not physically present with

each other so they communicate via a voice telephone call or online instant messaging, or

email; 5) technology-generated customer contact, such as bank’s ATMs, self-service

kiosks in the airport, automated car washes, and so on.

To encourage customer participation, to enhance operations efficiency and cost-

effectiveness, as well as to be more competitive (Fitzsimmons and Fitzsimmons, 2004;

Huete and Roth, 1988; Haynes and Thies, 1991; Hill et al., 2002; Oliveira et al., 2002;
 31

Roth, 2000; Boyer et al., 2002; Menon, 2003; Karmarkar and Pitbladdo, 1995; Heskett et

al. 1997), more and more service companies are encouraging customers to adopt self-care

or self-service. This trend has been called the “self-service revolution”, which provides a

multichannel service delivery system with a collection of employee and self-service

channels in both physical and virtual environments (Xue, et al. 2007).

Different theories help to explain customers’ decision making process of whether

to adopt self-care or not, and of which ones to choose. Customers’ technology readiness

(Parasuraman 2000) states that there are four characteristics that indicate whether

customers are ready for technology or not: optimism, innovativeness, discomfort, and

insecurity about technology. Optimism means customers think technology is a good thing;

Innovativeness refers to “a tendency to be a technology pioneer and thought leader”;

Discomfort is “a perceived lack of control over technology and a feeling of being

overwhelmed by it,” and insecurity is defined as the “distrust of technology and

skepticism about its ability to work properly.” Based on this concept, if a customer is

optimistic and innovative, he or she will adopt the newer technology such as using online

self-care while if a customer is feeling discomfort and insecure, he or she may opt to pick

up a phone and talk to a customer representative. Moreover, the Theory of Reasoned

Action (Fishbein and Azjen, 1975) and Theory of Planned Behavior (Ajzen, 1985, 1991)

from psychology hold that an individual’s intention, their perceived control over their

behavior, their attitude towards the behavior, as well as the rational cognitive assessments

of the behavior are essential in their decision whether to take action or not. So when an

individual feels he or she has the control over whether to perform or not to perform the
 32

behavior, the stronger the intention to engage in a behavior, the more likely the action is

taken. In the meantime, attitude, which is “a person’s general feelings of favorableness or

unfavorableness towards some stimulus object” (Fishbein and Azjen 1975), drives an

individual’s intentions for future behavior. Hence a onetime unhappy communication

experience with the service provider could result in customers’ attitude formation

towards the company and affect their future behavior and decision on whether they would

continue or terminate their service. Thus when customers first started to apply new

technology such as instant messaging or email to communicate with their service

providers a decade ago, companies regarded it as a challenge (Zeithaml et al. 2002).

In addition to customers’ heterogeneous behavioral characteristics, other factors

such as the availability of different channels, the cost and benefits of using different

channels, as well as the different activities that they perform through these channels also

impact their choice of self-care channel. If the activity they need to perform is to make

payment and it can be accomplished by simply logging onto their online account,

customers are not likely to bother to call a live agent.

Even though customers’ choice of whether to utilize self-care and which channel

to use is not intended to maximize service provider’s profits, prior studies have found a

causal link between utilization of self-service channels and the relationship between

customers and the service provider. For example, Hitt and Frei, (2002), Campbell and

Frei (2010), Xue et al. (2007, 2011) all find that employing a self-care channel can

increase customer retention rates. Based on these past findings and theories related, I

assume, given the different channels that customers can employ (e.g online help (CUWI),
 33

interactive voice response with call center (IVR), making calls to a live agent in a call

center, and different activities they perform, such as changing account information,

making payment, or changing service (e.g. upgrading or downgrading service plans), that

not only the employment of self-service but also the interactions between self-service

channels and activities are significant in predicting customers’ decision making of

conversion, churn and reactivation with their service provider. Furthermore, if a customer

utilized self-care channel during the trial period, and he or she converted to a paid

customer, it indicates a positive attitude being formed and a positive impact on his or her

future behavior. Thus, I expect that the customer will be less likely to churn and more

likely to be won back if for any reason they discontinued their service with the service

provider:

H1a: The employment of self-care channel in the conversion period and churn period

increases the likelihood of customer conversion, non-churn and subsequent reactivation

with the company.

H1b: The interaction of self-care channels and customers’ activities in both trial period

and churn period are highly predictive of customer retention and winback.

3.2 Marketing Initiatives

Customers choosing different self-care channels to communicate with their

service-providers are just one direction of the interactions between customers and firms.

To acquire customers more effectively, to maintain a long term relationship with them

afterwards, as well as to regain lost customers so that customers’ lifetime value with the

company can be maximized, firms also reach out to customers using various marketing
 34

initiatives. Past literature has shown that marketing actions are typically effective in

customer acquisition, retention, and winback (Bass 1969; Mahajan et al. 1982; Berger et

al. 2002; Danaher 2002; Lewis 2004, 2005, 2006) and such actions include pricing,

promotions and marketing outbound communications, and so on. In this dissertation, I

look into the following marketing initiatives and their respective impact on acquisition,

retention and winback: marketing communication efforts and customers’ responsiveness,

promotion depth and pricing.

3.2.1 Marketing Communication Channels and Customer Responsiveness

Marketing communication through different channels has become an

indispensible part of marketing strategy. With more availability of communication

channels to contact customers, the problem that firms are faced with is which channel

they should utilize to interact with which customers. One consideration is the cost-benefit

analysis of each communication channel and the other is the effectiveness of each

communication channels or the mix of them. Traditional communications media either

cannot be customized to meet each customer’s demand (e.g. television) or is expensive to

implement (e.g. sales forces, direct mail). With the availability of electronic media,

companies tend to send e-mails to customers to welcome them to join the club, to instruct

them how to start their service, to remind them of the service or payment due, or to send

promotional offers at certain occasions, based on the customer information stored in their

database. However, do emails work better than direct mail or telemarketing? What is the

effect when a firm applies all three communication channels to reach the same customer?
 35

Prior researchers have explored the volume and mixed effect of communication channel

in non-service settings.

Two different findings regarding the effects of marketing volume in the

relationship between customer and firms have been disclosed. Venkatesan and Kumar

(2004) discover a positive linear relationship between the level of bidirectional

communication between customers and firms and purchase frequencies. Godfrey et al.

(2011) find that an inverted-U shape between customers repurchase and the volume of

three individual communication channels: telephone, e-mail, and direct mail. Godfrey et

al (2011) further explain these two different phenomena by two social norm theories:

reciprocal action theory and reactance theory. Reciprocal action theory holds that people

feel obliged to proportionally respond to goodness by goodness (Bagozzi 1995; Becker

1990). Therefore when firms invest more on customers by increasing marketing volume,

customers would reciprocate by repeated purchase. Reactance theory on the other hand

explains why excessive marketing communication volume would diminish the effect of

marketing efforts: when the volume of the marketing communication increases to a

certain extent, customers would perceive the incoming marketing communication efforts

as persuasive attempts to force them to make purchase (Clee & Wicklund 1980) thus

respond negatively to all kinds of marketing efforts such as personal selling (e.g.,

Wicklund, Slattum, & Solomon 1970), advertising (e.g., Robertson & Rossiter 1974),

direct marketing (e.g., Morimoto & Chang 2006), and rewards programs (e.g., Kivetz

2005).
 36

However, what Venkatesan and Kumar (2004) and Godfrey et al. (2011) study as

the volume of communication is distinct from the total volume of marketing

communications defined in this dissertation. Venkatesan and Kumar (2004) count both

inbound and outbound communication by customers and the firm and Godfrey et al

(2011)’s volume is for each individual communication channel. What I refer to as the

total volume is the additive effect of multiple communication channels that occur in both

the conversion and retention period. Nonetheless, the reciprocal action theory and

reactance theory can be applied to explain the different effects of marketing

communications in different stages of CRM. In the trial period, when customers initially

receive the welcome or promotional communications from a firm, it helps them to learn

about the company and their products so the higher the communication volume, the

stronger the bond and thus higher probability for them to convert to a paid customer.

However, after customers have converted and started a steady relationship with the

company, the cumulative effect of the marketing efforts in both trial and paid period

makes the communication happening in the retention period easy to trigger the reactance

behavior and thus result in churn. As to winning back lost customers, since I use trial

period marketing volume to predict the probability of winning back unconverted

customers and both trial period and retention period marketing volume to forecast the

likelihood of recapturing churned customers, I would expect the same effects of

marketing communication volume on the winback period. Hence, I postulate the

following:
 37

H2a: The total volume of marketing outbound in the trial period is positively associated

customers’ acquisition, retention and winback probability respectively.

H2b: The volume of marketing communication in the paid period, particularly the ones

closer to the churn date or right censored date, is positively associated customer churn.

Customer responsiveness to firm’s marketing efforts is an indicator of customers’

interaction level with their service provider. The more responsive the customers are, the

stronger is the relationship between the customer and the firm, thus the more likely they

would convert from a free to fee customer, to be won back, and less probable to churn.

Accordingly I expect the following:

H3: The response rate of a customer during the both trial period and paid period is

positively associated with customer conversion and winback but negatively associated

with customer churn.

In addition to the total volume of contact channels, each individual channel (e.g.,

telephone, direct mail, e-mail) has been characterized as more or less interpersonal.

Email is viewed as the least interpersonal communication channel to reach customers

since most of them do not require any customers’ involvement but to open and read it. In

addition, due to the cheap cost associated with sending out emails, customers have been

bombarded with all kinds of commercial emails and would simply discard them as junk

mail and never care to open. Prior research shows that a more involving and

interpersonal contact channel have a much higher conversion rate on average than a less

involving contact channel (Anderson and Narus 1999, p. 302). Moreover, compared with

inexpensive and prevalent email communications, customers feel that they are more
 38

valued or cared about by the company when they receive direct mail since it shows that

the company is willing to invest in their relationship. As the direct mails are more and

more replaced by emails, the utilization of direct mail may actually enhances

relationships and thus have a better chance to retain them. Finally, regarded as the most

interpersonal channel among the three, the telephone channel requires both a service

company’s personnel and customers’ involvement. Customer representatives’ attitude as

well as their ability to solve customers’ issues varies among different representatives,

which could create uncertainty and negative impact on customer perception in firm’s

service quality if an encounter did not have a satisfactory outcome. Since each

communication channel has its own advantages and disadvantages, I expect different

marginal effects from each of them.

Adding to the complexity of the different impact of each individual

communication channel is the synergy effect that previous researchers have studied

(Reinartz, Thomas, and Kumar 2005, Godfrey et al. 2011). The common understanding is

that the interaction effect is greater than the sum of individual channel effect and previous

researchers again use reciprocal action theory and reactance theory to explicate the two

possible outcomes of the interaction effects of multichannel communication: the initial

combination of multichannel communication could elicit customers’ good feelings of

being invested but excessive communication coming in all kinds of forms can drive

customers away from rather than closer to a company. Therefore I postulate the following:

H4a: Different communication channels have different effects on customer conversion,

retention and winback.

 39

H4b: The interaction effect of multichannel communication is strongly significant in

predicting customer conversion, churn and winback.

3.2.2 Radio Density

Radio density is defined as the total number of radio stations in each zip code

which captures the regional differences in terms of product availability and product

variety in the telecommunication company studied. Product variety provided to

customers during their trial period largely impacts customers’ experience with the firm’s

products and thus affects their decision whether to convert from a free trial customer to a

paid customer. However, if a customer has opted to terminate their service with the firm,

the factor of radio density should be not significant in affecting their decision to

reactivate service with the company or not since the radio density is static for long period

of times.

H5a: The radio density is positively related with the probability of customer conversion,

and negatively related to customer churn.

H5b: The radio density is not significant in predicting winning back both of unconverted

and churned customers.

3.2.3 Promotional Depth

Promotional depth is one of the factors that early researchers have explored in its

association with customer lifetime value. Two contradictory findings concerning its

effect on customer lifetime value have been revealed. Analyzing the long-term behavior

of some newspaper and online grocery customers and their acquisition discount,

Anderson, et al. (2004) find a negative relationship between acquisition discounts and
 40

customer value; in the mean time, Anderson and Simester (2004) discover that customers

acquired through catalogs with more discounted items have higher long-term value.

Despite the disparity of the two outcomes, prior researchers are in agreement that deep

price cuts can increase the temporary sales for the first-time customers but decrease

future purchases by established customers. Applying this concept to the three stages of

CRM, one can reasonably expect that the promotion given out in the trial period would

encourage customers to convert, however, the discounts that feed the established paid

customers signal the segment of customer base that are price-sensitive and they are at

greater chance to churn once the promotional discount come to a halt. Furthermore, once

the customers who have enjoyed too many promotions churned, they would be hard to

gain back. In this dissertation, the promotional depth is denoted by two terms: the number

of promotional discounts which shows how many times a customer was offered the

promotional price, and the price range which indicate how deep the price cut a customer

received during the retention period. Hence I hypothesize:

H6a: Promotional discounts are positively associated with customer churn but negatively

associated with winback;

H6b: There is a positive relationship between price range and customer churn but a

negative relationship between price range and customer winback.

3.3.4 Pricing and Churn and Winback

Pricing is another “old” topic that researchers first started with to explore its

relationship with acquisition (Bass 1969; Mahajan, Muller et al. 1984; Norton and Bass

1987; Davis 1989; Davis, et al. 1989; Rogers 1995) and it is closely related to promotions.
 41

Most extant literature focuses on what is the optimal pricing strategy to acquire new

customers and even win back lost customers (Thomas, et al. 2004). This dissertation

examines how the average price that a customer pays for his or her overall services with

the service provider each month helps to predict their probability of churn or future

winback. The average price that a customer pays is connected to classifying customer

base that a company has and company usually would regard the higher price customers as

high value, while the ones who pay low average prices as low value customers. Therefore

studying how much the price a customer pays on average and its relationship with their

CRM targeting strategy is important in targeting at the right customer at right time.

Therefore one can expect that the higher price a customer pays each month, the happier

he or she is with the service that the company provides, and the lower probability he or

she would churn. However, once a customer churns, firm’s strategy to win back lost

customers is usually to offer deeper discounts (Thomas, et al. 2004) than to existing

customers. The customers who used to enjoy the discounts, or are of lower customer

value to the company, maybe enticed to accept the new discount and reactivate their

service with the company. As to the high value customers, since they are mostly loyal,

companies tend not to offer discounts to them until they decide to terminate the service

one day. The deep discount that the company offers to win them back could be tempting

since now they discover they could actually enjoy the same service with much lower

price than they usually paid. Hence I postulate the following:

H7a: The average price a customer pays to his or her service provider is negatively related

to their churn;
 42

H7b: The average price a customer pays before he or she churned has an U-shape

relationship with their probability of winback.

3.3 Deactivation Reasons

One very revealing information source is customers’ self-reported deactivation

reasons for relationship termination. Prior researchers have explored the various causes

that lead to customer churn (Keaveney 1995), and Braun and Schweidel (2011) even use

the multiple causes of churn to model customer lifetime. Combining both Keaveney

(1995) and Braun and Schweidel (2011)’s categorization methods, we group our

customer self-reported deactivation reasons into ten subcategories: "Service

Dissatisfaction", “Uncontrollable Reason”, "Product Dissatisfaction", "Cost/price/fee",

"Non Usage", "Prepaid period ended" (this one is grouped into “Free ended” for trial

deactivation reason), "Free ended", "Competition", "Unknown reason", and “NonPay”.

This grouping of the deactivation reasons covers from customer satisfaction for both

service and product, to competition, to customers’ usage of service, as well as to financial

reasons. They reveal whether these factors play role in influencing customers’ decision

making of whether to convert, churn or reactivate service with their company during each

stage of CRM. Particularly for conversion from free to fee, the reason of “Free ended”

should be the highly predictive of whether a customer converts or not. Therefore I

hypothesize the following:

H8a: Deactivation reasons should be among the top important factors that are predictive of

customer behavior of conversion, churn and being won back.

 43

H8b: Among all deactivation reasons, “Free ended” plays a major role in why a customer

would not convert.

3.4 Duration on Winback

Prior researchers have examined the effect of retention duration on customer

churn (Hughes 2006; Reichheld 1996), and the impact of the acquisition duration on

retention duration (Schweidel, et al. 2008). They find that while acquisition duration has

a positive relationship with retention, retention has a negative effect on churn. Therefore,

the longer the trial period, the longer the subsequent retention duration. On the other

hand, the longer a customer stays with the company, the less likely he or she will churn.

Building on these research findings, I speculate that on the one hand, the longer the

acquisition duration and retention duration, the stronger the relationship between the

customer and company it indicates, therefore the more likely that a customer could be

won back once they discontinued their service for any reason. However, if a customer has

had too long history with the company, he or she tends to have experienced the product

of the company so well that once he or she decides to churn, it could imply that he or she

has already had it all so it will be hard to win them back. Therefore, I postulate the

following:

H9: There is an inverted U-shape relationship between the customer acquisition and

retention duration and the probability of winning them back.

 44

CHAPTER 4

METHODOLOGY

4.1 Classification Methods

Based on the research objectives of this dissertation, as well as the four binary

outcome events that this study examines, namely, conversion, churn, winning back

unconverted customers and winning back churned customers, in this section, we discuss

three classification methods that are used in this study: Bayesian logit/probit, boosting,

and random forest. I focus on the binary class version of these methods.

4.1.1 Bayesian Logit/Probit Model for Classification

The standard approach in marketing for binary and multiclass classification are

generalized linear models. The commonly used generalized linear models include the

probit, logit, log-log and complementary log-log models. These models are all

exponential-family models and differ according to the choice of link function used to

transform the DV that will be expressed as a linear function of the Xs and they also may

differ on the assumptions underlying the regression structure on latent continuous data.

The most commonly used generalized linear models are probit and logit models, and

there is actually not much difference between them. The logistic distribution is similar to

the normal distribution except in the tails, which are considerably fatter (It more closely

resembles a t distribution with 7 degrees of freedom). Therefore, the two distributions

tend to give similar probabilities for the intermediate values of x '  such as between -1.2

and +1.2, where x is the vector of input variable and  are the parameters. When x ' 

gets extremely small or extremely large, the logistic distribution tends to give larger
 45

probabilities to y  0 than normal distribution. However, unless the data is extremely

unbalanced, such as too few responses with y  1 or y  0 , I expect little differences in

predictions from the probit or logit models.

The Bayesian versions of generalized linear models for classification usually

employ Bayesian logit or probit models for binary outcomes, and Bayesian multinomial

logit or probit for polychotomous response data. There are several possible prior

assumptions for Bayesian probit or logistic models. Readers who wish to pursue a full

discussion of the intricacies of choosing a prior can consult Gelman, Carlin, Stern, and

Rubin (2004). However, rather than provide a comprehensive discussion of how to

choose priors in a Bayesian context, here I provide a brief justification of my chosen prior.

Binary classification - hierarchical logit model

Given a set of training data D  {( x1 , y1 ),( x2 , y2 ),...( xn , yn )} , let yi {0,1} denote

the class label for each binary outcome with 1 being the code for membership and 0 for

the nonmembership, and define x as the n  p matrix of standardized input variables, then

the logistic classifier takes the form:

exp(x'i β)
Pr( yi  1| xi )  Z ( x'i β)  (1)
1  exp(x'i β)

where β is the vector for parameters, and Z  Z1 ,...Z n is the latent variable. Each Z i is

independent and follows a normal distribution: N (xi'  ,1) , and

yi  1 if Zi  0

yi  0 if Zi  0
 46

Therefore the decision boundary for classification is the set of points for which z(s)

are zero, or a hyperplane defined by {x | 0   T x  0} , and the conditional probability of

each outcome is ½.

The Bayesian approach to the logistic classifier assumes that each parameter  j

follows a prior distribution and this prior distribution is usually assumed to be normal

with mean  j and variance V :

1 (  j   j ) 2
p(  j |  j ,V ) ~ N ( j ,V )  exp( ), j  1,...d (2)
2 V 2V

Here I assume that each parameter  j is independent of all other parameters

conditional on hyperparameters. In this case, they are  j and V .

A full Bayesian approach specifies a second-stage prior on the hyperparameters to

make a hierarchical model. For the hierarchical logistic model with normal prior, I could

further assume that the hyperparameters  comes from a normal distribution and V from

inverse Wishart distribution of:

 ~ N (,V  A1 ),
(3)

V ~ IW ( ,V )

where A1  100I or larger to set a diffuse prior for the variance.

Since I assume that the  j are conditionally independent, then

p(  )   p(  j )

If I multiply it by the likelihood of the data, I get the posterior distribution of  :

 47

I ni
y 1 yij
p(  | D)  p( D |  ) p(  )  {  pij ij (1  pij ) } p(  ) (4)
i 1 j 1

p( ) ~ N (  , A1 )

4.1.2 Boosting

Boosting is one of the most powerful learning ideas introduced in the last twenty

years and it was specifically designed for classification problems. It is designed to

combine the outputs of many “weak” classifiers in order to generate a powerful

“committee.” From this perspective boosting is similar to bagging and other committee-

based approaches which take a simple unweighted average of the predictions from each

model, essentially giving equal probability to each model. However this resemblance is

only on the surface and boosting is fundamentally different from the committee-based

approaches. For example, compared with bagging which uses bootstrap to sample the

training set, boosting has no randomness at all.

There are a number of boosting algorithms but the most popular one is called

Adaptive Boosting (AdaBoost) introduced by Freund and Schapire (1996). Later on,

researchers also explored some algorithms to modify the original AdaBoost algorithm

such as Gentle AdaBoost, Logit AdaBoost, and Real AdaBoost (Friedman et al. 2000).

Recent developments in this area include the regularization of the boosting algorithm

(Friedman 2001; Rosset, Zhu, and Hastie 2004), such as utilizing a learning rate

parameter to regularize the boosting when classification trees constitutes the base

classifiers. Another proposed algorithm for regularization, which is gaining popularity, is

called stochastic gradient boosting (Friedman 2002, Ridgeway 2006). It employs a

 48

random permutation sampling strategy at each iteration to obtain a refined training set

and achieves significant computational savings.

Binary AdaBoost

The original AdaBoost algorithm by Freund and Schapire (1996) has two versions,

denoted as “AdaBoost.M1” and “AdaBoost.M2” respectively. These two versions are

equivalent in dealing with binary classification problems but differ in handling problems

with more than two classes. Here I only introduce “AdaBoost.M1” since Freund and

Schapire (1996) show that AdaBoost.M1 fits a forward stepwise additive logistic

regression model that minimizes the expectation of the exponential loss function, ,

with F(x) denoting the boosted classifier. Consider a two-class problem, with the output

variable coded as Y ∈ {−1, 1}. Given a vector of predictor variables X, a classifier C ( X )

produces a prediction taking one of the two values {−1, 1}. The error rate on the training

sample is

1 N
err   I ( yi  C ( xi )) ,
N i 1
(4)

and the expected error rate on future predictions is EXY I (Y  C ( X )) .

A weak classifier is defined as the one whose error rate is only a little better than

random guessing. The boosting algorithm has access to the weak learning algorithm and

sequentially applies the weak classification algorithm to repeatedly modified versions of

the data, thus producing a series of weak classifiers Cm ( x) , m  1, 2,..., M . The

predictions from all of the weak classifiers are then pooled through a weighted majority

vote to generate the final prediction:

 49

M
C ( x)  sign(  mCm ( x)) (5)
m 1

Here 1 ,  2 ,...,  M denote the contribution (or weights) of each weaker learner Cm ( x) and

are computed by the boosting algorithm. The more accurate classifier receives higher

weights.

The data are also modified at each boosting step. Initially all training observations

( xi , yi ) , i  1, 2,..., N , were given the same weights, that is wi  1/ N , i  1, 2,...N . For

each successive iteration m  2,3,..., M , the observation weights are individually

modified so that the observations that were misclassified at the previous step are given

more weights, whereas the weights are decreased for those that were classified correctly.

As a result as iterations proceed, observations that are hard to classify correctly receive

ever-increasing influence. Each successive classifier is thus forced to focus on those

training observations that are misclassified in the previous iteration in the sequence.

The power of AdaBoost is to considerably increase the performance of even a

very weak classifier. This AdaBoost.M1 algorithm is also called “Discrete AdaBoost”

since the base classifier Cm ( x) returns a discrete class label. Friedman et al. (2000)

modified the algorithm to enable the base classifier to return real-valued predictions thus

called “Real AdaBoost”. Other modifications such as Gentle AdaBoost requires fitting a

regressor at each iteration and results in the original GentleBoost algorithm whenever  m

=1.

As to the “base learner” Cm ( x) for AdaBoost algorithm, the most commonly used

one is classification trees, where improvements are often most dramatic. In fact, Breiman
 50

(1996, 1998) referred to AdaBoost with classification trees as the “best off-the-shelf

classifier in the world”.

4.1.3 Random Forest

Another ensemble classification method which aims at enhancing the

classification accuracy are random forests. The idea of a random forest is to combine tree

predictors so that each tree depends on the values of a random vector sample

independently and with the same distribution for all trees in the forest.

Random forests were developed by Breiman (2001), the same author who

introduced bagging. Compared with Adaboost, random forests are more robust with

respect to noise, which occurs because Adaboost has no random elements and grows an

ensemble of trees by successive reweightings of the training set where the current

weights depend on the past history of the ensemble formation whereas random forests do

not depend on the past history of the ensemble. Unlike SVM, random forests require little

tuning and is very easy to implement.

Like bagging, random forests also use the bootstrap to generate samples and to

grow a committee of classification trees. To classify a new object from an input vector,

one puts the input vector down each of the trees in the forest and each tree gives a

classification. Then the new object is classified into the class which “won the most votes”

from all the trees in the forest. Unlike bagging, the trees in random forests are de-

correlated and they allow random selection of features to split each node. Bagging can

actually be regarded as a special case of random forests, and random forests are the
 51

combination of bagging and random selection of subsets of features (Ho 1998; Amit &

Geman 1997).

4.1.3.1Construction of Random Forests

A random forest is constructed as follows. For each tree in the forests Tb (b = 1 to

B),

1) Draw a bootstrap sample Z* of size N from the original data to be the training set.

2) Grow a random forests tree Tb with the sample Z*. If there are p input variables, a

number m<<p is specified such that at each node, m variables are selected at

random out of p variables and the best split on these m is used to split the node

into daughter nodes. The value of m is held constant during the forest growing.

3) Each tree Tb is grown to the largest extent possible and there is no pruning.

Then output the ensemble of trees {Tb }1B . For the classification prediction at a new point x,

let Cˆb ( x) be the class prediction of the bth random forest tree, then

Cˆ rfB ( x) = majority vote of {Cˆb ( x)}1B . (6)

4.1.3.2 Strength and Correlation

In the original paper on random forests, Breiman (2001) shows that the forest

error rate relies on two parameters: the strength of each individual tree in the forest and

the correlation between any two trees in the forest, defined as follows.

Consider a random forest classifier consisting of a collection of tree-structured

classifiers {h(x, k ), k  1,...} where the {k } that are independently and identically
 52

distributed (i.i.d) random vectors and each tree Tb casts a unit vote for the most popular

class at input x. Define the margin function for a random forest as:

{mr ( X , Y )  P (h( X , )  Y )  max P (h( X , )  j) (7)

and the strength of the set of classifiers {h(x, k )} is

s  EX ,Y mr ( X , Y ) . (8)

Now define the raw margin function as

{rmg (, X , Y )  I (h( X , )  Y )  I (h( X , )  ˆj( X , Y )) (9)

where I is the indicator function. Hence mr ( X , Y )  Ermg (, X , Y ) . Since any two

values of (, ) are i.i.d, the variance of mr is:

var(mr )  E, (cov X ,Y rmg (, X , Y )rmg (, X , Y )) (10)

In brief, the higher the correlation, the greater the forest error rate and increasing

the strength of the individual trees decreases the forest error rate. Reducing the predictor

number m at each node reduces both the correlation and the strength and increasing it

increases both. For classification, the default value for m is and the minimum node

size is one (Hastie, et al. 2009). However, the "optimal" value of m can be found quickly

by using the oob (out of bag) error rate and this is the only adjustable parameter to which

random forests are somewhat sensitive.

In the following sections, I introduce several unique features or output that

random forests possess.

4.1.3.3 Use of Out of Bag Data

 53

Random forests use out of bag (oob) data as the test set; therefore cross-validation

is not needed to estimate the test set error. When the training set for the current tree is

drawn by sampling with replacement, about one-third of the cases are left out of the

sample. This oob (out-of-bag) data is used to get a running unbiased estimate of the

classification error as trees are added to the forest. It is also used to get estimates of

variable importance, which is a prominent feature of random forests.

During the run, each tree is constructed using a different bootstrap sample from

the original data but the oob data are not used in the construction of the kth tree. Put each

case left out in the construction of the kth tree down the kth tree to get a classification.

Thus, a test set classification is obtained for each case in about one-third of the trees. At

the end of the run, take j to be the class that got most of the votes every time case n was

oob. The proportion of times that j is not equal to the true class of n averaged over all

cases is the oob error estimate.

4.1.3.4 Variable Importance

Variable importance is a very important feature of random forests and it is used in

this dissertation to rank the factors that could influence customers’ decision making

during the three stages of CRM. The variable importance in random forest is calculated in

the following way: first, input the oob cases in every tree grown in the forest and count

the number of votes cast for the correct class; second, permute the values of variable m

randomly in the oob cases and enter these cases into the tree again; third, subtract the

number of votes for the correct class in the variable-m-permuted oob data from the
 54

number of votes for the correct class in the intact oob data; finally, average this number

over all trees in the forest to obtain the raw importance score for variable m.

If the values of this score from tree to tree are independent, one can calculate the

standard error by a standard computation. The correlations of these scores between trees

have been calculated for numerous data sets and proved to be quite low, therefore

standard errors is computed in the classical way: one can divide the raw score by its

standard error to get a z-score, and assign a significance level to the z-score assuming

normality. When the number of variables is very large, forests can be run with all the

variables once, and then be run again using only the most important variables from the

first run. For each case, a local importance score for variable m for this case is generated

by subtracting the percentage of votes for the correct class in the variable-m-permuted

oob data from the percentage of votes for the correct class in the untouched oob data.

Gini importance is another measure for variable importance and it is generated by

random forests in the following way: every time a split of a node is made on variable m

the gini impurity criterion decreases. Adding up the gini value decreases for each

individual variable over all trees in the forest, gives a Gini importance value.

4.1.3.5 Advantages of Random Forests

Even though Breiman (2001)’s claim that random forests is unexcelled in

accuracy among current algorithms, many later empirical studies showed that this is not

the case. However, random forests do possess the following advantages: 1) they run

efficiently on large data sets; 2) they can handle thousands of input variables without

variable deletion; 3) they give estimates of what variables are important in the
 55

classification; 4) they generate an internal unbiased estimate of the generalization error as

the forest building progresses; 5) they incorporate methods for balancing error in class

population unbalanced data sets; 6) the generated forests can be saved for future use on

other data; 7) they compute proximities between pairs of cases and hence offers an

experimental method for detecting variable interactions; 8) random forests do not overfit

(Breiman, 2001).

In this dissertation, I employ random forests, gradient boosting to perform

classification for the following three events: 1) conversion (or trial conversion), the event

when a subscriber who is not paying for a subscription out of his own pocket (but

provided free by the company) decides to become a paying subscriber and purchases a

subscription after his trial period ends; 2) churn, the event when a revenue-generating

subscriber who pays out-of-pocket decides to terminate service, for some reason; and 3)

winback, the event when a deactivated subscriber who either discontinued his service at

the end of trial-period or churned as a paid-subscriber reactivates his service with the

company.

4.2 Measuring Classifier Performance

There are quite a few measurements to evaluate performance of classification

methods. The commonly used classification performance measurements include accuracy

rate (1- error rate), precision and recall, sensitivity and specificity, F-measure, Youden’s

index (arithmetic mean between sensitivity and specificity), Matthews correlation

(correlation between the actual and predicted), as well as graphical tools such as ROC

curves and Cumulative Lift charts. However all of these performance check methods are
 56

based on a confusion matrix which consists of four dimensions: true positive (TP), true

negative (TN), false positive (FP), and false negative (FN). Each measurement has

strength and weaknesses, which I outline in the discussion of commonly used measures

of text classification accuracy below.

1) Accuracy rate = (TP + TN) / T (11)

where T=TP + TN + FP + FN, or the total size of the test set. This is most straight-

forward measurement, however, when a certain category comprises the majority of the

test set, the accuracy can easily reach the high percentage. Sometimes it is worth trying to

maximize the accuracy score, but accuracy (and its counterpart error) are considered

fairly crude scores that don't give much information about the performance of a classifier.

2) Precision (PPV) = TP / (TP + FP) (12)

is also called Positive predictive value (PPV), which measures the proportion of predicted

positives which are actually positive. The issue with precision is that when a

classification method outputs only confident categories (the categories that take the

majority), the precision rate will increase to a high percentage.

3) Recall (Sensitivity) = TP/(TP+FN) (13)

measures the proportion of actual positives which are predicted positive. The problem

with the measurement is that when the classifier outputs loosely, the recall rate can get

really high.

4) F measure= the harmonic mean between precision and recall

The F-scores are in the interval [0, 1]. The higher the score, the higher the

classification quality with 1 indicating perfect classification. Since there is a trade-off

 57

between recall and precision, the F-measure is usually used as a simple measure to

evaluate the classifier. Considering that you can get a perfect precision score by always

assigning zero categories, or a perfect recall score by always assigning every category.

However, to maximize the F-score, the classifier has to assign the correct categories and

only the correct categories, which maximizes both precision and recall at the same time.

There are two types of F-measures, resulting from two different ways of

averaging: micro-average F-measure and macro-average F-measure. The Micro-

averaging F-measure is computed globally over all category decisions. So if I have

multiclass classification, for each class i , I define TPi as the number of documents

assigned correctly to class i , FPi as the number of documents that do not belong to class

i but are assigned to class i , FNi as the number of documents that are not assigned to

class i by classifier but which actually belong to class i , and  as the precision, and  as

the recall, then

M M

 TPi  TP i
= M
i 1
,  = M
i 1

 (TPi  FPi )  (TP  FN )

i i
i 1 i 1 (14)

2
F (micro-averaging) =
  (15)

where M is the total number of categories. In contrast, the micro-averaging F-value gives

equal weight to each document, therefore it is the same as accuracy rate when you are

dealing with single-label classification. It tends to be dominated by the classifier’s

performance on common categories.

 58

The macro averaging F-measure is calculated locally over each category first and

then averaged over all categories.

2 i i F i
Fi  , and F (macro-averaging) = i 1

 i  i M (16)

where M is the number of categories.

 59

CHAPTER 5

DATA AND VARIABLES

5.1 Data

The data utilized in this dissertation comes from Sirius XM, the largest worldwide

satellite company, and was granted through the Wharton Customer Analysis Initiative

(WCAI). This data set provides a “360 view” of SiriusXM’s 1,000,000 customers who

first subscribed or tried in late 2009, and are tracked through roughly 2.5 years up to

February 2012. It includes information on subscribers, subscribers’ vehicles, subscribers’

radios, product usage, billing & payments, outbound direct marketing, and customer

service interactions (online and voice). While customers may access the service via the

Internet and mobile phones, the majority of subscribers listen to SiriusXM in their

vehicles.

The dataset I received includes 9 tables from Sirius XM: Services Table, MDB

(Outbound Marketing) Table, Billing Table, Collections Table, Notes Table, Self Care

Table, Listening Log Table, ESN Table (Device Information), VLC Table (Vehicle

Information), and one table created by WCAI: Radio Locator Table.

Drawing from this data, I first focus on customers who own one vehicle since the

sole vehicle owners’ activation, deactivation, or reactivation activities can provide clean

estimates of the effects of various predictors on conversion, retention, and winback.

Moreover, many of the multi-vehicle subscribers are business customers who are likely to

be qualitatively different from non-business subscribers. This reduces the sample size

from 1 million to less than 700,000 customers. Second I determine which customers are
 60

paid customers since only these customers should be examined in the retention period.

The number of paid customer with only one vehicle is 464,810. Third, I separate the data

into three parts: trial period, paid period and winback period by combining the

information from each customer’s trial period end date, first paid service start date,

deactivation date and reactivation date. For paid customers, their trial period records

consist of all interactions that occur before the first paid service start date and their

retention period starts from their first paid service start date; for non-converted customers

the trial period records consist of interactions either before their deactivation date or the

end date in the data. Using these definitions results in 392,532 trial customers, of whom

221,333 converted into paid customer. The reported conversion rate of the company for

2010 was 46.2%, which is similar to my data sample.

To decide which paid customers churned, I used their deactivation date, if it was

before the end of the data in the sample (March 1, 2012). I treat customers who have not

churned by this data as right-censored, meaning they are classified as non-churned

customers. Thus I obtain 228,597 churned customers and 236,213 non-churned customers.

Finally the winback customer base consists of customers who have a clear deactivation

date. Among them, customers who have a specific reactivation date are defined as the

ones that have been successfully won back while others are not. For my research

purposes, I further divided the winback customers into categories: the ones who

deactivated during or at the end of their trial period and the ones who churned after

conversion. There are 1,539 non-converted customers who were reacquired but 157,158

were not. Among the churned customers, 67,124 were recaptured and 148,047 were not.
 61

Table 4 below shows the final datasets that I used to predict conversion, churn and

winback both unconverted customers and churned customer. Table 4 below shows the

data for each of the three periods as well as the rate of conversion, churn and winback.

Table 4 Data for 4 events

Trial Period Paid Period Unconverted Churned

Winback Winback
Non-
Converted 221,333 236,213 Won back 1,539 67,124
churned
Not won
Unconverted 171,199 Churned 228,597 157,158 148,047
back
Total 392,532 Total 464,810 Total 158,697 215,171
Conversion Churn Winback
56.38% 49.18% 1% 31.2%
rate rate rate

5.2 Variables

In order to explore my research questions and hypotheses, I created four

categories of variables from the data: 1) Customer characteristics variables, which

include both a linear and quadratic form for mean-centered acquisition/conversion

duration, retention/paid/churn period duration, the dummy variable for usage of self-care

in both conversion and churn period, customer response rate to firm’s marketing actions,

three self-care activity variables (e.g "Change_AcctInfo", “Make_Payment”,

“Change_Service”), three self-care channel variables ( “Live_agent” for speaking to

customer representatives, “IVR” for interactive voice response, and “CUWI” for internet
 62

help), as well as the interactions between self-care channels and activities which count

how many time each customer use each self-care channel to do each activity; 2)

Marketing initiative variables, which consist of total number of marketing out-bound

contacts, the counts of each communication channel that each customer receives from the

firm, number of promotions that each customer gets, mean-centered average price in both

linear and quadratic forms, and total-radio number for each zip code which I treat as radio

density variable (Xue, et al. 2011); 3) Deactivation reasons, which have been grouped

into 10 subcategories: "Service Dissatisfaction", “Uncontrollable reason”, "Product

Dissatisfaction", "Cost/price/fee", "Non Usage", "Prepaid period ended" (this one is

grouped into “Free ended” for trial deactivation reason), "Free ended", "Competition",

"Unknown reason", and “NonPay”; 4) Text mining variables. I analyzed the call center

notes text using LIWC2007 (Linguistic Inquiry and Word Count). LIWC2007 is text

analysis software which is able to calculate the degree to which people use different

categories of words across text with 64 language dimensions. The variables generated by

LIWC include 22 linguistic dimensions such as percentage of words in the text that are

pronouns, articles, auxiliary verbs etc, 32 psychological process word categories

including social, affective, cognitive, perceptual, biological, and relativity processes, 7

personal concern categories (e.g., work, home, leisure activities), and 3 paralinguistic

dimensions (assents, fillers, nonfluencies). Therefore, these variables are clusters from

the results of a word extraction process, which groups the words and phrases that express

the same meaning or have the same linguistic function together. For a complete list of the

LIWC output variables and their examples, please see the Appendix 1.
 63

The variable values generated by LIWC are the percentage of total words for each

entry. For example, an incomplete snapshot of the LIWC output in Table 5 shows that

11.59% of note #1 is composed of pronouns compared with 15% of note #2. Also 5.92%

of note #5 consists of positive emotional words compared with 12.5% of those for note

#6. Since these values are the percentage of frequency in each note, the total word count

for each note has been taken into consideration.

Table 5 LIWC Output Snapshot

Notes funct pronoun they posemo negemo anger sad cogmech
1 54.32 11.59 0.45 4.32 1.59 0 0.23 12.05
2 57.5 15 0 5 0 0 0 15
3 50 10.94 0 6.25 0 0 0 14.06
4 48.94 19.15 0 4.26 2.13 0 0 17.02
5 55.62 10.06 0 5.92 0.59 0 0 13.61
6 53.12 15.62 3.12 12.5 0 0 0 15.62
7 60.4 21.78 0 1.98 4.95 0.99 0.99 17.82
8 64.85 18.18 1.21 3.64 1.82 1.82 0 18.79
9 48.94 19.15 0 4.26 2.13 0 0 17.02
10 60 14.29 0 2.86 5.71 2.86 0 20
11 75 30 5 0 0 0 0 30
12 54.76 7.14 0 0 0 0 0 19.05
13 59.33 20.1 3.35 1.91 3.35 0.96 0 17.22
14 46.43 10.71 0 4.46 0.89 0.89 0 12.5
15 53.33 13.33 0 6.67 0 0 0 20
16 48.94 19.15 0 4.26 2.13 0 0 17.02

5.3 Time Frame of Variables

All time-varying variables, including all self-care activities that each customer

performed, the self-care channels that each customer used, all marketing variables such as
 64

the total marketing outbound communication, customer responsiveness to marketing

communications, each marketing outbound channel), as well as the call center notes

during both conversion and retention period, were separated into two time frames: 60

days within the conversion and retention period end dates and 60 days beyond the end

date of conversion and retention period. Therefore, for these variables the number of

variables is doubled. The reason why I used 60 days instead of other time frame is

because that SiriusXM usually communicates with its customers 30 days before their

contract ends to ask them whether they want to renew their services or not and if not,

SiriusXM would take some action (such as offering a discount) to retain customers. In

addition, based on prior research results, I know that the recency effect plays a role in

customers’ decision making, so dividing the data into these time-based categories, I can

also test whether activities that happened closer to the conversion or churn dates are more

predictive of these events. It seems reasonable that activities that occurred further in the

past should exhibit decaying effects.

In order to model the winback event, I used the conversion period variables to

predict winning back non-converted customers and the retention period variables to

predict recapturing churned customers.

Table 6 shows some selected variables and their definition. The summary

statistics of the four tables are attached as Appendix 2-5.

 65

Table 6 Selected variables and their definitions

Variable Definition
RADIO_TOTAL The total number of radio in a given zip code
Acquisition_duration Mean centered trial period length in months

Acquisition_duration2 Quadratic form of Acquisition_duration

Retention_duration Mean centered paid period length in months
Retention_duration2 Quadratic form of Retention_duration
Unknown_reason 1 if customer deactivated for 'Unknown_reason', 0
otherwise
Free_ended 1 if customer deactivated for 'Free_ended', 0 otherwise
Uncontrollable_reason 1 if customer deactivated for 'Uncontrollable_reason', 0
otherwise
Service_Dissatisfaction 1 if customer deactivated for 'Service_Dissatisfaction', 0
otherwise
NonPay 1 if customer deactivated for 'NonPay', 0 otherwise
Non_Usage 1 if customer deactivated for 'Non_Usage', 0 otherwise
Product_Dissatisfaction 1 if customer deactivated for 'Product_Dissatisfaction', 0
otherwise
Cost_price_fee 1 if customer deactivated due to Cost or price or fee, 0
otherwise
Competition 1 if customer deactivated due to competition, 0 otherwise
Acquis_Live_Agent60 Count of customer calling customer representatives within
60 days of their trial period end date.
Acquis_CUWI60 Count of customer using Internet Help within 60 days of
their trial period end date.
Acquis_IVR60 Count of customer using Interactive Voice Response
within 60 days of trial period end date
Acquis_Live_Agent_gt60 Count of customer calling customer representatives
beyond 60 days of their trial period end date.
Acquis_CUWI_gt60 Count of customer called call center within 60 days of
their trial period end date.
Acquis_IVR_gt60 Count of customer using Interactive Voice Response
beyond 60 days of trial period end date
Acquis_ChangeAcctInfo6 Count of customer changing account info through all self-
0 care channels within 60 days of their trial period end date.
Acquis_Change_Service6 Count of customer upgrading or downgrading service
0 through all self-care channels within 60 days of their trial
period end date.
Acquis_Make_Payment60 Count of customer using self-care channels to make
payment within 60 days of their trial period end date.
Acquis_Make_Payment_g Count of customer using self-care channels to make
 66
(Table 6, continued)
t60 payment beyond 60 days of their trial period end date.
Acquis_ChangeAcctInfo_ Count of customer changing account info through all self-
gt60 care channels beyond 60 days of their trial period end
date.
Acquis_Change_Service_ Count of customer upgrading or downgrading service
gt60 through all self-care channels beyond 60 days of their trial
period end date.
Acq_LiveAg_ChangeInfo Count of customer calling customer rep to change account
60 info within 60 days of their trial period end date.
Acq_LiveAg_ChangeSer6 Count of customer calling customer rep to change service
0 plan within 60 days of their trial period end date.
Acq_LiveAg_MakePaymt Count of customer calling customer rep to make payment
60 within 60 days of their trial period end date.
Acq_CUWI_MakePaymt6 Count of customer using internet help channel to make
0 payment within 60 days of their trial period end date.
Acq_CUWI_ChangeInfo6 Count of customer using internet help channel to make
0 payment within 60 days of their trial period end date.
Acq_IVR_MakePaymt60 Count of customer using interactive voice message to
make payment within 60 days of their trial period end
date.
Acq_CUWI_ChangeSer60 Count of customer using internet help channel to change
service plans within 60 days of their trial period end date.
Acq_IVR_ChangeSer60 Count of customer using interactive voice message to
change service plan within 60 days of their trial period
end date.
Acq_IVR_ChangeInfo60 Count of customer using interactive voice message to
change account info within 60 days of their trial period
end date.
Acq_LiveAg_MakePaymt Count of customer calling customer rep to make payment
_gt60 beyond 60 days of their trial period end date.
Acq_CUWI_ChangeInfo_ Count of customer using internet help channel to make
gt60 payment beyond 60 days of their trial period end date.
Acq_CUWI_ChangeSer_g Count of customer using internet help channel to change
t60 service plans beyond 60 days of their trial period end date.
Acq_LiveAg_ChangeSer_ Count of customer calling customer rep to change service
gt60 plan beyond 60 days of their trial period end date.
Acq_CUWI_MakePaymt_ Count of customer using interactive voice message to
gt60 make payment beyond 60 days of their trial period end
date.
Acq_LiveAg_ChangeInfo Count of customer calling customer rep to change account
_gt60 info beyond 60 days of their trial period end date.
Acq_IVR_MakePaymt_gt Count of customer calling customer rep to change service
60 plan beyond 60 days of their trial period end date.
Acq_IVR_ChangeInfo_gt Count of customer using internet help channel to make
 67
(Table 6, continued)
60 payment beyond 60 days of their trial period end date.
Acq_IVR_ChangeSer_gt6 Count of customer using interactive voice message to
0 change service plan within 60 days of their trial period
end date.
Acquis_MDBcount60 Total count of marketing outbound initiatives within 60
days of customer's trial period end date
Acquis_MDBcount_gt60 Total count of marketing outbound initiatives beyond 60
days of customer's trial period end date
ACQUSI_EM60 Number of email sent to customers within 60 days of
customer's trial period end date
ACQUSI_DM60 Number of direct mail sent to customers within 60 days of
customer's trial period end date
ACQUSI_TELEM60 Number of telemarketing reaching customers within 60
days of customer's trial period end date
ACQUSI_EM_gt60 Number of email sent to customers beyond 60 days of
customer's trial period end date
ACQUSI_TELEM_gt60 Number of telemarketing reaching customers beyond 60
days of customer's trial period end date
ACQUSI_DM_gt60 number of direct mail sent to customers within 60 days of
customer's trial period end date
Trial60Response_Rate Customer's response rate to firm's marketing outbound
initiatives that was sent within 60 days of trial period end
date
Trial_gt60Response_Rate Customer's response rate to firm's marketing outbound
initiatives that was sent beyond 60 days of trial period end
date
ACQUSI_EMDM60 Interaction of email and direct mail sent to customer
within 60 days of trial period end date
ACQUSI_EMTELEM60 Interaction of email and telemarketing sent to customer
within 60 days of trial period end date
ACQUSI_DMTELEM60 Interaction of direct mail and telemarketing sent to
customer within 60 days of trial period end date
ACQUSI_EMDMTELEM Interaction of email, direct mail and telemarketing sent to
60 customer within 60 days of trial period end date
ACQUSI_EMDM_gt60 Interaction of email and direct mail sent to customer
beyond 60 days of trial period end date
ACQUSI_EMTELEM_gt Interaction of email, direct mail and telemarketing sent to
60 customer beyond 60 days of trial period end date
ACQUSI_DMTELEM_gt Interaction of email, direct mail and telemarketing sent to
60 customer beyond 60 days of trial period end date
ACQUSI_EMDMTELEM Interaction of email, direct mail and telemarketing sent to
_gt60 customer beyond 60 days of trial period end date
AcqSelfcare_dummy 1 if customer uses any self-care channel, 0 otherwise
Variables unique to retention/churn period
 68
(Table 6, continued)
Service_count Number of services that each customer had during paid
period
MC_Ave_Price Mean-centered average price a customer paid during
MC2_Ave_Price Quadratic form of MC_Ave_Price
Price_range The difference between the maxi and mini price a
customer paid
Promo_count Number of promotions a customer gets during paid period
 69

CHAPTER 6

RESULTS AND DISCUSSIONS

6.1 Variable Importance in Four Events

Four variable importance graphs were produced by random forests to exhibit the

30 most important variables for each of the four events: conversion, churn, winning back

of non-converted customers, and wining back of churned customers. Studying each of

them, I find:

1) For conversion period (see Figure 1 below): a) the 30 most predictive variables

are dominated by text mining variables from LIWC of both linguistic words and

psychological vocabulary regardless of whether they are within or beyond 60 days of trial

period end date. This result indicate that analyzing what and why customers made phone

calls to customer representatives is of critical importance in predicting conversion. b)

Other factors that play roles in converting customers include: marketing actions such as

the total number of outbound communications that company sent to customer beyond 60

days of trial period end date and customers’ responsiveness to them, the number of direct

mail that the company sent to customer within 60 days of the trial period end date, as well

as the number of phone calls that the company made to customers beyond the 60 days of

the trial period end date. This result shows that during the early stage of the trial period

when customers were trying out the company’s product, the additive effect of company’s

communications as well as whether customers are responsive to company’s marketing

 70

efforts are important in affecting customers’ conversion. In addition, marketing

communications by phone play more important role during this period than other

communication methods. However, when it is close to the date that trial period ends,

direct mail are more effective in converting customers. c) The duration of customers’ trial

period, the radio density in each zip code, whether customers apply self-care channel and

how many times they called customer care center are all also highly indicative of whether

customers would convert to paid customer or not. This result is consistent with the prior

research findings. d) Among all the deactivation reasons that customers report, only

“Free_ended” play an important role in predicting whether customers would convert or

not. This result supports Hypothesis 8b and shows that the major reason that customers

would not convert was due to the end of free trial period.
 71

Figure 1 Conversion Variable Importance Plot

2) For retention period: a) Firm’s marketing initiative variables play dominant

roles in shaping customers decision making on whether to churn or stay with the

company. The total number of outbound communications that company sent to customer

during the paid period regardless of the time frame, the total marketing communications

occurring during the trial period, the number of times the company reach customers by

email, direct mail, or phone call both within and beyond the paid period end date, as well

as the number of promotions each customers gets are all highly predicative of customers’

churn or no-churn behavior. b) From customer side, whether customers use self-care

services, their responsiveness to firm’s marketing actions throughout the whole retention
 72

period, the number of services they had with the company as well as the average price

and price range associated with them, both the trial period duration and paid period

duration, are all differentiating characteristics that reveal customers’ future behavior

direction. c) A few psycholinguistic variables such as the total number of function words,

the number of verbs, prepositions as well as the number of relativity words (e.g. area,

bend, exit, stop) that appear in the call center notes are also good predictor of whether

customer will churn or not.

 73

Figure 2 Churn Variable Importance Plot

3) To win back the customers who fail to become paid customers, the factors that

used to influence their initial decisions of conversion or non-conversion still play a large

role. Even though the number of text mining variables decrease by 2 and their importance

ranking also declined compared with marketing variables, they still constitute nearly half

of the top 30 important variable list which shows the importance of investing in studying

the content of calls that customers made during their trial period. Radio density becomes

the top one factor influencing whether company can win back customer or not, which
 74

indicates the regional difference in availability and variety of the product is the major

aspect that firms should consider when they allocate the marketing resources regarding

who should be won back. The total number of marketing initiatives and customers’

responsiveness to it, the trial period duration, as well as the communication channels that

the firm uses to reach out customer are all effective in winning back the non-converted

customers.

Figure 3 Non-Converted Winback Variable Importance Plot

 75

4) For winning back churned customers, whether customers used self-care during

both trial and paid periods, the number of promotions that customers used to get during

the paid period, customers’ responsiveness to marketing communications throughout the

retention period, the average price and price range that customers used to get during the

retention period, customers’ trial period duration, customers’ total service number with

the company, as well as firm’s marketing communication variables are all highly

predictive of whether customers could be won back or not. Particularly the number of

promotions that customers get during the paid period became the top predictor of

customer churn or not, which could imply that the customers who churned were mostly

price sensitive. The larger number of promotions each customer gets, the more price-

sensitive they are, and the more difficult for the firm to reacquire them. A few

psycholinguistic variables such as “cogmech”, “social”, functional words and verbs are

also among the top 30 predictors in winning back churned customers.

 76

Figure 4 Churned Winback Variable Importance Plot

5) Examining all four variable importance graphs together, I discovered that: all

marketing action variables, whether customers use self-care or not, customer duration in

both conversion and retention periods are strong predictors in classifying converted and

unconverted customer, churned and non-churned customer, as well as the customers that

could be won back or not. Finally, nearly all deactivation variables except ‘Free_ended”

are missing from the top 30 list. Some of them are significant in predicting one or two

events (see Table 9 below) but compared with other variables, they are not that
 77

‘important’ in terms of predicting customer conversion, churn and reacquisition.

Therefore H8a is not supported.

6.2 Hypothesis Testing

Random forests only give us the overall importance ranking of all variables in

terms of increasing classification accuracy. To examine individual variables’ marginal

effect and some interaction effects among certain variables on the probability of four

events, namely conversion, churn, winning back unconverted customers and winning

back churned customer, I rely on logistic regression to provide the parameter estimates

and their statistical significance.

6.2.1 Self-care

The major results for the analysis of self-care channels, and related activities, are

shown in Tables 7 and 8. Studying these two tables I find:

1) Usage of self-care: Whether customers employing self-care channel or not

during their trial period and subsequent paid period are strongly significant (p-value <

0.01) in predicting all four events: customer conversion, customer churn, customer non-

converted winback and customer churn winback. However the only positive effect of

whether customers adopting self-care is for winning back churned customers (β = 0.3862,

p-value < 0.01), which barely supports H1a. Customers using self-care during their trial

period actually decreases their likelihood of converting from free to fee and subsequent

reactivation of service with the company (β = -0.5640 and -0.7260 respectively, p-value <

0.01). In addition, the employment of self-care during the retention period increases

customer churn probability. All these negative effects of using self-care could suggest
 78

customers mainly voluntarily contact the service-provider when they encounter problems

and they use self-care to seek help from the company. To further investigate if this

assumption is true or not, I proceed to examine the interaction and marginal effects of

customer’s usage of each three self-care channel (e.g. call live agent, use online help

channel, or use interactive voice message) and the activity they performed (eg. Change

account information, change service plan, or make payment).

2) Interaction and marginal effects of self-care channel usage and activities: The

interactions between self-care channels and self-care activities here are not the

multiplication of their marginal effects but the number of times two events happen

concurrently in a given time period. It captures what activity a customer performed

through which channel at a given point of time. For example,

‘Retent_LiveAg_MakePa60’ refers to the count of times that a customer called a live

agent to make payment within 60 days of their churn or the right-censored date of March

1st, 2012. For Table 7, I discover that the estimates of all marginal effects and the

interaction of self-care channel and self-care activities are insignificant except the

‘Retent_LiveAg_MakePa60’. Furthermore, the parameter estimate shows a negative

value with regard to churn probability (β = -0.1722, p-value < 0.05). So statistically the

more a customer makes phone calls to a customer representative to make payment within

60 days of their churn period end date, the lower the probability he or she will churn. The

result of overall performance of the interaction and marginal effects of self-care channel

usage and activities fail to support H1b.

 79

Table 7 Self-care Channel Activity in 4 events

Variables Conver- NonCov Variables Churn Churn
sion erted WB
WB
AcqSelfcare_dummy - - Retent_Selfcare_dummy 0.4007* 0.3862*
0.5640* 0.7260* * *
* *
Acquis_Live_Agent60 -0.0655 -0.0453 Retent_Live_Agent60 0.0132 -0.0233
Acquis_CUWI60 0.7442 -6.5272 Retent_CUWI60 -0.1436 -0.0527
Acquis_IVR60 0.1764 -5.5450 Retent_IVR60 -0.1299 -0.0340
Acquis_Live_Agent_gt60 0.3278 3.1967 Retent_CUWI_gt60 0.0143 -0.0148
Acquis_CUWI_gt60 -0.1110 -1.1663 Retent_Live_Agent_gt60 0.0445 -0.1021
Acquis_IVR_gt60 -0.0105 -0.5491 Retent_IVR_gt60 0.0205 0.0349
Acquis_ChangeAcctInf60 -0.3589 -1.1416 Retent_Change_Servic60 0.1265 0.1619
Acquis_Change_Service60 -0.7268 5.5601 Retent_Make_Payment60 0.1602 0.0615
Acquis_Make_Payment60 0 0 Retent_ChangeAcctInf60 0 0
Acquis_Make_Payment_gt 0.0649 -3.2603 Retent_ChangeAcctInf_gt60 -0.0955 -0.0271
60
Acquis_ChangeAcctInfo_g -0.3300 -3.0921 Retent_Change_Servic_gt60 -0.0475 0.0909
t60
Acquis_Change_Service_g 0 0 Retent_Make_Payment_gt60 0 0
t60
Acq_LiveAg_ChangeInf60 0.1333 1.1226 Retent_LiveAg_Change60 -0.1378 -0.1446
Acq_LiveAg_ChangeSer6 0.5929 -5.6631 Retent_LiveAg_MakePa60 - -0.0335
0 0.1722*
Acq_LiveAg_MakePaymt6 0 0 Retent_CUWI_ChangeIn60 0.1437 0.0377
0
Acq_CUWI_MakePaymt6 -0.6080 7.2858 Retent_CUWI_ChangeSe60 0.00349 -0.0979
0
Acq_CUWI_ChangeInfo60 -0.5649 7.5668 Retent_CUWI_MakePaym60 0 0
Acq_IVR_MakePaymt60 0 0 Retent_LiveAg_Change60 0 0
Acq_CUWI_ChangeSer60 0 0 Retent_IVR_ChangeSer60 0 0
Acq_IVR_ChangeSer60 0 0 Retent_IVR_MakePaymt60 0 0
Acq_IVR_ChangeInfo60 0 0 Retent_IVR_ChangeInf60 0 0
Acq_LiveAg_MakePaymt_ -0.3953 0.0372 Retent_CUWI_ChangeIn_gt6 0.0713 0.0220
gt60 0
Acq_CUWI_ChangeInfo_g 0.4800 4.4124 Retent_CUWI_ChangeSe_gt6 0.0378 -0.0457
t60 0
Acq_CUWI_ChangeSer_gt 0.0470 1.0053 Retent_LiveAg_MakePa_gt60 -0.0464 0.0983
60
Acq_LiveAg_ChangeSer_ -0.2183 -2.9024 Retent_CUWI_MakePaym_gt 0 0
gt60 60
Acq_CUWI_MakePaymt_ 0 0 Retent_LiveAg_Change_gt60 0.0522 0.0840
gt60
Acq_LiveAg_ChangeInf_g 0 0 Retent_LiveAg_Change_gt60 0 0
t60
Acq_IVR_MakePaymt_gt6 0 0 Retent_IVR_ChangeSer_gt60 0 0
0
Acq_IVR_ChangeInfo_gt6 0 -3.0754 Retent_IVR_ChangeInf_gt60 0 0
0
Acq_IVR_ChangeSer_gt60 0 0 Retent_IVR_MakePaymt_gt6 0 0
 80
(Table 7, continued)
0
*p<.05
** p<.01

6.2.2 Marketing Initiatives

The results of analyzing marketing initiative variables in Table 8 suggest the

following:

1) Total marketing communication outbound: the parameter estimate of trial

period marketing volumes on conversion and churn (β = 0.00566, and -0.0169

respectively, p <0.001) indicates that there is a positive association between trial period

marketing volumes and conversion and retention. Also the marketing volume beyond the

60 days increase the probability of winning back both unconverted and churned

customers; hence the H2a is largely supported. For the marketing communication volume

in the churn period, the more outbound marketing, the higher the probability that the

customer will churn. This supports H2b and is also consistent with the reciprocal action

theory and reactance theory explanations.

Table 8 Marketing variable estimates

Variables Convers NonCon- Variables Churn Churned

-ion version Winbac
Winback k
Acquis_MDBcount60 0.00566** 0.000845 Retention_MDBcount60 0.0052** 0.00045
Acquis_MDBcount_gt6 0.0393** 0.0584** Retention_MDBcount_gt - 0.00149**
0 60 0.0463**
Trial60Response_Rate 0.0332 0.00964 Churn60Response_Rate 1.0879** -0.246**
Trial_gt60Response_Ra 0.3758** 1.7549** Churn_gt60Response_R 5.4881** -0.674**
te ate
ACQUSI_EM60 -0.0103* 0.00502 RETENTION_DM60 -0.016** -0.015**
 81
(Table 8, continued)
ACQUSI_DM60 -0.0130* -0.0274 RETENTION_EM60 -0.002** -0.003**
ACQUSI_TELEM60 -0.0797** -0.0996 RETENTION_TELEM6 - -0.020**
0 0.0157**
ACQUSI_EMDM60 0.00438 0.00925 RETENTION_EMDM6 0.00077* 0.000247
0 *
ACQUSI_EMTELEM6 0.0156* -0.00343 RETENTION_EMTELE 0.00051* 0.000182
0 M60
ACQUSI_DMTELEM6 0.0335** 0.0383 RETENTION_DMTELE 0.00363* 0.0061**
0 M60
ACQUSI_EMDMTEL -0.00263 -0.00395 RETENTION_EMDMT -0.00006 -0.00011
EM60 ELEM60
ACQUSI_EM_gt60 -0.00096 0.0124 RETENTION_DM_gt60 - -0.03**
0.0271**
ACQUSI_TELEM_gt6 -0.1833** 0.0239 RETENTION_EM_gt60 - -0.0026**
0 0.0026**
ACQUSI_DM_gt60 -0.0137** 0.0399 RETENTION_TELEM_ - -0.0687**
gt60 0.0296**
ACQUSI_EMDM_gt60 0.00130 -0.0150 RETENTION_EMDM_ 0.000425 0.00073
gt60
ACQUSI_EMTELEM_ 0.00676** 0.00295 RETENTION_EMTELE 0.00109* 0.00109*
gt60 M_gt60 *
ACQUSI_DMTELEM_ 0.0142** -0.0152 RETENTION_DMTELE 0.0116** 0.0325**
gt60 M_gt60
ACQUSI_EMDMTEL - 0.00207 RETENTION_EMDMT 2.44E-6 -0.00049
EM_gt60 0.00067** ELEM_gt60
Acq_MDBcount - -0.01**
0.0169**
Acq_MDB_ResRate - -0.164**
0.2058**
Price_range 0.0223** 0.0156**
Promo_count 0.4544** 0.5644**
MC_Ave_Price - -0.018**
0.0084**
MC2_Ave_Price 0.00017* 0.0004**
*
*p<.05
** p<.01

2) The response rate of customers to marketing efforts: all response rates except

customers’ response rate within 60 days of trial end period are highly significant in

predicting either conversion, or churn or winning back both unconverted customers and

churned customers. Moreover high response rate during the trial period beyond the 60

days of trial end date increases the chance of conversion but the response rate within 60
 82

days of trial end date is insignificant in predicting conversion. This could be explained by

the firm’s specific marketing action that they always send mail to customers 30 days

before their trial end date to remind them to convert to paid customer. Thus the increase

in marketing volume and in the response rate within the 60 days of the end of conversion

period fail to be effective to influence customers’ conversion or winback since customers

could depict the increase in marketing volume near their contract end date not real

investment in the relationship but compelling message to persuade them to become paid

customers. Customers’ response rate could also be forced to be high since they have to

communicate whether to terminate the service or convert to paid customer. Therefore H3

is partially supported.

3) The interaction and marginal effects of communication channels: From Table 8

we see that all communications channels, no matter whether looking at interaction or

marginal effects are not significant in predicting winning back unconverted customers.

This could suggest that the unconverted customers were the ones who were not really

interested in the product that the company provides so they could resist the company’s

marketing effort. The interaction effects of all three channels both within and beyond 60

days of both trial end date and churn end date are not significant in predicting conversion,

churn and winback, therefore, the synergy or multiplicative effect that prior researchers

((Naik and Raman 2003, Godfrey et al 2011) found in their studies is not supported by

this study. On the contrary, the additive effect of multichannel communication (denoted

as the marketing volume in this dissertation) is highly significant in influencing

customer’s decision on conversion, churn and winback. This result is consistent with the
 83

random forests importance ranking. As to the marginal effects of communication

channels, there is difference in each individual channel’s effect on customer conversion

and churn with the telemarketing being the most effective one. Therefore H4a is supported

by the evidence from this study but H4a is not.

4) Promotional depth: the parameter estimates for both Promo_count and

Price_range are highly significant in predicting customer churn and winback. Particularly,

the positive values for all four coeffients indicate that the more discounts that a customer

gets the more likely he is going churn and be won back. Therefore both H6a and H6b are

partially supported.

5) Average price a customer pays per month: A negative main effect and a

positive quadratic coefficient for the average price in predicting both churn and winback

shows that there is indeed a U-shape relationship between the average price that a

customer pays and the probability that he or she could be reacquired. Therefore H7b is

fully supported by the result. In the mean time, the average price that a customer pays per

month does have a negative impact on their churn. Therefore, the higher the price a

customer pays each month, the less likely he or she will churn. However, when this

diminishing effect reaches a certain point, the higher price will result in higher customer

churn. So H7a is partially support. Both results imply that: in order to win back lost

customers, company should target at the both low-value and high-value customers since

they are the ones who are most likely to be recapture. To reduce customer churn,

company has to control price increases so that they will not drive their most valuable

customers away.
 84

6.2.3 Radio Density

The results of the analysis of the effect of radio density on customer conversion,

churn and winback are displayed in Table 9. The negative coefficient of the Radio_Total

on predicting conversion shows that the availability and variety of products are not the

reason that customers convert from free to fee. In addition, the insignificant coefficient of

Radio_Total on predicting churned customer partially supports the H5b. Finally the

negative effect of radio density on customer churn partially supports H5a. In short, radio

density can indeed decrease customer churn and does not show effect in winning back

churned customers.

6.2.4 Duration on Winback

The positive main effects and negative quadratic coefficients of acquisition

duration on predicting winning back both unconverted customers and churned customers

indicate there is indeed an inverted U-shape relationship between prior duration and the

customer winback. This result indicates that companies should not target at the customers

who have the extremely short and long duration with them but focus on the customer

with medium duration.

Table 9 Variables predicting all 4 events

Variables Converted NonChurn NonConversio Churned

n Winback Winback
AcqSelfcare_dummy -0.5640* 0.8468* -0.7260* 0.679*
Acquisition_duration 0.00593* 0.00386* 0.0112* 0.00241
Acquisition_duration2 -0.00088* 0.00226* -0.00261* -0.00075*
 85
(Table 9, continued)
RADIO_TOTAL -0.00546* 0.000638* 0.0031* -.00046
Competition -0.5024 -0.0489 -8.2520 -0.2573
Cost_price_fee 0.3325 -0.0988 -7.7211 -0.0824
Free_ended -0.0684* -0.1387* -0.5755* -0.0233
NonPay -0.2629* -0.1011* -0.2417 -0.0426
Non_Usage -0.3807* -0.1042* -7.5816 -0.0903
Product_Dissatisfactio -0.2973 -0.0774 -7.9545 0.0275
n
Service_Dissatisfaction 0.0439 -0.1337* -1.2322 -.00275
Uncontrollable_reason -0.2351* -0.1283* -0.2361 -0.0621
Unknown_reason -0.1432* -0.1233* -0.5982* -0.1003
*p<.05
 86

CHAPTER 7

CONCLUSIONS, LIMITATIONS AND FUTURE DIRECTIONS

This study aims at exploring how different factors play roles during the three

stages of CRM by identifying the most influential factors for each stage, namely

acquisition, retention and winback of lost customers, and examining how each factor’s

marginal effect evolve in shaping customers’ decision making of conversion, churn and

reactivation of service with the company. With the grant of a big CRM data from a large

multi-media company in the country through WCAI, which covers all three stage of

CRM activities as well as the text data from call center notes, this dissertation examines

how customer characteristics variable, marketing action variables, psycholinguistic

variables from the text mining of call center notes, as well as the customer self-reported

deactivation reasons impact customers’ decision making through three stages of CRM of

whether to convert to a paid customer, to churn, or to reactivate their service with the

firm. In the mean time, applying a new machine learning method – random forest with its

variable importance ranking, this dissertation also shed light on which factors impact

customer conversion, churn and winback most among all the factors from both customer

side and firm side. In summary, this study obtained the following findings.

Findings and Implications

This study reveals that: 1) during the trial period of CRM, the content of the

customer call center notes are the top indicator of whether customers would convert or

not; 2) after customer have become paid customers, the marketing actions that the

company takes, which include the volume they communicate with the customer, the
 87

depth of the promotion they offer to each customer, the communication channels they

apply to reach out customers, as well as the radio density in different areas, all play major

roles in retaining customers and preventing them from churn; 3) the factors that influence

the most whether the company can win back the lost customers who only tried their

products but never converted to paid customers are the same ones that affect conversion

most; and the factors that are most predictive of churn or no-churn are also highly

foretelling of company’s chance of recapturing the churned paid customers; 4) customer

self-reported deactivation reasons are not top predictors compared with other factors

throughout all three stages of CRM. Since all deactivation reasons bear the negative

impact that could affect company’s CRM and it is company-specific, so the no effect of

these factors could be explained by the lack of service dissatisfaction or product

dissatisfaction or competition issue for the specific company I study with. 5) Customer

usage of self-care service increases the likelihood of customer retention and winning back

churned customers but decreases the likelihood of customer conversion and winning back

unconverted customers. 6) both acquisition and retention durations affect all three events

of conversion, churn and winback and they both have nonlinear relationship with the

probability of these three events. 7) The multiplicative effect of marketing

communications that were discovered by prior researchers is not significant in affecting

customer conversion, churn or winback. On the contrary, the additive effect of

communication channels plays a major role in shaping customers’ decision making of

conversion, churn and winback.

 88

As companies strive to cut costs and to maximize their profit by acquiring as

many customers as possible and retain them as long as they can, this study helps them to

identify which factors are more important in shaping customers’ decision making of

whether to stay with the company or terminate the service can help company better

allocate their resources. In addition, conducting a comprehensive analysis of CRM in

terms of data range and variety, as well as the possible factors that impact customer

acquisition, retention and winback, largely enriched the CRM literature and especially the

understudied winback literature. However, this study bears some limitations.

Limitations and Future Directions

There are several limitations that this study bears: first, this study only focuses on

predicting whether customer would convert, churn or be won back, but did not have

apply the same factors to calculate customer’s lifetime value which is what the CRM in

each company is most concerned with. Therefore, one future direction is to extend the

current research to customer lifetime value analysis. Second, due to time limitation, this

study could not conduct real time series study with machine learning methods but merely

divided the data into two time frames, namely within and beyond 60 days of trial and

paid period. Hence the second possible direction is to explore machine learning methods

that could deal with the real time series data. Third, the psycholinguistic analysis of the

customer’s conversion, churn and winback is limited and I included all 64 LIWC

variables into the prediction of conversion, churn and winback. A possible extension is to

only select certain variables from the 64 LIWC variables and examine how different

psycholinguistic factors evolve throughout the three stages of CRM and their impact on
 89

customers’ decision making. Finally, cross-selling or cross-buying of customers is

another important aspect in CRM, but I only used the data for customers who owns one

vehicle. Therefore, including the customers with more than one vehicle into analysis of

cross-buying can be another direction for my future research.

Summary

Even though CRM research has flourished in marketing with the availability of

CRM database, none of the existing research has examined all three stages of CRM

together and how factors’ effect evolve throughout customer acquisition, retention and

winback. Extremely under researched is the customer winback but as studies have shown

winback can greatly cut company’s cost and achieve greater return. This dissertation

enriched the literature in CRM and makes managerial contribution to companies’

customer relationship management.

 90

REFERENCES

Aitchison, J., and S. D. Silvey, (1957). “The Generalization of Probit Analysis to the
Case of Multiple Responses,” Biometrika. 57, 253–262.

Albert, J.H, and S. Chib, (1993). “Bayesian Analysis of Binary and Polychotomous
Response Data,” Journal of the American Statistical Association. 88(422):669–679.

Allenby, Greg M., Robert P. Leone, et al. (1999). “A Dynamic Model of Purchase
Timing with Application to Direct Marketing,” Journal of American Statistical
Association. 94 (June). 365–74.

Allwein, E., R. Schapire, et al. (2000). “Reducing multiclass to binary: a unifying

approach for margin classifiers,” Journal of Machine Learning Research. 1, 113–141.

Amit, Y. and Geman, D. (1997). “Shape quantization and recognition with randomized
trees,” Neural Computation. 9, 1545–1588.

Anderson, Eric and Duncan Simester (2004), "Long-Run Effects of Promotional Depth
on New Versus Established Customers: Three Field Studies," Marketing Science, 23 (1),
4-21.

Andrews, R. L., A. Ainslie, et al. (2002). “An empirical comparison of logit choice
models with discrete versus continuous representations of heterogeneity,” Journal of
Marketing Research. 39(4):479–487.

Bake, L. D. and A. K. McCallum, (1998). “Distribustional clustering of words for text

classification,” In Proceedings of SIGIR–98, 21ST ACM International Conference on
Research and Development in Information Retrieval. 96–103.

Bass, F. M. (1969). “A new product growth for model consumer durables,” Management
Science. 15(5):215–227.

Bass, F. M., N. Bruce, et al. (2007). “Wearout effects of different advertising themes: A
dynamic Bayesian model of the advertising–sales relationship,” Marketing Science.
26(2):179–195.

Berger, P., T. Magliozzi. (1992). “The effect of sample size and proportion of buyers in
the sample on the performance of list segmentation equations generated by regression
analysis,” Journal of Direct Marketing. 6(1):13–22.

Blattberg, Robert C. and John Deighton (1996). “Manage Marketing by the Customer
Equity Test,” Harvard Business Review. 74 (July/August). 13 6–44.
 91

Bock, H–H., (2002). The Goal of Classification. Handbook of Data Mining and
Knowledge Discovery. 254–258, Oxford University Press.

Bolton, R. N. (1998). “A dynamic model of the duration of the customer's relationship

with a continuous service provider: The role of satisfaction,” Marketing Science.
17(1):45–65.

Bottou, L. C. Cortes, J. Denker, et al. (1994). “Comparison of classifier methods: A case

study in handwriting digit recognition,” in Proc. Int. Conf. Pattern Recognition. 77–87.

Boutell, M. R., J. Luo, et al. (2004). “Learning multi–label scene classification,” Pattern
Recognition. 37(9):1757–1771.

Boulding, W., A. Kalra, et al. (1993). “A Dynamic Process Model of Service Quality –
from Expectations to Behavioral Intentions,” Journal of Marketing Research 30(1):7–27.

Boyer, K.K., Hallowell, R., et al. (2002). “E–Services: operating strategy—a case study
and a method for analyzing operational benefits, “ Journal of Operations Management.
20 (2):175–188.

Brank J., Grobelnik M., et al. (2002). “Interaction of Feature Selection Methods and
Linear Classification Models,” Proc. of the 19th International Conference on Machine
Learning, Australia, 2002.

Braun, M. and D. A. Schweidel (2011). “Modeling Customer Lifetimes with Multiple

Causes of Churn,” Marketing Science 30(5):881–902.

Breiman, Leo (1996). “Bagging predictors,” Machine Learning. 24(2):123–140.

Breiman, Leo (1998). “Arcing Classifiers,” The Annals of Statistics. 26(3):801–849.

Breiman, Leo (2001). “Random Forests,” Machine Learning. 45(1):5–32.

Berger, Paul, Ruth Bolton, Douglas Bowman, Elten Briggs, V. Kumer, A. Parasuraman,
and Creed Terry (2002), "Marketing Actions and the Value of Customer Assets: A
Framework for Customer Asset Management," Journal of Service Research, 5 (1), 39-55

Cecchini, M., H. Aytug, et al. (2010). “Detecting Management Fraud in Public

Companies,” Management Science. 56(7):1146–1160.

Campbell, D. and F. Frei (2010). “Cost Structure, Customer Profitability, and Retention
Implications of Self–Service Distribution Channels: Evidence from Customer Behavior
in an Online Banking Channel,” Management Science 56(1):4–24.
 92

Crammer, K. and Y. Singer (2000). “On the learnability and design of output codes for
multiclass problems,” Comput. Learing Theory. 35–46

Cui, D. P. and D. Curry (2005). “Prediction in marketing using the support vector
machine,” Marketing Science, 24(4):595–615.

Cui, G., M. L. Wong, et al. (2006). “Machine learning for direct marketing response
models: Bayesian networks with evolutionary programming,” Management Science,
52(4):597–612.

Danaher, P. J. (2002). “Optimal pricing of new subscription services: Analysis of a

market experiment,” Marketing Science. 21(2):119–138.

Darden, W. R. and W. D. Perreault, (1977). “Classification for Market–Segmentation –

Improved Linear–Model for Solving Problems of Arbitrary Origin,” Management
Science. 24(3):259–271.

Davis, D. 1989. Perceived usefulness, perceived ease of use and user acceptance of
information technology. MIS Quart. 13(3).318–339.

Davis, D., R. P. Bagozzi, P. R. Warshaw (1989). User acceptance of computer

technology: A comparison of two theoretical models. Management Science, 35(8).982–
1003.

Deerwester, S., S. T. Dumais, et al. (1990). “Indexing by latent semantic indexing,”

Journal of the American Society for Information Science. 41(6):391–407.

Dillon, W. R. (1979). “Performance of the Linear Discriminant Function in Non–Optimal

Situations and the Estimation of Classification Error Rates – Review of Recent Findings,”
Journal of Marketing Research. 16(3):370–381.

Dillon, W. R., M. Goldstein, et al. (1978). “Appropriateness of Linear Discriminant and

Multinomial Classification Analysis in Marketing–Research,” Journal of Marketing
Research. 15(1):103–112.

Elisseeff, A., and J. Weston, (2002). “A kernel method for multi–labelled classification,”
Advances in Neural Information Processing Systems. 14.

Fader, P. S., B. G. S. Hardie, K. L. Lee, (2005). “RFM and CLV: Using iso–value curves
for customer base analysis,” Journal of Marketing Research, 42(4):415–430.

Fitzsimmons, J.A. and M.J. Fitzsimmons. (2004). Service Management: Operations,

Strategy, and Information Technology. Irwin–McGraw Hill, New York, NY.
 93

Fletcher, Keith, and Colin Wheeler, and Julia Wright (1992). “Success in Database
Marketing: Some Critical Factors,” Marketing Intelligence & Planning. 10, 18–23.

Forman, G., (2003). “An Experimental Study of Feature Selection Metrics for Text
Categorization,” Journal of Machine Learning Research. 3:1289–1305.

Friedman J. H. (1996). “Another Approach to Polychotomous Classification”. Dept.

Statist., Stanford University., Stanford, CA. [Online]. Available: http://www–
stat.stanford.edu/reports/friedman/poly.ps.Z.

Friedman J. H. (1997). “Data Mining and Statistics: What’s the connection?” Available at:
http://www.stat.standford.edu/~jhf/.
Friedman J, Hastie T, Tibshirani R (2000). “Additive Logistic Regression: A Statistical
View of Boosting,” The Annals of Statistics. 28(2):337–407.
Friedman J. H. (2001). “Greedy Function Approximation: A Gradient Boosting Machine,”
The Annals of Statistics. 29(5):1189–1232.
Friedman J (2002). “Stochastic Gradient Boosting,” Computational Statistics & Data
Analysis, 38(4):367–378. doi:10.1016/S0167–9473(01)00065–2.
Freund Y, and R. Schapire. (1996). “Experiments with a New Boosting Algorithm,” In
International Conference on Machine Learning. 148–156.
Freund, Yoav and Robert E. Schapire (1997); “A Decision–Theoretic Generalization of
On–Line Learning and an Application to Boosting,” Journal of Computer and System
Sciences. 55(1):119–139

Froehle C., A. Roth. 2004. New measurement scales for evaluating perceptions of the
technology-mediated customer service experience. Journal of Operations. Management
22(1) 1–21.

Ganesh, J., M. J. Arnold, et al. (2000). “Understanding the customer base of service
providers: An examination of the differences between switchers and stayers,” Journal of
Marketing. 64(3):65–87.

Gelman, Andrew, J. B. Carlin, H.S. Stern, and D. B. Rubin (2004) Bayesian Data
Analysis, Second Edition, Chapman & Hall/CRC.

Ghose, A. and P. G. Ipeirotis (2011). “Estimating the Helpfulness and Economic Impact
of Product Reviews: Mining Text and Reviewer Characteristics,” IEEE Transactions on
Knowledge and Data Engineering. 23(10):1498–1512.
 94

Godbole, S., and Sarawagi, S. (2004). “Dis¬criminative methods for multi–labeled

classification,” Proceedings of the 8th Pacific–Asia Conference on Knowledge Discovery
and Data Mining.

Godfrey, A., K. Seiders, et al. (2011). “Enough Is Enough! The Fine Line in Executing
Multichannel Relational Communication,” Journal of Marketing. 75(4):94–109.

Goolsbee, A., P. J. Klenow. (2002). “Evidence on learning and network externalities in

the diffusion of home computers,” Journal of Law Economics. 45(2):317–343.

Grewal, R., M. Chandrashekaran, et al. (2010). “Customer Satisfaction Heterogeneity and

Shareholder Value,” Journal of Marketing Research. 47(4):612–626.

Griffin, Jill and Michael W. Lowenstein( 2001). Customer Win–back: How to Recapture
Lost Customers–And Keep Them Loyal. San Francisco: Jossey–Bass.

Guadagni, P. M., J. D. C. Little. (1983). “A logit model of brand choice calibrated on

scanner data,” Marketing Science. 2(3).203–238.

Gupta, S., D. R. Lehmann, et al. (2004). “Valuing customers,” Journal of Marketing

Research. 41(1):7–18.

Gurland, J., I. Lee, et al. (1960). “Polychotomous Quantal Response in Biological Assay,”
Biometrics. 16, 382–398.

Gustafsson, A., M. D. Johnson, et al. (2005). “The effects of customer satisfaction,

relationship commitment dimensions, and triggers on customer retention,” Journal of
Marketing. 69(4):210–218.

Haenlein, M., A. M. Kaplan, D. Schoder, (2006). “Valuing the real option of abandoning
unprofitable customers when calculating customer lifetime value,” Journal of Marketing.
70(3):5–20.

Hansotia, Behram and Paul Wang( 1997). “Analytical Challenges in Customer

Acquisition,” Journal of Interactive Marketing. 11 (Spring). 7–19.

Hastie, Trevor, Robert Tibshirani, Jerome Friedman (2009). The Elements of Statistical
Learning, Second Edition, Springer.

Hayes, P. J., P. M. Andersen, et al. (1990). “Tcs: a shell for content–based text
categorization,” In Proceedings of CAIA–90, 6th IEEE Conference on Artificial
Intelligence Applications. 320–326.
 95

Hill, T., M. OConnor, et al. (1996). “Neural network models for time series forecasts,”
Management Science. 42(7):1082–1092.

Hill, A.V., Collier, D.A., Froehle, et al. (2002),”Research opportunities in service process
design,” Journal of Operations Management. 20(2):189–202.

Hitt, L. M. and F. X. Frei (2002). “Do better customers utilize electronic distribution
channels? The case of PC banking,” Management Science. 48(6):732–748.

Ho, T. K. (1998). “The random subspace method for constructing decision forests,” IEEE
Trans. on Pattern Analysis and Machine Intelligence. 20(8). 832–844.

Hogan, John E., Donald R. Lehmann, Mario Merino, Rajendra K. Srivastava, Jacquelyn
S . Thomas, and Peter C. Verhoef (2002). “Linking Customer Assets to Financial
Performance,” Journal of Service Research. 5(August). 26–38.

Hornik, K., M. B. Stinchcombe, et al. (1989). “Multilayer feedforward networks are

universal approximators,” Neural Networks. 2: 359–366.
Jim Rosenfield (2002). “Customer relationship management: a brief history, and a big
mystery” Australasian Business Intelligence. Mar 31, 2002.

Kantor, J. R. (1953). The Logic of Modern Science. Bloomington IN: Principle Press.

Keaveney, S. M. (1995). “Customer Switching Behavior in–Service Industries – an

Exploratory Study,” Journal of Marketing. 59(2):71–82.

Kemeny, J. G. (1959). A Philosopher Looks at Science. New York: Van Norstand.

Kim, B. D., M. Shi, et al. (2001). “Reward programs and tacit collusion,” Marketing
Science. 20(2):99–120.

Kim, S. P., T. Pantel, et al. “Automatically Assessing Review Helpfulness,” Proceedings

of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP
2006). 423–430,

Kim, Y., W. N. Street, et al. (2005). “Customer targeting: A neural network approach
guided by genetic algorithms,” Management Science. 51(2):264–276.

Kinnear, T. C. and J. R. Taylor, (1971). “Multivariate Methods in Marketing Research –

Further Attempt at Classification,” Journal of Marketing. 35(4):56–59.

Kivetz, R. and I. Simonson (2002). “Earning the right to indulge: Effort as a determinant
of customer preferences toward frequency program rewards,” Journal of Marketing
Research. 39(2):155–170.
 96

Kivetz, R. and I. Simonson (2003). “The idiosyncratic fit heuristic: Effort advantage as a
determinant of consumer response to loyalty programs,” Journal of Marketing Research.
40(4):454–467.

Knerr, S., L. Personnaz, et al. (1990). Single–layer learning revisited: A stepwise

procedure for building and training a neural network, Neurocomputing: Algorithm,
Architectures and Applications, J. Fogelman, Ed. New York: Springer–Verlag, 1990.

Kreßel, U., (1999). Pairwise Classification and Support Vector Machines, in Advances in
Kernel Methods–Support Vector Learning, B. Schölkopf, C. J. C. Burges, and A. J.
Smola, Eds. Cambridge, MA: MIT Press, 1999, 255–268.

Lemmens, A. and C. Croux (2006). “Bagging and boosting classification trees to predict
churn,” Journal of Marketing Research. 43(2):276–286.

Lewis, D. D. (1992). “An evaluation of phrasal and clustered representations on a text

categorization task,” In Proceedings of SIGIR–92, 15th ACM International Conference
on Research and Development in Information Retrieval (Kobenhavn, DK, 1992). 37–50.
Lewis, M. (2004). “The influence of loyalty programs and short–term promotions on
customer retention,” Journal of Marketing Research. 41(3):281–292

Lewis, M. (2006). “Customer acquisition promotions and customer asset value,” Journal
of Marketing Research. 43(2):195–203.

Li, T., & Ogihara, M. (2003). Detecting emotion in music. Proceedings of the
International Symposium on Music Information Retrieval.

Li, Y. H. and A. K. Jain, (1998). “Classification of text documents,” The Computer

Journal. 41(8):537–546.

Liittschwager, J. M. and C. Wang, (1978). “Integer Programming Solution of a

Classification Problem,” Management Science. 24(14):1515–1525.

Luo, X., & Zincir–Heywood, A. N. (2005). “Evaluation of two systems on multi–class

multi–label document classifica¬tion,” Proceedings of the 15th Interna¬tional
Symposium on Methodologies for Intelligent Systems.

Mahajan, Vijay, Paul E. Green, S. M. Goldberg (1982). “A conjoint model for measuring
self– and cross–price/demand relationships,” Journal of Marketing Research. 19(3). 334–
342.

Mahajan, V., E. Muller, R. A. Kerin. (1984). “Introduction Strategy for New Products
with Positive and Negative Word–of–Mouth,” Management Science. 30(12):1389–1404.
 97

Malakooti, B. and Y. Q. Zhou, (1994). “Feedforward Artificial Neural Networks for

Solving Discrete Multiple Criteria Decision–Making Problems,” Management Science
40(11):1542–1561.
Mangasarian, O. L., and D. R. Musicant. (1999). “Successive overrelaxation for support
vector machines,” IEEE Transactions on Neural Networks. 10(5):1032–1037.

Maron, O., and Lozano–Perez, T. (1997). “A framework for multiple–instance learning,”

Proceedings of Neural In¬formation Processing Systems. (NIPS 1997).

McCullagh, P. (1980). “Regression Models for Ordinal Data,” Journal of the Royal
Statistical Society. Ser. B, 109–142.

McCullagh, P., and J. Nelder. (1983). Generalized Linear Models. Chapman and Hall,
London.

McGahan, A. M. and P. Ghemawat (1994). “Competition to Retain Customers,”

Marketing Science. 13(2):165–176.

McKelvey, R., and Zavoina, W. (1975). “A Statistical Model for the Analysis of Ordingal
Level Dependent Variables,” Journal of Mathematical Sociology. 4:103–120.

Mitchell, T.M. (1996). Machine Learning. McGraw Hill, New York, NY.

Musalem, A. and Y. V. Joshi (2009). “How Much Should You Invest in Each Customer
Relationship? A Competitive Strategic Approach,” Marketing Science. 28(3):555–565.
Nahshon Wingard, “CRM History: The Evolution Of Better Customer Service,”
http://www.streetdirectory.com/.

Nam, S., P. Manchanda, et al. (2010). “The Effect of Signal Quality and Contiguous
Word of Mouth on Customer Acquisition for a Video–on–Demand Service,” Marketing
Science. 29(4):690–700.

Norton, J. A. and F. M. Bass (1987). “A Diffusion–Theory Model of Adoption and

Substitution for Successive Generations of High–Technology Products,” Management
Science 33(9):1069–1086.

Oliveira, P., A.V. Roth, et al. (2002). “Achieving competitive capabilities in E–services,”
Technological Forecasting and Social Change. 69(7):721–731.

Petrison, Lisa A., Robert C. Blattberg, et al. (1997). “Database Marketing Past, Present,
and Future,” Journal of Direct Marketing. 7(3):27–43.
 98

Piramuthu, S., H. Ragavan, et al. (1998). “Using feature construction to improve the
performance of neural networks,” Management Science. 44(3):416–430.

Reinartz, W., J. S. Thomas, V. Kumar. (2005). “Balancing acquisition and retention

resources to maximize customer profitability,” Journal of Marketing. 69(1):63–79.

Reinartz, W. J. and V. Kumar (2003). “The impact of customer relationship

characteristics on profitable lifetime duration,” Journal of Marketing. 67(1):77–99.

Ridgeway G (2006). gbm: Generalized Boosted Regression Models. R package version

1.5–7, URL http://www.i–pensieri.com/gregr/gbm.shtml.

Riedmiller, M., and H. Braun. (1993). “A direct method for faster backpropagation
learning: the rprop algorithm,” Proceedings of the IEEE International Conference on
Neural Networks (ICNN). 1:586–591.

Ripley, B. D. (1996). Pattern recognition and neural networks. Cambridge University

Press.

Roehm, M. L., E. B. Pullins, et al. (2002). “Designing loyalty–building programs for

packaged goods brands,” Journal of Marketing Research. 39(2):202–213.

Rogers, E. M. (1995). Diffusion of Innovations, 4th ed. Free Press, New York.

Rosenblatt, F. (1958). “The Perceptron: A Probabilistic Model for Information Storage

and Organization in the Brain,” Psychological Review. 65: 386–408.

Roth, A.V. (2000). Service Strategy and the Technological Revolution: The 7 Myths of
E–Services. In: Machuca, J.A.D., Mandakovic, T. (Eds.). POM Facing the New
Millennium:
Evaluating the Past, Leading with the Present and Planning the Future of Operations.
POM Sevilla, 159–168.

Rust, RolandT ., ValarieA Zeithaml, and KatherineN Lemon (2004). “Return on

Marketing: Using Customer Equity to Focus Marketing Strategy,” Journal of Marketing.
68 (January). 23–53.

Rosset, S., Ji Zhu et al. (2004). “Boosting as a regularized path to a maximum margin
classifier”. Journal of Machine Learning Research. 5:941–973.
Rossi, P. E. and G. M. Allenby (1993). “A Bayesian–Approach to Estimating Household
Parameters,” Journal of Marketing Research. 30(2):171–182.

Rossi, P. E., G. M. Allenby, et al. (2005). Bayesian Statistics and Marketing. John Wiley
& Sons Ltd.
 99

Salton, G. and Buckley, C. (1988). “Term–weighting approaches in automatic text

retrieval”. Inform. Process. Man. 24(5):513–523. Also reprinted in Sparck Jones and
Willett [1997], 323–328.

Salton, G. and M. J. McGill, (1983). Introduction toModern Information Retrieval.

McGraw Hill.

Schapire, R. (1997). “Using output codes to boost multiclass learning problems,” in

Proceedings of the Fourteenth International Conference on Machine Learning. Morgan
Kauffman.

Schapire, R. and Singer, Y. (1999). “Improved boosting algorithms using confidence–

rated prediction,” Machine Learning 37: 297–336.

Schweidel, D. A., Peter. S. Fader, et al. (2008). “A Bivariate Timing Model of Customer
Acquisition and Retention,” Marketing Science. 27(5):829–843.

Sebastiani F. (2002). “Machine Learning in Automated Text Categorization,” ACM

Computing Surveys. 34 (1):1–47.

Shaw, R. and Stone, M. Database Marketing. New York: John Wiley & Sons, 1988.

Slonim, N. and N. Tishby, (2001). “The power of word clusters for text classification,” In
Proceedings of ECIR–01, 23rd European Colloquium on Information Retrieval Research
(Darmstadt, DE, 2001).

Soucy P. and Mineau G., (2003). “Feature Selection Strategies for Text Categorization,”
AI 2003, LNAI 2671, 505–509.

Sousa P., Pimentao J. P., et al. (2003). “Feature Selection Algorithms to Improve
Documents Classification Performance,” LNAI 2663, 288–296.

Stauss, Bernd and Christian Friege (1999). “Regaining Service Customers,” Journal of
Service Research. 1(4). 347–61.

Sun, M. H., A. Stam, et al. (1996). “Solving multiple objective programming problems
using feed–forward artificial neural networks: The Interactive FFANN Procedure,”
Management Science. 42(6):835–849.

Tellis, G. J. and F. S. Zufryden (1995). “Tackling the Retailer Decision Maze – Which
Brands to Discount, How Much, When and Why,” Marketing Science. 14(3):271–299.
 100

Thieme, R. J., M. Song, et al. (2000). “Artificial neural network decision support systems
for new product development project selection,” Journal of Marketing Research.
37(4):499–507.

Thomas, J. S. (2001). “A methodology for linking customer acquisition to customer

retention,” Journal of Marketing Research. 38(2):262–268.

Thomas, J. S., R. C. Blattberg, E. J. Fox, (2004). “Recapturing lost customers,” Journal of

Marketing Research. 41(1):31–45.

Tibshirani, R. (1996). “Regression Shrinkage and Selection via the Lasso,” Journal of the
Royal Statistical Society. Series B (Methodological). 58(1):267–288.

Torkkola K. (2002). “Discriminative Features for Text Document Classification,” Proc.

International Conference on Pattern Recognition, Canada.

Tsoumakas, G. (2007). “Multi–label classification: an overview,” International Joural of

Data Warehousing & Mining. 3(3):1–13.

Vapnik, V. (1998).Statistical Learning Theory. New York: Wiley, 1998.

Venkatesan, R. and V. Kumar (2004). “A Customer Lifetime Value Framework for

Customer Selection and Resource Allocation Strategy,” Journal of Marketing. 68
(October). 106–125.

Villanueva, J., S. Yoo, et al. (2008). “The impact of marketing–induced versus word–of–
mouth customer acquisition on customer equity growth,” Journal of Marketing Research.
45(1):48–59.
Walf, A. (1926). Essentials of Scientific Method. New York. Macmillan Company.

Wang, S. H. (1995). “The Unpredictability of Standard Back–Propagation Neural

Networks in Classification Applications,” Management Science. 41(3):555–559.

Wedel, M., W. A. Kamakura, et al. (2000). “Marketing data, models and decisions,”
International Journal of Research in Marketing. 17(2–3). 203–208.

West, P. M., P. L. Brockett, et al. (1997). “A comparative analysis of neural networks and
statistical methods for predicting consumer choice,” Marketing Science. 16(4):370–391.

Weston, J. and C. Watkins, (1999). “Multi–class support vector machines,” the Proc.
ESANN99. M. Verleysen, Ed., Brussels, Belgium, 1999.

Xue, M., L. M. Hitt, et al. (2011). “Determinants and Outcomes of Internet Banking
Adoption,” Management Science. 57(2):291–307.
 101

Yang, S., G. M. Allenby, et al. (2002). “Modeling variation in brand preference: The
roles of objective environment and motivating conditions,” Marketing Science. 21(1):14–
31.

Zeithaml, Valarie A., Berry, Leonard L. & Parasuraman, A. (1996). “The Behavioral
Consequences of Service Quality,” Journal of Marketing, 60, 31-46.

Zeithaml, V.A., A. Parasuraman, et al. (2002). “Service quality delivery through web
sites: a critical review of extant knowledge,” Journal of the Academy of Marketing
Science. 30(4):362–375.

Zhang, M.–L., and Z.–H. Zhou, (2005). “A k–nearest neighbor based algorithm for
multi–label classification,” Proceedings of the 1st IEEEInternational Confer¬ence on
Granular Computing.

Zhu, J., H. Zou, S. Rosset, et al. (2009). “Multi–class AdaBoost,” Statistics and Its
Interface. 2:349–360.

Zhu, K. and K. L. Kraemer (2005). “Post–adoption variations in usage and value of e–

business by organizations: Cross–country evidence from the retail industry,” Information
Systems Research. 16(1):61–84.

Zhu, K., K. L. Kraemer, S. Xu (2003). “Electronic business adoption by European firms:

A cross–country assessment of the facilitators and inhibitors,” European Journal of
Information Systems. 12(4):251–268.

Zou, H., and T. Hastie, (2005). “Regularization and variable selection via the elastic net,”
J.R. Statist. Soc B. 67(2):301–320.

Zou, H., J. Zhu, and T. Hastie, (2008). “New Multicategory Boosting Algorithms Based
on Multicategory Fisher–Consistent Losses,” The Annals of Applied Statistics.
2(4):1290–1306.
 102

Appendices:

Appendix A: LIWC2007 Output Variable Information

Category Abbrev Examples Words in category

Linguistic Processes
Total function words funct 464
Total pronouns pronoun I, them, itself 116
Personal pronouns ppron I, them, her 70
Impersonal pronouns ipron It, it’s, those 46
Articles article A, an, the 3
Common verbs verb Walk, went, see 383
Auxiliary verbs auxverb Am, will, have 144
Past tense past Went, ran, had 145
Present tense present Is, does, hear 169
Future tense future Will, gonna 48
Adverbs adverb Very, really, quickly 69
Prepositions prep To, with, above 60
Conjunctions conj And, but, whereas 28
Negations negate No, not, never 57
Quantifiers quant Few, many, much 89
Numbers number Second, thousand 34
Swear words swear Damn, piss, fuck 53
Psychological Processes
Social processes social Mate, talk, they, child 455
Family family Daughter, husband, 64
aunt
Friends friend Buddy, friend, 37
neighbor
Humans human Adult, baby, boy 61
Affective processes affect Happy, cried, abandon 915
Positive emotion posemo Love, nice, sweet 406
Negative emotion negemo Hurt, ugly, nasty 499
Anxiety anx Worried, fearful, 91
nervous
Anger anger Hate, kill, annoyed 184
Sadness sad Crying, grief, sad 101
Cognitive processes cogmech cause, know, ought 730
Insight insight think, know, consider 195
Causation cause because, effect, hence 108
Discrepancy discrep should, would, could 76
Tentative tentat maybe, perhaps, guess 155
Certainty certain always, never 83
Inhibition inhib block, constrain, stop 111
Inclusive incl And, with, include 18
 103
(Appendix A, continued)
Exclusive excl But, without, exclude 17
Perceptual processes percept Observing, heard, 273
feeling
See see View, saw, seen 72
Hear hear Listen, hearing 51
Feel feel Feels, touch 75
Biological processes bio Eat, blood, pain 567
Body body Cheek, hands, spit 180
Health health Clinic, flu, pill 236
Sexual sexual Horny, love, incest 96
Ingestion ingest Dish, eat, pizza 111
Relativity relativ Area, bend, exit, stop 638
Motion motion Arrive, car, go 168
Space space Down, in, thin 220
Time time End, until, season 239
Personal Concerns
Work work Job, majors, xerox 327
Achievement achieve Earn, hero, win 186
Leisure leisure Cook, chat, movie 229
Home home Apartment, kitchen, 93
family
Money money Audit, cash, owe 173
Religion relig Altar, church, mosque 159
Death death Bury, coffin, kill 62
Spoken categories
Assent assent Agree, OK, yes 30
Nonfluencies nonflu Er, hm, umm 8
Fillers filler Blah, Imean, 9
youknow
 104

Appendix B: Summary Statistics of Conversion table

Variable Mean Std Dev Minimum Maximum

RADIO_TOTAL 55.57648 20.28646 0 106
Acquisition_duration -0.00010 5.14175 -10.703 25.297
Acquisition_duration2 26.43733 37.17775 0 639.94
Unknown_reason 0.25696 0.43696 0 1
Free_ended 0.50762 0.49994 0 1
Uncontrollable_reason 0.00334 0.05767 0 1
Service_Dissatisfaction 0.00181 0.04249 0 1
NonPay 0.00259 0.05078 0 1
Non_Usage 0.00051 0.02262 0 1
Product_Dissatisfaction 0.00005 0.00714 0 1
Cost_price_fee 0.00005 0.00714 0 1
Competition 0.00001 0.00276 0 1
Acquis_Live_Agent60 0.04815 0.26629 0 10
Acquis_CUWI60 0.03363 0.26182 0 7
Acquis_IVR60 0.00038 0.02147 0 3
Acquis_Live_Agent_gt60 0.59940 0.82938 0 44
Acquis_CUWI_gt60 0.02305 0.22012 0 28
Acquis_IVR_gt60 0.00213 0.05469 0 4
Acquis_ChangeAcctInfo60 0.04900 0.28635 0 6
Acquis_Change_Service60 0.02609 0.18034 0 6
Acquis_Make_Payment60 0.00707 0.09589 0 7
Acquis_Make_Payment_gt60 0.02121 0.43458 0 44
Acquis_ChangeAcctInfo_gt60 0.02419 0.20928 0 8
Acquis_Change_Service_gt60 0.57919 0.65766 0 14
Acq_LiveAg_ChangeInfo60 0.01712 0.13579 0 5
Acq_LiveAg_ChangeSer60 0.02433 0.17185 0 6
Acq_LiveAg_MakePaymt60 0.00671 0.09186 0 6
Acq_CUWI_MakePaymt60 0.00031 0.01915 0 4
Acq_CUWI_ChangeInfo60 0.03183 0.24772 0 6
Acq_IVR_MakePaymt60 0.00006 0.01022 0 2
Acq_CUWI_ChangeSer60 0.00149 0.04110 0 4
Acq_IVR_ChangeSer60 0.00028 0.01719 0 2
Acq_IVR_ChangeInfo60 0.00005 0.00714 0 1
Acq_LiveAg_MakePaymt_gt60 0.02040 0.41930 0 44
Acq_CUWI_ChangeInfo_gt60 0.01553 0.17493 0 7
Acq_CUWI_ChangeSer_gt60 0.00690 0.09229 0 7
Acq_LiveAg_ChangeSer_gt60 0.57043 0.64800 0 14
Acq_CUWI_MakePaymt_gt60 0.00061 0.06012 0 28
Acq_LiveAg_ChangeInfo_gt60 0.00858 0.10710 0 7
Acq_IVR_MakePaymt_gt60 0.00020 0.02069 0 4
Acq_IVR_ChangeInfo_gt60 0.00008 0.00889 0 1
Acq_IVR_ChangeSer_gt60 0.00186 0.04777 0 4
Acquis_MDBcount60 3.18858 2.72539 0 27
Acquis_MDBcount_gt60 2.78640 3.37969 1 112
ACQUSI_EM60 0.86992 1.59616 0 15
 105
(Appendix B, continued)
ACQUSI_DM60 1.22405 1.43420 0 19
ACQUSI_TELEM60 0.32842 0.57997 0 9
ACQUSI_EM_gt60 1.24364 2.66804 0 107
ACQUSI_TELEM_gt60 0.25627 0.49826 0 17
ACQUSI_DM_gt60 0.47473 1.06432 0 27
Trial60Response_Rate 0.11573 0.19670 0 1
Trial_gt60Response_Rate 0.04397 0.15938 0 1
ACQUSI_EMDM60 1.38454 3.86736 0 133
ACQUSI_EMTELEM60 0.45155 1.52871 0 36
ACQUSI_DMTELEM60 0.79814 1.72681 0 54
ACQUSI_EMDMTELEM60 1.14175 4.44625 0 192
ACQUSI_EMDM_gt60 0.83004 8.42157 0 1475
ACQUSI_EMTELEM_gt60 0.43117 4.28087 0 528
ACQUSI_DMTELEM_gt60 0.30342 1.53058 0 200
ACQUSI_EMDMTELEM_gt60 1.00711 29.45525 0 11800
funct_trial60 10.34647 15.64231 0 100
pronoun_trial60 1.26627 3.36484 0 66.67
ppron_trial60 0.71227 2.36294 0 50
i_trial60 0.21798 1.06868 0 33.33
we_trial60 0.03136 0.36947 0 22.22
you_trial60 0.00138 0.09301 0 20
shehe_trial60 0.44192 1.77311 0 50
they_trial60 0.01962 0.30250 0 18.18
ipron_trial60 0.55400 2.02742 0 66.67
article_trial60 0.97883 2.92367 0 100
verb_trial60 2.92114 5.94516 0 100
auxverb_trial60 0.94131 3.08434 0 100
past_trial60 1.77291 4.20530 0 100
present_trial60 0.92016 3.01119 0 80
future_trial60 0.09920 0.71509 0 50
adverb_trial60 0.32238 1.56397 0 50
preps_trial60 5.34505 8.87729 0 100
conj_trial60 0.82022 2.37829 0 50
negate_trial60 0.51022 2.30106 0 100
quant_trial60 0.17527 1.22178 0 100
number_trial60 0.37952 2.63339 0 100
swear_trial60 0.00034 0.04512 0 11.11
social_trial60 3.29225 6.50796 0 100
family_trial60 0.03959 0.55702 0 100
friend_trial60 0.01013 0.31052 0 33.33
humans_trial60 0.06555 0.68509 0 33.33
affect_trial60 0.79390 2.64456 0 100
posemo_trial60 0.62632 2.30134 0 100
negemo_trial60 0.16694 1.20066 0 100
anx_trial60 0.00520 0.15261 0 20
anger_trial60 0.01729 0.38307 0 50
sad_trial60 0.05432 0.68632 0 50
cogmech_trial60 3.93714 7.26039 0 100
 106
(Appendix B, continued)
insight_trial60 0.28747 1.53418 0 100
cause_trial60 1.29756 3.66394 0 100
discrep_trial60 0.28806 1.41740 0 50
tentat_trial60 0.15004 1.28118 0 66.67
certain_trial60 0.43543 1.89064 0 100
inhib_trial60 0.06256 0.76647 0 100
incl_trial60 0.88547 2.67024 0 66.67
excl_trial60 0.44044 2.13987 0 50
percept_trial60 0.16330 1.30880 0 66.67
see_trial60 0.01837 0.29805 0 33.33
hear_trial60 0.13837 1.24583 0 66.67
feel_trial60 0.00610 0.21409 0 33.33
bio_trial60 0.07071 0.66603 0 50
body_trial60 0.00259 0.14251 0 25
health_trial60 0.06586 0.63788 0 50
sexual_trial60 0.00202 0.10782 0 16.67
ingest_trial60 0.00074 0.08105 0 33.33
relativ_trial60 5.36336 9.63670 0 100
motion_trial60 1.09291 3.02618 0 100
space_trial60 1.72588 4.37158 0 100
time_trial60 2.37067 6.30618 0 100
work_trial60 2.07597 5.09205 0 100
achieve_trial60 0.95585 3.10516 0 66.67
leisure_trial60 0.94802 2.94667 0 100
home_trial60 0.04528 0.57335 0 100
money_trial60 2.00394 5.01842 0 100
relig_trial60 0.07981 1.09529 0 66.67
death_trial60 0.00136 0.09012 0 18.18
assent_trial60 0.04508 0.70340 0 66.67
nonfl_trial60 0.31737 1.41531 0 50
filler_trial60 0.00278 0.11551 0 20
funct_trial_gt60 3.45811 10.28507 0 100
pronoun_trial_gt60 0.43576 2.04467 0 50
ppron_trial_gt60 0.31826 1.60634 0 40
i_trial_gt60 0.08976 0.68009 0 25
we_trial_gt60 0.01311 0.23094 0 25
you_trial_gt60 0.00058 0.05437 0 11.11
shehe_trial_gt60 0.20678 1.23483 0 36.36
they_trial_gt60 0.00803 0.19307 0 28.57
ipron_trial_gt60 0.11750 0.82112 0 33.33
article_trial_gt60 0.36313 1.75678 0 40
verb_trial_gt60 1.25592 4.27054 0 100
auxverb_trial_gt60 0.36100 1.78616 0 100
past_trial_gt60 0.83085 3.18807 0 100
present_trial_gt60 0.33123 1.70857 0 80
future_trial_gt60 0.04611 0.48900 0 33.33
adverb_trial_gt60 0.11898 0.91611 0 66.67
preps_trial_gt60 1.70195 5.22045 0 66.67
 107
(Appendix B, continued)
conj_trial_gt60 0.34520 1.59682 0 28.57
negate_trial_gt60 0.18870 1.24766 0 50
quant_trial_gt60 0.04961 0.52623 0 50
number_trial_gt60 0.03358 0.50699 0 66.67
swear_trial_gt60 0.00033 0.05031 0 14.29
social_trial_gt60 1.37350 4.61078 0 100
family_trial_gt60 0.01416 0.33459 0 50
friend_trial_gt60 0.00135 0.08762 0 14.29
humans_trial_gt60 0.02353 0.36546 0 25
affect_trial_gt60 0.26965 1.66342 0 100
posemo_trial_gt60 0.20756 1.27674 0 100
negemo_trial_gt60 0.06155 1.00593 0 100
anx_trial_gt60 0.00150 0.07966 0 20
anger_trial_gt60 0.00991 0.26877 0 50
sad_trial_gt60 0.00978 0.27431 0 50
cogmech_trial_gt60 1.34394 4.56618 0 100
insight_trial_gt60 0.14073 1.12705 0 100
cause_trial_gt60 0.43551 2.01468 0 100
discrep_trial_gt60 0.10915 0.86927 0 50
tentat_trial_gt60 0.04354 0.64453 0 50
certain_trial_gt60 0.04762 0.52297 0 40
inhib_trial_gt60 0.02604 0.54513 0 100
incl_trial_gt60 0.35199 1.67204 0 50
excl_trial_gt60 0.16595 1.16322 0 50
percept_trial_gt60 0.05079 0.54957 0 50
see_trial_gt60 0.01308 0.27467 0 50
hear_trial_gt60 0.03486 0.44624 0 33.33
feel_trial_gt60 0.00273 0.11291 0 25
bio_trial_gt60 0.01933 0.34940 0 33.33
body_trial_gt60 0.00178 0.10239 0 14.29
health_trial_gt60 0.01661 0.32395 0 33.33
sexual_trial_gt60 0.00147 0.09458 0 20
ingest_trial_gt60 0.00029 0.05108 0 25
relativ_trial_gt60 1.81440 5.83534 0 100
motion_trial_gt60 0.46561 2.07815 0 75
space_trial_gt60 0.71179 2.65124 0 66.67
time_trial_gt60 0.56307 2.43138 0 100
work_trial_gt60 0.44059 2.25539 0 100
achieve_trial_gt60 0.18811 1.17087 0 66.67
leisure_trial_gt60 0.35511 1.67137 0 50
home_trial_gt60 0.04566 0.70325 0 50
money_trial_gt60 0.51242 2.51850 0 100
relig_trial_gt60 0.00542 0.21446 0 33.33
death_trial_gt60 0.00087 0.06666 0 11.11
assent_trial_gt60 0.01640 0.33489 0 25
nonfl_trial_gt60 0.12170 0.86152 0 50
filler_trial_gt60 0.00081 0.05155 0 12.5
AcqSelfcare_dummy 0.62240 0.48479 0 1
 108

Appendix C: Summary Statistics of Churn table

Variable Mean Std Dev Minimum Maximum

RADIO_TOTAL 54.47975 20.28804 0 106
Acquisition_duration 0.02522 4.68418 -10.703 25.297
Acquisition_duration2 21.94201 35.44478 0 639.938
Retention_duration 0.02041 8.17695 -18.207 102.793
Retention_duration2 66.86294 207.95353 0 10566.4
Uncontrollable_reason 0.08264 0.27534 0 1
Non_Usage 0.04623 0.20998 0 1
Product_Dissatisfaction 0.00582 0.07606 0 1
Service_Dissatisfaction 0.02486 0.15570 0 1
NonPay 0.09900 0.29866 0 1
Unknown_reason 0.04385 0.20476 0 1
Free_ended 0.00529 0.07251 0 1
Prepaid_period_ended 0.00813 0.08979 0 1
Cost_price_fee 0.00406 0.06362 0 1
Competition 0.00042 0.02058 0 1
Service_count 1.79962 1.00699 1 15
MC_Ave_Price 0.00215 3.37437 -11.67 88.32
MC2_Ave_Price 11.38510 61.20399 0 7800.42
Retent_Live_Agent60 0.80906 2.44931 0 79
Retent_CUWI60 0.09264 0.55942 0 26
Retent_IVR60 0.02102 0.22750 0 14
Retent_CUWI_gt60 0.10369 0.63672 0 63
Retent_Live_Agent_gt60 1.35447 3.91411 0 121
Retent_IVR_gt60 0.03178 0.32865 0 34
Retent_Change_Service60 0.32264 0.82794 0 20
Retent_Make_Payment60 0.46393 1.93774 0 76
Retent_ChangeAcctInfo60 0.13616 0.53019 0 18
Retent_ChangeAcctInfo_gt60 0.15419 0.58628 0 16
Retent_Change_Service_gt60 0.31917 0.79760 0 36
Retent_Make_Payment_gt60 1.01658 3.71397 0 121
Retent_LiveAg_ChangeSer60 0.28982 0.75787 0 20
Retent_LiveAg_MakePaymt60 0.43935 1.88183 0 76
Retent_CUWI_ChangeInfo60 0.05320 0.33804 0 17
Retent_CUWI_ChangeSer60 0.02375 0.20576 0 8
Retent_CUWI_MakePaymt60 0.01569 0.16185 0 21
Retent_LiveAg_ChangeInfo60 0.07988 0.35446 0 15
Retent_IVR_ChangeSer60 0.00907 0.10342 0 8
Retent_IVR_MakePaymt60 0.00888 0.15604 0 14
Retent_IVR_ChangeInfo60 0.00307 0.05839 0 4
Retent_CUWI_ChangeInfo_gt60 0.06175 0.38242 0 11
Retent_CUWI_ChangeSer_gt60 0.01886 0.19527 0 14
Retent_LiveAg_MakePaymt_gt60 0.97436 3.62871 0 121
Retent_CUWI_MakePaymt_gt60 0.02308 0.28635 0 62
Retent_LiveAg_ChangeInfo_gt60 0.08689 0.38124 0 12
Retent_LiveAg_ChangeSer_gt60 0.29322 0.73391 0 21
 109
(Appendix C, continued)
Retent_IVR_ChangeSer_gt60 0.00709 0.09940 0 16
Retent_IVR_ChangeInfo_gt60 0.00554 0.07993 0 4
Retent_IVR_MakePaymt_gt60 0.01915 0.25915 0 34
funct_Churn60 12.66208 17.77446 0 100
pronoun_Churn60 1.75367 3.71579 0 50
ppron_Churn60 1.22364 2.76165 0 50
i_Churn60 0.32003 1.07588 0 26.32
we_Churn60 0.04972 0.39317 0 20
you_Churn60 0.00492 0.11816 0 15.38
shehe_Churn60 0.80135 2.12758 0 50
they_Churn60 0.04760 0.41779 0 38.46
ipron_Churn60 0.53003 1.55965 0 33.33
article_Churn60 1.45681 3.13394 0 42.86
verb_Churn60 4.19514 7.18431 0 100
auxverb_Churn60 1.75192 3.65209 0 100
past_Churn60 2.05181 4.39141 0 100
present_Churn60 1.69964 4.27259 0 100
future_Churn60 0.17304 0.79639 0 50
adverb_Churn60 0.48902 1.51933 0 50
preps_Churn60 5.28383 7.86707 0 66.67
conj_Churn60 1.02851 2.30112 0 40
negate_Churn60 1.08353 3.07817 0 100
quant_Churn60 0.42479 1.68596 0 66.67
number_Churn60 0.25901 1.09144 0 33.33
swear_Churn60 0.00062 0.05593 0 20
social_Churn60 4.19741 6.88559 0 100
family_Churn60 0.09647 0.68192 0 50
friend_Churn60 0.00609 0.17199 0 28.57
humans_Churn60 0.09992 0.71050 0 40
affect_Churn60 1.09466 2.85295 0 100
posemo_Churn60 0.85627 2.33540 0 100
negemo_Churn60 0.23618 1.47360 0 100
anx_Churn60 0.01309 0.18583 0 18.18
anger_Churn60 0.02775 0.55166 0 50
sad_Churn60 0.07512 0.75796 0 50
cogmech_Churn60 4.17810 6.97204 0 100
insight_Churn60 0.61368 2.07765 0 100
cause_Churn60 0.88602 2.46660 0 100
discrep_Churn60 0.46660 1.50688 0 50
tentat_Churn60 0.16579 0.96991 0 100
certain_Churn60 0.11737 0.73077 0 50
inhib_Churn60 0.17956 1.05385 0 100
incl_Churn60 0.88447 2.24823 0 66.67
excl_Churn60 0.75803 2.38607 0 50
percept_Churn60 0.41213 1.80707 0 66.67
see_Churn60 0.02724 0.39790 0 40
hear_Churn60 0.37665 1.74456 0 66.67
feel_Churn60 0.00761 0.16061 0 23.08
 110
(Appendix C, continued)
bio_Churn60 0.06941 0.66257 0 66.67
body_Churn60 0.00237 0.10139 0 22.22
health_Churn60 0.06420 0.64310 0 66.67
sexual_Churn60 0.00238 0.09951 0 20
ingest_Churn60 0.00090 0.05088 0 8.33
relativ_Churn60 4.50198 7.29048 0 100
motion_Churn60 0.72729 2.31177 0 100
space_Churn60 1.99210 3.74511 0 66.67
time_Churn60 1.69655 3.66511 0 100
work_Churn60 2.14742 4.60124 0 100
achieve_Churn60 0.32572 1.32823 0 100
leisure_Churn60 0.82532 2.13761 0 100
home_Churn60 0.06669 0.70229 0 50
money_Churn60 2.80244 5.76180 0 100
relig_Churn60 0.01998 0.35626 0 66.67
death_Churn60 0.01562 0.53583 0 80
assent_Churn60 0.02379 0.31848 0 50
nonfl_Churn60 0.38113 1.21258 0 50
filler_Churn60 0.00912 0.21116 0 40
funct_Churn_gt60 9.91040 16.38913 0 100
pronoun_Churn_gt60 1.42978 3.45653 0 60
ppron_Churn_gt60 1.03692 2.65430 0 60
i_Churn_gt60 0.25475 0.99569 0 50
we_Churn_gt60 0.05583 0.42662 0 22.22
you_Churn_gt60 0.00314 0.09919 0 16.33
shehe_Churn_gt60 0.69288 2.04906 0 50
they_Churn_gt60 0.03030 0.33728 0 18.18
ipron_Churn_gt60 0.39285 1.30591 0 33.33
article_Churn_gt60 1.29419 3.06111 0 66.67
verb_Churn_gt60 3.15886 6.08431 0 100
auxverb_Churn_gt60 1.20702 2.91155 0 100
past_Churn_gt60 1.77943 4.14342 0 100
present_Churn_gt60 1.05655 2.64037 0 80
future_Churn_gt60 0.14479 0.70681 0 33.33
adverb_Churn_gt60 0.44109 1.47309 0 50
preps_Churn_gt60 4.39429 7.65871 0 80
conj_Churn_gt60 0.87860 2.23931 0 33.33
negate_Churn_gt60 0.44647 1.46363 0 50
quant_Churn_gt60 0.22057 0.99770 0 50
number_Churn_gt60 0.18851 0.99394 0 66.67
swear_Churn_gt60 0.00037 0.03468 0 9.09
social_Churn_gt60 3.34020 6.34982 0 100
family_Churn_gt60 0.05746 0.49100 0 25
friend_Churn_gt60 0.00635 0.20772 0 66.67
humans_Churn_gt60 0.06811 0.63101 0 100
affect_Churn_gt60 1.04663 2.99821 0 100
posemo_Churn_gt60 0.87345 2.67134 0 100
negemo_Churn_gt60 0.17187 1.16691 0 100
 111
(Appendix C, continued)
anx_Churn_gt60 0.00954 0.15264 0 16.67
anger_Churn_gt60 0.02395 0.45496 0 50
sad_Churn_gt60 0.03000 0.41618 0 60
cogmech_Churn_gt60 3.37272 6.44217 0 100
insight_Churn_gt60 0.66727 2.22587 0 100
cause_Churn_gt60 0.78076 2.47541 0 100
discrep_Churn_gt60 0.33629 1.27911 0 50
tentat_Churn_gt60 0.15554 1.00690 0 100
certain_Churn_gt60 0.10715 0.68747 0 50
inhib_Churn_gt60 0.18815 1.21173 0 100
incl_Churn_gt60 0.74504 2.09983 0 66.67
excl_Churn_gt60 0.34085 1.23972 0 50
percept_Churn_gt60 0.14176 0.80032 0 50
see_Churn_gt60 0.02737 0.39130 0 50
hear_Churn_gt60 0.10760 0.66453 0 50
feel_Churn_gt60 0.00629 0.13340 0 14.29
bio_Churn_gt60 0.05736 0.53219 0 50
body_Churn_gt60 0.00200 0.09068 0 22.22
health_Churn_gt60 0.05252 0.51365 0 50
sexual_Churn_gt60 0.00219 0.10205 0 33.33
ingest_Churn_gt60 0.00108 0.05427 0 10
relativ_Churn_gt60 3.60496 6.87296 0 100
motion_Churn_gt60 0.58544 2.07920 0 100
space_Churn_gt60 1.53675 3.33913 0 66.67
time_Churn_gt60 1.42508 3.81987 0 100
work_Churn_gt60 1.71918 4.39740 0 100
achieve_Churn_gt60 0.32263 1.35865 0 100
leisure_Churn_gt60 0.53329 1.71723 0 100
home_Churn_gt60 0.06137 0.64298 0 50
money_Churn_gt60 2.18155 5.19021 0 100
relig_Churn_gt60 0.01918 0.32686 0 50
death_Churn_gt60 0.00231 0.16634 0 50
assent_Churn_gt60 0.03301 0.38858 0 25
nonfl_Churn_gt60 0.36579 1.28654 0 100
filler_Churn_gt60 0.00499 0.15611 0 33.33
Retention_MDBcount60 10.62872 13.97699 0 200
Retention_MDBcount_gt60 14.45388 17.35395 0 166
RETENTION_DM60 1.02347 2.02116 0 24
RETENTION_EM60 8.11318 12.64675 0 149
RETENTION_TELEM60 0.64619 1.29260 0 23
RETENTION_DM_gt60 0.45168 1.10973 0 57
RETENTION_EM_gt60 12.76222 16.46438 0 132
RETENTION_TELEM_gt60 0.20555 0.72194 0 34
Churn60Response_Rate 0.03134 0.11103 0 1
Churn_gt60Response_Rate 0.03859 0.12786 0 1
RETENTION_EMDM60 11.29186 42.22174 0 2793
RETENTION_EMTELEM60 6.53043 25.79364 0 2277
RETENTION_DMTELEM60 1.61831 5.34861 0 240
 112
(Appendix C, continued)
RETENTION_EMDMTELEM60 18.73759 148.54372 0 20700
RETENTION_EMDM_gt60 4.39596 19.65413 0 2580
RETENTION_EMTELEM_gt60 3.78496 21.60338 0 1802
RETENTION_DMTELEM_gt60 0.20184 1.33223 0 96
RETENTION_EMDMTELEM_gt60 2.50482 30.14699 0 5720
Selfcare_dummy 0.57855 0.49379 0 1
AcqSelfcare_dummy 0.24458 0.42984 0 1
Acq_MDBcount 4.52879 4.68952 0 118
Acq_MDB_ResRate 0.07273 0.12752 0 1
Price_range 2.47240 3.84743 0 93.51
Promo_count 0.52966 0.75499 0 7
 113

Appendix D: Summary Statistics of Unconverted Winback table

Variable Mean Std Dev Minimum Maximum

RADIO_TOTAL 56.93436 20.14063 0 106
Acquisition_duration -0.48011 4.94394 -9.703 25.297
Acquisition_duration2 24.67272 35.25138 0 639.938
Unknown_reason 0.19763 0.39822 0 1
Free_ended 0.48543 0.49979 0 1
Uncontrollable_reason 0.00283 0.05312 0 1
Service_Dissatisfaction 0.00137 0.03695 0 1
NonPay 0.00295 0.05422 0 1
Non_Usage 0.00055 0.02354 0 1
Product_Dissatisfaction 0.00004 0.00615 0 1
Cost_price_fee 0.00004 0.00615 0 1
Competition 0.00001 0.00251 0 1
Acquis_Live_Agent60 0.04245 0.25017 0 10
Acquis_CUWI60 0.02926 0.24368 0 5
Acquis_IVR60 0.00038 0.02174 0 3
Acquis_Live_Agent_gt60 0.38112 0.76000 0 42
Acquis_CUWI_gt60 0.01520 0.19113 0 28
Acquis_IVR_gt60 0.00144 0.04488 0 4
Acquis_ChangeAcctInfo60 0.04287 0.26821 0 6
Acquis_Change_Service60 0.02303 0.16886 0 6
Acquis_Make_Payment60 0.00618 0.08956 0 6
Acquis_Make_Payment_gt60 0.01582 0.39815 0 42
Acquis_ChangeAcctInfo_gt60 0.01511 0.16495 0 6
Acquis_Change_Service_gt60 0.36682 0.61600 0 14
Acq_LiveAg_ChangeInfo60 0.01493 0.12673 0 5
Acq_LiveAg_ChangeSer60 0.02161 0.16151 0 5
Acq_LiveAg_MakePaymt60 0.00590 0.08698 0 6
Acq_CUWI_MakePaymt60 0.00022 0.01485 0 1
Acq_CUWI_ChangeInfo60 0.02788 0.23219 0 5
Acq_IVR_MakePaymt60 0.00006 0.01004 0 2
Acq_CUWI_ChangeSer60 0.00115 0.03485 0 3
Acq_IVR_ChangeSer60 0.00026 0.01702 0 2
Acq_IVR_ChangeInfo60 0.00006 0.00753 0 1
Acq_LiveAg_MakePaymt_gt60 0.01501 0.37600 0 42
Acq_CUWI_ChangeInfo_gt60 0.00958 0.13649 0 6
Acq_CUWI_ChangeSer_gt60 0.00497 0.08068 0 7
Acq_LiveAg_ChangeSer_gt60 0.36064 0.60626 0 14
Acq_CUWI_MakePaymt_gt60 0.00064 0.08505 0 28
Acq_LiveAg_ChangeInfo_gt60 0.00547 0.08443 0 6
Acq_IVR_MakePaymt_gt60 0.00017 0.01992 0 4
Acq_IVR_ChangeInfo_gt60 0.00006 0.00753 0 1
Acq_IVR_ChangeSer_gt60 0.00121 0.03772 0 2
Acquis_MDBcount60 2.85318 2.75432 0 21
Acquis_MDBcount_gt60 2.60986 2.62832 1 99
ACQUSI_EM60 0.69466 1.51378 0 15
 114
(Appendix D, continued)
ACQUSI_DM60 0.98578 1.32000 0 18
ACQUSI_TELEM60 0.27657 0.54776 0 5
ACQUSI_EM_gt60 1.11811 2.71587 0 86
ACQUSI_TELEM_gt60 0.30209 0.51520 0 9
ACQUSI_DM_gt60 0.47374 1.01486 0 25
Trial60Response_Rate 0.11261 0.19737 0 1
Trial_gt60Response_Rate 0.05017 0.17689 0 1
ACQUSI_EMDM60 1.07270 3.45291 0 70
ACQUSI_EMTELEM60 0.37446 1.46548 0 36
ACQUSI_DMTELEM60 0.63463 1.55295 0 27
ACQUSI_EMDMTELEM60 0.90031 4.06694 0 192
ACQUSI_EMDM_gt60 0.83986 8.86491 0 1475
ACQUSI_EMTELEM_gt60 0.46697 4.16481 0 472
ACQUSI_DMTELEM_gt60 0.38653 1.75627 0 200
ACQUSI_EMDMTELEM_gt60 1.21315 35.30163 0 11800
funct_trial60 4.56711 11.85362 0 100
pronoun_trial60 0.51657 2.18941 0 42.86
ppron_trial60 0.27892 1.50891 0 33.33
i_trial60 0.06364 0.54170 0 25
we_trial60 0.01650 0.28036 0 20
you_trial60 0.00061 0.06976 0 20
shehe_trial60 0.18654 1.18202 0 33.33
they_trial60 0.01162 0.25691 0 18.18
ipron_trial60 0.23766 1.20464 0 42.86
article_trial60 0.37851 1.81012 0 100
verb_trial60 1.57350 4.89603 0 80
auxverb_trial60 0.63537 2.94733 0 50
past_trial60 0.74965 2.80629 0 66.67
present_trial60 0.67798 3.05342 0 80
future_trial60 0.05628 0.61743 0 50
adverb_trial60 0.14674 1.03739 0 50
preps_trial60 2.18097 5.90824 0 100
conj_trial60 0.32326 1.50243 0 28.57
negate_trial60 0.40833 2.48477 0 100
quant_trial60 0.08055 0.84965 0 100
number_trial60 0.09478 0.95040 0 66.67
swear_trial60 0.00018 0.03323 0 9.09
social_trial60 1.58697 5.21088 0 100
family_trial60 0.02080 0.41449 0 33.33
friend_trial60 0.00318 0.12903 0 12.5
humans_trial60 0.02226 0.35887 0 25
affect_trial60 0.34573 1.91471 0 100
posemo_trial60 0.25636 1.51209 0 66.67
negemo_trial60 0.08912 1.08978 0 100
anx_trial60 0.00229 0.09620 0 16.67
anger_trial60 0.00706 0.25498 0 50
sad_trial60 0.01750 0.36468 0 25
cogmech_trial60 1.83387 5.58648 0 100
 115
(Appendix D, continued)
insight_trial60 0.13704 1.08812 0 50
cause_trial60 0.52982 2.63200 0 100
discrep_trial60 0.17398 1.19407 0 50
tentat_trial60 0.06983 0.93603 0 66.67
certain_trial60 0.16560 0.98826 0 66.67
inhib_trial60 0.03527 0.74586 0 100
incl_trial60 0.32683 1.65959 0 66.67
excl_trial60 0.34685 2.29082 0 50
percept_trial60 0.07187 0.78570 0 50
see_trial60 0.01035 0.23860 0 33.33
hear_trial60 0.05854 0.72050 0 50
feel_trial60 0.00260 0.13887 0 25
bio_trial60 0.04185 0.58422 0 50
body_trial60 0.00105 0.10032 0 25
health_trial60 0.03996 0.56611 0 50
sexual_trial60 0.00028 0.02772 0 5
ingest_trial60 0.00064 0.09602 0 33.33
relativ_trial60 2.20419 6.49653 0 100
motion_trial60 0.58603 2.39381 0 50
space_trial60 0.75236 3.01510 0 100
time_trial60 0.80521 3.54030 0 100
work_trial60 1.00355 3.70819 0 100
achieve_trial60 0.35318 1.68476 0 66.67
leisure_trial60 0.30312 1.67941 0 50
home_trial60 0.02017 0.32862 0 25
money_trial60 1.03904 3.75244 0 66.67
relig_trial60 0.01839 0.42174 0 50
death_trial60 0.00044 0.04145 0 7.69
assent_trial60 0.01577 0.41792 0 66.67
nonfl_trial60 0.11350 0.83477 0 50
filler_trial60 0.00154 0.08401 0 11.11
funct_trial_gt60 2.99081 9.53919 0 75
pronoun_trial_gt60 0.37963 1.94168 0 40
ppron_trial_gt60 0.27696 1.52030 0 40
i_trial_gt60 0.07677 0.64322 0 25
we_trial_gt60 0.01069 0.20995 0 16.67
you_trial_gt60 0.00073 0.06852 0 11.11
shehe_trial_gt60 0.18141 1.17301 0 28.57
they_trial_gt60 0.00737 0.18153 0 18.18
ipron_trial_gt60 0.10266 0.78359 0 33.33
article_trial_gt60 0.31433 1.66462 0 40
verb_trial_gt60 1.15136 4.15946 0 80
auxverb_trial_gt60 0.30418 1.64537 0 50
past_trial_gt60 0.77170 3.17747 0 75
present_trial_gt60 0.30498 1.65138 0 80
future_trial_gt60 0.04021 0.46897 0 33.33
adverb_trial_gt60 0.09438 0.79895 0 66.67
preps_trial_gt60 1.47799 4.82178 0 66.67
 116
(Appendix D, continued)
conj_trial_gt60 0.29691 1.50553 0 28.57
negate_trial_gt60 0.17561 1.17119 0 40
quant_trial_gt60 0.03551 0.45242 0 33.33
number_trial_gt60 0.02099 0.34346 0 30.77
swear_trial_gt60 0.00012 0.02645 0 8.33
social_trial_gt60 1.21889 4.34485 0 100
family_trial_gt60 0.01006 0.27282 0 25
friend_trial_gt60 0.00107 0.07516 0 9.09
humans_trial_gt60 0.01803 0.31925 0 20
affect_trial_gt60 0.21857 1.58888 0 100
posemo_trial_gt60 0.15744 1.10906 0 50
negemo_trial_gt60 0.06077 1.09265 0 100
anx_trial_gt60 0.00120 0.07654 0 20
anger_trial_gt60 0.01089 0.28021 0 33.33
sad_trial_gt60 0.00900 0.27519 0 50
cogmech_trial_gt60 1.19075 4.34785 0 100
insight_trial_gt60 0.10669 0.98146 0 66.67
cause_trial_gt60 0.42225 2.00048 0 75
discrep_trial_gt60 0.09429 0.82056 0 33.33
tentat_trial_gt60 0.03912 0.63904 0 50
certain_trial_gt60 0.04036 0.46159 0 16.67
inhib_trial_gt60 0.02004 0.47893 0 66.67
incl_trial_gt60 0.29854 1.55727 0 33.33
excl_trial_gt60 0.15350 1.08238 0 40
percept_trial_gt60 0.04856 0.52653 0 25
see_trial_gt60 0.01201 0.26050 0 25
hear_trial_gt60 0.03387 0.43108 0 25
feel_trial_gt60 0.00247 0.09867 0 14.29
bio_trial_gt60 0.01516 0.30223 0 33.33
body_trial_gt60 0.00147 0.08913 0 12.5
health_trial_gt60 0.01272 0.27873 0 33.33
sexual_trial_gt60 0.00130 0.08284 0 9.09
ingest_trial_gt60 0.00028 0.03990 0 9.09
relativ_trial_gt60 1.68313 5.74917 0 100
motion_trial_gt60 0.50349 2.29602 0 75
space_trial_gt60 0.64523 2.51705 0 66.67
time_trial_gt60 0.46999 2.13504 0 100
work_trial_gt60 0.32330 1.85462 0 100
achieve_trial_gt60 0.17987 1.15478 0 66.67
leisure_trial_gt60 0.33131 1.64208 0 50
home_trial_gt60 0.08257 0.99113 0 50
money_trial_gt60 0.41276 2.30718 0 100
relig_trial_gt60 0.00290 0.13790 0 16.67
death_trial_gt60 0.00094 0.06925 0 9.09
assent_trial_gt60 0.01486 0.32424 0 25
nonfl_trial_gt60 0.11300 0.86455 0 50
filler_trial_gt60 0.00068 0.04786 0 7.14
AcqSelfcare_dummy 0.71287 0.45242 0 1
 117

Appendix E: Summary Statistics of Churned Winback table

Variable Mean Std Dev Minimum Maximum

RADIO_TOTAL 54.02857 20.42604 0 106
Acquisition_duration -0.14843 4.25878 -10.703 25.297
Acquisition_duration2 18.15901 31.17506 0 639.938
Retention_duration 0.05629 8.11540 -18.207 102.793
Retention_duration2 65.86273 209.98816 0 10566.4
Uncontrollable_reason 0.07210 0.25865 0 1
Non_Usage 0.03912 0.19389 0 1
Product_Dissatisfaction 0.00481 0.06916 0 1
Service_Dissatisfaction 0.02134 0.14452 0 1
NonPay 0.08403 0.27743 0 1
Unknown_reason 0.03728 0.18944 0 1
Free_ended 0.00484 0.06942 0 1
Prepaid_period_ended 0.00690 0.08279 0 1
Cost_price_fee 0.00336 0.05783 0 1
Competition 0.00033 0.01829 0 1
Service_count 1.82796 1.03039 1 15
MC_Ave_Price -0.32500 3.46000 -11.67 88.32
MC2_Ave_Price 12.07603 60.59619 0 7800.42
Retent_Live_Agent60 0.70786 2.33191 0 46
Retent_CUWI60 0.08103 0.52297 0 22
Retent_IVR60 0.01837 0.21123 0 14
Retent_CUWI_gt60 0.09320 0.60673 0 42
Retent_Live_Agent_gt60 1.19269 3.74798 0 121
Retent_IVR_gt60 0.02783 0.31078 0 34
Retent_Change_Service60 0.27565 0.77956 0 20
Retent_Make_Payment60 0.41256 1.84239 0 44
Retent_ChangeAcctInfo60 0.11907 0.50027 0 14
Retent_ChangeAcctInfo_gt60 0.13328 0.55059 0 16
Retent_Change_Service_gt60 0.27103 0.74619 0 20
Retent_Make_Payment_gt60 0.90943 3.55355 0 121
Retent_LiveAg_ChangeSer60 0.24746 0.71457 0 20
Retent_LiveAg_MakePaymt60 0.39066 1.78778 0 44
Retent_CUWI_ChangeInfo60 0.04675 0.31648 0 9
Retent_CUWI_ChangeSer60 0.02030 0.19017 0 8
Retent_CUWI_MakePaymt60 0.01398 0.15973 0 21
Retent_LiveAg_ChangeInfo60 0.06974 0.33572 0 13
Retent_IVR_ChangeSer60 0.00788 0.09560 0 7
Retent_IVR_MakePaymt60 0.00791 0.14656 0 14
Retent_IVR_ChangeInfo60 0.00257 0.05318 0 3
Retent_CUWI_ChangeInfo_gt60 0.05470 0.35919 0 11
Retent_CUWI_ChangeSer_gt60 0.01677 0.18264 0 12
Retent_LiveAg_MakePaymt_gt60 0.87066 3.47081 0 121
Retent_CUWI_MakePaymt_gt60 0.02174 0.28156 0 42
Retent_LiveAg_ChangeInfo_gt60 0.07389 0.35511 0 12
Retent_LiveAg_ChangeSer_gt60 0.24815 0.68750 0 19
 118
(Appendix E, continued)
Retent_IVR_ChangeSer_gt60 0.00611 0.09146 0 13
Retent_IVR_ChangeInfo_gt60 0.00469 0.07312 0 4
Retent_IVR_MakePaymt_gt60 0.01704 0.24962 0 34
funct_Churn60 10.72665 16.99245 0 100
pronoun_Churn60 1.49490 3.48090 0 42.86
ppron_Churn60 1.04274 2.58215 0 38.46
i_Churn60 0.27474 1.00434 0 25
we_Churn60 0.04242 0.35957 0 18.75
you_Churn60 0.00437 0.11128 0 15.38
shehe_Churn60 0.67984 1.97274 0 33.33
they_Churn60 0.04135 0.39193 0 38.46
ipron_Churn60 0.45216 1.44040 0 33.33
article_Churn60 1.24230 2.93237 0 42.86
verb_Churn60 3.52393 6.70964 0 100
auxverb_Churn60 1.48561 3.43225 0 100
past_Churn60 1.72273 4.07754 0 100
present_Churn60 1.42562 3.88863 0 100
future_Churn60 0.14607 0.71567 0 33.33
adverb_Churn60 0.41835 1.41500 0 50
preps_Churn60 4.46227 7.44710 0 66.67
conj_Churn60 0.87713 2.14798 0 37.5
negate_Churn60 0.90914 2.83215 0 50
quant_Churn60 0.35261 1.51971 0 66.67
number_Churn60 0.21532 0.98975 0 33.33
swear_Churn60 0.00042 0.03510 0 8.33
social_Churn60 3.54851 6.49367 0 66.67
family_Churn60 0.08000 0.60570 0 33.33
friend_Churn60 0.00451 0.13030 0 20
humans_Churn60 0.08182 0.63600 0 40
affect_Churn60 0.92167 2.60694 0 100
posemo_Churn60 0.72166 2.12829 0 100
negemo_Churn60 0.19816 1.33526 0 100
anx_Churn60 0.01080 0.16635 0 18.18
anger_Churn60 0.02512 0.54719 0 50
sad_Churn60 0.06178 0.66022 0 44.44
cogmech_Churn60 3.52407 6.53965 0 100
insight_Churn60 0.51787 1.84925 0 100
cause_Churn60 0.74540 2.26924 0 100
discrep_Churn60 0.39021 1.37432 0 50
tentat_Churn60 0.14099 0.87947 0 50
certain_Churn60 0.09940 0.66454 0 50
inhib_Churn60 0.14999 0.91904 0 100
incl_Churn60 0.75068 2.09473 0 66.67
excl_Churn60 0.64710 2.24934 0 50
percept_Churn60 0.33597 1.60150 0 50
see_Churn60 0.02253 0.36208 0 40
hear_Churn60 0.30623 1.54310 0 50
feel_Churn60 0.00660 0.14593 0 14.29
 119
(Appendix E, continued)
bio_Churn60 0.05772 0.59161 0 50
body_Churn60 0.00170 0.07502 0 11.11
health_Churn60 0.05356 0.57754 0 50
sexual_Churn60 0.00185 0.07983 0 9.52
ingest_Churn60 0.00095 0.05284 0 8.33
relativ_Churn60 3.80843 6.89520 0 100
motion_Churn60 0.61419 2.10344 0 50
space_Churn60 1.68765 3.51674 0 66.67
time_Churn60 1.43370 3.42329 0 100
work_Churn60 1.82464 4.29391 0 100
achieve_Churn60 0.27703 1.22665 0 100
leisure_Churn60 0.69477 1.95730 0 100
home_Churn60 0.05633 0.64256 0 50
money_Churn60 2.36575 5.37446 0 100
relig_Churn60 0.01619 0.29595 0 27.27
death_Churn60 0.01222 0.48101 0 80
assent_Churn60 0.01953 0.27790 0 25
nonfl_Churn60 0.31868 1.10667 0 50
filler_Churn60 0.00790 0.19974 0 40
funct_Churn_gt60 8.52940 15.59441 0 100
pronoun_Churn_gt60 1.22941 3.23210 0 60
ppron_Churn_gt60 0.89356 2.48283 0 60
i_Churn_gt60 0.22083 0.93129 0 50
we_Churn_gt60 0.04755 0.39153 0 22.22
you_Churn_gt60 0.00283 0.08698 0 10.42
shehe_Churn_gt60 0.59737 1.90947 0 40
they_Churn_gt60 0.02497 0.29916 0 16.67
ipron_Churn_gt60 0.33584 1.20862 0 33.33
article_Churn_gt60 1.11644 2.86742 0 50
verb_Churn_gt60 2.70549 5.72786 0 100
auxverb_Churn_gt60 1.04153 2.75298 0 100
past_Churn_gt60 1.51827 3.87816 0 100
present_Churn_gt60 0.90768 2.45582 0 66.67
future_Churn_gt60 0.12687 0.66376 0 33.33
adverb_Churn_gt60 0.37875 1.36759 0 50
preps_Churn_gt60 3.78384 7.25993 0 80
conj_Churn_gt60 0.75436 2.08902 0 33.33
negate_Churn_gt60 0.38287 1.35001 0 50
quant_Churn_gt60 0.19015 0.93559 0 42.86
number_Churn_gt60 0.16047 0.90056 0 40
swear_Churn_gt60 0.00024 0.02468 0 5.88
social_Churn_gt60 2.85670 5.98159 0 100
family_Churn_gt60 0.04961 0.45798 0 25
friend_Churn_gt60 0.00585 0.24071 0 66.67
humans_Churn_gt60 0.05833 0.56900 0 50
affect_Churn_gt60 0.88918 2.77697 0 100
posemo_Churn_gt60 0.74040 2.46604 0 100
negemo_Churn_gt60 0.14769 1.07692 0 100
 120
(Appendix E, continued)
anx_Churn_gt60 0.00822 0.13473 0 12.5
anger_Churn_gt60 0.01973 0.39942 0 50
sad_Churn_gt60 0.02528 0.36532 0 50
cogmech_Churn_gt60 2.89314 6.07282 0 100
insight_Churn_gt60 0.57526 2.08068 0 100
cause_Churn_gt60 0.66001 2.27693 0 100
discrep_Churn_gt60 0.28989 1.17915 0 50
tentat_Churn_gt60 0.13534 0.92790 0 100
certain_Churn_gt60 0.09251 0.64695 0 33.33
inhib_Churn_gt60 0.15994 1.08389 0 100
incl_Churn_gt60 0.64156 1.95751 0 40
excl_Churn_gt60 0.29521 1.15265 0 50
percept_Churn_gt60 0.12240 0.74470 0 50
see_Churn_gt60 0.02405 0.37960 0 50
hear_Churn_gt60 0.09281 0.61010 0 33.33
feel_Churn_gt60 0.00515 0.11984 0 11.11
bio_Churn_gt60 0.04936 0.50788 0 50
body_Churn_gt60 0.00176 0.09455 0 22.22
health_Churn_gt60 0.04518 0.49037 0 50
sexual_Churn_gt60 0.00188 0.08184 0 9.09
ingest_Churn_gt60 0.00090 0.04544 0 5.26
relativ_Churn_gt60 3.09286 6.49983 0 100
motion_Churn_gt60 0.50382 1.94456 0 100
space_Churn_gt60 1.32306 3.15720 0 66.67
time_Churn_gt60 1.21546 3.57158 0 100
work_Churn_gt60 1.46355 4.09769 0 100
achieve_Churn_gt60 0.27851 1.29773 0 100
leisure_Churn_gt60 0.45890 1.58367 0 50
home_Churn_gt60 0.05383 0.60281 0 50
money_Churn_gt60 1.84912 4.80259 0 100
relig_Churn_gt60 0.01647 0.31080 0 50
death_Churn_gt60 0.00205 0.12022 0 25
assent_Churn_gt60 0.02868 0.36688 0 25
nonfl_Churn_gt60 0.31426 1.17800 0 50
filler_Churn_gt60 0.00421 0.15326 0 33.33
Retention_MDBcount60 10.74277 14.63668 0 200
Retention_MDBcount_gt60 8.72867 12.55055 0 144
RETENTION_DM60 0.95491 1.99016 0 22
RETENTION_EM60 7.66921 12.64005 0 149
RETENTION_TELEM60 0.59634 1.28366 0 23
RETENTION_DM_gt60 0.40304 1.05420 0 57
RETENTION_EM_gt60 11.41462 16.25480 0 132
RETENTION_TELEM_gt60 0.19522 0.71442 0 34
Churn60Response_Rate 0.04033 0.12548 0 1
Churn_gt60Response_Rate 0.06494 0.17062 0 1
RETENTION_EMDM60 11.38902 44.98458 0 2793
RETENTION_EMTELEM60 6.36603 27.40730 0 2277
RETENTION_DMTELEM60 1.49181 5.39043 0 240
 121
(Appendix E, continued)
RETENTION_EMDMTELEM60 19.28264 173.58672 0 20700
RETENTION_EMDM_gt60 3.92684 17.86916 0 1104
RETENTION_EMTELEM_gt60 3.71668 21.97903 0 1802
RETENTION_DMTELEM_gt60 0.19123 1.30181 0 96
RETENTION_EMDMTELEM_gt60 2.49735 31.27355 0 3696
Selfcare_dummy 0.68393 0.46494 0 1
AcqSelfcare_dummy 0.22655 0.41860 0 1
Acq_MDBcount 3.93977 4.67568 0 118
Acq_MDB_ResRate 0.06584 0.12501 0 1
Price_range 2.71272 3.94645 0 85
Promo_count 0.65806 0.79883 0 7
 122

Appendix F: Conversion Variable Importance sorted by Mean Decrease Accuracy

Variable Converte Unconverte MeanDecreaseAccur MeanDecrease

d d acy Gini
Acquis_MDBcount_gt60 28.418 16.316 46.452 273.206
AcqSelfcare_dummy 30.086 21.682 43.778 150.018
Trial_gt60Response_Rate 27.612 -15.926 31.24 112.126
funct_trial60 -18.892 25.258 29.84 523.264
relativ_trial60 -15.608 20.192 24.71 392.866
funct_trial_gt60 16.372 7.464 24.3 151.034
cogmech_trial60 -13.856 18 22.016 224.416
preps_trial60 -15.118 18.482 21.87 435.95
time_trial60 -11.11 16.024 21.476 249.506
relativ_trial_gt60 13.49 8.488 20.998 137.426
preps_trial_gt60 14.964 6.872 20.992 140.456
Acquisition_duration 4.254 9.812 20.888 115.518
RADIO_TOTAL 12.246 17.984 19.68 274.308
ACQUSI_DM60 1.942 8.216 19.138 110.436
cogmech_trial_gt60 13.506 3.262 19.052 99.53
social_trial_gt60 18.896 -3.988 19.006 102.894
Acquisition_duration2 2.146 9.842 18.672 113.072
work_trial60 -15.136 16.802 18.496 133.812
verb_trial_gt60 13.484 3.644 17.118 106.276
social_trial60 -9.84 13.448 16.902 190.348
money_trial60 -12.868 14.554 16.218 115.526
Free_ended 4.088 4.976 16.166 39.446
money_trial_gt60 10.802 7.692 15.864 67.964
achieve_trial60 -12.5 13.938 15.576 87.412
negate_trial60 -5.794 12.73 15.382 70.68
verb_trial60 -8.34 11.612 15.25 151.358
ACQUSI_TELEM_gt60 2.592 7.584 15.108 56.13
motion_trial_gt60 5.446 12.132 14.96 77.872
home_trial_gt60 4.34 15.262 14.93 21.528
cause_trial60 -9.234 11.75 14.426 106.92
ACQUSI_EM60 -1.876 8.264 14.418 95.054
leisure_trial60 -10.258 12.206 14.188 96.378
work_trial_gt60 7.942 8.206 13.732 59.576
past_trial_gt60 10.404 0.912 13.666 73.82
past_trial60 -7.422 10.608 13.658 118.112
ACQUSI_DM_gt60 3.204 5.318 13.486 75.174
space_trial_gt60 7.906 6.032 13.332 87.074
Acquis_Change_Service_ 1.512 12.04 13.074 180.4
gt60
excl_trial60 -5.038 11.988 12.856 53.136
auxverb_trial60 -3.378 8.61 12.662 84.372
Acquis_Live_Agent_gt60 -0.582 10.816 12.63 160.962
Unknown_reason 1.198 5.616 12.596 38.452
 123
(Appendix F, continued)
Acq_LiveAg_ChangeSer_ 1.884 10.86 12.54 158.714
gt60
ACQUSI_EM_gt60 3.408 3.74 12.518 127.186
affect_trial60 -9.334 11.264 12.478 64.612
cause_trial_gt60 4.802 6.706 12.438 55.14
ACQUSI_DMTELEM60 0.012 6.354 12.176 71.788
time_trial_gt60 8.878 3.35 12.082 65.22
pronoun_trial60 -6.396 8.944 11.612 83.914
Acquis_ChangeAcctInfo6 8.124 4.406 11.534 17.352
0
number_trial60 -7.768 9.982 11.466 35.85
ACQUSI_DMTELEM_gt 2.838 5 11.442 46.162
60
ACQUSI_EMDM60 -2.94 8.204 11.172 69.732
ACQUSI_TELEM60 -0.41 5.454 11.15 53.134
achieve_trial_gt60 2.658 8.31 10.848 32.266
Acquis_Live_Agent60 7.992 5.08 10.808 21.124
posemo_trial60 -8.162 9.498 10.706 48.284
present_trial_gt60 3.756 8.702 10.69 44.374
motion_trial60 -3.824 7.006 10.556 83.534
certain_trial60 -0.526 5.532 10.438 32.572
Acquis_MDBcount60 3.692 4.73 10.242 161.888
present_trial60 -5.232 8.182 10.052 82.718
space_trial60 -7.328 8.59 9.954 125.952
Trial60Response_Rate 2.728 3.29 9.914 117.33
leisure_trial_gt60 3.106 5.416 9.76 44.224
ACQUSI_EMTELEM60 -1.86 6.604 9.518 47.656
ACQUSI_EMDMTELE -2.372 7.266 9.448 47.27
M60
affect_trial_gt60 3.714 4.708 9.224 33.41
ACQUSI_EMDM_gt60 -2.504 7.52 9.092 54.594
incl_trial60 -5.664 7.228 9.03 67.822
article_trial_gt60 4.336 4.682 8.954 38.67
ipron_trial60 -3.738 6.244 8.95 44.598
pronoun_trial_gt60 3.762 4.264 8.72 34.698
conj_trial_gt60 5.656 2.07 8.686 37.96
incl_trial_gt60 4.424 2.134 8.444 36.238
ppron_trial60 -2.768 5.368 7.912 44.744
Acq_LiveAg_ChangeInfo 5.008 3.708 7.738 7
60
conj_trial60 -4.22 5.804 7.628 60.554
auxverb_trial_gt60 1.958 5.282 7.616 34.398
Acquis_CUWI60 4.61 3.518 7.6 8.964
ACQUSI_EMTELEM_gt -1.094 5.842 7.496 49.548
60
Acq_CUWI_ChangeInfo6 4.28 3.714 7.432 8.512
0
Acq_LiveAg_ChangeSer6 5.9 2.314 7.334 9.266
 124
(Appendix F, continued)
0
Acquis_Change_Service6 5.652 2.6 7.274 9.874
0
ppron_trial_gt60 3.426 2.572 7.266 28.212
posemo_trial_gt60 2.096 3.478 7.204 25.57
article_trial60 -7.006 7.24 7.11 60.012
negate_trial_gt60 0.054 4.962 6.906 23.76
nonfl_trial_gt60 1.548 4.358 6.744 21.718
excl_trial_gt60 -1.638 5.472 5.988 20.48
nonfl_trial60 -8.504 7.882 5.954 33.356
insight_trial_gt60 2.18 2.808 5.822 20.068
ACQUSI_EMDMTELE -1.71 5.224 5.742 29.82
M_gt60
percept_trial60 -1.212 4.59 5.638 16.722
shehe_trial_gt60 2.12 2.152 5.436 19.562
relig_trial60 1.172 3.426 5.328 5.564
tentat_trial60 -1.568 4.526 4.944 18.298
Acquis_CUWI_gt60 -4.218 7.174 4.928 10.824
Acquis_ChangeAcctInfo_ -5.714 6.922 4.926 10.632
gt60
i_trial_gt60 -1.316 4.956 4.734 15.262
hear_trial60 -1.892 4.134 4.654 14.716
shehe_trial60 -1.258 3.206 4.624 27.078
ipron_trial_gt60 -2.534 5.586 4.584 17.716
adverb_trial_gt60 2.42 1.572 4.538 15.882
adverb_trial60 -4.394 4.686 4.384 28.772
negemo_trial_gt60 -1.338 5.558 4.278 12.232
i_trial60 -3.55 4.052 3.936 20.444
discrep_trial_gt60 -1.012 3.66 3.896 15.814
discrep_trial60 -7.008 9.328 3.892 38.386
Acquis_Make_Payment60 3.008 0.8 3.288 3.406
percept_trial_gt60 -1.5 4.518 3.246 9.864
quant_trial_gt60 2.158 0.364 3.06 10.124
Acq_CUWI_ChangeInfo_ -5.758 6.132 3.022 6.874
gt60
sad_trial60 -4.74 5.252 2.922 7.206
Acquis_Make_Payment_g -2.042 4.23 2.884 6.422
t60
Acq_LiveAg_MakePaymt 1.674 1.714 2.732 3.068
60
tentat_trial_gt60 -0.67 3.148 2.724 8.438
hear_trial_gt60 -2.342 4.746 2.66 7.604
humans_trial_gt60 4.322 -2.878 2.606 5.896
future_trial_gt60 -1.584 2.956 2.45 9.298
Acq_LiveAg_MakePaymt -2.024 3.734 2.282 6.428
_gt60
negemo_trial60 -5.108 4.326 1.836 19.358
certain_trial_gt60 -0.486 2.278 1.806 9.626
 125
(Appendix F, continued)
Acq_CUWI_ChangeSer_ -2.07 3.642 1.57 5.394
gt60
inhib_trial_gt60 0.304 1.238 1.462 5.286
insight_trial60 -6.478 4.876 1.336 28.998
Service_Dissatisfaction -2.272 4.062 1.332 3.806
Acq_LiveAg_ChangeInfo -3.392 3.418 1.326 5.408
_gt60
friend_trial_gt60 0.564 0.806 1.202 0.38
number_trial_gt60 -2.206 2.786 0.996 7.036
relig_trial_gt60 1.362 -0.5 0.866 0.908
you_trial_gt60 0.002 0.606 0.624 0.12
death_trial_gt60 -0.052 0.602 0.548 0.146
assent_trial_gt60 -0.338 0.632 0.41 3.19
Acq_CUWI_MakePaymt 0.17 0.224 0.394 0.358
_gt60
anger_trial_gt60 -2.86 3.038 0.348 2.274
family_trial_gt60 -0.994 1.56 0.284 2.92
Acq_CUWI_ChangeSer6 -0.404 0.696 0.274 0.838
0
Acquis_IVR_gt60 -2.998 3.032 0.266 2.026
Acq_CUWI_MakePaymt -0.17 0.284 0.228 0.152
60
Acq_IVR_ChangeSer60 0 0.22 0.22 0.152
sad_trial_gt60 -0.782 0.964 0.15 2.444
Acq_IVR_MakePaymt_gt 0.116 0 0.116 0.058
60
swear_trial60 -0.054 0.348 0.104 0.34
Acquis_IVR60 -0.348 0.252 0.01 0.194
Competition 0 0 0 0.002
Acq_IVR_MakePaymt60 0 0 0 0.016
Acq_IVR_ChangeInfo60 0 0 0 0.038
Acq_IVR_ChangeInfo_gt 0 0 0 0.02
60
swear_trial_gt60 0 0 0 0.044
ingest_trial_gt60 0 0 0 0.032
sexual_trial_gt60 -0.372 0.266 -0.012 0.358
bio_trial_gt60 -3.45 3.238 -0.11 4.472
ingest_trial60 -0.43 -0.002 -0.112 0.368
health_trial_gt60 -4.046 3.944 -0.158 4.05
feel_trial_gt60 -1.164 0.712 -0.222 0.884
body_trial_gt60 -0.676 0.242 -0.226 0.33
Cost_price_fee -0.084 -0.316 -0.248 0.238
we_trial_gt60 -0.752 0.388 -0.25 3.56
Product_Dissatisfaction -0.27 0 -0.27 0.118
filler_trial_gt60 -0.186 -0.228 -0.338 0.28
NonPay -1.33 0.668 -0.374 3.482
Acq_IVR_ChangeSer_gt6 -3.098 2.598 -0.416 1.836
0
 126
(Appendix F, continued)
see_trial_gt60 -2.47 1.864 -0.44 3.258
anx_trial_gt60 -0.912 0.088 -0.722 0.612
Non_Usage -0.936 -0.14 -0.748 0.868
Uncontrollable_reason -3.318 1.936 -0.764 4.758
they_trial_gt60 -3.794 2.736 -0.798 2.56
sexual_trial60 -1.098 -0.084 -0.99 0.144
death_trial60 -1.196 0.116 -1.034 0.406
health_trial60 -2.968 1.858 -1.248 10.398
bio_trial60 -2.17 1.308 -1.296 10.698
body_trial60 -2.008 0.892 -1.656 0.516
you_trial60 -1.938 -0.284 -1.832 0.272
future_trial60 -5.172 3.518 -2.184 16.904
quant_trial60 -7.35 3.74 -2.534 19.58
feel_trial60 -3.31 1.17 -2.79 1.314
home_trial60 -5.31 2.16 -2.868 6.682
see_trial60 -4.82 3.072 -2.942 4.43
they_trial60 -5.96 5.116 -3.116 6.438
inhib_trial60 -11.148 10.88 -3.442 12.69
filler_trial60 -4.318 1.542 -3.704 1.094
friend_trial60 -3.912 0.432 -3.72 1.62
assent_trial60 -7.334 3.058 -4.452 5.434
anx_trial60 -5.456 2.09 -4.604 1.932
humans_trial60 -10.654 5.426 -4.706 10.33
we_trial60 -6.59 2.014 -4.858 6.63
family_trial60 -7.68 3.91 -5.04 8.084
anger_trial60 -5.674 1.194 -5.05 2.814
 127

Appendix G: Churn Variable Importance sorted by Mean Decrease Accuracy

Variable Churn noChurn MeanDecreaseAccuracy MeanDecr

easeGini
Retention_MDBcount_gt60 37.1267 63.56 71.19 1951.73
Churn_gt60Response_Rate 55.2233 39.5033 70.7267 710.733
Promo_count 29.85 32.0233 46.2667 488.943
Selfcare_dummy -9.51667 29.6567 34.06 311.1
AcqSelfcare_dummy -11.1233 26.1833 31.1233 245.307
Churn60Response_Rate 13.11 17.05 22.8267 196.187
MC2_Ave_Price -2.42 17.7067 21.7567 317.117
Retention_MDBcount60 7.40667 13.3367 21.1333 426.227
Acquisition_duration2 11.5067 4.88667 19.6367 271.81
Acq_MDBcount 4.73 8.17 18.5633 339.493
MC_Ave_Price 10.87 9.78 18.42 299.207
Retention_duration2 10.03 2.91333 16.5767 368.273
Price_range 8.1 14.3367 15.6167 212.71
RETENTION_EM_gt60 11.34 -2.03333 14.63 361.027
Acquisition_duration 10.8233 -0.76333 13.4733 191.623
RETENTION_EM60 9.39 -1.95 12.7067 309.43
Service_count 6.04667 9.53667 12.2667 159.703
funct_Churn60 6.53 3.44333 12.1167 130.507
Retention_duration 7.83667 0.12 11.6233 316.927
RETENTION_EMDM60 6.17333 3.15667 10.71 154.107
RETENTION_EMDMTELE 4.80333 4.02 10.3233 93.5567
M60
RETENTION_DM60 6.33667 0.92 9.91 154.983
Acq_MDB_ResRate 6.93 -0.04667 9.8 210.89
RETENTION_DMTELEM6 5.23333 2.52 9.66 115.063
0
verb_Churn60 4.94 3.54667 9.62 116.923
RETENTION_DM_gt60 7.82 -1.7 9.51333 117.893
funct_Churn_gt60 3.02333 4.86 9.42667 77.5367
RETENTION_EMTELEM6 5.36333 1.77333 9.33 136.297
0
preps_Churn60 5.69667 2.90333 9.27333 125.097
relativ_Churn60 5.29333 2.76333 9.17 122.473
cogmech_Churn60 4.93667 4.16333 9.01667 119.68
RETENTION_TELEM60 6.58667 -0.95 8.85333 126.473
social_Churn60 4.85667 2.03667 8.16667 116.863
preps_Churn_gt60 4.36333 1.92667 7.77667 79.26
verb_Churn_gt60 1.32667 5.34 7.34667 71.26
Retent_Make_Payment_gt60 2.94667 3.50333 7.04 43.5333
relativ_Churn_gt60 2.33333 4.43 7.00667 76.3667
cogmech_Churn_gt60 2.15333 4.33667 6.85333 73.6
pronoun_Churn_gt60 1.69333 3.53667 6.82 52.5233
Retent_LiveAg_MakePaymt 2.14667 4.20667 6.80667 42.97
_gt60
 128
(Appendix G, continued)
Retent_Live_Agent_gt60 2.95667 3.26 6.73667 54.8733
work_Churn60 5.55333 -0.93 6.54 104.437
past_Churn60 4.52333 1 6.44333 99.0867
social_Churn_gt60 1.72333 3.71667 6.41667 74.55
pronoun_Churn60 3.17333 2.53333 6.37667 75
space_Churn60 3.99333 1.37667 6.36667 100.47
auxverb_Churn_gt60 1.53 3.04667 6.31333 57.6533
money_Churn60 5.19333 -0.51667 6.23 111.27
RETENTION_TELEM_gt6 5.01 -0.97333 6.13667 76.4467
0
ppron_Churn_gt60 1.82333 3.75333 6.12333 49.54
present_Churn60 2.45333 3.14667 5.87667 84.74
past_Churn_gt60 0.66 4.02667 5.81 65.61
RETENTION_EMDM_gt60 5.12667 -1.22 5.63667 125.353
Retent_Make_Payment60 0.913333 3.49333 5.51 29.8433
Retent_LiveAg_MakePaymt 0.666667 3.47333 5.49333 29.67
60
auxverb_Churn60 3.48333 0.82 5.46 85.8667
conj_Churn60 2.64333 0.59 5.32333 75.9533
Retent_Live_Agent60 1.39 2.72667 5.25333 40.5267
affect_Churn60 2.91 2.60667 5.23667 77.8367
RETENTION_EMTELEM_ 4.1 0.07 5.16 93.2667
gt60
time_Churn60 3.56667 1.11 5.04667 93.18
ppron_Churn60 1.96333 1.43667 4.92667 69.6633
space_Churn_gt60 2.09667 2.1 4.92667 67.3133
work_Churn_gt60 1.21333 2.99 4.88333 65.3967
conj_Churn_gt60 1.43333 2.23667 4.71667 53.0967
Retent_Change_Service60 0.52 2.97 4.59 27.2833
money_Churn_gt60 2.35 1.19667 4.56333 67.4033
time_Churn_gt60 1.33 2.19333 4.53667 63.0567
shehe_Churn_gt60 2.62333 1.45333 4.44667 42.9667
incl_Churn_gt60 1.45 1.83 4.44 49.8067
negate_Churn60 2.37667 1.47 4.38333 72.8867
present_Churn_gt60 2.51667 0.873333 4.34333 55.8333
cause_Churn_gt60 0.236667 2.72 4.31333 52.12
affect_Churn_gt60 1.57667 1.40333 4.21333 53.3033
negate_Churn_gt60 1.38667 1.78 4.04667 43.0033
shehe_Churn60 2.31667 1.34 3.94667 56.5433
RETENTION_EMDMTELE 2.61 1.90667 3.85 41.36
M_gt60
posemo_Churn60 2.30333 0.853333 3.82 70.0433
Retent_LiveAg_ChangeSer6 0.363333 2.96 3.77667 25.7933
0
leisure_Churn60 3.22333 -0.46 3.74333 71.7533
cause_Churn60 3.30667 -0.64 3.58333 71.3367
article_Churn_gt60 1.52333 0.986667 3.58333 61.0433
RETENTION_DMTELEM_ 3.24667 -0.32333 3.54 52.1433
 129
(Appendix G, continued)
gt60
Retent_Change_Service_gt6 2.15667 1.14 3.42333 29.2467
0
discrep_Churn60 0.843333 2.58333 3.41333 55.5767
excl_Churn60 2.22 0.313333 3.34 64.4767
ipron_Churn60 2.11 0.573333 3.29667 53.4467
NonPay 1.02667 1.73667 3.25 13.86
article_Churn60 3.38667 -0.37 3.22333 82.93
incl_Churn60 1.21 1.52333 3.22333 70.4433
insight_Churn_gt60 2.82 -0.09667 3.18333 51.1867
RADIO_TOTAL 1.97667 2.46 3.13667 369.73
excl_Churn_gt60 1.45667 1.48333 3.11667 37.7533
Retent_LiveAg_ChangeSer_ 1.29 0.93 3.05333 28.1633
gt60
posemo_Churn_gt60 0.35 2.01667 3.02667 50.7567
ipron_Churn_gt60 -0.83 2.99333 2.89667 37.93
leisure_Churn_gt60 1.04 1.15 2.83333 45.6067
discrep_Churn_gt60 1.68667 0.276667 2.79 38.3033
percept_Churn60 0.336667 2.06 2.63333 46.44
adverb_Churn_gt60 2.54667 -0.12333 2.63333 42.9933
motion_Churn_gt60 0.73 1.12333 2.51 47.0233
motion_Churn60 1.80333 -0.51 2.31333 66.2533
hear_Churn60 0.363333 1.84333 2.20333 42.6967
nonfl_Churn_gt60 1.98333 -0.81333 1.96333 44.6333
Uncontrollable_reason 0.403333 0.5 1.86333 13.07
achieve_Churn_gt60 0.0266667 1.65333 1.83 38.39
Retent_ChangeAcctInfo_gt6 0.0433333 1.33333 1.72667 19.6567
0
Retent_LiveAg_ChangeInfo 1.26333 0.276667 1.68 13.8467
60
percept_Churn_gt60 -0.523333 2.00667 1.57667 23.2467
Unknown_reason 1.33333 0.326667 1.49 10.6867
insight_Churn60 1.79333 -0.56 1.32667 62.1967
negemo_Churn_gt60 0.84 0.456667 1.28333 24.91
nonfl_Churn60 1.63333 -0.91333 1.24667 54.81
adverb_Churn60 0.606667 0.32 1.23333 55.4667
Retent_ChangeAcctInfo60 1.47 -0.48667 1.10333 17.9567
i_Churn_gt60 0.47 0.513333 1.03333 33.6967
quant_Churn60 3.77667 -3.11 0.836667 52.6667
future_Churn_gt60 0.943333 -0.54 0.746667 26.82
Retent_LiveAg_ChangeInfo 1.65 -1.03 0.693333 15.82
_gt60
hear_Churn_gt60 -0.84 1.31333 0.656667 20.4667
Retent_CUWI60 -0.243333 0.676667 0.603333 13.8833
tentat_Churn_gt60 1.22667 -0.65333 0.566667 25.32
achieve_Churn60 1.46333 -1.27333 0.433333 47.2
negemo_Churn60 1.04667 -0.54333 0.416667 36.22
Retent_IVR_ChangeInfo60 -0.25 0.483333 0.3 1.33667
 130
(Appendix G, continued)
Retent_CUWI_gt60 -0.393333 0.456667 0.283333 14.1733
Non_Usage 0.73 -0.59667 0.16 10.2267
quant_Churn_gt60 1.33333 -1.47333 0.103333 33.1233
Retent_CUWI_MakePaymt6 0.39 -0.43 -0.0533333 5.04667
0
Retent_IVR_gt60 0.54 -0.52 -0.176667 7.24667
number_Churn_gt60 -0.786667 0.48 -0.206667 29.3867
future_Churn60 3.34333 -3.52 -0.216667 35.32
ingest_Churn_gt60 0.6 -0.84 -0.296667 0.586667
Service_Dissatisfaction 0.556667 -0.90667 -0.313333 7.73
certain_Churn_gt60 -0.496667 -0.06 -0.386667 22.3267
Retent_CUWI_ChangeInfo_ -0.19 - -0.43 11.0367
gt60 0.216667
Retent_CUWI_ChangeSer60 -0.306667 -0.16667 -0.436667 7.16667
Free_ended -1.58333 0.563333 -0.483333 2.95667
we_Churn_gt60 2.41333 -2.85667 -0.486667 15.64
bio_Churn_gt60 0.57 -1.1 -0.486667 12.7867
Retent_IVR_ChangeSer60 -0.35 -0.49 -0.513333 3.29333
Retent_IVR_MakePaymt_gt 0.343333 -1.1 -0.52 3.76
60
swear_Churn_gt60 -0.473333 -0.33333 -0.58 0.17
sad_Churn60 0.546667 -1.27667 -0.61 16.5667
Retent_IVR_ChangeSer_gt6 0.173333 -0.66667 -0.646667 3.17667
0
swear_Churn60 -0.333333 -0.33333 -0.666667 0.19
Prepaid_period_ended 1.09333 -1.67667 -0.683333 4.74667
inhib_Churn_gt60 1.65 -1.91333 -0.686667 28.82
Retent_CUWI_ChangeInfo6 -0.01 -0.81667 -0.69 10.3367
0
death_Churn_gt60 -0.096667 -0.68 -0.696667 0.69
Competition -0.333333 -0.6 -0.756667 0.31
i_Churn60 1.65333 -1.99 -0.783333 47.5967
Retent_IVR_ChangeInfo_gt -0.19 -0.77667 -0.793333 2.45667
60
number_Churn60 0.843333 -2.12 -0.866667 44.2733
you_Churn_gt60 -0.9 -0.11 -0.88 1.24333
Retent_CUWI_MakePaymt_ 0.476667 -1.59667 -0.913333 5.86
gt60
Cost_price_fee 0.466667 -1.33 -0.916667 2.67333
sexual_Churn60 -0.016667 -0.96667 -0.98 1.04333
health_Churn60 -1.05667 -0.22 -1.02667 14.44
Retent_IVR60 -0.413333 -0.86 -1.04667 5.11333
sexual_Churn_gt60 -0.303333 -1.05 -1.09333 0.73
see_Churn_gt60 0.97 -2.00333 -1.1 9.34
tentat_Churn60 1.68667 -2.87 -1.13333 31.9367
bio_Churn60 -1.59333 0.063333 -1.15667 16.1067
family_Churn60 -0.65 -0.72333 -1.18333 21.05
feel_Churn_gt60 -0.903333 -0.68 -1.18667 2.77
 131
(Appendix G, continued)
body_Churn60 -0.596667 -0.85667 -1.25667 0.736667
health_Churn_gt60 1.22 -2.35667 -1.25667 11.79
relig_Churn60 -1.06 -0.72333 -1.26667 5.51667
relig_Churn_gt60 0.783333 -2.15333 -1.28 5.61
ingest_Churn60 0.106667 -1.78 -1.28333 0.52
friend_Churn_gt60 -0.2 -1.41333 -1.28333 2.08
body_Churn_gt60 -1.42333 -0.73667 -1.34 0.716667
inhib_Churn60 -0.063333 -1.4 -1.36667 33.79
we_Churn60 -1.09333 -0.66 -1.44333 16.49
certain_Churn60 0.0433333 -1.72667 -1.58333 26.4467
death_Churn60 -0.17 -1.81667 -1.63667 2.58667
Product_Dissatisfaction -0.046667 -1.94667 -1.64333 2.83333
they_Churn_gt60 -0.996667 -1.14333 -1.79333 9.63333
you_Churn60 -0.533333 -1.64 -1.84333 2.39667
filler_Churn_gt60 0.126667 -1.90333 -1.87667 2.47
home_Churn_gt60 -0.586667 -1.64333 -1.91333 12.3633
home_Churn60 0.03 -2.50333 -2.13 17.8333
assent_Churn_gt60 1.44 -3.98667 -2.13 9.5
Retent_IVR_MakePaymt60 0.426667 -2.80667 -2.14333 1.64667
humans_Churn60 0.643333 -3.35 -2.2 20.63
Retent_CUWI_ChangeSer_g -2.44333 -0.65 -2.25333 6.16333
t60
anx_Churn_gt60 -1.35333 -1.67 -2.27 4.68667
humans_Churn_gt60 -0.466667 -2.21 -2.36667 14.7667
sad_Churn_gt60 -1.03667 -1.91 -2.52 7.79333
family_Churn_gt60 -0.143333 -2.77 -2.54 13.3533
friend_Churn60 -1.47 -2.01667 -2.66 2.02333
feel_Churn60 0.636667 -3.49667 -2.66333 3.80667
anger_Churn60 0.93 -4.11667 -2.73 8
anger_Churn_gt60 -1.63 -2.10333 -2.99333 7.12333
filler_Churn60 1.1 -3.83 -3.01 4.23667
see_Churn60 -0.97 -2.65333 -3.09 9.6
assent_Churn60 -0.486667 -2.97667 -3.13667 9.48
they_Churn60 -1.08333 -2.75667 -3.14 13.86
anx_Churn60 -1.36 -3.15 -3.58667 6.48
 132

Appendix H: Unconverted Winback Variable Importance sorted by Mean Decrease

Accuracy

Variable NonWinba Winbac MeanDecreaseAccura MeanDecrease

ck k cy Gini
Trial_gt60Response_Rate 24.9 13.38 26.74 37.13
Acquis_MDBcount_gt60 18.72 15.12 19.79 46.12
ACQUSI_DM60 15.55 1.89 15.85 22.74
ACQUSI_EMDM60 15.19 -7.97 15.05 14.51
ACQUSI_EM60 15.01 -4.77 14.9 21.55
AcqSelfcare_dummy 14.56 8.23 14.7 13.84
Free_ended 12.46 1.01 12.65 7.85
Trial60Response_Rate 12.64 -1.8 12.65 27.34
ACQUSI_EMTELEM60 12.79 -4.85 12.65 9.42
ACQUSI_DMTELEM60 11.29 -2.37 11.3 15.15
cogmech_trial60 11.24 -6.45 11.18 17.07
ACQUSI_EM_gt60 11.06 -0.67 11.03 26.06
Unknown_reason 10.68 -3.34 10.66 7.67
funct_trial60 10.62 -9.03 10.63 21.75
ACQUSI_DMTELEM_gt 9.76 -1.47 9.73 9.41
60
relativ_trial60 9.01 -5.03 9.06 19.23
ACQUSI_EMDMTELEM 8.9 -4.17 8.87 9.12
60
cogmech_trial_gt60 8.53 -8.95 8.46 14.41
ACQUSI_EMDMTELEM 8.45 -2.83 8.34 5.9
_gt60
auxverb_trial60 7.73 -2.47 7.72 9.43
social_trial60 7.77 -5.99 7.71 15.97
funct_trial_gt60 7.5 -7.33 7.49 17.71
pronoun_trial60 7.37 -5.7 7.35 10.09
verb_trial60 6.78 -5.31 6.79 16.62
ACQUSI_TELEM60 6.44 2.4 6.73 10.8
Acquis_Change_Service_ 6.57 -4.1 6.59 9.46
gt60
pronoun_trial_gt60 6.1 -4.97 6.11 9.89
social_trial_gt60 6.14 -7.47 6.1 16.42
Acquis_Live_Agent_gt60 5.6 -4.14 5.61 9.56
home_trial_gt60 5.55 -0.59 5.56 1.77
Acq_LiveAg_ChangeInfo 5.26 1.03 5.3 2.03
_gt60
affect_trial60 5.23 -3.48 5.22 8.85
verb_trial_gt60 5.19 -5.34 5.17 13.37
money_trial60 4.93 1.38 5.07 15.33
ACQUSI_EMDM_gt60 4.94 -3.25 4.84 10.62
Acquis_Make_Payment_g 4.74 1.5 4.81 2.13
t60
 133
(Appendix H, continued)
Acq_LiveAg_ChangeSer_ 4.73 -2.14 4.74 10
gt60
ACQUSI_EMTELEM_gt 4.71 -0.46 4.72 11.51
60
relativ_trial_gt60 4.66 -4.21 4.64 17.96
preps_trial60 4.52 -3.98 4.51 20.28
present_trial60 4.46 -4.53 4.39 11.04
NonPay 4.53 -1.4 4.38 1.78
Acq_CUWI_ChangeInfo_ 4.39 -2.05 4.34 1.92
gt60
Acquisition_duration 4.09 2.68 4.3 23.01
past_trial60 4.22 1.78 4.3 14.94
ppron_trial_gt60 4.29 -4.5 4.26 8.09
conj_trial_gt60 4.24 -2.35 4.23 9.26
negate_trial60 4.13 -1.42 4.13 6.92
preps_trial_gt60 3.99 -5.24 3.98 16.4
ACQUSI_DM_gt60 3.79 0.07 3.89 15.99
Acq_LiveAg_MakePaymt 3.76 -0.39 3.7 1.71
_gt60
percept_trial60 3.67 0.33 3.7 4.06
hear_trial60 3.64 2.95 3.7 3.31
ipron_trial60 3.35 -1.1 3.33 7.34
motion_trial_gt60 3.28 -0.57 3.29 11.65
Acquis_ChangeAcctInfo_ 3.17 -0.56 3.18 3.13
gt60
past_trial_gt60 3.1 -5.98 3.06 12.67
ACQUSI_TELEM_gt60 2.93 -0.42 2.98 11.09
excl_trial60 2.92 -1 2.89 6.51
affect_trial_gt60 2.78 0.17 2.8 7.58
Acquis_CUWI_gt60 2.81 -0.89 2.78 2.42
time_trial60 2.63 0.53 2.65 13.74
percept_trial_gt60 2.51 0.11 2.55 3.32
Acquis_MDBcount60 2.45 1.3 2.49 36.87
cause_trial_gt60 2.45 1.45 2.47 14.15
leisure_trial_gt60 2.38 4.08 2.44 13.12
ppron_trial60 2.39 -3.11 2.38 8.17
Acq_LiveAg_ChangeSer6 2.3 1.32 2.33 2.81
0
shehe_trial60 2.28 -1.03 2.28 6.34
work_trial60 2.23 -0.41 2.25 14.22
space_trial60 2.22 -1.14 2.22 14.96
Acquisition_duration2 2.06 2.74 2.2 24.13
Acquis_Live_Agent60 2.15 1.97 2.2 3.32
space_trial_gt60 2.15 -1.04 2.15 13.69
incl_trial_gt60 2.13 -1.7 2.12 9.51
shehe_trial_gt60 2.01 -0.63 2.02 6.84
money_trial_gt60 1.96 -1.12 1.95 9.29
article_trial_gt60 1.86 0.11 1.87 9.83
 134
(Appendix H, continued)
discrep_trial_gt60 1.86 -2.46 1.81 4.33
negate_trial_gt60 1.79 -0.77 1.79 7.09
RADIO_TOTAL 1.45 1.99 1.77 63.06
Acquis_IVR_gt60 1.85 -1.74 1.65 0.55
present_trial_gt60 1.46 0.51 1.47 8.99
posemo_trial_gt60 1.36 1.1 1.43 6.69
auxverb_trial_gt60 1.41 -0.48 1.42 9.21
incl_trial60 1.38 -0.3 1.39 10.4
negemo_trial60 1.35 -0.66 1.35 4.79
Acq_IVR_ChangeSer_gt6 1.19 0 1.18 0.62
0
i_trial60 1.04 1.77 1.09 6.81
posemo_trial60 1.03 -0.06 1.04 7.25
sexual_trial_gt60 1 0 1 0.01
article_trial60 0.64 2.8 0.71 10.4
tentat_trial60 0.54 -0.04 0.54 4.06
health_trial60 0.46 1.47 0.5 2.36
family_trial60 0.45 1.29 0.48 2.21
future_trial_gt60 0.46 1.76 0.48 3.21
bio_trial_gt60 0.4 -1.04 0.38 2.02
cause_trial60 0.2 3.22 0.33 12.93
motion_trial60 0.22 3.57 0.3 13.2
they_trial60 0.11 1.57 0.16 2.28
Acquis_ChangeAcctInfo6 -0.03 2.12 0.03 3.82
0
Non_Usage 0 0 0 0.02
Product_Dissatisfaction 0 0 0 0
Cost_price_fee 0 0 0 0
Competition 0 0 0 0
Acq_IVR_MakePaymt60 0 0 0 0
Acq_IVR_ChangeInfo60 0 0 0 0
Acq_CUWI_MakePaymt_ 0 0 0 0.02
gt60
Acq_IVR_MakePaymt_gt 0 0 0 0.01
60
Acq_IVR_ChangeInfo_gt 0 0 0 0.01
60
swear_trial60 0 0 0 0
sexual_trial60 0 0 0 0
you_trial_gt60 0 0 0 0
swear_trial_gt60 0 0 0 0
friend_trial_gt60 0 0 0 0.01
ingest_trial_gt60 0 0 0 0
conj_trial60 -0.04 -0.19 -0.05 10.46
i_trial_gt60 -0.14 2.18 -0.07 5.82
excl_trial_gt60 -0.26 1.92 -0.12 6.29
Uncontrollable_reason -0.75 3.55 -0.18 1.79
insight_trial_gt60 -0.38 0.12 -0.38 4.57
 135
(Appendix H, continued)
Service_Dissatisfaction -0.45 0 -0.45 0.33
Acquis_CUWI60 -0.49 1.39 -0.47 2.99
family_trial_gt60 -0.48 0.4 -0.47 2.25
time_trial_gt60 -0.84 2.54 -0.75 13.55
anx_trial_gt60 -0.82 1.42 -0.76 0.53
discrep_trial60 -0.78 0.11 -0.78 6.76
achieve_trial_gt60 -1.01 2.86 -0.83 7.88
adverb_trial_gt60 -0.9 2.53 -0.84 4.94
Acq_CUWI_MakePaymt6 -0.91 -1 -1.03 0.44
0
Acquis_Make_Payment60 -1.1 0.28 -1.09 1.83
filler_trial_gt60 -1.13 1 -1.09 0.91
Acq_CUWI_ChangeSer_g -1.06 -1.05 -1.14 1.01
t60
bio_trial60 -1.27 1.51 -1.22 2.45
ipron_trial_gt60 -1.34 2.16 -1.26 6.14
number_trial_gt60 -1.34 2.84 -1.26 4.21
inhib_trial_gt60 -1.46 2.3 -1.37 4.18
filler_trial60 -1.53 0.54 -1.49 0.74
see_trial60 -1.74 0.69 -1.73 1.2
Acquis_IVR60 -1.98 0 -1.98 0.23
health_trial_gt60 -1.95 -0.68 -2 1.94
Acq_CUWI_ChangeInfo6 -2.28 1.54 -2.21 2.64
0
ingest_trial60 -2.44 0 -2.44 0.52
future_trial60 -2.52 1.34 -2.49 4.2
body_trial60 -2.81 0 -2.81 0.82
Acq_IVR_ChangeSer60 -2.85 0 -2.85 0.29
Acq_LiveAg_MakePaymt -3.13 2.14 -2.92 1.62
60
feel_trial60 -2.94 0 -2.94 1.02
see_trial_gt60 -2.98 0.69 -2.95 1.24
Acquis_Change_Service6 -3.38 3.16 -3.04 2.67
0
nonfl_trial_gt60 -3.27 2.72 -3.13 6.76
Acq_LiveAg_ChangeInfo -3.36 2.7 -3.18 2.35
60
tentat_trial_gt60 -3.55 2.46 -3.38 4.51
feel_trial_gt60 -3.21 0.01 -3.4 1.19
hear_trial_gt60 -3.63 2.84 -3.41 3.09
Acq_CUWI_ChangeSer60 -3.52 0 -3.52 0.27
adverb_trial60 -3.72 2.97 -3.57 6.24
negemo_trial_gt60 -3.84 1.91 -3.76 4.74
relig_trial60 -3.96 1.01 -3.89 1.31
you_trial60 -4.15 0 -4.15 0.38
friend_trial60 -4.78 1.07 -4.61 0.79
relig_trial_gt60 -4.71 0 -4.68 0.95
assent_trial_gt60 -5.01 2.25 -4.78 1.07
 136
(Appendix H, continued)
they_trial_gt60 -5.16 4.18 -4.87 2.59
anger_trial_gt60 -5.19 0.78 -5.07 1.1
death_trial60 -5.35 0 -5.35 0.81
achieve_trial60 -5.59 3.59 -5.44 11.52
death_trial_gt60 -6.09 2.73 -5.48 1.25
body_trial_gt60 -6.05 0 -6.05 0.55
we_trial_gt60 -6.27 1.02 -6.19 1.3
sad_trial60 -6.6 1.26 -6.38 2.57
work_trial_gt60 -6.77 5.15 -6.4 9.58
anx_trial60 -6.57 0.56 -6.49 1.64
leisure_trial60 -6.79 2.99 -6.58 9.88
insight_trial60 -8.01 4.52 -7.62 6.04
anger_trial60 -8.29 2.15 -7.99 1.68
nonfl_trial60 -8.33 1.71 -8.15 7.33
assent_trial60 -8.44 1.41 -8.26 1.43
sad_trial_gt60 -8.55 1.67 -8.33 1.77
quant_trial_gt60 -9.11 3.95 -8.72 3.38
humans_trial60 -9 1.98 -8.96 3.03
inhib_trial60 -9.66 2.33 -9.46 2.73
humans_trial_gt60 -10.1 2.69 -9.85 1.88
certain_trial_gt60 -10.36 4.04 -10.04 3.65
we_trial60 -10.5 0.22 -10.52 3.28
certain_trial60 -11.35 4.34 -10.94 7.45
home_trial60 -12.39 5.02 -11.45 4.18
number_trial60 -12.05 4.26 -11.66 6.58
quant_trial60 -14.26 2.09 -14.11 5.84
 137

Appendix I: Churned Winback Variable Importance sorted by Mean Decrease Accuracy

Variable NonWinback Winback MeanDecrease MeanDecrease

Accuracy Gini
Selfcare_dummy 58 1.53 71.87 914.32
Promo_count 33.18 33.92 42.89 2832.13
Churn_gt60Response_Rate 17.65 10.42 30.44 732.97
AcqSelfcare_dummy -3.49 22.76 28.63 1440.8
RETENTION_EMDMTEL 17.33 -3.87 21.89 293.91
EM60
Churn60Response_Rate 9.31 8.25 20.49 438.33
MC_Ave_Price 4.81 12 20.4 1005.14
RETENTION_EMDM60 14.48 -0.72 19.33 467.44
RETENTION_EMTELEM6 13.1 0.02 19.1 417.78
0
MC2_Ave_Price 10.29 8.61 19.08 950.37
funct_Churn60 15.72 -4.02 18.92 437.79
RETENTION_DMTELEM6 13.4 -2.03 17.83 356.39
0
cogmech_Churn60 12.96 -1.21 17.14 373.01
Price_range 10.09 9.71 16.2 588.96
RETENTION_EM60 4.07 4.15 15.41 907.53
Retention_duration 3.81 4.16 15.05 906.37
verb_Churn60 12.51 -4.37 14.57 374.66
preps_Churn60 10.8 -1.07 14.21 404.64
Retention_MDBcount_gt60 0.55 9.3 14 1446.5
RETENTION_DM60 5.79 2.22 13.94 461.77
relativ_Churn60 11.75 -3.97 13.8 408.43
social_Churn60 11.48 -3.22 13.78 383.44
relativ_Churn_gt60 10.42 -1.03 13.48 253.67
Service_count 5.42 11.11 13.14 488.73
verb_Churn_gt60 11.8 -4.03 13.08 220.93
RETENTION_EM_gt60 1.12 6.25 12.36 1024.47
work_Churn60 9.09 -0.72 12.19 324.31
cogmech_Churn_gt60 10.4 -2.55 11.8 237.74
funct_Churn_gt60 10.31 -5.17 11.74 268.25
Acquisition_duration 6.12 4.05 11.48 488.39
RETENTION_EMTELEM_ 8.9 -0.28 11.29 285.4
gt60
RETENTION_EMDMTEL 11.21 -4.4 10.37 143.36
EM_gt60
affect_Churn60 9.2 -3.24 10.29 228.01
Retent_Live_Agent_gt60 8.02 -1.2 9.97 186.93
social_Churn_gt60 8.57 -2.45 9.93 238.33
Retent_Make_Payment_gt6 8.88 -3.26 9.59 134.67
0
preps_Churn_gt60 6.54 -1.23 9.39 252.18
 138
(Appendix I, continued)
past_Churn_gt60 6.63 -0.2 9.1 203.18
RETENTION_TELEM60 3.89 1.98 9.08 382.32
conj_Churn60 8.16 -3.32 9.05 231.02
auxverb_Churn_gt60 7.3 -0.94 8.93 174.53
pronoun_Churn60 8.32 -5.28 8.85 231.92
pronoun_Churn_gt60 9.24 -2.9 8.85 161.95
money_Churn60 6.55 -0.41 8.69 354.82
Retention_duration2 0.6 5 8.57 1056.17
auxverb_Churn60 7.04 -2.52 8.52 266.36
present_Churn60 6.87 -2.47 8.37 251.3
affect_Churn_gt60 7.68 -1.13 8.29 167.35
Acq_MDB_ResRate 3.2 1.49 8.13 605
ppron_Churn60 7.67 -4.22 7.84 209.55
RETENTION_DM_gt60 0.4 4.07 7.82 360.39
Retent_Live_Agent60 4.48 0.63 7.73 142.71
space_Churn60 6.6 -2.65 7.65 310.23
past_Churn60 5.72 -0.92 7.5 299.28
ppron_Churn_gt60 6.86 -3.95 7.5 148.68
RETENTION_TELEM_gt6 2.6 2.57 7.19 231.97
0
shehe_Churn60 6.93 -5.3 7.04 172.2
negate_Churn60 7.17 -4.57 6.85 218.76
space_Churn_gt60 5.62 -1.72 6.59 208.03
time_Churn_gt60 3.81 0.68 6.56 191.2
Retent_LiveAg_MakePaymt 5.65 -2.71 6.4 131.92
_gt60
Retention_MDBcount60 -1.63 5.34 6.19 1190.73
Acquisition_duration2 -2.06 6.1 6.14 717.81
RETENTION_EMDM_gt60 3.8 0.62 5.89 388.27
percept_Churn60 6.31 -4.83 5.82 140.75
Retent_Change_Service_gt6 5.14 -0.76 5.6 89.93
0
present_Churn_gt60 3.7 -0.97 5.25 166.19
RETENTION_DMTELEM_ 4.8 0.02 5.16 169.17
gt60
incl_Churn60 4.22 -0.27 4.96 216.76
Retent_LiveAg_MakePaymt 4.04 -1.37 4.94 91.7
60
excl_Churn60 4.55 -3.54 4.93 193.51
conj_Churn_gt60 4.68 -2.25 4.92 159.95
NonPay 4.09 -1.8 4.79 42.49
Retent_LiveAg_ChangeSer6 3.81 -2.27 4.78 78.9
0
Retent_ChangeAcctInfo_gt6 3.77 -0.41 4.75 61.32
0
Retent_LiveAg_ChangeSer_ 4.29 -2.83 4.56 85.04
gt60
incl_Churn_gt60 3.16 0.15 4.5 153.08
 139
(Appendix I, continued)
article_Churn_gt60 3.99 -2.33 4.47 186.08
Retent_Change_Service60 4.3 -2.54 4.37 84.19
Retent_Make_Payment60 4 -2.66 4.24 94.76
shehe_Churn_gt60 3.53 -0.79 4.24 129.65
Non_Usage 3.14 0.16 4.11 31.49
article_Churn60 3.63 -1.75 4.08 254.65
time_Churn60 2.76 -0.57 4.01 286.53
leisure_Churn_gt60 4.01 -2.32 3.97 140.91
money_Churn_gt60 2.34 -0.44 3.75 214.13
hear_Churn60 4.29 -3.18 3.74 133.15
posemo_Churn60 3.7 -3.16 3.73 210.71
work_Churn_gt60 1.24 1.68 3.73 198.62
cause_Churn60 2.8 -1.23 3.63 214.21
Uncontrollable_reason 3.11 -0.7 3.58 40.01
nonfl_Churn60 4.87 -4.92 3.41 166.48
posemo_Churn_gt60 2.41 0.77 3.38 155.36
ipron_Churn_gt60 2.51 -0.25 3.36 121.55
Retent_LiveAg_ChangeInfo 3.1 -2.5 3.01 45.31
_gt60
Unknown_reason 2.14 -0.05 2.59 29.62
cause_Churn_gt60 1.66 0.35 2.59 159.63
Retent_ChangeAcctInfo60 2.44 -1.61 2.36 58.13
RADIO_TOTAL 1 2.54 2.33 1096.98
negate_Churn_gt60 2.26 -1.4 2.31 131.63
adverb_Churn_gt60 1.72 -0.33 2.17 131.58
insight_Churn_gt60 1.58 0.09 2.13 150.98
quant_Churn60 1.91 -1.24 2.05 157.3
discrep_Churn_gt60 1.8 -1.3 2.01 116.15
leisure_Churn60 2.05 -1.32 1.98 222.12
motion_Churn60 1.09 0.56 1.96 205.38
ipron_Churn60 1.87 -1.5 1.63 165.76
motion_Churn_gt60 1.31 -1.05 1.4 144.04
Retent_CUWI_gt60 1.59 -1.55 1.28 48.76
discrep_Churn60 0.58 0.46 1.27 172.73
percept_Churn_gt60 0.58 0.51 0.97 74.27
Retent_LiveAg_ChangeInfo -0.26 2.27 0.89 45.17
60
insight_Churn60 0.77 -0.38 0.85 192.34
Acq_MDBcount -1.7 2.16 0.08 989.92
Retent_CUWI60 0.04 -0.22 -0.05 45.01
Free_ended -0.12 0 -0.13 14.04
tentat_Churn60 0.54 -1.46 -0.18 97.75
excl_Churn_gt60 0.48 -1.9 -0.22 119.04
i_Churn_gt60 -0.42 0.26 -0.45 112.04
i_Churn60 -0.55 0.45 -0.51 142.91
Retent_CUWI_ChangeInfo_ -1.03 0.78 -0.92 36.82
gt60
Retent_CUWI_MakePaymt -0.67 -0.35 -0.93 22.4
 140
(Appendix I, continued)
_gt60
adverb_Churn60 -1.18 0.86 -0.96 164.51
Retent_CUWI_ChangeSer_ -1.27 0.95 -1.22 25.98
gt60
Retent_CUWI_MakePaymt -1.89 2.2 -1.32 18.21
60
negemo_Churn_gt60 -2.03 1.28 -1.7 79.88
hear_Churn_gt60 -3.15 4.12 -1.85 67.21
Retent_IVR_ChangeInfo60 -3.59 4.24 -2.12 6.59
Service_Dissatisfaction -2.52 1.75 -2.2 25.69
Retent_CUWI_ChangeSer6 -1.85 0.46 -2.24 25.85
0
inhib_Churn_gt60 -2.54 1.15 -2.69 88.69
Retent_IVR_ChangeSer_gt6 -4.26 4.2 -2.78 14.51
0
Retent_IVR_MakePaymt_gt -4.13 3.71 -2.88 15.87
60
achieve_Churn_gt60 -2.95 1.91 -2.9 117.79
Competition -3.27 1.74 -2.92 1.62
Retent_IVR_ChangeSer60 -3.16 1.78 -3.01 16.26
quant_Churn_gt60 -2.32 0.53 -3.06 99.97
Retent_IVR_gt60 -4.35 4.16 -3.07 29.04
Retent_CUWI_ChangeInfo6 -2.51 -0.21 -3.28 33.57
0
swear_Churn_gt60 -4.88 3.01 -3.29 1.97
swear_Churn60 -4 1.14 -3.34 1.42
tentat_Churn_gt60 -4.14 3.58 -3.44 80.68
future_Churn_gt60 -3.79 2.76 -3.46 87.2
Retent_IVR_ChangeInfo_gt -4.44 1.7 -3.67 9.89
60
negemo_Churn60 -3.81 1.32 -3.77 113.31
nonfl_Churn_gt60 -4.23 2.68 -3.9 129.55
body_Churn60 -4.81 2 -4.01 4.32
future_Churn60 -4.2 3.24 -4.04 108.48
Retent_IVR60 -5.63 4 -4.07 21.98
Product_Dissatisfaction -5.21 2.73 -4.46 13.85
Cost_price_fee -4.58 2.33 -4.49 9.86
bio_Churn_gt60 -4.17 1.02 -4.49 39.57
humans_Churn_gt60 -5.33 3.64 -4.56 49.39
filler_Churn60 -6.64 4.66 -4.71 15.39
Retent_IVR_MakePaymt60 -6.31 3.75 -4.94 9.49
ingest_Churn60 -7.44 3.91 -5.32 5.51
health_Churn_gt60 -6.35 4.18 -5.36 36.75
family_Churn_gt60 -4.1 0.43 -5.37 46.57
sexual_Churn60 -7.1 3.14 -5.4 6.05
death_Churn_gt60 -7.83 4.75 -5.51 5.86
Prepaid_period_ended -7.15 4.86 -5.61 14.58
ingest_Churn_gt60 -7.47 3.58 -5.79 4.77
 141
(Appendix I, continued)
achieve_Churn60 -7.85 5.66 -5.94 151.96
certain_Churn_gt60 -7.65 3.26 -6.27 72.57
number_Churn_gt60 -7.12 3.94 -6.49 90.39
anger_Churn_gt60 -7.84 5.15 -6.9 28.59
body_Churn_gt60 -8 2.35 -6.99 3.67
sexual_Churn_gt60 -9.35 4.71 -7.51 5.53
health_Churn60 -8.68 4.66 -7.68 51.6
friend_Churn_gt60 -10.03 5.48 -7.85 12.76
relig_Churn_gt60 -10.37 6.47 -8.22 21.19
bio_Churn60 -10.82 6.67 -8.26 55.06
home_Churn_gt60 -9.92 5.44 -8.66 46.47
death_Churn60 -12.17 5.05 -8.8 15.73
you_Churn60 -11.54 5.83 -8.98 13.19
sad_Churn_gt60 -13.31 11.3 -9.05 35.83
assent_Churn_gt60 -9.88 6.14 -9.1 41.78
inhib_Churn60 -10.55 7.36 -9.18 108.43
we_Churn_gt60 -11.47 8.19 -9.18 51.9
feel_Churn60 -9.6 3.22 -9.39 15.32
sad_Churn60 -11.54 7.47 -9.4 61.62
you_Churn_gt60 -10.29 2.53 -9.4 9.3
humans_Churn60 -9.77 4.56 -9.45 70.83
friend_Churn60 -12.44 5.88 -9.54 13.4
number_Churn60 -9.72 5.45 -9.57 135.24
they_Churn_gt60 -11.08 4.05 -10 31.06
see_Churn_gt60 -11.75 6.96 -10 34.75
filler_Churn_gt60 -14.5 9.32 -10.14 14.79
we_Churn60 -10.71 3.9 -10.31 55.15
certain_Churn60 -11.28 4.91 -10.76 88.38
anx_Churn_gt60 -14.82 9.47 -11.47 25.6
feel_Churn_gt60 -14 4.26 -11.8 12.55
anx_Churn60 -15.51 7.49 -12.08 26.9
see_Churn60 -13.92 7.48 -12.3 39.22
relig_Churn60 -15.56 6.53 -12.42 25.15
family_Churn60 -15.17 9.13 -12.59 71.36
home_Churn60 -17.84 11.48 -12.68 67.72
assent_Churn60 -18.84 12.23 -12.72 39.67
they_Churn60 -15.94 6.88 -13.85 51.06
anger_Churn60 -16.7 4.72 -14.62 29.5