Neural Computing and Applications (2021) 33:4097–4110

https://doi.org/10.1007/s00521-020-05548-5(0123456789().,-volV)(0123456789().
,- volV)

S.I.: SPIOT 2020

An intelligent optimization method of E-commerce product marketing

Fang Cui1 • Haihua Hu2 • Ying Xie3

Received: 6 September 2020 / Accepted: 18 November 2020 / Published online: 3 January 2021
Ó Springer-Verlag London Ltd., part of Springer Nature 2021

Abstract
In order to improve the marketing effect of e-commerce products, based on machine learning algorithms, this paper
constructs an e-commerce product marketing model based on machine learning and SVM. Moreover, this paper studies the
classic reinforcement learning algorithm Q-learning and proposes an improved Q-learning algorithm. In addition, this
paper uses the method of mean standardization to reduce the noise impact of the reward signal caused by the unfixed time
interval between decision points, further constructs a standardization factor for the deviation caused by the asynchronous
update of the time interval in the iterative process of the Q value function and updates the standardization factor according
to the update method of the value function, and proposes the Interval-Q algorithm. At the same time, in view of the fact that
traditional reinforcement learning algorithms cannot effectively deal with the observable part of customer status in direct
marketing scenarios, based on the deep reinforcement learning DQN model, this paper combines with the idea of hybrid
model to propose a DQN model based on dual networks. Finally, this paper uses public data sets for model training and
simulation environment construction and then evaluates the algorithm proposed in this paper from different angles and
analyses model performance based on examples. From the research results, it can be seen that the precision marketing
model constructed in this paper has a good effect and can be applied to practice.

Keywords Machine learning  SVM  E-commerce  Product marketing

1 Introduction the following steps: the first is to collect data and establish
a database; the second is to segment the market and target
In simple terms, precision marketing is to use the results of customers; the third is to provide targeted products and
big data analysis and the Internet and big data mining services based on the results of the segmentation; the fourth
technology to segment the market and customers and have is to implement different marketing programs for different
targeted and in-depth communication with segmented target groups and provide personalized marketing services;
customers to understand their needs, so as to push them the the fifth is to continuously improve products and services
most needed products and services at the most suit- based on marketing results to meet user needs [1].
able time. In this way, it cannot only improve marketing In the context of the Internet, precision marketing uses
efficiency but also greatly reduce the company’s operating cloud computing, data mining and other technologies based
costs, and it will help companies achieve low-cost and on big data to segment markets and target customers and
high-efficiency marketing goals. Under normal circum- provide users with personalized products and services.
stances, a company’s precision marketing is divided into Moreover, it deeply explores the needs of users, and
achieves accurate products and services on the basis of
meeting user needs, and achieves the purpose of precision
& Haihua Hu
marketing. The core of precision marketing is consumer-
[email protected]
centric, and everything is consumer-oriented. First, seg-
1
International Business School, Shaanxi Normal University, ment the market and conduct in-depth communication with
Xi’an 710119, China target customers to understand their needs. Then, it for-
2
School of Management, Xi’an University of Architecture and mulates marketing strategies based on the target customers’
Technology, Xi’an 710055, China behavioral characteristics and consumption habits. Whe-
3
School of Economic and Management, Northwest University, ther it is a real customer or a potential customer, precision
Xi’an 710127, China

123
4098 Neural Computing and Applications (2021) 33:4097–4110

marketing must formulate different marketing plans based communicate with enterprises. Based on this communica-
on their behavioral characteristics and habit preferences to tion mode, enterprises can achieve precise marketing based
achieve personalized recommendations. Finally, consider- on the content of communication. The literature [5] pro-
ing the long-term interests of enterprises, the purpose of posed the marketing concept of segmented marketing.
precision marketing is to improve marketing efficiency. For Segmented marketing is specifically to segment consumers,
real customers, it is very necessary to cultivate their loyalty label them differently in accordance with established
to products and services, so precision marketing on the standards, and then provide precise and personalized ser-
other hand can enhance customer loyalty and form a vices. The literature [6] introduced the two concepts of big
stable customer base [2]. data and precision marketing and made a case analysis on
In the context of the Internet era, the application of big how big data is applied to precision marketing, and studied
data is becoming more and more important for an enter- the relationship between big data and precision marketing
prise. The application of big data to marketing is now a through case analysis. The literature [7] studied the con-
new marketing method that many companies are trying. In ditions for big data technology to achieve precision mar-
the context of increasing data on the Internet, precision keting and believed that in the Internet era, precision
marketing has data support and technical support. The marketing should be combined with mobile Internet tech-
company organizes and analyses the collected customer nology to accurately locate targets. The literature [8]
information and formulates different marketing plans for studied the role of big data in precision marketing of media
different groups of customers to achieve the goal of pre- and believed that social media still has a large potential
cision marketing. The reason why big data can be used in market. Using big data technology to explore these
precision marketing is not only that the use of big data potential markets can better help the media’s precision
allows companies to have a definite target direction for marketing. The literature [9] combined big data with
marketing, but also has a great role in achieving precision management, and studies data mining of enterprise pro-
marketing and improving overall marketing efficiency. duction and operation information through big data tech-
There are four main significances for precision mar- nology and then grasped consumer behavior preferences.
keting through big data. First, the company uses the data Moreover, it used the results of the analysis as an important
accumulated in the past to analyse the needs of customers reference for the application of big data to precision mar-
at different levels, thereby launching a variety of suit- keting. The literature [10] studied the factors that affect the
able products and services and promoting the efficiency of results of precision marketing and believed that the dif-
corporate marketing. Second, through big data technology, ferentiation of target customers is an important factor in
companies organize and analyse collected data and infor- determining whether precision marketing can be achieved.
mation, and the resulting analysis results are fed back to all Only by categorizing target customers can the big data
marketing links of the company, thereby improving the precision marketing work efficiently and orderly. The lit-
quality of products and services. Third, corporate data are erature [11] analyzed the security of big data precision
divided into structured data and unstructured data. Struc- marketing and believed that under the background of the
tured data are better for statistics. For unstructured data, Internet, big data marketing should pay attention to pro-
special technical means are needed, and big data technol- tecting corporate secrets and data security. The literature
ogy can process various web pages, pictures, sounds and [12] studied data mining methods, and believed that data
other unstructured data to help companies solve data pro- cannot be separated from the technical support of cloud
cessing problems. Fourth, through the analysis results of computing, and believed that cloud computing technology
big data, companies can understand their product market- is the most effective among many big data technology
ing effects, predict in advance which products consumers mining methods. The literature [13] discussed and analyzed
are interested in, and what products and services are suit- the application scenarios and related fields of big data. The
able for people at different levels, so as to prepare for literature [14] analyzed the validity and safety of data and
marketing in advance [3]. put forward corresponding suggestions on how to ensure
the safety and validity of data. The literature [15] studied
the impact of big data technology on the development of
2 Related work enterprises from the perspective of business model, and
proposed a new business model combined with big data
The literature [4] proposed that precision marketing is very technology. The literature [16] discussed the relationship
important for corporate marketing and proposed the con- between big data and marketing from the perspective of
cept of Internet precision marketing. In the context of the corporate marketing, specifically analyzed the application
Internet, many traditional communication methods of of big data in this field, and put forward suggestions on
enterprises have been broken, and consumers can directly how to use big data to achieve precision and efficiency in

123
Neural Computing and Applications (2021) 33:4097–4110 4099

marketing. The literature [17] discussed the importance of 3 Elements of reinforcement learning
big data, connects big data with the commercial value of
enterprises, and believed that using big data to improve the Because the theoretical framework of reinforcement
commercial value of enterprises has high requirements for learning is MDP, MDP is introduced first, and then the
the application of big data. The literature [18] studied the basic components of reinforcement learning: strategies,
specific application of big data and introduced the appli- rewards, and value functions are elaborated in order to
cation of big data technology in audit work. This innova- discuss the reinforcement learning methods later.
tive work model has brought many conveniences to audit
(1) Markov decision process
work. The literature [18] pointed out that in the era of big
data, rural small and medium financial institutions should MDP is generally used to formalize the description of
seize development opportunities, use big data technology reinforcement learning problems. Further, if both the state
to mine user information, explore the development of space and the behavior space are finite, then the finite
value-added business services and create new marketing Markov Decision Process (finite MDP) is used to describe.
management paths for business operations. The literature Unless otherwise specified, the MDP mentioned in this
[19] studied the advantages of big data in insurance mar- paper refers to finite MDP. Under normal circumstances,
keting, including improving corporate risk management we can use a four-tuple hS; A; p; r i to represent finite MDP,
and control capabilities, improving the marketing effi- among them: s represents a finite state space, defined as a
ciency of various marketing channels and providing con- finite set fS1 ; S2 ; . . .; SN g. N is the size of the state space.
ditions for precision marketing. The literature [20] studied At the same time, A represents a finite behavior space,
the application of precision marketing in e-commerce. The which is defined as a finite set fA1 ; A2 ; . . .; AM g. M is the
literature [21] analyzed how precision marketing is size of the behavior space. P represents the state transition
implemented in the Internet, including the establishment of function, and r represents the instant reward function [26].
databases and the application of related technologies. The Consider that the state of the environment is St at any
literature [22] studied the advantages of precision market- discrete time point t in the MDP. At this moment, if the
ing from personalized services and believed that person- agent takes action At , it will get a limited instant reward Rt ,
alized precision marketing is an effective means to improve and the environment state will also change accordingly and
customer service levels and help companies to accurately move to the next state Stþ1 . Among them, the state Stþ1 at
locate. Moreover, it believed that personalized precision the next moment is determined according to the state
marketing is a key measure for e-commerce-related com- transition function P, and the limited reward Rt is obtained
panies to carry out precision marketing, and the coopera- according to the reward function r.
tion between personalized precision marketing and In addition, according to the different ways of state
e-commerce platforms is a new direction for future mar- transition, it can be divided into two types: deterministic
keting. The literature [23] proposed that the essence of state transition and random state transition. Among them,
precision marketing is data marketing whether it is with the in the deterministic state transition, the next state to which
help of Internet platforms or e-commerce platforms. To the transition is made is deterministic. The reward function
successfully realize precision marketing, we need to con- is only related to the current state and the behavior to be
sider from the following perspectives, namely, market, taken, and it can be regarded as an evaluation of instant
customers, marketing channels and marketing strategy, reward. In the random state transition, the next state
market segmentation, target customer locking, marketing transferred to is uncertain, but a random variable and the
channel innovation and marketing strategy formulation. reward function depend on the next state in addition to the
These are inseparable from massive data support, and these current state and the behavior taken. Therefore, it can be
data are obtained through the user’s buying behavior and regarded as an evaluation of long-term effects.
browsing behavior. Moreover, analyzing and predicting the
(2) Strategy and return
collected data can provide theoretical support for precision
marketing. The literature [24] focused on the relationship The so-called strategy refers to the mapping from state s
between big data and precision marketing, how to apply to behavior a, which is usually represented by the symbol
big data mining technology to process and analyse large p. Similarly, strategies also have deterministic strategies
amounts of data, so as to achieve precision marketing. The and random strategies. Among them, the deterministic
literature [25] summarized the marketing concept of pre- strategy is defined as p : S ! A, and its output is an action
cision marketing, and believed that precision marketing sequence. The randomness strategy is defined as
should pay attention to the three factors of ‘‘time’’, ‘‘de- p : S  A 2 ½0; 1, and its output is the probability of the
gree’’ and ‘‘efficiency’’. ‘‘Time’’ is timing and rhythm, action taken in this state [27]:
‘‘degree’’ is precision and frequency, and ‘‘effect’’ is effect.

123
4100 Neural Computing and Applications (2021) 33:4097–4110

pðs; aÞ ¼ p½At ¼ ajSt ¼ s ð1Þ vp ðsÞ ¼ Ep ½Gt jSt ¼ s

" #
The task of reinforcement learning is to learn a strategy X
1
k
¼ Ep c Rtþk jSt ¼ s
p from the continuous interaction with the environment to
k¼0
maximize the cumulative reward (cumulative reward) " #
X
1
generated, that is, maximize the return. We assume that the ¼ Ep Rt þ c k
c Rtþkþ1 jSt ¼ s
reward sequence received after the moment of death is k¼0
X X
fRt ; Rtþ1 ; Rtþ2 ; . . .g, then the calculation of the reward ¼ pðajsÞ pðs0 js; aÞ ð5Þ
using the discount cumulative reward method can be a s0
expressed as: " " ##
X
1
0 k 0
X1 r ðs; a; s Þ þ cEp c Rtþkþ1 jStþ1 ¼ s
Gt ¼ ck Rtþk ð2Þ X X
k¼0
k¼0 ¼ pðajsÞ pðs0 js; aÞ½r ðs; a; s0 Þ þ cvp ðs0 Þ
In the above formula, Gt is the return and c\1 is a con- a s0

stant, which is called the discount factor. In particular, in 8s 2 S

the process of interaction between the system and the
In the above formula:
environment, if a task can be naturally divided into seg-
ments with an end time slice, the task is called an episode pðs0 js; aÞ ¼ pr fStþ1 ¼ s0 jSt ¼ s; At ¼ ag ð6Þ
task. If the task cannot be decomposed into several frag-
Indicates the probability that the environment will
ments, and the entire task needs to be continued continu-
transition to the next state s0 after the agent executes the
ously, the task is called a continuous task.
behavior in the state s.
(3) Value function r ðs; a; s0 Þ ¼ EfRt jSt ¼ s; At ¼ a; Stþ1 ¼ s0 g ð7Þ
Because the goal of reinforcement learning is to maxi- represents the expected reward generated when the envi-
mize the cumulative reward, it is natural to use the above- ronment moves to the next state s0 after the agent takes
mentioned return calculation method to measure the value action a in the current state s.
of a state. However, since the state sequence generated in a In the same way, the Bellman equation of the state
certain state can have many situations, the return Gt is a behavior value function can also be obtained as follows:
random variable, not a definite value, and it cannot be " #
optimized as an objective function. However, its expecta- X X
0 0 0 0 0 0
qp ðs; aÞ ¼ pðs js; aÞ r ðs; a; s Þ þ c pða js Þqp ðs ; a Þ
tion is a certain value, so it can be used as the definition of s0 a0
the state value function: when the agent adopts strategy p,
8 s 2 S; 8 a 2 A
its return obeys a distribution, and the expected return
value of state s is defined as the value function of state s. ð8Þ
The formula is defined as: The purpose of calculating the value function is to learn
" #
X1 the optimal strategy from the data when constructing the
k
vp ðsÞ ¼ Ep ½Gt jSt ¼ s ¼ Ep c Rtþk jSt ¼ s ð3Þ learning algorithm. Because each strategy corresponds to a
k¼0 state value function, the optimal strategy corresponds to the
In some cases, it is more useful to record the value of the optimal state value function. Therefore, the optimal state
state behavior than just the value of the state. Therefore, in value function v ðsÞ is defined as the value function with
the state s, the expected value of the return generated by the the largest value among all strategies, that is:
agent after taking action a is defined as the state behavior v ðsÞ ¼ max vp ðsÞ ð9Þ
p
value function, and the formula is defined as:
Similarly, the optimal state behavior value function
qp ðs; aÞ ¼ Ep ½Gt jSt ¼ s; At ¼ a
" # q ðs; aÞ is defined as the state behavior value function with
X
1
ð4Þ the largest value among all strategies, that is:
¼ Ep ck Rtþk jSt ¼ s; At ¼ a
k¼0 q ðs; aÞ ¼ max qp ðs; aÞ ð10Þ
p
The expression form of the above state value function is The Bellman optimal equations of the optimal state
very inconvenient in actual calculation and programming. value function v ðsÞ and the optimal state behavior value
Therefore, through further derivation, the Bellman equa- function q ðs; aÞ can be obtained as shown in the following
tion of the state value function can be obtained as follows: formula:

123
Neural Computing and Applications (2021) 33:4097–4110 4101

X
v ðsÞ ¼ max pðs0 js; aÞ½r ðs; a; s0 Þ þ cv ðs0 Þ ð11Þ E I E I E I E I
p0 ! qp0 ! p1 ! qp1 ! p2 ! qp2 ! p3 !    ! p ! qp
E
a
s0
  ð14Þ
X
0 0 0 0
q ðs; aÞ ¼ pðs js; aÞ r ðs; a; s Þ þ c max
0
q ð s ; a Þ ð12Þ is all random. In the strategy evaluation, for each state
s
s0
behavior value, the Bellman equation is used to update the
Finally, if the optimal state behavior value function is value function until qkþ1 is stable, and the current round of
obtained, the optimal strategy can be determined by iteration ends, as shown in the following equation. Among
directly maximizing q ðs; aÞ, as shown in the following them, k represents the number of iterations in this round.
formula: " #
 X X
0 0 0 0 0 0
1; if a ¼ arg maxa2A q ðs; aÞ qkþ1 ðs;aÞ ¼ pðs js;aÞ r ðs;a;s Þ þ c pða js Þqk ðs ;a Þ
p ðajsÞ ¼ ð13Þ s0 a0
0: otherwise
ð15Þ
It can be seen that the value function is actually a predic-
tion of future rewards (returns). Then, for a state s, if the When the strategy is improved, for each state, the
instant reward it receives at this time is relatively low, it greedy strategy is used to improve the current strategy, as
does not mean that its state value is low. Because if a shown below.
higher reward is produced in the state after the state s, it pðsÞ ¼ arg max qðs; aÞ ð16Þ
a
will still achieve a high state value. This is the reason why
reinforcement learning can consider the long-term profit Before converging to the optimal strategy, the strategy
maximization in the learning process, and it is also the of each round is better than the strategy of the previous
reason why this paper chooses to use reinforcement round. When strategy p is stable, the dynamic program-
learning technology to apply to marketing problems to ming process ends, and the optimal value function q ðs; aÞ
obtain long-term profit maximization. and optimal strategy p can be obtained.
(2) Value iteration
4 Reinforcement learning algorithm based The value iteration algorithm is a simplification of the
on value function strategy iteration algorithm process, and it also includes
two steps of strategy evaluation and strategy improvement.
The strategy iteration method mainly includes two steps of Different from strategy iteration, when making strategy
strategy evaluation and strategy improvement, and these improvement, it does not need to wait until the value
two steps are alternated in sequence during the learning function completely converges before making the
process. Specifically, when evaluating strategies, the improvement. Instead, it performs a strategy improvement
algorithm needs to perform multiple iterations in each every time the state behavior space is scanned (updated). In
round. Moreover, in each iteration, it is necessary to scan this way, the convergence speed of the value function is
each state in the state space according to the improved accelerated. For each state-behavior pair, the update for-
strategy in the previous round, and then use the Bellman mula for value iteration is as follows. Among them, l is the
equation to update the state value. After constant iteration, number of iterations.
the value function finally converges to a fixed point, and X  
0 0 0 0
then enters this round of strategy improvement steps. When qlþ1 ðs; aÞ ¼ pðs js; aÞ r ðs; a; s Þ þ c max
0
ql ð s ; a Þ
a
improving the strategy, the algorithm uses the value s0

function obtained from the previous round of strategy ð17Þ

evaluation to generate a new strategy in a greedy manner.
Until qlþ1 stabilizes, the algorithm ends. At this point,
Then, the above two steps are continuously looped until it
because the value function has converged, the optimal
converges to the optimal strategy. In particular, when
strategy can be obtained by directly applying the greedy
evaluating the strategy, the calculation of the state value
method to the value function.
vp ðsÞ needs to use the value function vp ðs0 Þ of its subse-
The Monte Carlo method uses the idea of averaging
quent state, so the idea of bootstrapping is used.
experience to replace the idea of random variables, so there
In the following, the strategy iteration process of the
is no need to provide environmental models. Among them,
state behavior value function is taken as an example to
experience refers to doing many experiments according to
introduce in detail:
this strategy, resulting in many episodes. Each plot is an
experiment, and the average refers to the average value.
According to the different solution methods, it can be

123
4102 Neural Computing and Applications (2021) 33:4097–4110

divided into the first-visit Monte Carlo method and the v^ðs; hÞ  vp ðsÞ ð21Þ
every-visit Monte Carlo method. In the process of inter-
q^ðs; a; hÞ  qp ðs; aÞ ð22Þ
acting with the environment, when a plot is over, MC will
use the data in the plot to update the value function and It can be found that in the approximate state value function
improve the strategy. In addition, the update of the value v^ðs; hÞ, for each state, it no longer stores its state value, but
function uses an incremental (Incremental) method of only stores a set of parameters. The value function obtained
calculating the mean. Taking the state value function as an from the known state is extended to those untouched states.
example, the update formula is: For a large-scale state space, this is equivalent to com-
Vkþ1 ðSt Þ ¼ Vk ðSt Þ þ aðGt  Vk ðSt ÞÞ ð18Þ pressing its state space. Because of h 2 Rn , the space
storage overhead is OðnÞ. When the state scale is large and
Among them, Vkþ1 ðSt Þ represents the estimated value of discrete, n is much smaller than the cost jSj of storing
Vkþ1 ðSt Þ at the k þ 1 th iteration, a represents the learning values for each state.
rate, and Gt represents the expected reward from the From the table-based value function update process, it
beginning of the state s to the end of the plot. can be seen that both the MC method and the TD method
The so-called time difference refers to the deviation of are updated toward a certain target value. This target value
the observation of the same event or variable at two con- is Gt in the Monte Carlo method. In the time difference
secutive times. In this paper, unless otherwise specified, the method, it is:
time difference algorithm mentioned refers to a one-step
Rt þcQðStþ1 ; Atþ1 Þ ð23Þ
update algorithm, that is, the value function is directly
learned according to the next time step, and then the value Therefore, if the update process of the tabular value
function of the next state is estimated. We assume that in function is extended to the value function approximation
state St , the agent takes action At , and the state transfers to process, then the process of approximating the value
Stþ1 when reward Rt is obtained. Then, the time difference function v^ðs; hÞ can actually be regarded as a supervised
dt at time t is expressed as: learning process. The data and label pair is hSt ; vp ðSt Þi.
dt ¼ Rt þcV ðStþ1 Þ  V ðSt Þ ð19Þ Among them, vp ðSt Þ is equivalent to Gt in the Monte Carlo
method and Rt þcQðStþ1 ; Atþ1 Þ in the time difference
The update formula of the value function can be expressed method. Taking the approximation of the state value
as: function as an example, the objective function to be trained
V ðSt Þ V ðSt ÞþaðRt þcV ðStþ1 Þ  V ðSt ÞÞ ð20Þ can be expressed as the following formula:

a is the step length parameter, which controls the learning arg minðvp ðsÞ  v^ðs; hÞÞ2 ð24Þ
h
rate. Rt þcV ðStþ1 Þ is called TD target, which corresponds to
Gt in formula (18). The difference between the two is that That is, by finding the parameter vector h, the mean square
the TD target uses the bootstrapping method to estimate the error between the approximate value function v^ðs; hÞ and
current value function. the true value function vp ðsÞ is minimized. In addition,
From a mathematical point of view, function approxi- depending on the update method, the update method of the
mation methods include parametric function approximation value function can be divided into an incremental update
methods and non-parametric function approximation method and a batch update method.
methods. Among them, parameterized function approxi- The most commonly used incremental update method is
mation can be divided into linear function approximation the gradient descent method. Therefore, the update formula
and nonlinear function approximation. of the parameters can be obtained from the above formula:
(1) Parametric linear function approximation h hþa½vp ðsÞ  v^ðSt ; hÞrh v^ðSt ; hÞ ð25Þ

Parametric function approximation can be defined as a In the incremental learning method, each update of the
mapping from the parameter space to the approximate model requires interaction with the environment, and the
value function space. Therefore, the state value function data are discarded once it is used, which leads to inefficient
and the state behavior value function can be approximated use of samples. The so-called batch update method is to
by a set of parameters h 2 Rn , respectively. In this paper, extract a data set from a given period of experience:
the approximate state value function and the approximate      
D ¼ S1 ; vp1 ; S2 ; vp2 ; . . .; ST ; vpT ð26Þ
state behavior value function are denoted as: v^ðs; hÞ and
q^ðs; a; hÞ, as shown below: The best fitting function v^ðs; hÞ is found.
X
2
T p p
LSðhÞ ¼ t¼1 vt  v^t ðSt ; hÞ ð27Þ

123
Neural Computing and Applications (2021) 33:4097–4110 4103

This method is equivalent to Experience Replay. It revisits audience selection, communication strategy selection and
the experience over a period of time and updates the marketing result measurement, so as to conduct accurate
parameters, so the sample utilization efficiency is high. mining of potential customers, accurate selection of mar-
Taking the state value function as an example, the linear keting methods and accurate evaluation of marketing
function approximation method can be expressed as: effects. The precision marketing process under big data is
shown in Fig. 1.
Xn
v^ðs; hÞ ¼ /i ðsÞhi ¼ hT /ðSÞ ð28Þ According to the behavior of potential consumers, all
i¼1 potential customers are divided into multiple groups,
Customers in each group have several obvious consump-
Among them, h ¼ ðh1 ;    ; hn Þ 2 R. tion and life characteristics. There are many kinds of
/ðsÞ ¼ ð/1 ðsÞ; . . .; /n ðsÞÞT ð29Þ marketing promotion methods, which can be selected
according to the characteristics of the potential customer
The above formula is the eigenvector of state s, and /i ðsÞ is group, so as to maximize the marketing effect and mini-
called eigenfunction or basis function. Common basis mize the marketing cost. In the process of selecting mar-
functions are: polynomial basis functions, Fourier basis keting methods, factors such as potential customer groups,
functions and radial basis functions. The linear function marketing methods, marketing effects, and marketing costs
approximation method is not only simple and easy to need to be considered, as shown in Fig. 2.
understand, but also theoretically converges to the global Accurate customer management is the result of quanti-
optimum. The disadvantage is that the characterization tative and precise marketing activities under the Internet
ability is weak, and the type of basis function and the and big data technology. Through the digital quantitative
number of parameters need to be specified in advance, calculation of each element of marketing activities, it
which limits the approximation ability of the function. realizes the optimization control of the marketing process,
(2) Parameterized nonlinear function approximation the accurate management of precise customers, and finally
realizes the optimization of marketing results. In the con-
Neural network can be seen as a method of parameter- text of big data, the online accurate customer management
izing basis functions. Since the parameters of the basis plan is shown in Fig. 3.
function are learned from the data, its representation ability In the era of big data, the browsing, clicking, leaving
is greatly improved. Each layer of the neural network can messages, comments, consumption and other information
be regarded as a new basis function based on the previous of hundreds of millions of consumers in the physical and
layer, that is, a feature. In this sense, the latter layer is the online markets are silently recorded, which constitutes a
abstraction of the previous layer, which can achieve the
effect of dimensionality reduction for high-dimensional
input. Therefore, compared with the linear approximator,
the advantage of neural network is that it has strong
characterization ability and approximation ability, and can
approximate the objective function with arbitrary precision.
However, it also faces many problems: the samples of deep
learning are independent, and the target distribution is
fixed, while the state before and after reinforcement
learning is related, and the target distribution is constantly
changing, which often leads to unstable convergence, etc.,
problem. Therefore, a training method suitable for non-
static, non-independent uniformly distributed data is nee-
ded to obtain the approximate function.

5 Model construction

The three main steps in precision marketing are the

selection of target audiences, the selection of communi-
cation strategies and the measurement of marketing results.
Using big data theory to conduct information mining on
consumption data can provide data support for target Fig. 1 The precision marketing process under big data

123
4104 Neural Computing and Applications (2021) 33:4097–4110

Fig. 4 Consumer Information Structure

Fig. 2 Considerations for choosing marketing methods

digital activity map of consumers. The digital activity map

records basic information, such as consumer living habits,
social data and consumption information such as con-
sumption records and consumption characteristics, as
shown in Fig. 4. It records consumer life, work and con-
sumption in a comprehensive and three-dimensional way.
Through the analysis of the digital activity map, the
potential consumer demand of consumers can be accurately
obtained.
We need to segment hundreds of millions of consumers
based on the huge amount of fragmented information and
discover potential customers of Baby Online.For different

Fig. 5 Consumer segmentation process

Fig. 3 Customer relationship management scheme in the context of

big data Fig. 6 Marketing strategy selection process

123
Neural Computing and Applications (2021) 33:4097–4110 4105

Table 1 Statistical table of structural domain names

Chinese domain English domain Chinese domain English domain

Area code ORG_CODE Required quantity QTY_DEMOND

Sales area code SALE_REG_CODE The actual amount QTY_ORDER
Retail Account Code CUST_CODE Record time INSERT_TIME
Address CUST_ADDRESS Store ID SHOP_ID
Phone CUST_PHONE Execution ID REC_ID
Store type CUST_TYPE_CODE User name USER_NAME
Sales area code SALE_REG_CODE User ID USER_ID
Years YEAR_MONTH Store country bureau code NATIONAL_CODE
Which week YEAR_WEEK Store Name SHOP_NAME
Order number ORDER_ID Store owner name SHOP_USER
Order date ORDER_DATE Shop address SHOP_ADDRESS
Retail Account Code CUST_CODE Is it a new store ISNEW
Order amount AMT_ORDER_SUM Shop phone SHOP_PHONE
Sales area code SALE_REG_CODE Longitude LONGITUDE
Years YEAR_MONTH Latitude LATITUDE
Which week YEAR_WEEK Total purchases of retail households ORDER_QTY
Order number ORDER_ID Number of purchases by retail households ORDER_NUM
Cigarette Specification Code BRAND_CODE

Table 2 Statistical diagram of the final cluster centers Table 3 Statistical table of the
Clustering
number of cases in each cluster
Clustering 1 10,666
2 7227
1 1
3 8205
ORDER_QTY 3.5499 ORDER_QTY 3.5499 Effective 25,839
ORDER_NUM 46 ORDER_NUM 46 Missing 0

Fig. 7 Final cluster center

(a) (b)

products, its potential customers are different and have discovered from historical customer and marketing records.
different characteristics. These characteristics can be Based on these characteristics, combined with big data

123
4106 Neural Computing and Applications (2021) 33:4097–4110

Fig. 8 Statistical diagram of the number of cases in each cluster

Fig. 10 Statistical diagram of the final cluster centers

Table 4 Cluster analysis table
Cluster center Number of cases Percentage Value type
Table 6 Value clustering results of retail households
1 10,665.6 41.28 High value
Cluster center Number of cases Percentage Value type
2 8205.24 27.97 MIDDLE value
3 7226.55 31.75 Low value 1 2358 9.13 Low
2 3980 15.40 In
3 2726 10.55 Low
4 3246 12.56 In
5 1783 6.90 High
6 2533 9.81 Low
7 3018 11.68 Low
8 3238 12.53 In
9 3215 12.44 Low

Fig. 9 Cluster analysis diagram

information, consumers can be segmented, and then

potential maternal and child consumption customers of
Baby Online can be determined. The process is shown in
Fig. 5.
Through the analysis of the consumption and social
characteristics of potential customer groups, Baobao can
Fig. 11 Statistical diagram of the value clustering results of retail
choose suitable and effective marketing strategies online.
households

Table 5 Statistical table of the

1 2 3 4 5 6 7 8 9
final cluster centers
ORDER_QTY 0.705 3.275 0.401 2.036 7.231 0.182 1.028 2.667 1.454
ORDER_NUM 18 48 11 38 51 4 24 43 31

123
Neural Computing and Applications (2021) 33:4097–4110 4107

Table 7 Value classification

Value type Number of cases Proportion of cases (%) Proportion of sales (%)
table of regional retail
households High 1783 6.90 24.96
In 10,465 40.50 54.75
Low 13,850 53.61 21.29

country’s tax profits. For company A’s precision market-

ing, the relevant structured domain names are roughly as
shown in Table 1.
Drawing lessons from the idea of the RFM model, this
paper selects the total purchase amount (ORDER_QTY),
and the number of purchases (ORDER_NUM) corre-
sponding to retail households as two index parameters, as
the analysis dimension, to perform cluster analysis. Before
clustering, relevant data have been cleaned up, and outliers
have been processed. Therefore, for each dimension, the
result set of clustering is determined by the data itself,
which allows the function of the clustering algorithm to be
fully utilized to ensure that customer clustering in each
dimension achieves better results. This paper uses SPSS
Fig. 12 Value classification diagram of regional retail households statistical tools to cluster the data in 2014, and tries to
cluster three categories according to the dimensions in the
Using appropriate channels for promotion and publicity at above table, and divide the retail households into three
the right time can achieve a multiplier effect. On the pre- value segments: high, medium and low. The results
mise of ensuring the marketing effect, it effectively reduces obtained are shown in Table 2 and Fig. 7.
the marketing cost of Baby Online. The process is shown in The number of cases in each cluster is shown in Table 3
Fig. 6. and Fig. 8.
Through the analysis of the clustering results, the fol-
lowing rules are found in Table 4 and Fig. 9.
6 E-commerce precision marketing case According to the above classification results, there is an
analysis unreasonable distribution of the number of cases of the
three value types, and the number of high-value customer
Company A mainly sells products through e-commerce cases is too high, which does not meet the 28 principle, and
model. As a top-ranking company in the industry, after the result of the solid classification is not accepted. After
years of development, it has achieved tremendous devel- adjusting the number of clusters and combining multiple
opment and has made significant contributions to the clustering analyses, the final clustering results are shown in
Table 5 and Fig. 10.

Table 8 A brand sales forecast in the study area (unit: ten thousand boxes)
Expert group Predictive classification Regional sales Expert group Predictive classification Regional sales

Sales area salesman Highest sales 14.14 Sales management planner Highest sales 14.645
Most likely sales 13.13 Most likely sales 13.938
Minimum sales 12.625 Minimum sales 13.635
City Manager Highest sales 14.645 Data analyst Highest sales 14.645
Most likely sales 14.14 Most likely sales 14.342
Minimum sales 13.635 Minimum sales 14.14
Provincial planner Highest sales 14.443
Most likely sales 14.14
Minimum sales 13.635

123
4108 Neural Computing and Applications (2021) 33:4097–4110

Fig. 13 Forecast of brand sales

in the study region

(a) (b)

(c) (d)

(e)

According to empirical judgment, the final clustering The value of regional retail households is classified, and
results of the order quantity (ORDER_QTY) and the order the results are shown in Table 7 and Fig. 12.
number (ORDER_NUM) are shown in Table 6 and As can be seen from the above figure, the sales of Brand
Fig. 11. A of Company A are mainly concentrated in high-value

123
Neural Computing and Applications (2021) 33:4097–4110 4109

and medium-value retail households. The number of high- precision marketing system of A company; uses retail
value and medium-value retail households accounted for household data and the KMEANS algorithm to subdivide
about 46% of the total retail households, but the sales of the value of retail households; uses the Boston matrix to
high-value and medium-value retail households accounted classify the market; combines with the ARMA model,
for nearly 80%, so they are the key retail households of Holt-Winter model and Delphi expert survey method to
Company A. comprehensively predict sales. In summary, this paper
From the above analysis results, we can identify the key provides a reference for company A’s brand marketing to
retail households of Company A. Next, we will study the improve the efficiency and quality of the company’s pre-
value changes of these retail households. cision marketing, and perfect its existing precision mar-
For the sales forecast in the study area, we divide it into keting system.
three levels, namely, the highest sales volume, the most
likely sales volume, and the lowest sales volume, which are Acknowledgements This work was supported by National Project of
the Key Research Base for Philosophy and Social Sciences in Shaanxi
scored by selected group members. At the same time, the (ID: 18JZ037), Natural Science Foundation of China (71802158,
sales weights of the three gears are different, the highest 71502070), Shaanxi Social Science Fund (2018S42), Special research
sales weight is 0.2, the most likely sales weight is 0.5, and projects of Shaanxi Provincial Department of Education (739) and
the lowest sales weight is 0.3. Finally, the weighted aver- Northwestern University National Social Science Fund project incu-
bation project (17XNFH060).
age method is used to obtain the sales forecast. The whole
process adopts a recurring method. After 3 rounds of
investigation, the average value of the 3 results is collected,
Compliance with ethical standards
and the final result is obtained, as shown in Table 8 below, Conflict of interest The authors declare that they have no competing
and the corresponding statistical diagram is shown in interests.
Fig. 13.
When forecasting sales, it is required that the team
members must not discuss with each other, do not have References
horizontal contact, and can only communicate with the
sales management planner. The final results and opinions 1. Kozlenkova IV, Samaha SA, Palmatier RW (2014) Resource-
based theory in marketing. J Acad Mark Sci 42(1):1–21
have been repeatedly consulted, summarized and revised. 2. Eckhardt GM, Houston MB, Jiang B et al (2019) Marketing in the
Moreover, the final forecast data can provide a basis for the sharing economy. J Mark 83(5):5–27
decision-making of superior leaders. 3. Tiago MTPMB, Verı́ssimo JMC (2014) Digital marketing and
social media: why bother? Bus Horiz 57(6):703–708
4. Moorman C, Day GS (2016) Organizing for marketing excel-
lence. J Mark 80(6):6–35
7 Conclusion 5. Harmeling CM, Moffett JW, Arnold MJ et al (2017) Toward a
theory of customer engagement marketing. J Acad Mark Sci
Aiming at the problem that traditional supervised learning 45(3):312–335
6. Kannan PK (2017) Digital marketing: a framework, review and
and unsupervised learning methods can only maximize the research agenda. Int J Res Mark 34(1):22–45
instantaneous revenue of a single event when dealing with 7. Wedel M, Kannan PK (2016) Marketing analytics for data-rich
direct marketing problems, but cannot guarantee that the environments. J Mark 80(6):97–121
long-term revenue can be maximized in sequential deci- 8. Leeflang PSH, Verhoef PC, Dahlström P et al (2014) Challenges
and solutions for marketing in a digital era. Eur Manag J
sion-making problems, this paper chooses to use the value 32(1):1–12
function-based reinforcement learning method for research. 9. Germann F, Ebbes P, Grewal R (2015) The chief marketing
Then, in order to better adapt to actual needs, based on officer matters! J Mark 79(3):1–22
three aspects: the unfixed time interval of marketing 10. McDonagh P, Prothero A (2014) Sustainability marketing
research: past, present and future. J Mark Manag
decision points in the direct marketing scenario, the large 30(11–12):1186–1219
data load and the low learning efficiency, and the partial 11. Hillebrand B, Driessen PH, Koll O (2015) Stakeholder market-
observability of customer status, this paper improves the ing: theoretical foundations and required capabilities. J Acad
reinforcement learning algorithm based on the value Mark Sci 43(4):411–428
12. Barrales-Molina V, Martı́nez-López FJ, Gázquez-Abad JC (2014)
function, and uses a simulation environment to verify the Dynamic marketing capabilities: toward an integrative frame-
effectiveness of the proposed method. work. Int J Manag Rev 16(4):397–416
Among them, this paper mainly introduces the devel- 13. Samaha SA, Beck JT, Palmatier RW (2014) The role of culture in
opment and application of big data, and the concept and international relationship marketing. J Mark 78(5):78–98
14. Voorhees CM, Brady MK, Calantone R et al (2016) Discriminant
theory of precision marketing; analyses the bottlenecks and validity testing in marketing: an analysis, causes for concern, and
problems encountered by company A in the development proposed remedies. J Acad Mark Sci 44(1):119–134
of precision marketing; introduces the design of the

123
4110 Neural Computing and Applications (2021) 33:4097–4110

15. Krishna A, Schwarz N (2014) Sensory marketing, embodiment, 23. Chang YT, Yu H, Lu HP (2015) Persuasive messages, popularity
and grounded cognition: a review and introduction. J Consum cohesion, and message diffusion in social media marketing. J Bus
Psychol 24(2):159–168 Res 68(4):777–782
16. Gneezy A (2017) Field experimentation in marketing research. 24. Luxton S, Reid M, Mavondo F (2015) Integrated marketing
J Mark Res 54(1):140–143 communication capability and brand performance. J Advert
17. Ashley C, Tuten T (2015) Creative strategies in social media 44(1):37–46
marketing: an exploratory study of branded social content and 25. Buhalis D, Foerste M (2015) SoCoMo marketing for travel and
consumer engagement. Psychol Mark 32(1):15–27 tourism: empowering co-creation of value. J Destin Mark Manag
18. Key TM, Czaplewski AJ (2017) Upstream social marketing 4(3):151–161
strategy: an integrated marketing communications approach. Bus 26. Huang J, Duan Z, Kwok J et al (2019) Vaping versus JUULing:
Horiz 60(3):325–333 how the extraordinary growth and marketing of JUUL trans-
19. Berger J, Humphreys A, Ludwig S et al (2020) Uniting the tribes: formed the US retail e-cigarette market. Tobacco Control
using text for marketing insight. J Mark 84(1):1–25 28(2):146–151
20. Carins JE, Rundle-Thiele SR (2014) Eating for the better: a social 27. Appel G, Grewal L, Hadi R et al (2020) The future of social
marketing review (2000–2012). Public Health Nutr media in marketing. J Acad Mark Sci 48(1):79–95
17(7):1628–1639
21. Kozlenkova IV, Hult GTM, Lund DJ et al (2015) The role of Publisher’s Note Springer Nature remains neutral with regard to
marketing channels in supply chain management. J Retail jurisdictional claims in published maps and institutional affiliations.
91(4):586–609
22. Heimbach I, Kostyra DS, Hinz O (2015) Marketing automation.
Bus Inf Syst Eng 57(2):129–133

123