Stock Market Value Prediction Using Neural Networks

Mahdi Pakdaman Naeini

IT & Computer Engineering
Department
Islamic Azad University
Parand Branch
Tehran, Iran
e-mail: [email protected]
Hamidreza Taremian
Engineering Department
Islamic Azad University
Tehran East Branch
Tehran, Iran
email:
[email protected]
Homa Baradaran Hashemi
School of Electrical and Computer
Engineering
University of Tehran
Tehran, Iran
e-mail: [email protected]
Abstract Neural networks, as an intelligent data mining
method, have been used in many different challenging pattern
recognition problems such as stock market prediction.
However, there is no formal method to determine the optimal
neural network for prediction purpose in the literatur. In this
paper, two kinds of neural networks, a feed forward multi
layer Perceptron (MLP) and an Elman recurrent network, are
used to predict a companys stock value based on its stock
share value history. The experimental results show that the
application of MLP neural network is more promising in
predicting stock value changes rather than Elman recurrent
network and linear regression method. However, based on the
standard measures that will be presented in the paper we find
that the Elman recurrent network and linear regression can
predict the direction of the changes of the stock value better
than the MLP.
Keywords- Stock market prediction; Data mining; neural
networks
I. INTRODUCTION
From the beginning of time it has been mans common
goal to make his life easier. The prevailing notion in society
is that wealth brings comfort and luxury, so it is not
surprising that there has been so much work done on ways to
predict the markets. Therefore forecasting stock price or
financial markets has been one of the biggest challenges to
the AI community. Various technical, fundamental, and
statistical indicators have been proposed and used with
varying results. However, none of these techniques or
combination of techniques has been successful enough. The
objective of forecasting research has been largely beyond the
capability of traditional AI research which has mainly
focused on developing intelligent systems that are supposed
to emulate human intelligence. By its nature the stock market
is mostly complex (non-linear) and volatile. With the
development of neural networks, researchers and investors
are hoping that the market mysteries can be unraveled .
Artificial Neural networks inspired by human brain cells
activity can learn the data patterns and generalize their
knowledge to recognize the future new patterns.
Researches on neural networks show that Neural
Networks have great capability in pattern recognition and
machine learning problems such as classification and
regression. These days Neural Networks are considered as a
common Data Mining method in different fields like
economy, business, industry, and science. [6]
The application of neural networks in prediction
problems is very promising due to some of their special
characteristics.
First, traditional methods such as linear regression and
logistic regression are model based while Neural Networks
are self-adjusting methods based on training data, so they
have the ability to solve the problem with a little knowledge
about its model and without constraining the prediction
model by adding any extra assumptions. Bsides, neural
networks can find the relationship between the input and
output of the system even if this relationship might be very
complicated because they are general function
approximators. Consequently, neural networks are well
applied to the problems in which extracting the relationships
among data is really difficult but on the other hand there
exists a large enough training data sets. It should be
mentioned that, although sometimes the rules or patterns that
we are looking for might not be easily found or the data
could be corrupted due to the process or measurement noise
of the system, it is still believed that the inductive learning
or data driven methods are the best way to deal with real
world prediction problems.
Second, Neural Networks have generalization ability
meaning that after training they can recognize the new
patterns even if they havent been in training set. Since in
most of the pattern recognition problems predicting future
events (unseen data) is based on previous data (training set),
the application of neural networks would be very beneficial.
Third, neural networks have been claimed to be general
function approximators. It is proved that an MLP neural
network can approximate any complex continuous function
that enables us to learn any complicated relationship between
the input and the output of the system.
The idea of using neural networks for predicting
problems was first expressed by Hu in 1964 which was used
for weather forecasting [8]. The absence of any learning
method for multi layer networks made it impossible to apply
these networks to complex prediction problems. But in 1980s
the back propagation algorithm was introduced for training
an MLP neural network. Werbos used this technique to train
a neural network in 1988 and claimed that neural networks
132 978-1-4244-7818-7/10/$26.00 c 2010 IEEE
are better than regression methods and Box-Jenkins model in
prediction problems [15]. The research on neural network
applications continued up to the point that all the winners of
the prediction contest in Santafa institute had used neural
networks [14].
In the recent decade so many researches have been done
on neural networks to predict the stock market changes. One
of the first efforts was by Kimmoto and his colleagues in
which they used neural networks to predict the index of
Tokyo stock market [10]. Mizuno and his colleagues also
used neural networks to predict the trade of stocks in Tokyo
stock market. Their method was able to predict with 63%
precision [12]. By combining Neural Networks and genetic
algorithms, Phau and his colleagues managed to predict the
direction of Singapore stock market with 81% precision.
In this paper we have suggested a predictive model based
on MLP neural network for predicting stock market changes
in Tehran Stock Exchange Corporation (TSEC). Using this
model, one can predict the next day stock value of a
company only based on its stock trade history and without
any information of the current market. Our experiments
show that the prediction error of this model is around 1.5%.
In the following we will briefly introduce the idea of
MLP neural network in the second section. The third section
presents the architecture of the proposed prediction model,
data preparation methods used in this research and the
evaluation criteria used for the evaluation of different
models. In the fourth section the experimental results of the
simulations on a companys data will be analyzed using
different models. Finally the fifth section concludes the
papers describing the future works of the study.
II. NEURAL NETWORKS
The idea of neural networks was first inspired by human
beings nervous system which consists of a number of simple
processing units called neuron (figure 1). Each neuron
receives some signals from outside or from other neurons
and then by processing them in activation function produces
its output and sends it to other neurons. Each input impact is
different from other inputs. For example in figure two the
impact of the i
th
neuron on j
th
neuron is shown with w
ij
, the
weight of the connection between neuron i and j.
Figure 1. P
Consequently the more is the weight w
ij
the stronger would
the connection be and vice versa.
In this paper, we focus on feed forward multi layer neural
networks. These networks are made of layers of neurons.
The first layer is the layer connected to the input data. After
that there could be one or more middle layers called hidden
layers. The last layer is the output layer which shows the
results. In feedback networks in contrast with recurrent
networks all the connections are toward the output layer.
Figure one shows a three layer feed forward Perceptron
network.
One of the learning methods in multi layer Perceptron
Neural Networks is the error back propagation in which the
network learns the pattern in data set and justifies the weight
of the connections in the inverse direction respect to the
gradient vector of Error function which is usually regularized
Figure 2: Perceptron neurons connections
sum of squared error. The back propagation method picks a
training vector from training data set and moves it from the
input layer toward the output layer. In the output layer the
error is calculated and propagated backward so the weight of
the connections will be corrected. This will usually go on
until the error reaches a pre defined value. Its proved that
we can approximate any continuous function with a three
layer feedback network with any precision. It should be said
that the learning speed will dramatically decrease according
to the increase of the number of neurons and layers of the
networks.
III. THE SUGGESTED NEURAL NETWORK
In spite of all the features mentioned for neural networks,
building a neural network for prediction is somehow
complicated. In order to have a satisfactory performance one
must consider some crucial factors in designing of such a
prediction model. One of the main factors is the network
structure including number of layers, neurons, and the
connections. Other factors to be considered are the activation
functions in each neuron, the training algorithm, data
normalization, selecting training and test set and also
evaluation measurements.
In the suggested model two neural networks, a multilayer
Perceptorn feed-forward and an Elman recurrent are used
and the back propagation algorithm is used to train these
networks.
The inputs to the neural networks are the lowest, the
highest and the average value in the d previous days. Other
information available about the stock market is not used
because our goal is to predict the value of the stock share
only based on the stock value history. In other words, the
inputs
outputs
Hidden layer Output layer Input layer
Figure 1: Architecture of a feed forward MLP
2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM) 133
proposed model can be viewed as a time series prediction
model.
This model uses a three layer neural network in which the
input layer has 3d neurons which get the lowest, the highest
and the average stock value in the last d days. In the hidden
layer there are h neurons which are fully connected to the
input and output layers. There is one neuron in output layer
which predicts the expected stock value of the next day of
the stock market.
A. Data Preparation
In this paper the lowest, the highest and the average value
of the stock market in the last d days are used to predict the
next days market value. The stock market data have been
extracted from Tehran Stock Market website. In this method
in contrast with other methods the disorders in the market
caused by social or political reasons are not omitted from the
data set because we want to predict the value based on the
value history. The simulation data was extracted in 2000 to
2005. In this period of time 1094 companies shares were
traded in Tehran Stock Market. The data used as input to the
system are the lowest, the highest, and the average value in
the last d days (d= {1, 2,, 10}). The prediction system
predicts the next days value using the above data.
In neural networks applications the input data is usually
normalized into the range of [0, 1] or [-1,1] according to the
activation function of the neurons. So in this paper the value
of the stock market is normalized into the range of [-1, 1]
using the (1) and then the neural networks are trained and
tested using the back propagation algorithm.
( )
ice ice
ice ice
Min Max
Min Max price
price
Pr Pr
Pr Pr
2

+
=
(1)
B. Evaluation criteria
In prediction problems general criteria like mean absolute
deviation, mean absolute percentage error, mean squared
error, and root mean squared error are calculated based on (2,
3, 4, 5). These criteria are preferred to be smaller since they
indicate the prediction error of the system.
In addition to the above criteria three other measures are
used to compare stock value prediction methods. The correct
forecast trend measure shows the percentage of correct
ssssss
_

=
nSet Validatio ys for all da
today
real
tomorrow
forecast
price price T
Set Validation
nd recast Tre Correct Fo ) , (
1
_

=
nSet Validatio ys for all da
tomorrow
real
tomorrow
forecast
price price
Set Validation
MAD
1
(2)
_

=
nSet Validatio ys for all da
tomorrow
real
tomorrow
real
tomorrow
forecast
price
price price
Set Validation
MAPE
1 (3)
( )
2 1
_

=
nSet Validatio ys for all da
tomorrow
real
tomorrow
forecast
price price
Set Validation
MSE
) 4 (
( )
2 1
_

=
nSet Validatio ys for all da
tomorrow
real
tomorrow
forecast
price price
Set Validation
RMSE
) 5 (
prediction of the changes in n+1
th
day relative to n
th
day (6).
When the prediction is completely random this number
would be around 0.5. As a result, in order to have a reliable
prediction method this feature should be at least above 0.5.
Although knowing the direction of the changes is an
important factor for decision making, we also need to know
the amount of the changes. There will be two other criteria to
determine the ratio of correct forecast trend to the real trend
of stock changes (7) and the ratio of incorrect forecast trend
to the real trend of stock changes (8). In the Ideal case, the
predicted ratio of correct forecast trend to the real stock
changes in (7) should be equal to one. In addition, if the
quantity of this ratio is smaller (or greater) than one, it will
indicate that the direction of the changes is predicted
correctly while the amount of changes has been predicted
less (or more). In the other hand, when the direction of stock
changes is predicted incorrectly, the quantity of the (8) is
desired to be closer to one as much as possible which shows
the prediction error is minimum in this case.
IV. SIMULATION RESULTS
In this section the prediction results of the two suggested
methods using multi layer Perceptron neural networks and
Elman recurrent network are compared to linear regression
method results.
The training algorithm in multi layer neural network is
Levenberg-Marquardt back propagation which can train any
neural networking using differentiable activity functions. In
this kind of error back propagation algorithm we use both the
gradient and the Jacobean of the performance measure (error
function)of the training set respect to the connection weights,
to justify the network weights [9, 11].
(6)
(7)
(8)

=
0
0 ) ).( ( 1
) , (
otherwise
price price price price if
price price T
today
real
tomorrow
real
today
real
tomorrow
forecast
today
real
tomorrow
forecast
_

=
nSet Validatio ys for all da
today
real
tomorrow
real
today
real
tomorrow
forecast
price price
price price
Set Validation al Trend
nd recast Tre Correct Fo 1
Re
_
(
(

+ =
nSet Validatio ys for all da
today
real
tomorrow
real
today
real
tomorrow
forecast
price price
price price
Set Validation al Trend
rend Forecast T Incorrect
1
1
Re
134 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM)
Figure 2. comparing minimum abstract deviation (MAD) in Elman and
MLP.
Figure 3. comparing mean absolute percentage error (MAPE) in Elman
and MLP
On the other hand, to train Elman network, the error back
propagation with momentum and adaptive learning rate is
used. This algorithm like Levenberg-Marquardt has the
capability to train any network using differentiable activity
functions. The weights of the network in this algorithm are
adjusted according to Gradient decent (with momentum)
based on the (9) in which mc is the momentum, d
Xprev
is the
previous change in the network weights and lr is the learning
rate.
X
pref
Xprev X
d
d
mc lr d mc d . . + = (9)
In each epoch if the performance measure (Mean squared
error) is moving toward its goal value, the learning rate will
Figure 4. comparing mean squared error (MSE) in Elman and MLP.
increase (in this simulation lr_inc=1.05). On the other hand
if the performance measure increases more than a threshold
(max_perf_inc=1.04), leaning rate will decrease with the rate
of lr_dec (in this simulation lr_dec=0.7) and the related
change which has increased the performance measure, will
not be applied to the network weights.
When one of the below happens the algorithm will stop.
The training epochs reach its maximum (in this
simulation 1000 epochs).
The performance measure reaches its goal.
(MSE=10to-6)
The gradient of the performance measure gets under
a threshold. (10to-6)
In the following the results of these two methods are
going to be compared to the result of the linear regression
method. In figure 3, 4, and 5 its clearly shown that the MLP
Neural Network has less MSE, MAPE, and MSE comparing
to Elman and linear regression though this method cannot
predict the direction of the changes as well as Elman and
regression (figure 6). However the linear regression method
predicts the direction of the changes well (figure 6), the error
in the prediction of the value is much more than multilayer
Perceptron and Elman (figure 7). The Elman network can
predict the direction of the changes better than multilayer
Perceptron (figure 6) but suffers from greater error in
prediction (figure 7) .
V. CONCLUSIONS
In this paper we used neural networks model to predict
the value of stock share in the next day using the previous
data about stock market value. For this purpose two different
well known types of neural networks were applied to the
problem. The obtained results show that for predicting the
direction of changes of the values in the next day none of
these methods are better than simple linear regression model.
But the error of the prediction of the amount of value
changes using MLP neural network is less than both Elman
network and linear regression method. In addition to this,
when the feed forward MLP neural network predicts the
direction of the changes correctly, the amount of change is
Mean squared error
0
10,00
20,00
30,00
40,00
50,00
60,00
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
M
e
a
n

s
q
u
a
r
e
d

e
r
r
o
r
Mean absolute percentage
0.00
0.00
0.01
0.01
0.02
0.02
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
M
e
a
n

a
b
s
o
l
u
t
e

p
e
r
c
e
n
t
a
g
e

e
r
r
o
r
Mean abstract deviation
50
55
60
65
70
75
80
85
90
95
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
M
e
a
n

a
b
s
t
r
a
c
t

d
e
v
i
a
t
i
o
n
MLP (1 hidden layer)
Elman (1 hidden layer)
Linear regression
2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM) 135
Figure 5. Maximum number of correct change direction forecast in Elman
and MLP.
Figure 6. Ratio of correct forecast of direction change to real direction
change in Elman and MLP.
completely close to the real one in comparison to the other
two mentioned methods. In future works of this study we
are going to apply other recently proposed regression
methods such as Support Vector Regression models which is
newer in the field of machine learning researches and
claimed to have good generalization ability due to
application of large margin concept.
REFERENCES
[1] Chen An-Sing, Leung Mark, Daouk Hazem, Application of Neural
Networks to an Emerging Financial Market: Forecasting and Trading
the Taiwan Stock Index, Computers & Operations Research, vol. 30,
2003, pp. 901923.
[2] G. Armano, M. Marchesi, and A. Murru, A Hybrid Genetic-Neural
Architecture for Stock Indexes Forecasting, Information Sciences,
vol. 170, 2005, pp 3-33.
Figure 7. Ratio of incorrect forecast of direction change to real direction
change in Elman and MLP.
[3] Qing Cao, Leggio Karyl, Marc Schniederjans, A Comparison
Between Fama and Frenchs Model and Artificial Neural Networks in
Predicting the Chinese Stock Market, Computers & Operations
Research, vol. 32, 2005, pp. 2499-2512.
[4] Olson Dennis, Mossman Charles,Neural Network Forecasts of
Canadian Stock Returns Using Accounting Ratios, International
Journal of Forecasting, vol.19, 2003, pp. 453-465.
[5] C.W.J. Granger and A.P. Anderson, An Introduction to Bilinear
Time Series Models, Vandenhoeck and Ruprecht, Gottingen, 1978.
[6] G. Grudnitzky and L. Osburn, Forecasting S&P and Gold Futures
Prices: An Application of Neural Networks, Journal of Futures
Markets, vol. 13, No. 6, pp. 631-643, September 1993.
[7] Zhang Guoqiang, Patuwo Eddy, Hu Michael, Forecasting with
Artificial Neural Networks: The State of the Art, International
Journal of Forecasting, vol. 14, 1998, pp 35-62.
[8] M.J.C. Hu, Application of the Adaline System to Weather
Forecasting, Master Thesis, Technical Report 6775-1, Stanford
Electronic Laboratories, Stanford, CA, June 1964.
[9] Levenberg K., A Method for the Solution of Certain Problems in
Least Squares, Quart. Appl. Math. , no. 2, 1944, pp. 164-168.
[10] T. Kimoto, K. Asakawa, M. Yoda, and M. Takeoka, "Stock market
prediction system with modular neural network," Proceedings of the
International Joint Conference on Neural Networks, 1990, pp. 1-6.
[11] D. Marquardt, An Algorithm for Least Squares Estimation of
Nonlinear Parameters, SIAM Journal of Applied Mathematics, no.
11, 1963, pp. 431-441.
[12] H. Mizuno, M. Kosaka, H. Yajima, and N. Komoda, Application of
Neural Network to Technical Analysis of Stock Market Prediction,
Studies in Informatic and Control, vol.7, no.3, 1998, pp.111-120.
[13] H. Tong and K.S. Lim, Threshold Autoregressive, Limit Cycles and
Cyclical Data, Journal of the Royal Statistical Society Series, vol. B-
42, no. 3, 1980, pp. 245292.
[14] A.S. Weigend and N.A. Gershenfeld, Time Series Prediction:
Forecasting the Future and Understanding the Past, Addison-
Wesley, Reading, MA, 1993.
[15] P.J. Werbos, Generalization of back propagation with application to
a recurrent gas market model, Neural Networks, vol. 1, pp. 339-356,
1988.
[16] Md. Rafiul Hassan and Baikunth Nath, Stock Market Forecasting
Using Hidden Markov Model: A New Approach, 5th International
Conference on Intelligent Systems Design and Applications, 2005.
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
d
i
r
e
c
t
i
o
n

c
h
a
n
g
e
R
a
t
i
o

o
f

i
n
c
o
r
r
e
c
t

f
r
o
e
c
a
s
t

d
i
r
e
c
t
i
o
n

c
h
a
n
g
e

t
o

r
e
a
l

Ratio of correct forecast direction change to real direction change
0.1
1.0
10.0
100.0
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
d
i
r
e
c
t
i
o
n

c
h
a
n
g
e
R
a
t
i
o

o
f

c
o
r
r
e
c
t

f
o
r
e
c
a
s
t

d
i
r
e
c
t
i
o
n

c
h
a
n
g
e

t
o

r
e
a
l

Correct direction change
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1 2 3 4 5 6 7 8 9 10
Past days (as NN input)
C
o
r
r
e
c
t

d
i
r
e
c
t
i
o
n

c
h
a
n
g
e

f
o
r
e
c
a
s
t
Ratio of incorrect forecast direction change to real direction change
136 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM)