Scribd Logo
Upload

Welcome to Scribd!

  • 71 views

    Octave MLP Neural Networks

    Uploaded by

    RIska dewi
    This document discusses using multilayer perceptron (MLP) neural networks in Octave. It includes: 1. Steps for using neural networks as classifiers including preparing input-output patterns, pre-processing data, creating the neural network classifier, training and validating the network. 2. Details on creating an MLP network in Octave using the newff function including specifying the network structure and training parameters. 3. Training the network using the train function, which stops when validation performance does not improve over multiple epochs or other stopping criteria are met. 4. Testing the trained network on standardized test data using the sim function to generate predictions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Octave MLP Neural Networks

    Uploaded by

    RIska dewi
    71 views25 pages
    This document discusses using multilayer perceptron (MLP) neural networks in Octave. It includes: 1. Steps for using neural networks as classifiers including preparing input-output patterns, pre-processing data, creating the neural network classifier, training and validating the network. 2. Details on creating an MLP network in Octave using the newff function including specifying the network structure and training parameters. 3. Training the network using the train function, which stops when validation performance does not improve over multiple epochs or other stopping criteria are met. 4. Testing the trained network on standardized test data using the sim function to generate predictions.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses using multilayer perceptron (MLP) neural networks in Octave. It includes: 1. Steps for using neural networks as classifiers including preparing input-output patterns, pre-processing data, creating the neural network classifier, training and validating the network. 2. Details on creating an MLP network in Octave using the newff function including specifying the network structure and training parameters. 3. Training the network using the train function, which stops when validation performance does not improve over multiple epochs or other stopping criteria are met. 4. Testing the trained network on standardized test data using the sim function to generate predictions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    71 views25 pages

    Octave MLP Neural Networks

    Uploaded by

    RIska dewi
    This document discusses using multilayer perceptron (MLP) neural networks in Octave. It includes: 1. Steps for using neural networks as classifiers including preparing input-output patterns, pre-processing data, creating the neural network classifier, training and validating the network. 2. Details on creating an MLP network in Octave using the newff function including specifying the network structure and training parameters. 3. Training the network using the train function, which stops when validation performance does not improve over multiple epochs or other stopping criteria are met. 4. Testing the trained network on standardized test data using the sim function to generate predictions.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 25

    MLP Neural Networks

    Using Octave NN
    Package
    Nung Kion, Lee
    Faculty of Cognitive Sciences and Human Development
    UNIVERSITI MALAYSIA SARAWAK
    Introduction
     Octave provides a simple neural network package to construct the Multilayer
    Perceptron Neural Networks which is compatible (partially) with Matlab.
     Only feedforward backprogation neural network is implemented.
     Only one training algorithm is available (the Levenberg-Marquardt)
    Steps of Using Neural Networks as Classifier

    Prepare input- Pre-processing Creating Neural


    output patterns data Network classifier

    Testing for Training/cross


    performance validation
    Steps in creating NN in Octave
    • standardize inputs and Predict class of sim function
    Pre-processing outputs (if necessary). inputs of test
    • E.g. normalization dataset
    • prestd function

    Divide data into


    train, cross ,
    Use subset function or own
    validation, test coding

    Create the neural newff function


    network structure

    train function
    Train the neural
    network
    Take Note!!!
     In Octave, the training, test, or
    validation data must be prepared in
    columns. That is the rows are features
    and the columns are training input
    patterns.
     The target patterns/desired output of
    corresponding to input patterns must
    be arranged in columns as well.
    Input-output patterns format for Octave NN
    package Input Target

    Examples of Iris data

    Input
    Input output pairs for
    Octave NN

    Target
    Exercise 1
     Download the iris.data from Morpheus and extract the input and output
    patterns into inputdata and outputdata respectively.
     Prepare the inputdata and outputdata in the format suitable for Octave NN
    input.
    Min_Max Function
     Pulangkan nilai minimum dan maximum setiap baris dalam matrix untuk
    kegunaan fungsi newff.
    Exercise 2
     Use the min_max function to determine the min and max value of each
    feature value of the iris dataset you have loaded into Octave workspace in
    Exercise 1.

     Record your result.


    subset function
     Use to divide dataset into train, test and cross-validation.
     Recall that cross-validation data is used during training to achieve generalization.
     Reference: http://octave.sourceforge.net/nnet/function/subset.html

     Syntax:
    [mTrain, mTest, mVali] = subset (mData,nTargets,iOpti,fTest,fVali)
    mData – the complete training data with input and output values. E.g. in iris data, the mData is a 7
    x 150 matrix (4 inputs and 3 outputs).
    nTargets – the number of outputs (e.g. 3 in iris data).
    iOpti – randomize the columns of data (permute). 0- No; 1 or 2 – Yes (see ref)
    fTest – percentage of data use for testing (default is 1/3 of data)
    fVali – percentage of data use for cross-validation (default is 1/6 of data).
    Exercise 3
    Note that: the value of fTest + fVali < 1. If fTest + fVali = 1, then no data for
    training. It is suggested to use the default.

    E.g.: [mTrain, mTest, mVal] = subset(mData,1);


    returns three subsets mTrain – 0.5, mTest – 1/3 and mVal – 1/6 of the mData
    without random shuffling. mData has one output value.

    Load the iris.data into mData variable. Create train, test, validation
    subsets with 0.6, 0.2, 0.1 percentage. Verify your subsets using the size
    command on the produced subsets.
    Standardization of inputs
     Read the following articles to understand why standardization of input feature
    values.
    1. http://stackoverflow.com/questions/4674623/why-do-we-have-to-normalize-
    the-input-for-an-artificial-neural-network
    2. http://stats.stackexchange.com/questions/7757/data-normalization-and-
    standardization-in-neural-networks
    3. ftp://ftp.sas.com/pub/neural/FAQ2.html#A_std
     Standardization converts feature values to have zero mean and standard deviation
    is 1.
     We can standardize both input feature values and/or target values depending on
    the data used.
     The prestd is available by Octave for standardization which syntax can be found at
    http://dali.feld.cvut.cz/ucebna/matlab/toolbox/nnet/prestd.html?cmdname=prestd
    Exercise 4
     Standardize the input values of iris data you have loaded.
     Check the result for correctness.
    Creating MLP neural networks
    The MLP NN implemented by Octave is very limited. It only support the
    Levenberg-Marquardt (LM) backpropagation training algorithm, not the gradient
    descent method discussed in the class.

    See
    http://nopr.niscair.res.in/bitstream/123456789/8460/1/IJEMS%2012(5)%20434-
    442.pdf
    for LM training method.
    Syntax for creating MLP NN

    net = newff(PR,[S1 S2...SNl],{TF1 TF2...TFNl},BTF,BLF,PF)


    Description
    PR - R x 2 matrix of min and max values for R input elements.
    Si - Size of ith layer, for Nl layers.
    TFi - Transfer function of ith layer, default = 'tansig'.
    BTF - Backprop network training function, default = 'trainlm'.
    BLF - Backprop weight/bias learning function, default = 'learngdm'.
    PF - Performance function, default = 'mse'. Means square error.
    and returns an N layer feed-forward backprop network.
    The transfer functions TFi can be any differentiable transfer function such as tansig, logsig, or purelin.

    The net is a struct data type (e.g. struct in c++).


    Newff example
     Here is a problem consisting of inputs P and targets T that we would like to
    solve with a neural-network.
    P = [0 1 2 3 4 5 6 7 8 9 10]; #one input with eleven examples
    T = [0 1 2 3 4 3 2 1 2 3 4]; #one output
     Here a two-layer feed-forward network is created (one hidden layer, one
    output layer). The network's input ranges from [0 to 10]. The first layer has
    five tansig neurons, the second layer has one purelin neuron. The trainlm
    network training function is to be used.

    MLPnet = newff([0 10],[5 1],{'tansig' 'purelin'}); or


    MLPnet = newff([0 10],[5 1],{'tansig' 'purelin'},’trainlm’,’learngdm’,’mse’);
     MLPnet = newff([0 10],[5 1],{'tansig' 'purelin'});
    Pay attention to which activation function
    to be used for output neurons.
    tansig Different activation functions have
    different output range values. E.g. tansig
    is -1 to 1 and logsig is 0 to 1.
    purelin

    Input Hidden Output layer


    layer layer
    Training the neural network
    [net,tr,out,E] = train(MLPnet,mInputN,mOutput,[],[],VV);
    left side arguments:
    net: the trained network of the net structure MLPnet
    right side arguments:
    MLPnet : the untrained network, created with newff
    mInputN: normalized input matrix
    mOutput: output matrix (normalized or not)
    [] : unused parameter
    [] : unused parameter
    VV : validation data as struct. VV has two members:
    VV.P and VV.T. VV.P is the validation input data. VV.T is the validation target output.
    Training neural network (continue)
    Example:
     Suppose using Iris data.
    [Traindata, Testdata, Validation] = subset(Inputdata, 3,1, 0.4, 0.1);
    VV.P = Validation(1:4, : ); #first four rows are input features
    VV.T = Validation(5:7, : ); #5th to 8 rows are target output values
    trainsubset = Traindata(1:4, : );
    traintarget = Traindata(5:7, : );
    trainednet = train(MLPNet, trainsubset, traintarget, [], [], VV);

    Training will stop once one of the stopping criteria is met (see next slides).
    Training neural network (continue)
     The training parameters are stored in the neural network trainParam struct.
     Default values for trainParam are:
    net.trainParam.epochs = 100; #number of epochs
    net.trainParam.goal = 0; #minimum means square error to achieve
    net.trainParam.max_fail = 5;
    net.trainParam.min_grad = 1.0000e-010; #gradient of the mse graph
    net.trainParam.show = 50; #show mse every 50 epochs
    net.trainParam.time = Inf; #maximum training time in seconds
    net.show = 50; #how frequent the mse is displayed in training progress
    graph?
    max_fail: Validation vectors are used to stop training early if the network
    performance on the validation vectors fails to improve or remains the same for
    max_fail epochs in a row.
    Training neural networks (continue)
     Training stops when any of these conditions occurs:
     The maximum number of epochs (repetitions) is reached.
     The maximum amount of time is exceeded.
     Performance is minimized to the goal.
     The performance gradient falls below min_grad.
     Validation performance has increased more than max_fail times since the last time
    it decreased (when using validation).
    Training Neural Networks (continue)
    You should change the trainParam before using the train function to be effective.
    E.g.:
    MLPNet.trainParam.epochs = 500; #change maximum epochs to 500
    MLPNet.trainParam.goal = 0.01; #stop training if mse is <= 0.01
    trainednet = train(MLPNet, trainsubset, traintarget, [], [], VV);
    Test trained neural networks
     To test the trained neural network use the sim function
     Syntax:

    netoutput = sim (net, mInput)

    net : trained neural network using train function


    mInput: standardized test input values
    Exercise 5
     Download the mlpnn.m file from Morpheus and run the program in Octave.
     Please also download the input data file mData.txt.
     The input data has 13 columns: 12 inputs and 1 output.
     Put the mData.txt and mlpnn.m in the same folder.
     Read through the codes carefully and modify the training parameters.
    1. Change the neural networks number of hidden neurons.
    2. Run the network several times and observe the training progress graph.
    3. Change the trainParam with different values. Make sure suitable values are
    used.
    Exercise 6 – Using Iris.data
     Create a suitable neural network for the Iris.data file.
     Create suitable train, test, validation subsets.
     Train the neural networks using suitable parameters.
     Determine the accuracy of the neural network you have created.

    You might also like