Scribd
Upload
2 views

Pertemuan 2 (2)

Uploaded by

Mardi Yanto
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Pertemuan 2 (2)

Uploaded by

Mardi Yanto
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

PSIKS2W002 - Data Science

Pertemuan 10: Model Evaluation

PROGRAM MAGISTER ILMU KOMPUTER


FAKULTAS ILMU KOMPUTER
UNIVERSITAS AMIKOM PURWOKERTO
PROSES PEMBELAJARAN MESIN

Source: (Gupta & Bagchi, 2024)


JENIS PEMBELAJARAN MESIN

(Source: Peng et. al., 2021)


SUPERVISED LEARNING
If output space contains  categorical random
variables:
The task is called classification, and the model is
called classifier.

If output space contains  numeric random


variables (continuous or discrete):
The task is known as regression, and the model is
called regressor.

(Source: Zollanvari, 2023)


UNSUPERVISED LEARNING
The clustering goal is to discover
groups of similar observations within
a given data. As a result, the output
space includes all possible partitions
of the given sample

(Source: Zollanvari, 2023)


Forecasting/Prediction
• Prediction or forecasting in machine learning involves using computational algorithms
to analyze historical data and make informed guesses about future outcomes (Ranjan
& Goldsztein, 2022).
• Machine learning models are trained on data to recognize patterns and relationships,
enabling them to predict outcomes based on new, unseen data (Maeda, 2018).
• These predictions are made by leveraging the inherent relationships between input
and output data, with the predictive accuracy being influenced by factors such as data
features and size (Maeda, 2018).
Forecasting/Prediction
• Forecasting/prediction digunakan untuk memperkirakan nilai-nilai data time series (deret
waktu) di masa depan.
• Contoh data time series yang dapat digunakan dalam pembelajaran mesin:
1. Data harga saham harian selama beberapa tahun. Digunakan untuk memprediksi harga
saham di masa depan.
2. Data suhu, curah hujan, tekanan udara harian dari stasiun cuaca selama beberapa
tahun. Digunakan untuk peramalan cuaca di masa depan.
3. Data konsumsi bahan bakar harian dari sebuah pabrik atau armada kendaraan.
Digunakan untuk memprediksi jumlah konsumsi bahan bakar di masa depan.
REGRESSION MODEL
• A regression model in machine learning is a statistical technique used to
understand the relationship between a dependent variable and one or more
independent variables.
• It aims to predict the value of the dependent variable based on the values of the
independent variables.
• Regression models can be broadly categorized into linear and nonlinear models.
Linear regression models assume a linear relationship between variables, while
non-linear regression models allow for more complex and non-linear relationships
to be captured in the data.
JENIS MODEL REGRESI
• Linear Regression (Simple Linear Regression & Multiple Linear Regression)
• Polynomial Regression
• Stepwise Regression
• Decision Tree Regression
• Random Forest Regression
• Support Vector Regression
• Ridge Regression
• Lasso Regression
• ElasticNet Regression
• Bayesian Linear Regression

https://medium.com/@simranjeetsingh1497/the-ultimate-guide-to-machine-learning-machine-learning-
algorithms-part-4-597fbf8bd5dd
EVALUASI MODEL REGRESI
• Forecast Error
• Mean Absolute Error
• Mean Squared Error
• Root Mean Squared Error
• R-squared

Sumber Pustaka:
1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8279135/pdf/peerj-cs-07-623.pdf
2. Brownlee, J. 2018. Introduction to Time Series Forecasting with Python.
FORECAST ERROR
• The forecast error is calculated as the expected value minus the predicted value.
forecast error = expected value - predicted value
• The example below demonstrates how the forecast error can be calculated for a series of 5
predictions compared to 5 expected values.

Running the example calculates the forecast error for each of the 5 predictions:

A forecast error of zero indicates no error.


MEAN ABSOLUTE ERROR (MAE)
MAE is calculated as the average of the forecast error values, where all of the forecast values
are forced to be positive.
mean absolute error = mean(abs(forecast_error))

Example:

Output:
A mean absolute error of zero indicates no error.
MEAN SQUARED ERROR (MSE)
MSE is calculated as the average of the squared forecast error values. Squaring the forecast
error values forces them to be positive.
mean_squared_error = mean(forecast_error²)

Example:

Output:
A mean squared error of zero indicates perfect skill, or no error.
(RMSE)
The MSE is transformed back into the original units of the predictions by taking the square root of the
mean squared error score. This is called the RMSE.
𝑟𝑚𝑠𝑒 = 𝑚𝑒𝑎𝑛_𝑠𝑞𝑢𝑎𝑟𝑒𝑑_𝑒𝑟𝑟𝑜𝑟

Example:

Output:

RMSE of zero indicates no error.


R-SQUARED
Coefficient of determination (R² or R-squared):

The coefficient of determination (Wright, 1921) can be interpreted as the proportion of the
variance in the dependent variable that is predictable from the independent variables.
(worst value = −∞; best value = +1)
TUGAS KELOMPOK (1)
1. Gunakan dataset IceCreamData.csv untuk membuat model prediksi Tingkat penjualan
ice cream pada kondisi temperatur suhu lingkungan tertentu.
2. Gunakan data latih: 70% dan data uji: 30%.
3. Gunakan model prediksi Simple Linear Regression, Decision Trees Regression, Random
Forest Regression, dan Support Vector Regression (SVR) untuk memprediksi
“Revenue.” Gunakan kernel SVR: ‘linear’, ‘poly’, ‘rbf’ dalam eksperimen. Berdasarkan
metrik evaluasi Mean Squared Error, R-squared, Mean Absolute Percentage Error, dan
RMSE; Jelaskan model regresi mana yang terbaik dari eksperimen untuk keempat
model regresi tersebut!
TUGAS KELOMPOK (2)
Jelaskan metode evaluasi untuk model klasifikasi menggunakan:
 Confusion Matrix
 Precision
 Recall
 F-measure/F1-score
 Accuracy
 ROC (receiver operator characteristic) curves
 AUC (area under the curve)
 Hamming Loss
PENGUMPULAN TUGAS KELOMPOK
1. Pengumpulan tugas dalam bentuk softcopy: (1) Laporan Project; (2) File Source Code. Laporan

project berisi penjelasan hasil eksperimen model pembelajaran mesin.

2. Softcopy diunggah melalui link berikut: https://forms.gle/mhQSiDsQvz7otVAJA. Pilih penugasan

“Pertemuan 10.”

3. Pengunggahan softcopy paling lambat hari Jumat, 27 Desember 2024.

4. Pada pertemuan minggu depan, perwakilan kelas yang ditunjuk mendemonstrasikan dan

menjelaskan hasil eksperimen model prediksi yang telah diuji.

You might also like