Ask Ghassem - Recent questions in Machine Learning

Step-by-Step Hidden State Calculation in a Recurrent Neural Network

Mon, 01 Dec 2025 18:32:24 +0000

Consider a simplified Recurrent Neural Network (RNN) with a single input and a single output. The hidden state is updated using the recurrence:

$$ h_t = \text{ReLU}(W_{ih} \cdot x_t + W_{hh} \cdot h_{t-1}) $$

Assume the following:

$ x_t = 3 $ for every time step
$ h_0 = 0 $
$ W_{ih} = 0.4 $
$ W_{hh} = 0.6 $
Activation function: ReLU

Compute the value of the hidden state $ h_4 $ at time $ t = 4 $.

How to calculate feed-forward (forward-propagation) in neural network for classification?

Wed, 02 Oct 2024 14:47:26 +0000

For the following neural network, calculate accuracy of classification, given these settings

When to use one hot encode a category and when to segment by category?

Wed, 22 Feb 2023 20:30:38 +0000

When pre processing data for machine learning. Is there any difference in using one hot encoding to turn categoric variables into numeric variables or to segment the data and the model being used along the category. So say you run a multivariate regression model on data covering 5 cities. Would a single model with one variable for each city be more better or worse than having 5 models specific for each city? Or is there no difference? Or does it depend on certain factors and intuition?

How to calculate the residual errors, (MSE),(MAE), and (RMSE)?

Fri, 27 Jan 2023 04:09:28 +0000

Given the following sample dataset with 5 samples and 2 features:

Sample	Feature 1	Feature 2	Actual Value	Predicted Value
1	2	3	4	6
2	3	4	5	6
3	4	5	6	7
4	5	6	7	8
5	6	7	8	9

Calculate the residual errors, mean squared error (MSE), mean absolute error (MAE), and root mean squared error (RMSE) using a sample model.

Creating tables from unstructured texts about stock market

Tue, 02 Aug 2022 00:47:49 +0000

I am trying to extract information such as profits, revenues and others along with their corresponding dates and quarters from an unstructured text about stock market and convert it into a report in the table form but as there is not format of the input text, it is hard to know which entity belong to what date and quarters and which value belong to which entity. Chunking works on few documents but not enough. Is there any unsupervised way to linking entities with their corresponding dates, values and quarters?

Kmeans clustering in python - Giving original labels to predicted clusters

Wed, 27 Apr 2022 05:32:54 +0000

I have a dataset with 7 labels in the target variable.

X = data.drop('target', axis=1)
Y = data['target']
Y.unique()

array(['Normal_Weight', 'Overweight_Level_I', 'Overweight_Level_II',
'Obesity_Type_I', 'Insufficient_Weight', 'Obesity_Type_II',
'Obesity_Type_III'], dtype=object)

km = KMeans(n_clusters=7, init="k-means++", random_state=300)
km.fit_predict(X)
np.unique(km.labels_)

array([0, 1, 2, 3, 4, 5, 6])

After performing KMean clustering algorithm with number of clusters as 7, the resulted clusters are labeled as 0,1,2,3,4,5,6. But how to know which real label matches with the predicted label.

In other words, I want to know how to give original label names to new predicted labels, so that they can be compared like how many values are clustered correctly (Accuracy).

Bankruptcy prediction and credit card

Sun, 10 Apr 2022 05:50:14 +0000

Hello everyone newbie data scientist here.
I'm working on a project to predict companies (probability of default) bankruptcy probability and to assign them a credit rating/score based on that :
For example below 50 probability is good and above is bad ( just for the example)
I have a dataset contains financial ratios and a class refers if the company is bankrupted or not (0 and one).
I'm planning to use this models:
Logistic regression linear discrimination analysis, decision trees, random forest, ANN, adaboost, Svm.

The question is and i know it is a dumb question:
Does those models return a probability? Which i can transform to labels, I saw that in a thesis and I'm not sure about it.

Otherwise, any guidance,tips anything will be appreciated.

When dealing with categorical values, should the 'year' column be encoded using OHE or OrdinalEncoder?

Sat, 18 Dec 2021 18:46:07 +0000

It's a car prices dataset, and so I'm assuming that the more recent the more value a car should have. The values in the 'year' column simply consist of years from 1995 to 2020.
I am trying to predict the selling price of the car.

I'm a bit new to ML, currently still doing my undergraduate so any help / tips are appreciated. Thank you.

How to create a Decision Tree using the ID3 algorithm?

Wed, 01 Dec 2021 11:26:02 +0000

NASA wants to be able to discriminate between Martians (M) and Humans (H) based on the
following characteristics: Green ∈{N, Y }, Legs ∈{2, 3}, Height ∈{S, T}, Smelly ∈{N, Y }.
Our available training data is as follows:

https://i.imgur.com/3bC391L.png

a) Greedily learn a decision tree using the ID3 algorithm and draw the tree.
b) Write the learned concept for Martian as a set of conjunctive rules (e.g., if (green=Y
and legs=2 and height=T and smelly=N), then Martian; else if ... then Martian; ...; else
Human).

Which algorithm is best to detect anomalies within a data set of 5k+ user-login events?

Tue, 05 Oct 2021 17:45:38 +0000

I am trying to build an unsupervised ML model to detect anomalies within 5000+ users' login data. I selected 5 features contained within each of the user-login events (e.g. IP, hour of day, day of week, device_id, OS). I am looking for the best algorithm to use. I am considering using density function to determine probabilities of the feature values and whether an event is an outlier. The problem is that feature values are only relevant to the specific user. For example, you cannot compare login IP across users, login IP is only applicable to the user.
Ultimately, I want to detect events that are changes in a user login behavior, like different IP, day, hour, device_id, or OS, where the more features that have changed increase the probability of an outlier.
At this point, I am not sure how to build a model with data that contains multiple users, because I don't know how to separate the user data so the model is trained per user and finding anomalies within the individual user's features.

I also don't have any labeled data to use for testing, should I fabricate some?

Any advice greatly appreciated.

Thank you!

How we incorporate the polyline in machine learnning tools

Wed, 29 Sep 2021 06:16:30 +0000

Suppose I have to predict the traffic of a road segment based on available data such as number of houses and business along the road segment. Which machine learning tool would be the option to use that can incorporate the road segment (polylines) through coordinates in the attributes.

Do I need to save the standardization transformation?

Tue, 15 Dec 2020 13:06:48 +0000

When I standardized my data when I created my model. Do I need to save the standardization transformation when I want to predict with my model new data ?

Why should I use Dynamic Time Warping over GMM for timer series clustering?

Fri, 04 Dec 2020 03:19:16 +0000

How to predict from unseen data?

Tue, 17 Nov 2020 16:18:28 +0000

Hi. I have a question about model-based predictions when data is only available after the fact. Let me give you an example. I try to predict the result (HOME, AWAY or a DRAW) of the match based on data like number of shots, ball possession, number of fouls, etc.

TARGET	TEAM 1	TEAM 2	possesion team 1	possesion team 2	shots team 1	shots team 2	fouls team 1	fouls team 2
HOME	Arsenal	Chelsea	60	40	12	8	5	7

TARGET

TEAM 1

TEAM 2

possesion

team 1

possesion

team 2

shots

team 1

shots

team 2

fouls

team 1

fouls

team 2

HOME

Arsenal

Chelsea

Let's say I'm already after training the model and I want to see if I can predict the upcoming match. However, this match is only a few days away and I want to know the result of the model today. I understand that if the match had already taken place and I had the data, I could test it on the model and get the result. The goal is for the model to predict what will happen before the match.

Is it possible at all? What are my options? Should I only select pre-match variables? For example, last game form, match referee etc or should I aggregate the variables and include average possession, average shots and average number of fouls from recent matches?

How to model unknown yet data

Tue, 27 Oct 2020 10:39:47 +0000

So far, I have modeled on known historical data. What if there are variables known only after the fact?
Let me give you an example. I want to predict the outcome of the match, win, lose or draw. I use variables from previous games such as ball possession, number of shots, corners, etc. Let's say the Chelsea-Arsenal game is approaching Saturday. How am I supposed to build a model and predict the result if this data is not yet available for my event? What to do in such cases, is it possible to forecast such data?

From microarray data, which tools of pattern recognition can you apply to identify the genes responsible for diseases?

Thu, 15 Oct 2020 20:11:31 +0000

“During the last decade, the advent of microarray datasets stimulated a new line of research called Bioinformatics. A microarray database is a repository containing microarray gene expression data. Microarray data pose a great challenge for computational techniques, due to their large dimensionality (up to several tens of thousands of genes) and their sample sizes. Furthermore, additional experimental complications like noise and variability render the analysis of microarray data an exciting domain [Saeys et al. 2007, Bioinformatics]".

In light of the aforesaid excerpt, from microarray data which tools of the pattern recognition can you apply to identify the genes responsible for diseases like cancer? Explain how.

Can we use a trained model to supervise the other machine learning models?

Mon, 28 Sep 2020 14:17:37 +0000

Is that possible to train a machine using another trained machine?

Where can I find illustrative real life machine learning examples (In business, work. etc.)?

Tue, 22 Sep 2020 00:47:09 +0000

Is there a website for finding illustrative real-life examples of using machine learning? For instance: for End to End Machine Learning, End to End Machine Learning, Classification, Clustering, and Unsupervised Learning.

Where can I find simple machine learning mathematics explained visually?

Mon, 21 Sep 2020 23:55:12 +0000

Could you please let me know where I can find simple machine learning mathematics explained visually?

How to update the weights in backpropagation algorithm when activation function in not linear?

Mon, 10 Aug 2020 21:55:19 +0000

The goal of backpropagation is to optimize the weights so that the neural network can learn how to correctly map arbitrary inputs to outputs.

Assume for the following neural network, inputs = [$i_1,i_2$] = [0.05, 0.10], we want the neural network to output = [$o_1$,$o_2$] = [0.01, 0.99], and for learning rate, $\alpha=0.5$.
In addition, the activation function for the hidden layer (both $h_1$ and $h_2$) is sigmoid (logistic):

$S(x)=\frac{1}{1+e^{-x}}$

https://i.imgur.com/cnY5feu.png

Hint:
$w_{new} = w_{old} - \alpha \frac{\partial E}{\partial w}$

$E_{\text {total}}=\sum \frac{1}{2}(\text {target}-\text {output})^{2}$

a) Show step by step solution to calculate weights $w_1$ to $w_8$ after one update in table below.
b) Calculate initial error and error after one update (assume biases $[b_1,b_2]$ are not changing during the updates).

Updating weights in backpropagation algorithm
Weights	Initialization	New weights after one step
$w1$	0.15	?
$w2$	0.20	?
$w3$	0.25	?
$w4$	0.30	?
$w5$	0.40	?
$w6$	0.45	?
$w7$	0.50	?
$w8$	0.55	?

How to calculate the class probabilities and classify using Naive Bayes classifier?

Mon, 10 Aug 2020 21:26:28 +0000

We have data on 1000 pieces of fruit. The fruit being a Banana, Orange or some Other fruit and imagine we know 3 features of each fruit, whether it’s long or not, sweet or not and yellow or not, as displayed in the table below:

https://i.imgur.com/gOFzVXL.png

A piece of an unknown fruit with these features are provided: Long, Sweet and Yellow.

Calculate probability of each of these 3 classes based on Naive Bayes Classification algorithm and report the class.

What is difference between Support vector machine and Support Vector Classification?

Wed, 13 May 2020 20:22:23 +0000

Pre trainned word Embeddings and Preproceess

Fri, 10 Apr 2020 12:08:09 +0000

How should i preprocess my data if i am gonna use a pretrainned word embedding like glove or word2vec?Should I use stemming or stopword removal techniques?

Can PCA be used for supervised learning?

Tue, 18 Feb 2020 21:49:18 +0000

Can PCA be used for supervised learning???

I've seen some data scientists using PCA to transform their data for only numerical variables.

However, some other data scientists say that it is only used for unsupervised ML techniques.

How to calculate residual errors for linear regression and interpret regression metrics?

Tue, 18 Feb 2020 18:30:51 +0000

Assuming we have a linear regression equation and some data points (sample), how can we calculate residual error for each data point, and total cost based on the metrics such as MAE, MSE, RMSE, MAPE, or MPE if we have their formula?

Can I use a single Pipeline for multiple estimators in scikit-learn?

Tue, 18 Feb 2020 14:14:30 +0000

Is there any proper way to combine multiple classifiers and their parameter grids in one Pipeline?

why after applied the OneHotEncoder, it will create one more column whatis that column for?

Thu, 13 Feb 2020 02:53:52 +0000

How to map (string compare) a string with 10000+ strings in DB? which is the best way to do it?

Mon, 10 Feb 2020 06:41:55 +0000

I have a result string like this A, C, D, F, B,

In my DB I have 10,000+ combinations like this 1. A, B, C, D 2. A, C, B, I, D, W, Z etc....

Now I want to search for my result string with 10,000+ strings.

Which is the best way to map (string compare) and get the result match.

How can I find the "Sate of the art" approaches in Machine Learning?

Sat, 08 Feb 2020 00:56:39 +0000

If I want to find the latest trends in Machine Learning and best approaches known as the "State of the art" approach, what resources I can use?

How to calculate the probability and accuracy of a Logistic Regression classifier?

Mon, 03 Feb 2020 20:31:49 +0000

How to solve this problem?

https://i.imgur.com/8urywpf.jpg

Q1) Complete the ? sections

Q2) Accuracy of system if threshold = 0.5?

Q3) Accuracy of system if threshold = 0.95?

How to calculate Accuracy, Precision, Recall or F1?

Mon, 27 Jan 2020 19:22:26 +0000

In the following example, calculate Accuracy, Precision, Recall or F1?

https://i.imgur.com/OezFpqC.png

score() vs accuracy_score() in sklearn

Tue, 21 Jan 2020 21:28:11 +0000

Hi,

Since I still have confuse to use the score() and accuracy_score(), so I want to confirm my test assumption.
Q1: score(), we use the split data to test the accuracy by knn.score(X_test, y_test) to prevent bias using the same training data, right? here knn.score(X_test, y_test) just compare the pair of test value.

Q2: accuracy_score from sklearn.metrics to test the predicted output of target value "y_pred" with the y_test, using accuracy_score(y_test, y_pred), just compare the actual target value and predicted target value?

Q3.My result is the same after using both methods, are they doing the same thing?

Q4.using accuracy_score(), I can using to compare the split training target data y_train with the y_train_pred(return form knn.predict(X_train) ). Then it should be OK now, using it to show the accuracy by accuracy_score(y_train, y_train_pred), since the prediction is done and just compare the original data, then the bias does not exist?

Thanks.

Best algorithm for table reservation

Mon, 21 Oct 2019 18:03:19 +0000

What kind of algorithm would best for following problem.
I try to forecast reservation of different kind of tables. Let's say I have 100 different tables, which are reserved for from 17.00-22.00 daily. Each table is either reserved (1) or available (0) on certain hour. I'm interested in to forecast each table based on history data from 2 previous weeks. So for example result is that tomorrow 18.00-19.00 certain table is either 0 (available) or 1 (reserved).

What are the types of Classification and regression algorithms in Machine learning ?

Thu, 27 Jun 2019 21:00:05 +0000

For example, Logistic regression is classification likewise what are the other types? I am a bit confused.

How to perform a classification or regression using k-NN?

Thu, 27 Jun 2019 02:54:42 +0000

Suppose, you have given the following dataset where x and y are the 2 features and color Red or Blue is the target variable.

a) A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN, what you predict as the color for this data point?

Dataset
x	y	Color
-1	1	Red
0	1	Blue
0	2	Red
1	-1	Red
1	0	Blue
1	2	Blue
2	2	Red
2	3	Blue

b) Now assume we have the following dataset and the target value is the price. A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN. What would be the estimated price?

Dataset
x	y	Price
-1	1	$100
0	1	$50
0	2	$20
1	-1	$40
1	0	$30
1	2	$40
2	2	$70
2	3	$30

How to calculate k-means clustering with a numerical example?

Thu, 27 Jun 2019 02:16:32 +0000

Use the k-means algorithm and Euclidean distance to cluster the following 8 examples into 3 clusters:

$A1=(2,10), A2=(2,5), A3=(8,4), A4=(5,8), A5=(7,5), A6=(6,4), A7=(1,2), A8=(4,9)$.

Suppose that the initial seeds (centers of each cluster) are $A1$, $A4$ and $A7$. Run the k-means algorithm for 1 epoch only. At the end of this epoch show:

a) The new clusters (i.e. the examples belonging to each cluster)

b) The centers of the new clusters

c) Draw a 10 by 10 space with all the 8 points and show the clusters after the first epoch and the new centroids.

d) How many more iterations are needed to converge? Draw the result for each epoch

How to calculate the class probabilities and classify using Naive Bayes classifier for NLP?

Wed, 26 Jun 2019 19:43:41 +0000

We want to use Naive Bayes for tagging documents. It is a classification task that we want to assign a class (tag) to each string. We currently have two tags: Sport and Not Sport

Which tag does the sentence A very close game belong to? Using Naive Bayes classifier, calculate the class probability for Sport and Not sport for this sentence based on the dataset and decide about the tag.

Text	Tag
“A great game”	Sports
“The election was over”	Not sports
“Very clean match”	Sports
“A clean but forgettable game”	Sports
“It was a close election”	Not sports

How to calculate Covariance Matrix and Principal Components for PCA?

Wed, 26 Jun 2019 10:40:02 +0000

The dataset with two features $(x,y)$ is shown as follows (note $y$ in this example is the second feature, not a target value):

x	y
2.5	2.4
0.5	0.7
2.2	2.9
1.9	2.2
3.1	3.0
2.3	2.7
2.0	1.6
1.0	1.1
1.5	1.6
1.1	0.9

a) Calculate the Covariance Matrix.
b) Calculate eigenvalues and eigenvectors
c) Calculate all the PCs
d) How much percent of the total variance in the dataset is explained by each PC?

What is the difference between cross-validation and validation set?

Wed, 19 Jun 2019 18:39:39 +0000

I am confused about this figure. Is not this a cross-validation test or we have a fixed few examples for which it is tested while you also have various folds being tested at the same time?

https://i.imgur.com/aVru1MX.png

In DBSCAN algorithm, how should we choose optimal eps and minimum points?

Thu, 13 Jun 2019 17:22:08 +0000

How to optimize weights in Logistic Regression?

Wed, 05 Jun 2019 17:38:50 +0000

The hypothesis (model) of Logistic Regression which is a binary classifier ( $y =\{0,1\} $) is given in the equation below:

Hypothesis

$S(z)=P(y=1 | x)=h_{\theta}(x)=\frac{1}{1+\exp \left(-\theta^{\top} x\right)}$

Which calculates probability of Class 1, and by setting a threshold (such as $h_{\theta}(x) > 0.5 $) we can classify to 1, or 0.

Cost function

The cost function for Logistic Regression is defined as below. It is called binary cross entropy loss function:

$J(\theta)=-\frac{1}{m} \sum_{i}^{m}\left(y^{(i)} \log \left(h_{\theta}\left(x^{(i)}\right)\right)+\left(1-y^{(i)}\right) \log \left(1-h_{\theta}\left(x^{(i)}\right)\right)\right)$

Iterative updates

Assume we start all the model parameters with a random number (in this case the only model parameters we have are $\theta_j$ and assume we initialized all of them with 1: for all $\theta_j = 1$ for $j=\{0,1,...,n\}$ and $n$ is the number of features we have)

$\theta_{j_{n e w}} \leftarrow \theta_{j_{o l d}}+\alpha \times \frac{1}{m} \sum_{i=1}^{m}\left[y^{(i)}-\sigma\left(\theta_{j_{o l d}}^{\top}\left(x^{(i)}\right)\right)\right] x_{j}^{(i)}$

Where:
$m =$ number of rows in the training batch
$x^{(i)} = $ the feature vector for sample $i$
$\theta_j = $ the coefficient vector corresponding the features
$y^{(i)} = $ actual class label for sample $i$ in the training batch
$x_{j}^{(i)} = $ the element (column) $j$ in the feature vector for sample $i$
$\alpha =$ the learning rate

Dataset

The training dataset of pass/fail in an exam for 5 students is given in the table below:

If we initialize all the model parameters with 1 (all $\theta_j = 1$), and the learning rate is $\alpha = 0.1$, and if we use batch gradient descent, what will be the:

$a)$ Accuracy of the model at initialization of the train set ($\text{accuracy} = \frac{\text{number of correct classifications}}{\text{all classifications}}$)?
$b)$ Cost at initialization?
$c)$ Cost after 1 epoch?
$d)$ Repeat all $a,b,c$ steps if we use mini-batch gradient descent and $\text{batch size} = 2$

(Hint: For $x_{j}^{(i)}$ when $j=0$ we have $x_{0}^{(i)} = 1$ for all $i$ )

Could you please explain math symbols behind Machine Learning equations?

Sat, 18 May 2019 19:56:35 +0000

How do I Plot the linear classifier calculated with LIBLINEAR using sklearn?

Thu, 16 May 2019 08:13:06 +0000

Make a scatter plot where the x-axis is the height of the citizens and the y-axis is the weight of the citizens. The color of the points need to be different for males and females. In the same figure, plot the linear classifier calculated with LIBLINEAR using sklearn

how can i convert LSTM model to linear regression model?

Mon, 29 Apr 2019 11:50:07 +0000

Here is LSTM predict model and i want to convert Linear Regression.

...
model.fit(x_train, y_train, epochs=10, batch_size=16)

trainPredict = model.predict(x_train)
testPredict = model.predict(x_test)
# invert predictions
trainPredict = scaler.inverse_transform(trainPredict)
trainY = scaler.inverse_transform([y_train])
testPredict = scaler.inverse_transform(testPredict)
testY = scaler.inverse_transform([y_test])

I tried,

y = trainPredict
x = range(0,len(y))
XGBModel = XGBRegressor()
XGBModel.fit(x,y, verbose=False)

And the result is :

Check failed: preds.Size() == info.labels_.Size() (1 vs. 56969) labels are not correctly providedpreds.size=1, label.size=56969'

I don't know why this error occurs. How can I solve this problem?

How to update weights in backpropagation algorithm (a numerical example)?

Thu, 11 Apr 2019 17:02:04 +0000

Assume we have the following neural network and all activation functions are $f(z)=z$. If the weights are initialized with the values you see in table below, what will be new updated weights after one step if learning rate, $\alpha = 0.05$?

Assume the input values are [$i_1$,$i_2$] = [2,3] and target value $out = 1$.

Hint:
$w_{new} = w_{old} - \alpha \frac{\partial E}{\partial w}$

$E_{\text {total}}=\sum \frac{1}{2}(\text {target}-\text {output})^{2}$

Updating weights in backpropagation algorithm
Weights	Initialization	New weights after one step
$w1$	0.11	?
$w2$	0.21	?
$w3$	0.12	?
$w4$	0.08	?
$w5$	0.14	?
$w6$	0.15	?

https://i.imgur.com/v0RMeOQ.png

How to calculate univariate linear regression?

Thu, 11 Apr 2019 16:46:47 +0000

For the following dataset, calculate the regression equation $\hat{y} = ax+b$

dataset
x	y
1	42
3	50
10	75
16	100
26	150
36	200

How to calculate Softmax Regression probabilities in this example?

Thu, 04 Apr 2019 18:20:53 +0000

The scatter plot of Iris Dataset is shown in the figure below. Assume Softmax Regression is used to classify Iris to Setosa, Versicolor, or Viriginica using just petal length and petal width. If weights required for Softmax Regression initialized to 1 for class Setosa, 2 for class Versicolor, and 3 for Virginica,

1) What will be the probability of an iris with petal length = 4.6 and petal width = 1.7 to be classified as Virginica?

2) What will be the probability of Virginica, if we use all features petal length = 4.6 and petal width = 1.7, sepal length = 5.5 and sepal width = 3.0 with the same weight initialization?

How to calculate feed-forward (forward-propagation) in neural network?

Thu, 04 Apr 2019 15:54:17 +0000

In the figure below, a neural network is shown. Calculate the following:

1) How many neurons do we have in the input layer and the output layer?

2) How many hidden layers do we have?

3) If all the weights initialized with 1 ($w1=w2=w3=...=w19=1$), what is the output of this network after feed-forward for the sample shown in the figure (X = (x1,x2,x3) = (2,5,3) and y=10)? What is the error of the network ($\text { Error }=\frac{1}{2}(\hat{y}-y)^{2}$)? Assume activation functions for all neurons except the output neuron is $f(z)=z$.

4) If we change the activation function of all the neurons in the second hidden layer to Sigmoid ($S(x)=\frac{1}{1+e^{-x}}=\frac{e^{x}}{e^{x}+1}$), what would be the output of the network after this change? Calculate the error as well.

https://i.imgur.com/rtqPiRa.jpg

How do I reduce RMSE in a Random Forest Regressor?

Wed, 03 Apr 2019 12:24:09 +0000

I preprocessed the data, normalized the numerical features, and did one hot encoding for the categorical ones. I end up with a model with R^2=0.7 and RMSE which is 15% of the range of values.
I'm okay with the accuracy but I was wondering if there's a way to reduce RMSE to maybe ~7%?

Let me know please.

Thanks!

How to update weights using gradient decent algorithm?

Thu, 28 Mar 2019 17:17:39 +0000

For the below neural network, imagine we are going to use the backpropagation algorithm to update weights. If the Bias (b) in this problem is always 0 (ignore bias when you solve the problem), and we have a dataset with only one record of $x=2$ and the target value of $y=5$ as you can see in the following table, and activation function is defined as $f(z) = z$

feature (x)	Target (y)
2	5

1) Define the cost function, $J(w)$, based on the error in backpropagation algorithm: $J(w) = E = \frac{1}{2}(predicted - target)^2$, and draw it

2) Initialize the weight by $w=3$, and calculate the error

3) Calculate updated weights using the gradient decent algorithm after three updates if we have the following values for learning rate ($\alpha$)

$\alpha$ = 1
$\alpha$ = 0.1
$\alpha$ = 0.5

Hint: $w_{new} = w_{old} - \alpha \frac{\partial E}{\partial w}$

https://i.imgur.com/uohFS6l.png

x	y
2.5	2.4
0.5	0.7
2.2	2.9
1.9	2.2
3.1	3.0
2.3	2.7
2.0	1.6
1.0	1.1
1.5	1.6
1.1	0.9

x	y
2.5	2.4
0.5	0.7
2.2	2.9
1.9	2.2
3.1	3.0
2.3	2.7
2.0	1.6
1.0	1.1
1.5	1.6
1.1	0.9

x	y
2.5	2.4
0.5	0.7
2.2	2.9
1.9	2.2
3.1	3.0
2.3	2.7
2.0	1.6
1.0	1.1
1.5	1.6
1.1	0.9