Category Archives: machine learning

Bellman optimal equation for Q

  Q(s, a), the expected return from starting state s, by taking action a at time t. r(s, a), reward at state s, by taking action a maxQ(s’, a’),  maximized expected return for next state-action(s’,a’). Need to find the a’, which maximizes it.  

liner equation, non-linear equation

Single layer machine learning is actually linear equation. Linear equation is at most multi-variant polynomial equation, which at most include multiplication and adding. Deep learning uses multi layers. The multi layers is considered to be non-linear equation.  Non-linear equation may include sin, cos, square etc on variances. Both linear and non-linear equation fits curvature. However, non-linear is more… Read More »

Logistic Regression

Logistic Regression answer the YES/NO question. For example, giving a set of size of tumor, it answers if it is a tumor. Giving height and weight of a person, answer if it is a man. Hypothesis We have hypothesis function , and it ranges  . And we define the answer is YES when hypothesis is greater… Read More »

Simple Linear Regression

I’m learning machine learning these days. Here let me write down the note for this. Suppose we have m training points  in x-y coordination, and we want to find the best line fit for these points . Since it is the simplest line, we can define the line function, and call it hypothesis: . A way… Read More »

Example of Gradient Descent

Let’s talk about this function: It has shape like this: Obviously, it has extreme point (0, 0, 0). Let’s use gradient descent to get the extreme point. Calculate the partial derivative: Then, let’s take any point in (x, y), and run like below. Lambda is a tiny value, which controls how fast P goes to the… Read More »