TensorFlow2_200729系列---3、梯度下降求简单线性回归实例

一、总结

一句话总结:

梯度下降:梯度下降是对loss函数做的,对loss的w和b,比如w = w - learningRate*loss对w的梯度
画动态图:在动态图中更新y轴数据即可,如果需要更新text标注,那就也更新



import numpy as np
from matplotlib import pyplot as plt
from matplotlib import animation
from matplotlib import pylab
%pylab

fig,ax=plt.subplots()

# 画散点图
x = data.iloc[:,0]
y = data.iloc[:,-1]
ax.scatter(x,y)

# 画直线图

line,=ax.plot(x,0*x+0)
# 存这个text1,方便动画的时候修改
text1 = ax.text(30,110,"w=0.0000,b=0.0000", fontdict={'size':16,'color':'r'})

def animate(i):
text1.set_text('w=%.4f,b=%.4f' %(bw_list[i][1],bw_list[i][0]))
line.set_ydata(bw_list[i][1]*x+bw_list[i][0])
return line,

def init():
line.set_ydata(0*x+0)
return line,

ani=animation.FuncAnimation(fig=fig,func=animate,frames=999,init_func=init,interval=5,blit=False)
ani.save('line_model.gif', writer='imagemagick', fps=30)
plt.show()


 

 

1、梯度下降为什么会在极值点那里收敛?

因为极值点处斜率(梯度)为0,这样学习率*梯度也是0

 

2、梯度下降中为什么乘上步长?

不乘上步长,有些情况下,导数太大了

 

3、matplotlib画动态图如何更新文本标注?

text1 = ax.text(30,110,"w=0.0000,b=0.0000", fontdict={'size':16,'color':'r'})
text1.set_text('w=%.4f,b=%.4f' %(bw_list[i][1],bw_list[i][0]))

 

4、matplotlib画动态图?

主要是初始化函数和更新函数,在更新函数中更新y数据:line.set_ydata(bw_list[i][1]*x+bw_list[i][0])

 

5、本题梯度下降核心代码(和公式推导一样)?

grad_b = 2(wx+b-y):b_gradient += (2/N) * ((w_current * x + b_current) - y)
grad_w = 2(wx+b-y)*x:w_gradient += (2/N) * x * ((w_current * x + b_current) - y)



def step_gradient(b_current, w_current, points, learningRate):
b_gradient = 0
w_gradient = 0
N = float(len(points))
for i in range(0, len(points)):
x = points[i, 0]
y = points[i, 1]
# grad_b = 2(wx+b-y)
b_gradient += (2/N) * ((w_current * x + b_current) - y)
# grad_w = 2(wx+b-y)*x
w_gradient += (2/N) * x * ((w_current * x + b_current) - y)
# update w'
new_b = b_current - (learningRate * b_gradient)
new_w = w_current - (learningRate * w_gradient)
return [new_b, new_w]


 

 

 

6、本题损失函数最小二乘法实例?

totalError += (y - (w * x + b)) ** 2



# y = wx + b
def compute_error_for_line_given_points(b, w, points):
totalError = 0
for i in range(0, len(points)):
x = points[i, 0]
y = points[i, 1]
# computer mean-squared-error
totalError += (y - (w * x + b)) ** 2
# average loss for each point
return totalError / float(len(points))


 

 

 

二、梯度下降求简单线性回归实例

博客对应课程的视频位置:

 




In [1]:





import numpy as np


# data = []
# for i in range(100):
# x = np.random.uniform(3., 12.)
# # mean=0, std=0.1
# eps = np.random.normal(0., 0.1)
# y = 1.477 * x + 0.089 + eps
# data.append([x, y])
# data = np.array(data)
# print(data.shape, data)

# y = wx + b
def compute_error_for_line_given_points(b, w, points):
totalError = 0
for i in range(0, len(points)):
x = points[i, 0]
y = points[i, 1]
# computer mean-squared-error
totalError += (y - (w * x + b)) ** 2
# average loss for each point
return totalError / float(len(points))



def step_gradient(b_current, w_current, points, learningRate):
b_gradient = 0
w_gradient = 0
N = float(len(points))
for i in range(0, len(points)):
x = points[i, 0]
y = points[i, 1]
# grad_b = 2(wx+b-y)
b_gradient += (2/N) * ((w_current * x + b_current) - y)
# grad_w = 2(wx+b-y)*x
w_gradient += (2/N) * x * ((w_current * x + b_current) - y)
# update w'
new_b = b_current - (learningRate * b_gradient)
new_w = w_current - (learningRate * w_gradient)
return [new_b, new_w]

def gradient_descent_runner(points, starting_b, starting_w, learning_rate, num_iterations):
b = starting_b
w = starting_w
# update for several times
for i in range(num_iterations):
b, w = step_gradient(b, w, np.array(points), learning_rate)