Codes: https://strutive07.github.io/assets/images/4_5_How_Gradient_method_works/Week_4.zip Files: https://strutive07.github.io/assets/images/4_5_How_Gradient_method_works/IE661-Week_4-Part_2-icmoon-ver-1.pdf last update datetime: Jan 17, 2020 12:05 AM
4.4 에서 배웠던 gradient descent 를 rosenbrock function 예시를 통해서 알아보자.
Rosenbrock function
이 식은 (1,1) 에서 global minimum = 0 을 달성한다.
자 이제 우리는 이걸 gradient descent 로 찾아보자.
초기에 (-1.9, 0.9) 에서 시작했다고 가정해보자.
x1 = linspace(-1.5,1.5); x2 = linspace(-1,3);
[xx1,xx2] = meshgrid(x1,x2); f = Rosenbrock(xx1,xx2);
levels = 10:10:300;
LW = 'linewidth'; FS = 'fontsize'; MS = 'markersize';
figure, contour(x1,x2,f,levels,LW,1.2), colorbar
axis([-1.5 1.5 -1 3]), axis square, hold on
itr = 5000;
theta = zeros(itr,2);
theta(1,:) = [-1.3 0.9];
h=0.01; % 속력
for i = 2:itr
[derivX1, derivX2] = RosenbrockDeriv(theta(i-1,1),theta(i-1,2));
scale = sqrt(derivX1^2+derivX2^2);
derivX1=derivX1/scale; % X1 방향 -> u1
derivX2=derivX2/scale; % X2 방향 -> u2
theta(i,1) = theta(i-1,1)-h*derivX1; % gradient descent!
theta(i,2) = theta(i-1,2)-h*derivX2; % gradient descent!
end
h=scatter(theta(:,1),theta(:,2));
set(h(1),'MarkerFaceColor','g')
set(h(1),'MarkerEdgeColor','g')
hold on;
plot(theta(:,1),theta(:,2),'g');
figure
surf(x1,x2,f);
hold on;
value = zeros(itr,1);
for i = 1:itr
value(i)=Rosenbrock(theta(i,1),theta(i,2));
end
h=scatter3(theta(:,1),theta(:,2),value);
set(h(1),'MarkerFaceColor','g')
set(h(1),'MarkerEdgeColor','g')
hold on;
plot3(theta(:,1),theta(:,2),value,'g');
iter가 증가함에따라 1,1로 수렴하는모습을 볼 수 있다.