Summary - Autograd

In this group you learned autograd, requires_grad, and backward().

Summary: autograd

In this group:

Tensors with requires_grad=True are tracked; derived tensors are tracked too.
backward() computes gradients from a scalar result and fills the leaves' .grad.
Gradients accumulate; they must be zeroed every step.
no_grad() and detach() stop tracking.
The computational graph is built dynamically; grad_fn shows the nodes, the chain rule is applied backwards.

Exercise Gradient descent by hand

Minimise kayip = (w - 3) ** 2 with 50 steps of gradient descent starting from w = 0 with a learning rate of 0.1: at each step compute the loss, call backward(), update inside torch.no_grad() and zero the gradient. Print the final w with round(..., 3).

Show example solution

import torch

w = torch.tensor(0.0, requires_grad=True)

for _ in range(50):
    kayip = (w - 3) ** 2
    kayip.backward()
    with torch.no_grad():
        w -= 0.1 * w.grad
    w.grad.zero_()
print(round(w.item(), 3))

Test yourself

Why is the update in gradient descent with autograd done inside <code>torch.no_grad()</code>?
- It isn't required
- To zero the gradient
- So the update itself isn't recorded in the graph
- Because it is more accurate
Show answer

So the update itself isn't recorded in the graph — w -= lr·grad is a parameter adjustment, not a computation to differentiate.

Do the update without tracking.

with torch.no_grad():
    w ____ 0.1 * w.grad

Show answer

with torch.no_grad():
    w -= 0.1 * w.grad

Match the tools.

backward()

Gradient computation
.grad

Gradient value
zero_()

Zeroing
no_grad()

Untracked block

Next lesson: Preparing tensors

We Are Enhancing Your Experience

Cookie Preferences

Strictly Necessary Cookies

Analytical Cookies