gen-define-gradient-descent-and-give-one-short-worked-example-36c3

Define gradient descent and give one short worked example.

# Gradient Descent **Gradient descent** is an optimization algorithm that iteratively updates parameters to minimize a function by moving in the direction of steepest descent (negative gradient). **Update rule:** θ_new = θ_old - α∇f(θ) where α is the learning rate and ∇f(θ) is the gradient. ## Worked Example Minimize f(x) = x² starting at x₀ = 4 with learning rate α = 0.1 **Gradient:** f'(x) = 2x **Iterations:** - **Step 0:** x₀ = 4 - **Step 1:** x₁ = 4 - 0.1(2·4) = 4 - 0.8 = 3.2 - **Step 2:** x₂ = 3.2 - 0.1(2·3.2) = 3.2 - 0.64 = 2.56 - **Step 3:** x₃ = 2.56 - 0.1(2·2.56) = 2.56 - 0.512 = 2.048 The algorithm converges toward x = 0 (the minimum of f(x) = x²).