Gradient Operator

The gradient operator is a differentiation operator that applies partial derivatives to a function. It returns the gradient vector of a function.

Definition §

Gradient Operator ^definition

Let $f$ be a multivariate scalar function in Euclidean space, $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ where $f(x_1,\dots ,x_n)$. If $f$ is $C^1$, then the gradient of $f$ is:

$$\nabla f=\text{grad }f=\left(\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\dots ,\frac{\partial f}{\partial x_n}\right)$$

• While $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$, then $\nabla f:D\to {\mathbb{R}}^n$
• In the case of a real scalar function $f:{\mathbb{R}}^3\to \mathbb{R}$, if we have $f(x,y,z)$ then:

$$\nabla f=\left(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z}\right)$$

Applications §

• Used in gradient descent when training neural networks.