Derivative

As a limit

A function of a real variable ${\displaystyle f(x)}$ is differentiable at a point ${\displaystyle a}$ of its domain, if its domain contains an open interval containing ⁠${\displaystyle a}$⁠, and the limit $${\displaystyle L=\lim _{h\to 0}{\frac {f(a+h)-f(a)}{h}}}$$ exists.^[2] This means that, for every positive real number ⁠${\displaystyle \varepsilon }$⁠, there exists a positive real number ${\displaystyle \delta }$ such that, for every ${\displaystyle h}$ such that ${\displaystyle |h|<\delta }$ and ${\displaystyle h\neq 0}$ then ${\displaystyle f(a+h)}$ is defined, and $${\displaystyle \left|L-{\frac {f(a+h)-f(a)}{h}}\right|<\varepsilon ,}$$ where the vertical bars denote the absolute value. This is an example of the (ε, δ)-definition of limit.^[3]

If the function ${\displaystyle f}$ is differentiable at ⁠${\displaystyle a}$⁠, that is if the limit ${\displaystyle L}$ exists, then this limit is called the derivative of ${\displaystyle f}$ at ${\displaystyle a}$. Multiple notations for the derivative exist.^[4] The derivative of ${\displaystyle f}$ at ${\displaystyle a}$ can be denoted ⁠${\displaystyle f'(a)}$⁠, read as "⁠${\displaystyle f}$⁠ prime of ⁠${\displaystyle a}$⁠"; or it can be denoted ⁠${\displaystyle \textstyle {\frac {df}{dx}}(a)}$⁠, read as "the derivative of ${\displaystyle f}$ with respect to ${\displaystyle x}$ at ⁠${\displaystyle a}$⁠" or "⁠${\displaystyle df}$⁠ by (or over) ${\displaystyle dx}$ at ⁠${\displaystyle a}$⁠". See § Notation below. If ${\displaystyle f}$ is a function that has a derivative at every point in its domain, then a function can be defined by mapping every point ${\displaystyle x}$ to the value of the derivative of ${\displaystyle f}$ at ${\displaystyle x}$. This function is written ${\displaystyle f'}$ and is called the derivative function or the derivative of ⁠${\displaystyle f}$⁠. The function ${\displaystyle f}$ sometimes has a derivative at most, but not all, points of its domain. The function whose value at ${\displaystyle a}$ equals ${\displaystyle f'(a)}$ whenever ${\displaystyle f'(a)}$ is defined and elsewhere is undefined is also called the derivative of ⁠${\displaystyle f}$⁠. It is still a function, but its domain may be smaller than the domain of ${\displaystyle f}$.^[5]

For example, let ${\displaystyle f}$ be the squaring function: ${\displaystyle f(x)=x^{2}}$. Then the quotient in the definition of the derivative is^[6] $${\displaystyle {\frac {f(a+h)-f(a)}{h}}={\frac {(a+h)^{2}-a^{2}}{h}}={\frac {a^{2}+2ah+h^{2}-a^{2}}{h}}=2a+h.}$$ The division in the last step is valid as long as ${\displaystyle h\neq 0}$. The closer ${\displaystyle h}$ is to ⁠${\displaystyle 0}$⁠, the closer this expression becomes to the value ${\displaystyle 2a}$. The limit exists, and for every input ${\displaystyle a}$ the limit is ${\displaystyle 2a}$. So, the derivative of the squaring function is the doubling function: ⁠${\displaystyle f'(x)=2x}$⁠.

The graph of a function, drawn in black, and a tangent line to that graph, drawn in red. The slope of the tangent line is equal to the derivative of the function at the marked point.

The derivative at different points of a differentiable function. In this case, the derivative is equal to ${\displaystyle \sin \left(x^{2}\right)+2x^{2}\cos \left(x^{2}\right)}$.

The ratio in the definition of the derivative is the slope of the line through two points on the graph of the function ⁠${\displaystyle f}$⁠, specifically the points ${\displaystyle (a,f(a))}$ and ${\displaystyle (a+h,f(a+h))}$. As ${\displaystyle h}$ is made smaller, these points grow closer together, and the slope of this line approaches the limiting value, the slope of the tangent to the graph of ${\displaystyle f}$ at ${\displaystyle a}$. In other words, the derivative is the slope of the tangent.^[7]

Using infinitesimals

One way to think of the derivative ${\textstyle {\frac {df}{dx}}(a)}$ is as the ratio of an infinitesimal change in the output of the function ${\displaystyle f}$ to an infinitesimal change in its input.^[8] In order to make this intuition rigorous, a system of rules for manipulating infinitesimal quantities is required.^[9] The system of hyperreal numbers is a way of treating infinite and infinitesimal quantities. The hyperreals are an extension of the real numbers that contain numbers greater than anything of the form ${\displaystyle 1+1+\cdots +1}$ for any finite number of terms. Such numbers are infinite, and their reciprocals are infinitesimals. The application of hyperreal numbers to the foundations of calculus is called nonstandard analysis. This provides a way to define the basic concepts of calculus such as the derivative and integral in terms of infinitesimals, thereby giving a precise meaning to the ${\displaystyle d}$ in the Leibniz notation. Thus, the derivative of ${\displaystyle f(x)}$ becomes $${\displaystyle f'(x)=\operatorname {st} \left({\frac {f(x+dx)-f(x)}{dx}}\right)}$$ for an arbitrary infinitesimal ⁠${\displaystyle dx}$⁠, where ${\displaystyle \operatorname {st} }$ denotes the standard part function, which "rounds off" each finite hyperreal to the nearest real.^[10] Taking the squaring function ${\displaystyle f(x)=x^{2}}$ as an example again, $${\displaystyle {\begin{aligned}f'(x)&=\operatorname {st} \left({\frac {x^{2}+2x\cdot dx+(dx)^{2}-x^{2}}{dx}}\right)\\&=\operatorname {st} \left({\frac {2x\cdot dx+(dx)^{2}}{dx}}\right)\\&=\operatorname {st} \left({\frac {2x\cdot dx}{dx}}+{\frac {(dx)^{2}}{dx}}\right)\\&=\operatorname {st} \left(2x+dx\right)\\&=2x.\end{aligned}}}$$

Rules for basic functions

The following are the rules for the derivatives of the most common basic functions. Here, ${\displaystyle a}$ is a real number, and ${\displaystyle e}$ is the base of the natural logarithm, approximately 2.71828.^[30]

• Derivatives of powers:

  ${\displaystyle {\frac {d}{dx}}x^{a}=ax^{a-1}}$

• Functions of exponential, natural logarithm, and logarithm with general base:

  ${\displaystyle {\frac {d}{dx}}e^{x}=e^{x}}$

  ${\displaystyle {\frac {d}{dx}}a^{x}=a^{x}\ln(a)}$, for ${\displaystyle a>0}$

  ${\displaystyle {\frac {d}{dx}}\ln(x)={\frac {1}{x}}}$, for ${\displaystyle x>0}$

  ${\displaystyle {\frac {d}{dx}}\log _{a}(x)={\frac {1}{x\ln(a)}}}$, for ${\displaystyle x,a>0}$

• Trigonometric functions:

  ${\displaystyle {\frac {d}{dx}}\sin(x)=\cos(x)}$

  ${\displaystyle {\frac {d}{dx}}\cos(x)=-\sin(x)}$

  ${\displaystyle {\frac {d}{dx}}\tan(x)=\sec ^{2}(x)={\frac {1}{\cos ^{2}(x)}}=1+\tan ^{2}(x)}$

• Inverse trigonometric functions:

  ${\displaystyle {\frac {d}{dx}}\arcsin(x)={\frac {1}{\sqrt {1-x^{2}}}}}$, for ${\displaystyle -1<x<1}$

  ${\displaystyle {\frac {d}{dx}}\arccos(x)=-{\frac {1}{\sqrt {1-x^{2}}}}}$, for ${\displaystyle -1<x<1}$

  ${\displaystyle {\frac {d}{dx}}\arctan(x)={\frac {1}{1+x^{2}}}}$

Rules for combined functions

Given that the ${\displaystyle f}$ and ${\displaystyle g}$ are the functions. The following are some of the most basic rules for deducing the derivative of functions from derivatives of basic functions.^[31]

• Constant rule: if ${\displaystyle f}$ is constant, then for all ⁠${\displaystyle x}$⁠,

  ${\displaystyle f'(x)=0.}$

• Sum rule:

  ${\displaystyle (\alpha f+\beta g)'=\alpha f'+\beta g'}$ for all functions ${\displaystyle f}$ and ${\displaystyle g}$ and all real numbers ${\displaystyle \alpha }$ and ⁠${\displaystyle \beta }$⁠.

• Product rule:

  ${\displaystyle (fg)'=f'g+fg'}$ for all functions ${\displaystyle f}$ and ⁠${\displaystyle g}$⁠. As a special case, this rule includes the fact ${\displaystyle (\alpha f)'=\alpha f'}$ whenever ${\displaystyle \alpha }$ is a constant because ${\displaystyle \alpha 'f=0\cdot f=0}$ by the constant rule.

• Quotient rule:

  ${\displaystyle \left({\frac {f}{g}}\right)'={\frac {f'g-fg'}{g^{2}}}}$ for all functions ${\displaystyle f}$ and ${\displaystyle g}$ at all inputs where g ≠ 0.

• Chain rule for composite functions: If ⁠${\displaystyle f(x)=h(g(x))}$⁠, then

  ${\displaystyle f'(x)=h'(g(x))\cdot g'(x).}$

Computation example

The derivative of the function given by ${\displaystyle f(x)=x^{4}+\sin \left(x^{2}\right)-\ln(x)e^{x}+7}$ is $${\displaystyle {\begin{aligned}f'(x)&=4x^{(4-1)}+{\frac {d\left(x^{2}\right)}{dx}}\cos \left(x^{2}\right)-{\frac {d\left(\ln {x}\right)}{dx}}e^{x}-\ln(x){\frac {d\left(e^{x}\right)}{dx}}+0\\&=4x^{3}+2x\cos \left(x^{2}\right)-{\frac {1}{x}}e^{x}-\ln(x)e^{x}.\end{aligned}}}$$ Here the second term was computed using the chain rule and the third term using the product rule. The known derivatives of the elementary functions ${\displaystyle x^{2}}$, ${\displaystyle x^{4}}$, ${\displaystyle \sin(x)}$, ${\displaystyle \ln(x)}$, and ${\displaystyle \exp(x)=e^{x}}$, as well as the constant ${\displaystyle 7}$, were also used.

Vector-valued functions

A vector-valued function ${\displaystyle \mathbf {y} }$ of a real variable sends real numbers to vectors in some vector space ${\displaystyle \mathbb {R} ^{n}}$. A vector-valued function can be split up into its coordinate functions ${\displaystyle y_{1}(t),y_{2}(t),\dots ,y_{n}(t)}$, meaning that ${\displaystyle \mathbf {y} =(y_{1}(t),y_{2}(t),\dots ,y_{n}(t))}$. This includes, for example, parametric curves in ${\displaystyle \mathbb {R} ^{2}}$ or ${\displaystyle \mathbb {R} ^{3}}$. The coordinate functions are real-valued functions, so the above definition of derivative applies to them. The derivative of ${\displaystyle \mathbf {y} (t)}$ is defined to be the vector, called the tangent vector, whose coordinates are the derivatives of the coordinate functions. That is,^[37] $${\displaystyle \mathbf {y} '(t)=\lim _{h\to 0}{\frac {\mathbf {y} (t+h)-\mathbf {y} (t)}{h}},}$$ if the limit exists. The subtraction in the numerator is the subtraction of vectors, not scalars. If the derivative of ${\displaystyle \mathbf {y} }$ exists for every value of ⁠${\displaystyle t}$⁠, then ${\displaystyle \mathbf {y} '}$ is another vector-valued function.^[37]

Partial derivatives

Functions can depend upon more than one variable. A partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant. Partial derivatives are used in vector calculus and differential geometry. As with ordinary derivatives, multiple notations exist: the partial derivative of a function ${\displaystyle f(x,y,\dots )}$ with respect to the variable ${\displaystyle x}$ is variously denoted by

${\displaystyle f_{x}}$, ${\displaystyle f'_{x}}$, ${\displaystyle \partial _{x}f}$, ${\displaystyle {\frac {\partial }{\partial x}}f}$, or ${\displaystyle {\frac {\partial f}{\partial x}}}$,

among other possibilities.^[38] It can be thought of as the rate of change of the function in the ${\displaystyle x}$-direction.^[39] Here ∂ is a rounded d called the partial derivative symbol. To distinguish it from the letter d, ∂ is sometimes pronounced "der", "del", or "partial" instead of "dee".^[40] For example, let ⁠${\displaystyle f(x,y)=x^{2}+xy+y^{2}}$⁠, then the partial derivative of function ${\displaystyle f}$ with respect to both variables ${\displaystyle x}$ and ${\displaystyle y}$ are, respectively: $${\displaystyle {\frac {\partial f}{\partial x}}=2x+y,\qquad {\frac {\partial f}{\partial y}}=x+2y.}$$ In general, the partial derivative of a function ${\displaystyle f(x_{1},\dots ,x_{n})}$ in the direction ${\displaystyle x_{i}}$ at the point ${\displaystyle (a_{1},\dots ,a_{n})}$ is defined to be:^[41] $${\displaystyle {\frac {\partial f}{\partial x_{i}}}(a_{1},\ldots ,a_{n})=\lim _{h\to 0}{\frac {f(a_{1},\ldots ,a_{i}+h,\ldots ,a_{n})-f(a_{1},\ldots ,a_{i},\ldots ,a_{n})}{h}}.}$$

This is fundamental for the study of the functions of several real variables. Let ${\displaystyle f(x_{1},\dots ,x_{n})}$ be such a real-valued function. If all partial derivatives ${\displaystyle f}$ with respect to ${\displaystyle x_{j}}$ are defined at the point ⁠${\displaystyle (a_{1},\dots ,a_{n})}$⁠, these partial derivatives define the vector $${\displaystyle \nabla f(a_{1},\ldots ,a_{n})=\left({\frac {\partial f}{\partial x_{1}}}(a_{1},\ldots ,a_{n}),\ldots ,{\frac {\partial f}{\partial x_{n}}}(a_{1},\ldots ,a_{n})\right),}$$ which is called the gradient of ${\displaystyle f}$ at ${\displaystyle a}$. If ${\displaystyle f}$ is differentiable at every point in some domain, then the gradient is a vector-valued function ${\displaystyle \nabla f}$ that maps the point ${\displaystyle (a_{1},\dots ,a_{n})}$ to the vector ${\displaystyle \nabla f(a_{1},\dots ,a_{n})}$. Consequently, the gradient determines a vector field.^[42]

Directional derivatives

If ${\displaystyle f}$ is a real-valued function on ⁠${\displaystyle \mathbb {R} ^{n}}$⁠, then the partial derivatives of ${\displaystyle f}$ measure its variation in the direction of the coordinate axes. For example, if ${\displaystyle f}$ is a function of ${\displaystyle x}$ and ⁠${\displaystyle y}$⁠, then its partial derivatives measure the variation in ${\displaystyle f}$ in the ${\displaystyle x}$ and ${\displaystyle y}$ direction. However, they do not directly measure the variation of ${\displaystyle f}$ in any other direction, such as along the diagonal line ⁠${\displaystyle y=x}$⁠. These are measured using directional derivatives. Given a vector ⁠${\displaystyle \mathbf {v} =(v_{1},\ldots ,v_{n})}$⁠, then the directional derivative of ${\displaystyle f}$ in the direction of ${\displaystyle \mathbf {v} }$ at the point ${\displaystyle \mathbf {x} }$ is:^[43] $${\displaystyle D_{\mathbf {v} }{f}(\mathbf {x} )=\lim _{h\rightarrow 0}{\frac {f(\mathbf {x} +h\mathbf {v} )-f(\mathbf {x} )}{h}}.}$$

If all the partial derivatives of ${\displaystyle f}$ exist and are continuous at ⁠${\displaystyle \mathbf {x} }$⁠, then they determine the directional derivative of ${\displaystyle f}$ in the direction ${\displaystyle \mathbf {v} }$ by the formula:^[44] $${\displaystyle D_{\mathbf {v} }{f}(\mathbf {x} )=\sum _{j=1}^{n}v_{j}{\frac {\partial f}{\partial x_{j}}}.}$$

Total derivative, total differential and Jacobian matrix

When ${\displaystyle f}$ is a function from an open subset of ${\displaystyle \mathbb {R} ^{n}}$ to ⁠${\displaystyle \mathbb {R} ^{m}}$⁠, then the directional derivative of ${\displaystyle f}$ in a chosen direction is the best linear approximation to ${\displaystyle f}$ at that point and in that direction. However, when ⁠${\displaystyle n>1}$⁠, no single directional derivative can give a complete picture of the behavior of ${\displaystyle f}$. The total derivative gives a complete picture by considering all directions at once. That is, for any vector ${\displaystyle \mathbf {v} }$ starting at ⁠${\displaystyle \mathbf {a} }$⁠, the linear approximation formula holds:^[45] $${\displaystyle f(\mathbf {a} +\mathbf {v} )\approx f(\mathbf {a} )+f'(\mathbf {a} )\mathbf {v} .}$$ Similarly with the single-variable derivative, ${\displaystyle f'(\mathbf {a} )}$ is chosen so that the error in this approximation is as small as possible. The total derivative of ${\displaystyle f}$ at ${\displaystyle \mathbf {a} }$ is the unique linear transformation ${\displaystyle f'(\mathbf {a} )\colon \mathbb {R} ^{n}\to \mathbb {R} ^{m}}$ such that^[45] $${\displaystyle \lim _{\mathbf {h} \to 0}{\frac {\lVert f(\mathbf {a} +\mathbf {h} )-(f(\mathbf {a} )+f'(\mathbf {a} )\mathbf {h} )\rVert }{\lVert \mathbf {h} \rVert }}=0.}$$ Here ${\displaystyle \mathbf {h} }$ is a vector in ⁠${\displaystyle \mathbb {R} ^{n}}$⁠, so the norm in the denominator is the standard length on ${\displaystyle \mathbb {R} ^{n}}$. However, ${\displaystyle f'(\mathbf {a} )\mathbf {h} }$ is a vector in ⁠${\displaystyle \mathbb {R} ^{m}}$⁠, and the norm in the numerator is the standard length on ${\displaystyle \mathbb {R} ^{m}}$.^[45] If ${\displaystyle v}$ is a vector starting at ⁠${\displaystyle a}$⁠, then ${\displaystyle f'(\mathbf {a} )\mathbf {v} }$ is called the pushforward of ${\displaystyle \mathbf {v} }$ by ${\displaystyle f}$.^[46]

If the total derivative exists at ⁠${\displaystyle \mathbf {a} }$⁠, then all the partial derivatives and directional derivatives of ${\displaystyle f}$ exist at ⁠${\displaystyle \mathbf {a} }$⁠, and for all ⁠${\displaystyle \mathbf {v} }$⁠, ${\displaystyle f'(\mathbf {a} )\mathbf {v} }$ is the directional derivative of ${\displaystyle f}$ in the direction ⁠${\displaystyle \mathbf {v} }$⁠. If ${\displaystyle f}$ is written using coordinate functions, so that ⁠${\displaystyle f=(f_{1},f_{2},\dots ,f_{m})}$⁠, then the total derivative can be expressed using the partial derivatives as a matrix. This matrix is called the Jacobian matrix of ${\displaystyle f}$ at ${\displaystyle \mathbf {a} }$:^[47] $${\displaystyle f'(\mathbf {a} )=\operatorname {Jac} _{\mathbf {a} }=\left({\frac {\partial f_{i}}{\partial x_{j}}}\right)_{ij}.}$$