Derivatives

What is it?

Derivatives is a fundamental of Calculus and Differentiation which defines the rate of change in a function and the slope of a tangent line at any given point. Using a derivative, one could calculate the minimum and maximum values of a function, making it a basic mathematical tool used in Machine Learning for Gradient Descent methods.

Another fundamental concept for derivatives is its tangent line, which intersects the curve at a point and has the same instantaneous slope and that point. The slope of this line, it’s the first derivative! In turn, the secant line, touches the curve at two different points.

---

Notation and Definition

If we keep building our knowledge after learning Limits, one could draw two different points at a given function and approximate it until one overlaps with the other. This would, in turn, make the image also overlap itself. We could express this as the limit of the rate of change of $y$ by $x$. When approximating:

$$f^{\prime }\left(x_0\right)=\underset{x\to x_0}{\lim }\frac{f(x)-f\left(x_0\right)}{x-x_0}=\underset{\Delta x\to 0}{\lim }\frac{\Delta y}{\Delta x}$$

The notation $f^{\prime }(x)$, — with the single quote — describes the derivate of $f(x)$. But it could also use the notation of $\frac{dy}{dx}$, $\frac{d}{dx}f(x)$, $Df$ and so on. For now, let’s use the standard of $f^{\prime }(x)$.

---

Using the secant line

But there’s another expression to calculate the slope of the tangent line. If we draw a secant line, we can use it to approximate to the point $(x,f(x))$ as a limit, once that as $x\to x_0$, the value of $h$ also approaches $0$, resulting in:

$$f^{\prime }(x_0)=\underset{h\to 0}{\lim }\frac{f(x_0+h)-f(x_0)}{h}$$

---

Finding the derivatives with limits

For example, to calculate the derivative of a given a function $f(x)=x^2-1$, at $x_0=2$:

$$f^{\prime }\left(2\right)=\underset{x\to 2}{\lim }\frac{f(x)-f\left(2\right)}{x-2}=\underset{x\to 2}{\lim }\frac{x^2-4}{x-2}=\underset{x\to 2}{\lim }x+2=4$$

The **derivative of the function** is $f'(x) = x + 2$, and because $x \to 2$, the derivative of the function at **the given point** would be **4**, which is the **slope** at this given point.

Now that we know $f^{\prime }(x_0)$, we can find the equation of the tangent line to the curve of our polynomial function. To do this, apply the point-slope formula, which requires only one point and the slope of the line.

&\textit{Point-slope formula: }\quad y - y_0 = m(x - x_0) \\ \\ &(x_0, y_0) =(x_0,f(x_0)) \quad \text{and} \quad m = f'(x_0) \end{align}

Now, to calculate the equation of the tangent line to the curve:

$$\begin{array}{rl}& y-f(2)=4(x-2)\\ & y-(3)=4x-8\\ & y=4x-8+3\\ & y=4x-5\end{array}$$

Finally, to visualize the curve, one could easily input the found solutions and plot the functions:

x_values = np.arange(-7 ,8 ,1)  
line_values = np.arange(0, 5, 1)  
  
function = lambda x: x**2 - 1  
line_function = lambda x: 4*x - 5  
  
sns.lineplot(x=x_values, y=function(x_values))  
sns.lineplot(x=line_values, y=line_function(line_values));

Reference

• Derivada is All You Need — Anwar Mermuche — Medium.

• James Stewart’s “Calculus - Early Transcendentals” — Section 2.7, p. 141.

• Matemática Essencial — Universidade Estadual de Londrina.