The Taylor series method

Let us first consider a Taylor expansion of an analytical function $u$ .

$$u(x+\triangle x)=u(x)+\sum_{n=1}^{\infty}\frac{\triangle x^n}{n!}... ...\partial x^2}+\frac{\triangle x^3}{3!} \frac{\partial^3 u}{\partial x^3}+\ldots$$ (1.1)

Then for the first derivative one obtains:

$$\frac{\partial u}{\partial x}=\frac{u(x+\triangle x)-u(x)}{\trian... ...{\partial x^2}-\frac{\triangle x^2}{3!}\frac{\partial^3 u}{\partial x^3}-\ldots$$ (1.2)

If we break the right hand side of the last equation after the first term, for $\triangle x\ll 1$ the last equation becomes

$$\boxed{\frac{\partial u}{\partial x}=\frac{u(x+\triangle x)-u(x)}... ...al{O}(\triangle x)=\frac{\triangle_i u}{\triangle x}+\mathcal{O}(\triangle x)},$$ (1.3)

where

$$\triangle_i u=u(x+\triangle x)-u(x)=u_{i+1}-u_i$$

is called a forward difference. The backward expansion of the function $u$ can be written as $\triangle x\ll 1$ the last equation reads

$$u(x+(-\triangle x))=u(x)-\triangle x\frac{\partial u}{\partial x}... ...partial x^2}-\frac{\triangle x^3}{3!} \frac{\partial^3 u}{\partial x^3}+\ldots,$$ (1.4)

so for the first derivative one obtains

$$\boxed{\frac{\partial u}{\partial x}=\frac{u(x)-u(x-\triangle x)}... ...thcal{O}(\triangle x)=\frac{\nabla_i u}{\triangle x}+\mathcal{O}(\triangle x)},$$ (1.5)

where

$$\nabla_i u=u(x)-u(x-\triangle x)=u_i-u_{i-1}$$

is called a backward difference. if we substract Eq. (1.5) from Eq. (1.3) one obtains

$$u(x+\triangle x)-u(x-\triangle x)=2\triangle x\frac{\partial u}{\partial x}+2\frac{\triangle x^3}{3!} \frac{\partial^3 u}{\partial x^3}+\ldots,$$ (1.6)

what is equivalent to

$$\boxed{\frac{\partial u}{\partial x}=\frac{u(x+\triangle x)-u(x-\triangle x)}{2\triangle x}+\mathcal{O}(\triangle x^2)}$$ (1.7)

The second derivative can be found in the same way using the linear combination of different Taylor expansions. For instance, consider

$$u(x+2\triangle x)=u(x)+2\triangle x\frac{\partial u}{\partial x}+... ...rtial x^2}+\frac{(2\triangle x)^3}{3!} \frac{\partial^3 u}{\partial x^3}+\ldots$$ (1.8)

Substracting from the last equation Eq. (1.1), multiplied by two, one gets the following equation

$$u(x+2\triangle x)-2u(x+\triangle x)=-u(x)+\frac{\triangle x^2}{2!... ...\partial x^2}+\frac{\triangle x^3}{3!} \frac{\partial^3 u}{\partial x^3}+\ldots$$ (1.9)

So, one can approximate the second derivative as

$$\boxed{\frac{\partial^2 u}{\partial x^2}=\frac{u(x+2\triangle x)-2u(x+\triangle x)+u(x)}{\triangle x^2}+\mathcal{O}(\triangle x).}$$ (1.10)

Similarly one can obtain the expression for the second derivative in terms of backward expansion, i.e.,

$$\boxed{\frac{\partial^2 u}{\partial x^2}=\frac{u(x-2\triangle x)-2u(x-\triangle x)+u(x)}{\triangle x^2}+\mathcal{O}(\triangle x).}$$ (1.11)

Finally, if we add Eqn. (1.3) and 1.5 expression for the second derivative reads

$$\boxed{\frac{\partial^2 u}{\partial x^2}=\frac{u(x+\triangle x)-2u(x)+u(x-\triangle x)}{\triangle x^2}+\mathcal{O}(\triangle x^2).}$$ (1.12)

In an analogous way one can obtain finite difference approximations to higher order derivatives and differential operators. The short overview of the forward, backward and central differences for first three derivatives can be found in Tables 1.1, 1.2, 1.3.

Table: Forward difference quotient, $\mathcal{O}(\triangle x)$

	$u_{i}$	$u_{i+1}$	$u_{i+2}$	$u_{i+3}$	$u_{i+4}$
$\triangle x\frac{\partial u}{\partial x}$	-1	1
$\triangle x^2\frac{\partial^2 u}{\partial x^2}$	1	-2	1
$\triangle x^3\frac{\partial^3 u}{\partial x^3}$	-1	3	-3	1
$\triangle x^4\frac{\partial^4 u}{\partial x^4}$	1	-4	6	-4	1

Table: Backward difference quotient, $\mathcal{O}(\triangle x)$

	$u_{i-4}$	$u_{i-3}$	$u_{i-2}$	$u_{i-1}$	$u_{i}$
$\triangle x\frac{\partial u}{\partial x}$				-1	1
$\triangle x^2\frac{\partial^2 u}{\partial x^2}$			1	-2	1
$\triangle x^3\frac{\partial^3 u}{\partial x^3}$		-1	3	-3	1
$\triangle x^4\frac{\partial^4 u}{\partial x^4}$	1	-4	6	-4	1

Table: Central difference quotient, $\mathcal{O}(\triangle x^2)$

	$u_{i-2}$	$u_{i-1}$	$u_{i}$	$u_{i+1}$	$u_{i+2}$
$2\triangle x\frac{\partial u}{\partial x}$		-1	0	1
$\triangle x^2\frac{\partial^2 u}{\partial x^2}$		1	-2	1
$2\triangle x^3\frac{\partial^3 u}{\partial x^3}$	-1	2	0	-2	1
$\triangle x^4\frac{\partial^4 u}{\partial x^4}$	1	-4	6	-4	1