Newton’s Method is traditionally used to find the roots of a non-linear equation.

**Definition 1** (**Newton’s Method**): Let *f*(*x*) = 0 be an equation. Define *x _{n}* recursively as follows:

Here *f′*(x_{n}) refers to the derivative *f*(*x*) of at *x _{n}*.

**Property 1**: Let *x _{n}* be defined from

*f*(

*x*) as in Definition 1. As long as function

*f*is well behaved and the initial guess is suitable, then

*f*(

*x*) ≈ 0 for sufficiently large

_{n}*n.*

**Observation**: Newton’s method requires that *f*(*x*) has a derivative and that the derivative not be zero. Usually the method converges quickly to a solution, but this can depend on the initial guess. Also in cases where there are multiple solutions, different initial guesses can lead to different solutions.

**Example 1**: Use Newton’s Method to find the square root of 25.

The problem is equivalent to solving the equation *f*(*x*) = 0 where *f*(*x*) = x^{2} – 25.

From calculus, *f′*(*x*) = 2*x*, and so

Suppose we start the iteration with *x _{0}* = 2, then as we see in Figure 1, the iterations converge to 5 as expected.

**Figure 1 – Newton’s Method for Example 1**

Note that if we select *x _{0}* = 0 the algorithm won’t converge to a solution since would be undefined. Also if we set

*x*= -2 (or any negative value) then the procedure iterates to -5, which is the other solution to

_{0}*f*(

*x*) = 0.

We now extend Newton’s method to *m* equations in *m* unknowns.

**Definition 2** (**Newton’s Method**): Let *X* = [*x _{i}*] be an

*m*× 1 column vector and let

*F*= [

*f*] be an

_{i}*m*× 1 column vector of functions. Our objective is to solve the set of

*m*equations given by

*F*(

*x*) =

*Ο*(here

*Ο*is the zero vector).

Let *J* = [*q _{ij}*] be the Jacobian matrix, defined by the partial derivatives

*q*= . Now define the

_{ij}*m*× 1 column vectors

*X*and the

_{n}*m*×

*m*matrices

*J*as follows

_{n}**Property 2**: Using the terminology in Definition 2, then for a well-behaved *F*, a reasonable guess and sufficiently large *n*, *F*(*X _{n}*) ≈

*Ο*.

**Example 2**: Find the solution to the equations y – *e ^{x} *= 0 and

*x*+ y = 3.

Set *f*(*x*, y) = y – *e ^{x} *and

*g*(

*x*, y) =

*x*+ y – 3. Then we need to find the solution to

*f*(

*x*, y) = 0 and

*g*(

*x*, y) = 0 , i.e.

*F*(

*X*) =

*Ο*where

*X*= and

*F*= . Now

where we use the notation *f_{x}*(

*x*,y) to mean . Starting with

*X*= , we see in Figure 2 that the procedure converges to

_{0}*x*= 0.79206 and y = 2.20794 after 3 iterations.

**Figure 2 – Newton’s Method for Example 2**

Representative formulas for the worksheet in Figure 2 (for iteration 2) are shown in Figure 3.

**Figure 3 – Formulas for 2 ^{nd} iteration**

The example was chosen so that we could check the result using Newton’s method in one variable since the problem is equivalent to *e ^{x }*–

*x*– 3 = 0 and y = 3 –

*x*. We can therefore apply Newton’s method as in Example 1, with

*f*(

*x*) =

*e*–

^{x}*x*– 3,

*f′*(

*x*) =

*e*– 1, and so

^{x}The iteration proceeds as in Figure 4.

**Figure 4 – Alternative use of Newton’s Method for Example 2**

Thus *x* = 0.79206 and y = 3 – *x *= 3 – 0.79206 = 2.20794, as before.