Lecture Note by S.

M Alay-e-Abbas: Interpolation and Data Fitting

Polynomial Interpolation Suppose that we have a set of n discrete data points (xi, yi = f(xi)) and it is required to find the value of y at any x between xi and xi+1. One way to achieve this is by constructing an (n-1)th order polynomial that goes through all these points and then evaluate the polynomial at the x we want. By definition, given any n point it is always possible to find a unique polynomial of the order (n-1) going through these points which is called INTERPOLATING POLYNOMIAL. The interpolating polynomial needs to go through all the data points and should exactly be the same as the tabulated points whereas the BEST FIT is a function which is predetermined function that can model a given data.

Figure shows the polynomial interpolation and data fit to a discrete set of data. The solid line shows the polynomial interpolation whereas the dashed line displays the fit given by the function shown on graph that is being used to model the data. It is to be remembered that the polynomial we will obtain is also an approximation to the true function f(x) which
1

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

produces the point (xi, yi). Whereas the fit is a requirement which is used when we do not have confidence in our collected data. Finding the Unique Polynomial Suppose there is an (n-1)th order polynomial Pn-1(xi) = yi for (i = 0 to n) that represents all the data points shown above is called interpolating polynomial. We need to develop the techniques that can be used to approximate these polynomials. The unique polynomial can be represented in many ways. Well look at the polynomial development methods which are easier for numerical computations. The basic technique is underlined below. Let us start by considering the simple polynomial P(x) to the function f(x) in the form of P(x) = a0 + a1x + .+ anxn The coefficient ais can be determined by equating P(xi) = f(xi). But due to loss of significant figure this method is not efficient. Example 1: Consider the function f(x) such that it is given that f(6000) = 1/3 and f(6001) = -2/3 (which are infinite digits floating point numbers). The polynomial interpolation function will be of order 1 given by P(x) = a0 + a1x Using five decimal place floating point arithmetic and solving the two equation 1/3 = a0 + a16000 -2/3 = a0 + a16001 we get
2

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

P(x) = 6000.3 x But now using this P(x) if we compute P(6000) we get a loss of four decimal point digits i.e P(6000) = 0.3 and P(6001) = -0.7 only.

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Newtons Polynomial Approximation using Taylor Series We know that if f(x) is a function then its Taylor expansion around a point a is given by

Where (Rn

0 as n

infinity)

Since the series is in form of a polynomial, it can be used to approximate polynomials of a smooth functions by expanding the known points f(xi) around the point xi. This utilization of Taylors series to write a polynomial results in SHIFTED FORM,

P ( x ) = a 0 + a1 ( x x 0 ) + a 2 ( x x1 ) 2 + .... + a n ( x x n 1 ) n
The coefficients ao, a1 etc are actually the derivatives and the factorial quantities of the function f(x) at xi

f i (c ) ai = i!
According to NEWTONS FORM we do not write the P(x) in shifted form as given above but instead we use

P( x) = a 0 + a1 ( x x0 ) + a 2 ( x x0 )( x x1 ) + .... + a n ( x x0 )( x x1 )...( x x n 1 )
Which will become SHIFTED FORM if all the center points xi were to be the same. And it will be so called POWER FORM if all the xi = 0.
4

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Another way to write this polynomial is in the NESTED FORM as

P( x) = a0 + ( x x0 ){a1 + a 2 ( x x1 )[a 2 + ( x x2 ) {a3 + .... + ( x x n 2 )(a n 1 + ( x xn 1 )a n )....]}

Where common factors are taken out in all terms. This form needs to perform 2n multiplications and n additions but it is easy to program on the computers. The question is now how to compute the coefficients? One way to solve this polynomial is by taking x = x1 then x = x2 and so on getting the equations P(x = x1) = a0 P(x = x2) = a0 + a1 (x2 x1) and so on But this is not a good way of getting along. DIVIDED DIFFERENCE METHOD is used to perform this process. Let us call the function values at x0 as f0, at x1 as f1 and so on.
x0 x1 x2 x3 x4 f0 f1 f2 f3 f4

From

P( x) = a0 + a1 ( x x0 ) + a2 ( x x0 )( x x1 ) + ... ...... + an ( x x0 )( x x1 )...( x xn1 )

Then Newtons the notation will be

P ( x) = a 0 + ( x x0 ) f [ x0 , x1 ] + ( x x 0 )( x x1 ) f [ x0 , x1 , x 2 ] + .. ..... + ( x x0 )( x x1 )...( x x n 1 ) f [ x 0 ...x n ]

Now
5

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

f0 = a o

f1 = a0 + ( x1 x0 ) f [ x0 x1 ]
which gives

f [ x0 x1 ] =

f1 f 0 x1 x0

(a)

now f2 will be calculated in the same way

f 2 = a0 + ( x2 x0 ) f [ x0 x1 ] + ( x2 x0 )( x2 x1 ) f [ x0 , x1 , x2 ]
and compute f[x0, x1, x2] like before. The second difference can also be taken and also the third one as shown in the table below 1st DD
x0 x1 x2 x3 x4 f0 f1 f2 f3 f4

2nd DD
f1 f 0 x1 x 0 f [ x0 , x1 , x 2 ] = f [ x1 , x 2 ] f [ x0 , x1 ] x 2 x0

f [ x0 , x1 ] =

f [ x1 , x 2 ] = f [ x 2 , x3 ] = f [ x3 , x 4 ] =

f1 x 2 x1
f2

f [ x1 , x 2 , x3 ] = f [ x 2 , x3 , x 4 ] =

f [ x 2 , x3 ] f [ x1 , x 2 ] x3 x1 f [ x3 , x 4 ] f [ x 2 , x 3 ] x4 x2

f3 f2 x3 x 2 f4 f3 x 4 x3

From
f [ x0 , x1 , x 2 ] = f [ x1 , x 2 ] f [ x 0 , x1 ] x1 x 0

In general we can write for all the coefficients

6

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

f [ x0 , x1 ...x n ] =

f [ x1 , x 2 ....x n ] f [ x0 , x1 .....x n 1 ] x n x0

Example 2: Find the Newton divided difference method, the polynomial of the order three that passes through the points,
X 2.5 3.75 5.0 6.25 F(x) -28.62 -159.36 -513.97 -1265.45 1st DD 104.592 -283.688 -601.184 2nd DD -71.64 -126.998 3rd DD -14.76224

All upper values are the coefficients of the nested polynomial.

P ( x) = a 0 + ( x x 0 )[ f [ x 0 , x1 ] + ( x x1 ){ f [ x 0 , x1 , x 2 ] + ( x x 2 )[ f [ x 0 , x1 , x 2 , x 3 ]}]
P3(x) = -28.62+(x-2.5)[-104.592+(x-3.75){-71.64+ (x-5.0)[-14.762]}] Exercise 1: Find the sixth order polynomial approximation of log(x) between x = 1 and x = 8 using the Newton divided difference method. Choose different values of xi, i = 1 to 8 in the interval 1 < x <2 and compare the function value at x = 1.43.

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Newton-Gregory Forward Polynomial Approximation We know that the Newtons polynomial in its non-nested form can be written as

P( x) = a 0 + ( x x 0 ) f [ x 0 , x1 ] + ( x x 0 )( x x1 ) f [ x 0 , x1 , x 2 ] + ... .... + ( x x 0 )( x x1 )...( x x n 1 ) f [ x 0 ...x n ]

Consider the special case when the differences between all the data points of x is the same i.e. xi xj = h or xn = x0 + n h. Then we have
f [ x0 , x1 ] = f [ x1 , x 2 ] = f1 f 0 f f0 = 1 x1 x0 h f 2 f1 f f1 = 2 x 2 x1 h

Therefore
f [ x0 , x1 , x 2 ] = f [ x1 , x 2 ] f [ x 0 , x1 ] f 2 f 1 f 1 f 0 f 2 f1 f 0 = = 2 x 2 x0 2h 2 2h 2

Using these values in the Newtons polynomial equation we get

P( x) = a 0 + ( x x 0 ) f 2 f1 f 0 f1 f 0 + ( x x 0 )( x x1 ) 2 + .... h 2h 2
f1 f 0 = f

Writing the function differences as

f i + 2 2 f i +1 f i = 2 f
.

f i + n nf i + n 1 +

n(n 1) f i + n 2 ..... f i = n f 2!

And
x x0 = S etc. 2
8

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

We get
P( x) = f ( x 0 ) + Sf + S ( S 1) 2 S ( S 1)....(S n 1) n f ... + f 2 n!

Which is the Newton-Gregory forward polynomial. Lagrange Interpolation Polynomial This is a simple reformulation of the Newton polynomial that avoids the computation of divided difference. It can be easily implemented in a computer program as compared to Newtons algorithm. To understand this interpolation consider we have three data point x0, x1 and x2 and correspondingly the function has values f0, f1 and f3. Then the Lagrange polynomial can be written as

f 2 ( x) = ( x x1 )( x x 2 )a 0 + ( x x0 )( x x 2 )a1 + ( x x0 )( x x1 )a 2
Putting x = x0 give

f 2 ( x0 ) = f 0 = ( x x1 )( x x 2 )a 0
a0 =
Similarly we can get

f0 ( x0 x1 )( x0 x 2 )

a1 =

f1 ( x1 x0 )( x1 x 2 )
And

a2 =

f2 ( x 2 x0 )( x 2 x1 )

Putting these back in the equation we get

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

f 2 ( x) =

( x x0 )( x x 2 ) ( x x0 )( x x1 ) ( x x1 )( x x 2 ) f0 + f1 + f2 ( x0 x1 )( x0 x 2 ) ( x1 x0 )( x1 x 2 ) ( x2 x0 )( x 2 x1 )
n

This can also be written as

f n1 ( x) = Li ( x) f ( xi )
i =0

Where

Li ( x) =
j =0 j i

x xj xi x j

Note that Lagrange formulation is that each term Li(x) will be one at x = xi and zero at all the other sample points. Thus each product takes on the value f(xi) at the sample point. Exercise 1: Compare the results by plotting the error between actual and interpolated values of example 1 using Newtons, Newton-Gregory and Lagrange polynomials.

10

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Data Fitting Given a discrete set of data points we can interpolate to find the function in between the points. This is what we do when we have complete confidence on the accuracy of given data. Many a time the data may not be accurate and may be noisy, in these cases it is beneficial to use a fit which is a smooth curve that runs through the data points. In such cases we are actually expecting our data to be like a certain function with adjustable parameters. Let us say we expect in an experiment that our measured quantity should behave like = at , where t might be temperature or time etc. In a typical experiment if we measure as a function of t we may get a fluctuating curve. Taking log on both sides we get

log = log a + log t

And plot log ( ) Vs log(t) we must get a straight line. We do not do this visually because there are many such ways to draw a straight line between the points and we can make error. We need a quantitative way to find out which is the correct line that can go through our data points and can also give us an estimate of the maximum error too.

11

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Chi Square Fitting Suppose we have n data points (xi, yi) and we wish to fit a real linear model function f(x)=a1x2 + a2x + a3 through these points. If y = f(x) has some adjustable parameters ais then we can ask ourselves the following questions What are the values of ais that best represent this data? Or how good is the choice of these values? And having selected some values of ais what is the probability that our function will give values close to the measured values? Or what is the difference of yi from y. Or how is y distributed in yi = y + y. If we assume that each data point yi has an independent random error that is Gaussian distribution P(yi) around the true value of f(x).

P( yi ) = y
Or

1 y yi 2

P( yi ) = y

1 f ( x ) yi 2

Then the standard deviation of this distribution is same at all points. Now the total probability is then simply proportional to the multiplication of all the probabilities from i = 1 to i = n given by

P( yi ) y
i =1

1 f ( x ) yi 2

12

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Hence given a model f(x) and measured value yi, then f(x) is in the vicinity of yi with a difference + y. In order to maximize the probability P of getting the true value from our function using our choice of the parameters ais we need to minimize the term appearing as the power of exponential i.e.
1 f ( x ) yi 2
2

. We

call this value as the Chi squared written as

1 n = 2 i =1
2

f ( xi ) y i i

Whose sum over all the standard deviations for our data points should be minimized to get maximum P. This method is therefore called chi square fit. Differentiating with respect to the parameters and equating the Chi square equal to zero gives us minimum chi value. For a good fit we must have

2 = nm
Where n is the number of data points and m is the number of parameters that we have chosen. In that case we have a common given by

1 n = 2 i =1
2

( f ( xi ) yi )2
nm

Which tells that high number of n gives better fit.

13

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

Linear Fit using Chi square fitting Consider we want to fit a model function of line to our data given by

f ( x) = ax + b
The Chi square now becomes

1 n = 2 i =1
2

yi axi b i

Differentiating w.r.t a and b we get

n 2 = a i =1

yi axi b xi = 0 2 i

And
n 2 = b i =1

yi axi b =0 2 i

These two equations will give n simultaneous linear equations which can be solved to find the best values of the parameters a and b. Note that the above equations can be written also as

S xy + aS xx + bS x = 0 S y + aS x + bS = 0
Where following parts can then be individually computed

14

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

S =
i =1

i
n

,
2

S xy =
i =1

xi y i

Sx =
i =1

xi

Sy =
i =1

yi

S xx =
i =1

xi

i2

and

S yy =
i =1

yi

i2

Then the parameter s a and b are simply given by

a=

SS xy S x S y SS xx S x
2

and

b=

S x S xy S xx S y S x SS xx
2

Which will give us the model function we have considered. Example: Consider that a data set of 10 points is xi yi 1 2 3 4 5 6 28.5 7 35.9 8 9 10 48

6.775 11.12 12.45 19.81 26.5

37.19 45

Consider that i is not given then we can assume that the sum of all the i is equal to 1. Then S = 1, Sx = 55, Sy = 271.245, Sxy = 1882.925, Sxx = 385 and Syy = 9232.4 Which gives a = 4.9377083 b = -0.00587 Giving f(x) = 4.9377083 x - 0.00587
15

Lecture Note by S.M Alay-e-Abbas: Interpolation and Data Fitting

The Graph is

Where dotted line is our best fit. We can even fit a general linear function to these data points which gives better results!!! We can also do data fitting for nonlinear functions!!!

16