CSC 5800:
Intelligent Systems:
Algorithms and Tools
Lecture 2:
Linear algebra Review
Acknowledgement: This lecture is partially based on the slides from
http://chortle.ccsu.edu/VectorLessons/index.html
Some Terminology
• Point — a geometric object; a location in 3D (or 2D) space.
• Vector — a geometric object that has properties of direction and
length, but not location.
• Column matrix — an ordered list of numbers arranged into a
column.
• Row matrix — an ordered list of numbers arranged into a row.
• Element — one of the numbers that makes up a column or row
matrix.
• Dimension — the number of elements in a column or row matrix.
1
Matrices
• Matrix is a tabular representation of a set of numbers
as a collection of rows and columns
• “m by n matrix” means it has m rows and n columns
• Transpose of matrix is obtained by interchanging
rows by columns
– (AT)T =A
• A(2, :) means all elements of row 2
• A(:,3) means all elements of column 3
Matrix Addition
• Two matrices should be of same dimension
• Commutative (A+B = B+A)
• Associative ((A+B)+C=A+(B+C))
• Zero matrix is the identity element A+0=A
• Additive Inverse of A is –A
2
Scalar Multiplication
• The product of a scalar α and matrix A, is the matrix B= αA
• Associativity – α(βA) = (αβ)A
• Distributivity (Addition over Multiplication)– (α+β)A= αA+βA
• Distributivity (Multiplication over Addition)–α(A+B)= αA+ αB
• Scalar Identity - if α=1, then for any A, αA=A.
Matrix Multiplication
• To multiply two matrices
– No. of columns in the First matrix must the same as the No. of
rows in the Second Matrix
• A (m X n) B (n X k) = C (m X k)
• Generally, it is not Commutative AB<>BA
• Associativity, (AB)C=A(BC)
• Distributivity,
– A(B+C)=AB+AC
– (B+C)A=BA+CA
• Identity Matrix I (n Xn) is a diagonal matrix with 1 in the
diagonal
3
Matrix Properties
• The column rank of a matrix A is the maximal
number of linearly independent columns of A.
Likewise, the row rank is the maximal number
of linearly independent rows of A.
• The rank of an m-by-n matrix is at most the
lesser of m and n.
• A matrix that has as large a rank as possible is
said to have full rank; otherwise, the matrix is
rank deficient.
Matrices
• Singular Matrix have no inverse
– Determinant is zero
• Inverse of a Matrix
• AA-1 = I
• (ABC)-1 = C-1B-1A-1
4
Determinant
Determinant
5
Statistical Operations (from Wikipedia.org)
• Mean is the sum of the observations divided by the number of
observations. It describes the central location of the data
• Variance is the average of the squared differences between data points
and the mean. Standard deviation is the square root of the variance. It is a
measure of the spread of the values.
• Median is a type of average that is described as the number dividing the
higher half of a sample (or a population), from the lower half.
– More robust to outliers
• Mode of a data sample is the element that occurs most often in the
collection.
– For example, the mode of the sample [1, 3, 6, 6, 6, 6, 7, 7, 12, 12, 17]
is 6.
– Given the list of data [1, 1, 2, 4, 4] the mode is not unique, unlike the
arithmetic mean.
Percentile
• A percentile is the value of a variable below which a
certain percent of observations fall.
• E.g. 20th percentile is the value (or score) below which 20
percent of the observations may be found.
• The 25th percentile is also known as the first quartile; the
50th percentile as the median. Decile is equal to 10th
percentile.
• pth percentile of n ordered values is obtained by first
calculating the rank k=p(n+1)/100 , rounded to the
nearest integer and then taking the value that
corresponds to that rank
6
Harmonic Mean
• The harmonic mean H of the positive real numbers a1, ...,
an is defined to be
• If for half the distance of a trip you travel at 40 kilometres
per hour and for the other half of the distance you travel
at 60 kilometres per hour,
• Your average speed for the trip is given by the harmonic
mean of 40 and 60, which is 48; (not 50!!)
• i.e. the total amount of time for the trip is the same as if
you travelled the entire trip at 48 kilometres per hour.
Geometric Mean
• The nth root of the product of all the samples of the data
set, where n is the number of samples.
• The geometric mean is useful to determine "average
factors".
• For example, if a stock rose 10% in the first year, 20% in
the second year and fell 15% in the third year, then we
compute the geometric mean of the factors 1.10, 1.20 and
0.85 as (1.10 × 1.20 × 0.85)1/3 = 1.0391. We conclude
that the stock rose 3.91 percent per year, on average.
7
Relationship of the means
• In case of two numbers
• For all other cases,
Arithmetic Mean > Geometric Mean > Harmonic Mean
chi-square statistic
• X2 is the chi-square statistic is given by
X2 = Sigma [ (O-E)2 / E ]
where, O is the observed frequency and E is the expected frequency.
• For Heads, X2= (47-50)2/50=0.18
• For Tails, X2= (53-50)2/50=0.18
• The sum of these categories is 0.18 + 0.18 = 0.36
• Assess the significance levels and see if it is a fair coin
• More details about chi-square test are available at :
– http://cnx.org/content/m13487/latest/
