Eigenvalue Problems
1.
Eigenvalues and eigenvectors
2.
Vector spaces
3.
Linear transformations
4.
Matrix diagonalization
Eigenvalue Problems
1.
Eigenvalues and eigenvectors
2.
Vector spaces
3.
Linear transformations
4.
Matrix diagonalization
The Eigenvalue Problem
Consider a nxn matrix A
Vector equation: Ax = x
» Seek solutions for x and
» satisfying the equation are the eigenvalues
» Eigenvalues can be real and/or imaginary;
distinct and/or repeated
» x satisfying the equation are the eigenvectors
Nomenclature
» The set of all eigenvalues is called the spectrum
» Absolute value of an eigenvalue:
» The largest of the absolute values of the
eigenvalues is called the spectral radius
2
2
b
a
ib
a
j
j
Determining Eigenvalues
Vector equation
» Ax = x (A-x = 0
» A- is called the characteristic matrix
Non-trivial solutions exist if and only if:
» This is called the characteristic equation
Characteristic polynomial
» nth-order polynomial in
» Roots are the eigenvalues {
1
,
2
, …,
n
}
0
)
det(
2
1
2
22
21
1
12
11
nn
n
n
n
n
a
a
a
a
a
a
a
a
a
I
A
Eigenvalue Example
Characteristic matrix
Characteristic equation
Eigenvalues:
1
= -5,
2
= 2
4
3
2
1
1
0
0
1
4
3
2
1
I
A
0
10
3
)
3
)(
2
(
)
4
)(
1
(
2
I
A
Eigenvalue Properties
Eigenvalues of A and A
T
are equal
Singular matrix has at least one zero
eigenvalue
Eigenvalues of A
-1
: 1/
1
, 1/
2
, …, 1/
n
Eigenvalues of diagonal & triangular matrices
are equal to the diagonal elements
Trace
Determinant
n
j
j
Tr
1
)
(
A
n
j
j
1
A
Determining Eigenvectors
First determine eigenvalues: {
1
,
2
, …,
n
}
Then determine eigenvector
corresponding to each eigenvalue:
Eigenvectors determined up to scalar
multiple
Distinct eigenvalues
» Produce linearly independent eigenvectors
Repeated eigenvalues
» Produce linearly dependent eigenvectors
» Procedure to determine eigenvectors more
complex (see text)
» Will demonstrate in Matlab
0
)
(
0
)
(
k
k
x
I
A
x
I
A
Eigenvector Example
Eigenvalues
Determine eigenvectors: Ax = x
Eigenvector for
1
= -5
Eigenvector for
1
= 2
3
1
or
9487
.
0
3162
.
0
0
3
0
2
6
1
1
2
1
2
1
x
x
x
x
x
x
1
2
or
4472
.
0
8944
.
0
0
6
3
0
2
2
2
2
1
2
1
x
x
x
x
x
x
2
5
4
3
2
1
2
1
A
0
)
4
(
3
0
2
)
1
(
4
3
2
2
1
2
1
2
2
1
1
2
1
x
x
x
x
x
x
x
x
x
x
Matlab Examples
>> A=[ 1 2; 3 -4];
>> e=eig(A)
e =
2
-5
>> [X,e] = eig(A)
X =
0.8944 -0.3162
0.4472 0.9487
e =
2 0
0 -5
>> A=[2 5; 0 2];
>> e=eig(A)
e =
2
2
>> [X,e]=eig(A)
X =
1.0000 -1.0000
0 0.0000
e =
2 0
0 2
Vector Spaces
Real vector space V
» Set of all n-dimensional vectors with real
elements
» Often denoted R
n
» Element of real vector space denoted
Properties of a real vector space
» Vector addition
» Scalar multiplication
V
x
0
a
a
w
v
u
w
v
u
a
0
a
a
b
b
a
)
(
)
(
)
(
a
a
a
a
a
a
a
b
a
b
a
1
)
(
)
(
)
(
)
(
k
c
k
c
ck
k
c
c
c
c
Vector Spaces cont.
Linearly independent vectors
» Elements:
» Linear combination:
» Equation satisfied only for c
j
= 0
Basis
» n-dimensional vector space V contains
exactly n linearly independent vectors
» Any n linearly independent vectors form a
basis for V
» Any element of V can be expressed as a
linear combination of the basis vectors
Example: unit basis vectors in R
3
0
2
1
(m)
(2)
(1)
a
a
a
m
c
c
c
V
(m)
(2)
(1)
a
a
a
,
,
,
3
2
1
3
2
1
3
3
2
1
1
0
0
0
1
0
0
0
1
c
c
c
c
c
c
c
c
c
)
(
(2)
(1)
a
a
a
x
Inner Product Spaces
Inner product
Properties of an inner product space
Two vectors with zero inner product are called
orthogonal
Relationship to vector norm
» Euclidean norm
» General norm
» Unit vector: ||a|| = 1
n
k
n
n
k
k
T
b
a
b
a
b
a
b
a
1
2
2
1
1
)
,
(
b
a
b
a
b
a
0
if
only
and
if
0
)
,
(
0
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
2
1
2
1
a
a
a
a
a
a
c
c
a
c
b
c
a
c
b
a
q
q
q
q
2
2
2
2
1
)
,
(
n
T
a
a
a
a
a
a
a
a
b
a
b
a
b
a
b
a
)
,
(
Linear Transformation
Properties of a linear operator F
» Linear operator example: multiplication by a matrix
» Nonlinear operator example: Euclidean norm
Linear transformation
Invertible transformation
» Often called a coordinate transformation
)
(
)
(
)
(
)
(
)
(
x
x
x
v
x
v
cF
c
F
F
F
F
Ax
y
A
y
x
tion
Transforma
Operator
,
Elements
xn
m
m
n
R
R
R
y
A
x
Ax
y
A
y
x
1
x
tion
Transforma
Inverse
tion
Transforma
,
,
Dimensions
n
n
n
n
R
R
R
Orthogonal Transformations
Orthogonal matrix
»
A square matrix satisfying: A
T
= A
-1
»
Determinant has value +1 or -1
»
Eigenvalues are real or complex conjugate pairs
with absolute value of unity
»
A square matrix is orthonormal if:
Orthogonal transformation
»
y = Ax where A is an orthogonal matrix
»
Preserves the inner product between any two
vectors
»
The norm is also invariant to orthogonal
transformation
b
a
v
u
Ab
v
Aa
u
,
k
j
k
j
k
T
j
if
1
if
0
a
a
Similarity Transformations
Eigenbasis
» If a nxn matrix has n distinct eigenvalues,
the eigenvectors form a basis for R
n
» The eigenvectors of a symmetric matrix
form an orthonormal basis for R
n
» If a nxn matrix has repeated eigenvalues,
the eigenvectors may not form a basis for
R
n
(see text)
Similar matrices
» Two nxn matrices are similar if there exists
a nonsingular nxn matrix P such that:
» Similar matrices have the same eigenvalues
» If x is an eigenvector of A, then y = P
-1
x is
an eigenvector of the similar matrix
AP
P
A
1
ˆ
Matrix Diagonalization
Assume the nxn matrix A has an eigenbasis
Form the nxn modal matrix X with the
eigenvectors of A as column vectors: X = [x
1
,
x
2
, …, x
n
]
Then the similar matrix D = X
-1
AX is diagonal
with the eigenvalues of A as the diagonal
elements
Companion relation: XDX
-1
= A
n
nn
n
n
n
n
a
a
a
a
a
a
a
a
a
0
0
0
0
0
0
2
1
1
2
1
2
22
21
1
12
11
AX
X
D
A
Matrix Diagonalization
Example
2
0
0
5
2
15
4
5
1
3
2
1
7
1
1
3
2
1
4
3
2
1
1
3
2
1
7
1
1
3
2
1
7
1
1
3
2
1
1
2
,
2
3
1
,
5
4
3
2
1
1
1
1
2
1
2
2
1
1
AX
X
D
AX
X
D
X
x
x
X
x
x
A
Matlab Example
>> A=[-1 2 3; 4 -5 6; 7 8 -9];
>> [X,e]=eig(A)
X =
-0.5250 -0.6019 -0.1182
-0.5918 0.7045 -0.4929
-0.6116 0.3760 0.8620
e =
4.7494 0 0
0 -5.2152 0
0 0 -14.5343
>> D=inv(X)*A*X
D =
4.7494 -0.0000 -0.0000
-0.0000 -5.2152 -0.0000
0.0000 -0.0000 -14.5343