Professor: J.P. Pretti | Term: Winter 2023

Vectors in $R^{n}$

A member of $R^{n}$ is called a vector. We often use overline arrows. Let $x \in R^{n}$ . Can also write as a row, but use $^{⊺}$ superscript.

40 - 3 = [40 - 3]^{⊤}

Two ways to think about vectors.

Algebraic. List of $n$ real numbers
Geometric. Directed line segment between 2 points

Given $u, v \in R^{n}$ with $u = u_{i} ⋮ u_{n}$ $v = v_{i} ⋮ v_{n}$ then $u = v$ if $u_{i} = v_{i}$ for $i = 1, \dots, n$

Two operations:

Addition: Input $2$ vectors in $R^{n}$ outputs 1 vector in $R^{n}$ .

Scalar Multiplication: 1 vector in $R^{n}$ & 1 real number outputs 1 vector in $R^{n}$

u + v = u_{1} + v_{1} ⋮ u_{n} + v_{n}

Properties: Let $u, v, w \in R^{n}$

$u + v = v + u$
$u + (v + w) = (u + v) + w$
$0 ⋮ 0 is the additive identity of R^{n} : u + 0 ⋮ 0 = u$
$u$ has an additive inverse. If $u = u_{1} ⋮ u_{n}$ , then $- u_{1} ⋮ - u_{n}$ is the additive inverse of $u = 0$

Definition

Let $c \in R, u \in R^{n} .$ Say $u = u_{1} ⋮ u_{n}$ , then $c u = c u_{1} ⋮ c u_{n}$ . This stretches $u$ by $∣ c ∣$ . If $c < 0$ , then switch direction of $u$ .

Properties: Let $c, d \in R, u, v \in R^{n}$

$(c + d) v = c v + d v$
$(c d) v = c (d v)$
$c (u + v) = c u + c v$
$0 v = 0$
If $c v = 0$ then $c = 0$ or $v = 0$

Proof of the third point above.

c u_{1} ⋮ u_{n} + v_{1} ⋮ v_{n} = c u_{1} + v_{1} ⋮ u_{n} + v_{n} = c (u_{1} + v_{1}) ⋮ c (u_{n} + v_{n}) = c u_{1} + c v_{1} ⋮ c u_{n} + c v_{n} = c u_{1} ⋮ c u_{n} + c v_{1} ⋮ c v_{n} = c u_{1} ⋮ u_{n} + c v_{1} ⋮ v_{n}

Standard Basis

Standard Basis for $R^{2}$

[34] = 3 [10] + 4 [01] = 3 e_{1} + 4 e_{2}

$E = {e_{1}, e_{2}, \dots, e_{n}}$ is the standard basis.

Dot Product

Vector $\cdot$ Vector $⟹$ Real Number

$u \cdot v = u_{1} v_{1} + u_{2} v_{2} + \dots + u_{n} v_{n}$

Properties

$u \cdot v = v \cdot u$
$(u + v) \cdot w = u \cdot w + v \cdot w$
$(c u) \cdot v = c (u \cdot v)$

$v \cdot v \geq 0$ (only $= 0$ iff $v = 0$ )

Length of Vector

$v \in R^{n}$ is $∣∣ v ∣∣ = v \cdot v$ (magnitude).

[34] = [34] \cdot [34] = 5

If $c \in R$ and $v \in R^{n}$ , then $∣∣ c v ∣∣ = ∣ c ∣∣∣ v ∣∣$

Unit Vectors

$v \in R^{n}$ is a unit vector if $∣∣ v ∣∣ = 1$

Normalization

When $v \in R^{n}$ is a non-zero vector, we can produce a unit vector.

\overset{v}{^} = \frac{v}{∣∣ v ∣∣}

\hat{[34]} = \frac{1}{5} [34] = [\frac{3}{5} \frac{4}{5}]

Angle

∣∣ u - v ∣ ∣^{2} = ∣∣ u ∣ ∣^{2} + ∣∣ v ∣ ∣^{2} - 2∣∣ u ∣∣∣∣ v ∣∣ cos θ

Also

∣∣ u - v ∣ ∣^{2} = (u - v) \cdot (u - v) = u \cdot u - v \cdot u - u \cdot v + v \cdot v = ∣∣ u ∣ ∣^{2} - 2 u \cdot v + ∣∣ v ∣ ∣^{2}

Angle between $u$ and $v$ $(0 \leq θ \leq π)$ .

$u \cdot v = ∣∣ u ∣∣∣∣ v ∣∣ cos θ$

$θ = arccos (\frac{u \cdot v}{∣∣ vv u ∣∣∣∣ v ∣∣})$

Orthogonality

Two vectors are perpendicular if $u \cdot v = 0$ .

More rules:

$e_{1} \cdot e_{2} = 0$
$v \cdot 0 = 0, \forall v \in R^{n}$

Find 2 vectors orthogonal to $[12]$ . $[- 4 2], [- 6 3]$

Without inspection, $v_{1} + 2 v_{2} = 0$

Find the angle in-between $[13] [31]$

θ = arccos \frac{[ 1 3 ] \cdot [ 3 1 ]}{[ 1 3 ] [ 3 1 ]} = arccos (\frac{2 3}{2 \cdot 2}) = \frac{π}{6}

Projection and Perpendicular

We want $p ro j_{w} v = c w$ for some $c \in R$ and $p ro j_{w} v \cdot p er p_{w} v = 0$ .

Then

v \cdot w = (c w + p er p_{w} v) \cdot w = (c w) \cdot w + p er p_{w} v \cdot w = c ∣∣ w ∣ ∣^{2} + 0

Definition

Let $v, w \in R^{n}$ and $w \neq = 0$ . Projection of $v$ onto $w$ is
$p ro j_{w} (v) = \frac{( v \cdot w )}{∣∣ w ∣ ∣ ^{2}}$

The length of $w$ does not impact length of projection.

Definition

Perpendicular of $v$ onto $w$ is defined by
$p er p_{w} (v) = v - p ro j_{w} (v)$

We have $p ro j_{w} v = (v \cdot \overset{w}{^}) \overset{w}{^}$

Also

p ro j_{w} v = \frac{∣∣ v ∣∣∣∣ w ∣∣ cos θ}{∣∣ w ∣ ∣ ^{2}} \cdot w = ∣∣ v ∣∣ cos θ \overset{w}{^}

Let $v = [v_{1} v_{2}] \in R^{2}$

then

p ro j_{e_{1}} v = \frac{[ v _{1} v _{2} ] \cdot [ 1 0 ]}{[ 1 0 ] [ 1 0 ]} [10] = v_{1} [10] = [v_{1} 0]

and $p er p_{e_{1}} v = [v_{1} v_{2}] - [v_{1} 0] = [0 v_{2}]$

$p er p_{w} v \cdot p ro j_{w} w = 0$
$p ro j_{w} (p ro j_{w} v) = p ro j_{w} v$
$p ro j_{w} (c_{1} v_{1} + c_{2} v_{2}) = c_{1} p ro j_{w} v_{1} + c_{2} p ro j_{w} v_{2}$

Fields

$R$ and $C$ are fields

Can multiply, add, subtract, and stay in set
Can divide by non-zero and stay in set
”Everything works” (commutative, associative, distributive, inverse).

Cross Product

Let $u = u_{1} u_{2} u_{3}, v = v_{1} v_{2} v_{3} \in R^{3}$ (only defined for $R^{3}$ )

Cross product is given by

u \times v = u_{2} v_{3} - u_{3} v_{2} - (u_{1} v_{3} - u_{3} v_{1}) u_{1} v_{2} - u_{2} v_{1}

Properties

$u, v \in R^{3}$ and let $z = u \times v$

$z \cdot v = 0$ and $z \cdot u = 0$
$v \times u = - z = - u \times v$
$∣∣ u \times v ∣∣ = ∣∣ u ∣∣∣∣ v ∣∣ sin θ$ , where $sin θ$ is the angle between $u, v$ (and $u, v$ are not zero vectors).

$∣∣ u \times v ∣∣ =$ area of parallelogram

$e_{1} \times e_{2} = e_{3}$ $e_{3} \times e_{2} = - e_{1}$ $(c u) \times v = c (u \times v)$ $u \times (c v) = c (u \times v)$

Spans, Lines, and Planes

Linear Combinations and Spans

Example

i [1 - i] + (2 + i) [3 i 0] = [- 3 + 7 i 1]

$[- 3 + 7 i 1]$ is a linear combination of $[1 - 1]$ and $[3 i 0]$ .

$[- 3 + 7 i 1] \in Span {[1 - i], [3 i 0]}$

Linear combination is a vector, span is a set.

Examples - True or False?

Every vector in $R^{2}$ is an element of Span ${e_{1}, e_{2}}$
Every vector in $R$ is an element of Span ${[10], [11]}$
Every vector in $R^{3}$ is an element of Span $⎩ ⎨ ⎧ 100, 110 ⎭ ⎬ ⎫$
True - $[v_{1} v_{2}] = v_{1} [10] + v_{2} [01]$
True - $[v_{1} v_{2}] = c_{1} [10] + c_{2} [11], v_{1} = c_{1} + c_{2}, v_{2} = c_{2}$
False

Geometry

Span ${0} = {0}$

For $v \neq = 0$ , Span ${v}$ is a line through origin with direction $v$ .

For $v, w \neq = 0$ , if $v, w$ are not parallel, then Span ${v, w}$ is a plane

Lines

Parametric Equations

$p, q$ are fixed numbers ( $q \neq = 0$ ). Parametric equations of line in $R^{2}$ through point $(x, y)$ with slope $\frac{p}{q}$ are

x y = x_{1} + qt t \in R = y_{1} + pt

Vector Equation of a Line in $R^{2}$

[x y] = [x_{1} y_{1}] + t [p q], t \in R

This is a vector equation of line $L$ in $R^{2}$ through $[x_{1} y_{1}]$ with direction $[p q]$ .

Line in $R^{2}$

$L = {u + t v : t \in R}$

Parametric Equations of a Line

Parameter $t$ “generates” points on the line. When $q = 0$ , we have $x = x_{1}, y = y_{1} + pt$ which is a vertical line with undefined slope.

Vector Equation of a Line

Let $u = [x_{1} y_{1}]$ and $v = [q p]$ .

$ℓ = u + t v, t \in R$

$u$ is on
$u - v$ is on $ℓ$
$v$ usually is not on $ℓ$
$v$ is on $ℓ$ iff $v \in$ Span ${u}$

Exercises:

Give parametric equations for the line $y = 3 x + 1$
Give a vector equation of a line through $[13]$ and $[22]$
Show that $L_{1} = {[43] + t [1 - 1] : t \in R}$ is the same line as $L_{2} = {[1 - 2] + t [2 - 2] : t \in R}$

[x_{1} y_{1}] = [01]

[q p] = [13]

$x = t, y = 1 + 3 t$

l = [13] + t ([13] - [22]) t \in R

Prove parallel and coincident (pass through the same point) or,

Can set up an algebraic system of equations using set equality.

Direction vectors are scalar multiples, immediately parallel.

Equations in $R^{n}$

Vector Equations

$l = u + t v, t \in R$

Parametric Equations

l_{1} l_{2} l_{n} = u_{1} + t v_{1} = u_{2} + t v_{2} ⋮ = u_{n} + t v_{n}

Lines

$L = {u + t v : t \in R}$

Lines through the origin can be described as the span.

Example of a Line in $R^{5}$

L = Span ⎩ ⎨ ⎧ 12345 ⎭ ⎬ ⎫

Line through the origin and $(1, 2, 3, 4, 5)$ .

Parametric equations: $x_{1} = t \dots x_{5} = 5 t t \in R$

Planes Through the Origin

$v, w$ (non-zero), $v \neq = c w$ for $c \in R$

$P = Span {v, w} = {s v + t w : s, t \in R}$

Plane Examples

P_{1} = s 102 + t 0 - 1 1

P_{2} = 0100 + s 102 + t 0 - 1 1

These two are not on $p_{2}$ . When scaled and added to $0100$ , then it is on it.

$v + w$ are on $p ⟺ u \in Span {v, w}$

Normal Vectors

102 \times 0 - 1 1 = 2 - 1 - 1

\in p_{1} x y z \cdot 2 - 1 - 1 = 0

$n = v \times w$ (normal vector to plane)

n = a b c, u \in P and p = x y z \in P

Normal form of $P$

n \cdot (p - u) = 0

Expanding $= a x + b y + cz = d$

$d = n \cdot u$

Exercise

Find a vector equation and a scalar equation of the plane containing $A (1, 0, 2), B (- 3, - 2, 4)$ and $C (1, 8, 5)$ .

Can find direction vectors:

102 - - 3 - 2 4 = 42 - 2

102 - 18 - 5 = 0 - 8 7

So a vector equation is:

p = 102 + s 42 - 2 + t 0 - 8 7 s, t \in R

Finding Scalar Equation, we can find a normal vector:

Orthogonal to the 2 direction vectors

n = 42 - 2 \times 0 - 8 7 = - 2 - 28 - 32

Our equation is thus

- 2 x - 28 y - 32 z = - 2 - 28 - 32 \cdot 102 = - 66

Simplifies to $x + 14 y + 16 z = 33$

Systems of Linear Equations

These show up everywhere.

Examples from linear algebra

Is a vector in the span of a given set?
Find a vector orthogonal to another vector(s)?
Determine the intersection of two planes

Linear Equation

a_{1} x_{1} + a_{2} x_{2} + \dots a_{n} x_{n} = b

where $a_{1}, a_{2}, \dots b \in F$

$a \to$ coefficients (known)

$x \to$ variables (unknown)

A system is multiple of these. For $a_{ij}, i$ is the equation and $j$ gives the variable.

Solving means we found $y_{1} \dots y_{n}$ into $x_{1} \dots x_{n}$ and LHS=RHS.

We view a system as simultaneous.

Solution Set

a) The solution set of $x_{1} + 2 x_{2} + x_{3} = 7$ .

S = ⎩ ⎨ ⎧ 7 - 2 s - t s t : s, t \in R ⎭ ⎬ ⎫ Scalar Equation = ⎩ ⎨ ⎧ 700 + s - 2 10 + t - 1 01 : s, t \in R ⎭ ⎬ ⎫ Vector Equation

b) Show that

x_{1} x_{2} x_{3} = 031 + t 1 - 2 3, t \in R

is a solution to

x_{1} + 2 x_{2} + x_{3} x_{1} - x_{2} - x_{3} = 7 = - 4

Let $t \in R$ . Then $x_{1} = t, x_{2} = 3 - 2 t, x_{3} = 1 + 3 t$ .

And indeed,

t + 2 (3 - 2 t) + 1 + 3 t = 7 + 0 t t - (3 - 2 t) - (1 + 3 t) = - 4 + 0 t = 7 = - 4

This shows that the given line is a solution.

Solution is either empty, one element (unique), infinite number of equations.

Solving Systems: Two big ideas.

Some systems are easy to solve ( $x_{1} = 7, x_{2} = i, x_{3} = 2 - i$ ).

Solve a “hard” system by changing it to a “simpler” but equivalent system. Two systems are equivalent if they have the same solution set.

How do we do this? We can multiply both sides by $c \neq = 0$ , and we can add equations. We cannot multiply by zero, square an equation, remove an equation, or insert an equation.

Elementary Operations

Swap elementary operation

e_{i} \leftrightarrow e_{j}

Scale elementary operation

e_{i} \to m e_{i}, m \in F ∖ {0}

Addition elementary operation

e_{j} \to c e_{i} + e_{j} i \neq = j, c \in F

A single elementary operation performed produces an equivalent system.

Unique solution (consistent)

x_{1} + x_{2} x_{1} - x_{2} = 1 = 1

Infinite solutions (consistent)

x_{1} + x_{2} - x_{1} - x_{2} = 1 = - 1

No solutions (inconsistent)

x_{1} + x_{2} x_{1} + x_{2} = 1 = 0

Solving Systems using EO’s

x_{1} + 2 x_{2} + x_{3} x_{1} - x_{2} - x_{3} = 7 = - 4

e_{2} \to (- 1) e_{1} + e_{2} gives: x_{1} + 2 x_{2} + x_{3} - 3 x_{2} - 2 x_{3} = 7 = - 11

e_{1} \to \frac{2}{3} e_{2} + e_{1} gives: x_{1} - \frac{1}{3} x_{3} - 3 x_{2} - 2 x_{3} = - \frac{1}{3} = - 11

e_{2} \to \frac{1}{3} e_{2} gives: x_{1} - \frac{1}{3} x_{3} x_{2} + \frac{2}{3} x_{3} = - \frac{1}{3} = \frac{11}{3}

Let $x_{3} = t$ for some $t \in R$ .

We get the solution set:

S = ⎩ ⎨ ⎧ - \frac{1}{3} + t \frac{1}{3} \frac{11}{3} - t \frac{2}{3} t : t \in R ⎭ ⎬ ⎫

Question

Is $(3, 2, 1)$ on $p = s 01 + s 235 + t 467, s, t \in R ?$

We need to solve

2 s + 4 t 3 s + 6 t 5 s + 7 t = - 2 = 2 = 0

Has a solution $⟹$ on the plane.

e_{1} \to \frac{1}{2} e_{1} gives: s + 2 t 3 s + 6 t 5 s + 7 t = - 1 = 2 = 0

e_{2} \to - 3 e_{1} + e_{2} gives: s + 2 t 0 5 s + 7 t = - 1 = 5 = 0

There is no solution. That is $(3, 2, 1)$ is not on $p$ .

Prove that the only vector orthogonal to

- 2 - 4 - 6, 372, and - 3 50

is $0$ .

We must solve

- 2 x_{1} - 4 x_{2} - 6 x_{3} 3 x_{1} + 7 x_{2} + 2 x_{3} - 3 x_{1} + 5 x_{2} = 0 = 0 = 0

We will work with an augmented matrix:

- 2 3 - 3 - 4 75 - 6 20 000

Matrix

$m \times n$ matrix, array of $m$ rows, $n$ columns. $a_{ij}$ is coefficient of $x_{j}$ in the $i$ th equation.

Elementary Row Operations

Row swap: $e_{i} \leftrightarrow e_{j} R_{i} \leftrightarrow R_{j}$
Row scale: $e_{i} \to c e_{i} R_{i} \to c R_{i}$
Row addition: $e_{i} \to c e_{j} + e_{i} R_{i} \to c R_{j} \to R_{i}$

Zero row: Row that has all zero entries.

If $B$ is from $A$ by finite ERO, $A$ is equivalent to $B$ .

Gauss-Jordan Algorithm (Big Picture)

Assume $x$ , “really exists”. Begin by forward elimination phase (the goal is REF).

Scale so all leading coefficients are 1. We can now use back substitution to solve or, continue with backwards elimination phase. The goal is RREF (Reduced Row Echelon Form). Place 0s above all leading coefficients.

Leftmost non-zero entry of a non-zero matrix is the leading entry.

Definition

Row Echelon Form: When all zero rows are at the bottom, leading entries appear in a column to the right of the columns entering the leading entries from columns above.

Gauss-Jordan Algorithm Examples

- 2 3 - 3 - 4 75 - 6 20 000 R_{1} \to - \frac{1}{2} R_{1} 13 - 3 275320000 R_{2} \to - 3 R_{1} + R_{2} R_{3} \to R_{3} + 3 R_{1} 1002111 3 - 7 9 000 R_{3} \to R_{3} - 11 R_{2} 100210 3 - 7 86 000 R_{3} \to \frac{1}{86} R_{3} 100210 3 - 7 1 000 R_{1} \to R_{1} - 3 R_{3} R_{2} \to R_{2} + 7 R_{3} 100210001000 R_{1} \to - 2 R_{2} + R_{1} gives: 100010001000 RREF

This tells us that $⎩ ⎨ ⎧ 000 ⎭ ⎬ ⎫$ is the unique solution to the system and so $0$ is indeed the only vector orthogonal to the three given vectors.

021 2 - 6 - 2 37541510 R_{1} \leftrightarrow R_{3} gives: 120 - 2 - 6 2 57310154 R_{2} \to - 2 R_{1} + R_{2} 100 - 2 - 2 2 5 - 3 3 10 - 5 4 R_{3} \to R_{2} + R_{3} gives: 100 - 2 - 2 0 5 - 3 0 10 - 5 - 1 REF

Hence there is no solution.

1 - 2 1 2 - 4 2 - 3 6 - 3 023 4 - 2 13 R_{2} \to 2 R_{1} + R_{2} R_{3} \to R_{3} - R_{1} 100200 - 3 00 023469 R_{3} \to \frac{1}{3} R_{3} 100200 - 3 00 021463 R_{2} \to R_{2} - 2 R_{3} R_{2} \to \frac{1}{2} R_{2} 100200 - 3 00 010430 RREF

The first and fourth columns are pivot columns. The first two rows are pivot rows.

$x_{1}$ and $x_{3}$ are basic variables. $x_{2}$ and $x_{3}$ are free variables. When we see that the system is consistent, we assign parameters to the free variables and express the basic variables in terms of these parameters.

Notation: $M_{m \times n} (F)$ .

Definition

Let $A$ be any matrix, the rank is the number of pivots $rank (A) = r$ .

There are bounds to the rank. $rank (A) \leq min {m, n}$ .

Definition

System is consistent iff $rank (A) = rank ([A b])$ .

Proof:

Let $A$ and $[A b]$ be as given.

Let $[R c]$ be $RREF ([A b])$ .

Note that every pivot of $R$ is a pivot of $[R c]$ . This means $rank (A) \leq rank ([A b])$ .

$(⟹)$ First assume the system is consistent. Then $[R c]$ does not have a row of the form $[0 \dots 01]$ . That is, $[R c]$ does not have a pivot of $R$ . Hence $rank (A) = rank ([A b])$ .

$(⟸)$ Assume the system is inconsistent.

Therefore $[R c]$ has a row of the form $[0 \dots 01]$ . That $1$ is a pivot of $[R c]$ that is not a pivot of $R$ . Hence $rank (A) < rank ([A b])$ . That is $rank (A) = rank ([A b])$ .

Theorem

System Rank Theorem: Let $A \in M_{m \times n} (F)$ with $rank (A) = r$ . a) Let $b \in F^{m}$ . If the system of equations with augmented matrix $[A b]$ is consistent, then the solution set to this system will contain $n - r$ parameters. b) System with augmented matrix $[A b]$ is consistent for every $b \in F^{m}$ if and only if $r = m$ .

Sketch proof

$n$ is the number of columns $⟹$ $n$ is the number of variables. Rank is $r$ means $r$ pivots $⟹$ $r$ basic variables. $⟹$ $n - r$ free variables, $⟹$ $n - r$ parameters

$(⟸)$

Assume $r = m$ . Let $b \in F^{m}$ . Let $[R c]$ be $RREF ([A b])$ pivot in every row of $R$ . No $[0 \dots 01] ⟹$ consistent.

$(⟹)$

Assume $r \neq = m$ . Let $R = RREF (A)$ . There is a row without a pivot in $R$ . Hence $R$ has a zero-row. Say it is the $i$ th row. Hence $[R e_{i}]$ is inconsistent.

Using ERO in reverse, we get $[A b]$ which is inconsistent.

$nullity (A) = n - r$ .

Homogeneous Solutions: All elements on the right hand side are zero. Otherwise it is a non-homogeneous solution.

For homogeneous solutions, trivial solution $x_{1} = x_{2} = \dots = 0$ .

Solution to homogeneous system is called the nullspace of $A$ .

Matrix-Vector Multiplication

230 1 - 2 1 014 201 = 2 \cdot 2 + 1 \cdot 0 + 0 (- 1) 3 \cdot 2 + (- 2) 0 + 1 (- 1) 0 \cdot 2 + 1 \cdot 0 + 4 (- 1) = 2230 + 0 1 - 2 1 + (- 1) 614 = 45 - 4 = 45 - 4

Row Vector

$row_{i} (A) = [a_{i 1} a_{i 2} \dots a_{in}]$ .

Number of columns must be the same but $m$ doesn’t have to equal $n$ .

$[A b]$ can be written as $A x = b$ .

GJA works over $C$ also.

Example

$x_{1} - x_{2} - x_{3} - x_{4} = 1 x_{1} + x_{3} + x_{4} = 2 2 x_{1} - x_{2} = 3$

A = 112 - 1 0 - 1 - 1 10 - 1 10 b = 123

[A b] rref 100010120120210 x_{1} = - x_{3} - x_{4} + 2 x_{2} = - 2 x_{3} - 2 x_{4} + 1

x = x_{1} x_{2} x_{3} x_{4} = - x_{3} - x_{4} + 2 - 2 x_{3} - 2 x_{4} + 1 x_{3} x_{4} = 2100 + x_{3} - 1 210 + x_{4} - 1 - 2 01

$b$ determines whether or not the system is consistent. Structure really depends on $A$ .

[A 0] rref x = x_{3} - 1 210 + x_{4} - 1 - 2 01

Can partially/mostly understand $[A b]$ once we understand $[A 0]$ .

Notice $[A 0]$ is consistent. Note $0$ is a solution. Call it the trivial solution.

The set of all solutions to $A x = 0$ is called the nullspace of $A$ , written as $Null (A)$ .

Example

$Null (A) = ⎩ ⎨ ⎧ r - 1 210 + s - 1 - 2 01 : r, s \in R ⎭ ⎬ ⎫ = Span ⎩ ⎨ ⎧ - 1 210, - 1 - 2 01 ⎭ ⎬ ⎫$

Proposition: Let $S = Null (A)$ for some $A \in M_{m \times n}$ .

Let $u, v \in S$ . Let $c \in F$ . Then $u + v \in S$ and $c u \in S$ .

Proof:

To show $u + v \in Null (A)$ , we must show $A (u + v) = 0$ . Well, $A (u + v) = A u + A v = 0 + 0 = 0$ .

Proposition: Let $A \in M_{m \times n}$ . Let $b \in R^{m}$ such that $A x = b$ is consistent.

Let $c$ be a solution to $A x = b$ .

Let $T$ be the solution set to $A x = b$ .

Then $T = {c + y : y \in Null (A)} \leftarrow X$ .

So every solution to $A x = b$ is $c \cdot ($ a solution to $A x = 0)$ .

Proof:

Let $z \in X$ . So $z = c + v : v \in Null (A)$ .

We want to show $z \in T$ , i.e. that $A z = b$ .

Well, $A z = A (c + v) = A c + A v = b + 0 = b$ .

Let $z \in T$ . So $A z = b$ .

We want to show $z \in X$ , we want to show $z = c + Null (A)$ .

Note $A (z - c) = A z - A c = b - b = 0$ . So $z - c \in Null (A)$ .

So $z = c + \in Null (z - c)$ , so $z \in X$ .

$A x = b$ and $A x = 0$ are associated homogeneous systems.

Let $S$ be the solution set of $A x = b$ .

Let $T$ be the solution set of $A x = 0$ .

Suppose $c$ is a solution of $A x = b$ .

Then, $S = {c + z : z \in T}$ .

Suppose $A x = 0$ has solution set

⎩ ⎨ ⎧ r 1100 + s 1234 : s, r, \in R ⎭ ⎬ ⎫

Suppose $9999$ is a solution to $A x = b$ .

Give two more solutions.

101099 \to r = 1, s = 0 11121213 \to r = 1, s = 1

Consider now $A x = b_{1}$ , solution set $S_{1} \neq = \emptyset$ . Consider $A x = b_{2}$ , solution set $S_{2} \neq = \emptyset$ .

Suppose $c_{1}$ is a solution to $A x = b_{1}$ and $c_{2}$ is a solution to $A x = b_{2}$ .

Suppose $z$ is a solution to $A x = b_{1}$ .

Then $z - c_{1}$ is a solution to $A x = 0$ .

Then $z - c_{1} + c_{2}$ is a solution to $A x = b_{2}$ .

Theorem

$S_{2} = {z - c_{1} + c_{2} : z \in S_{1}}$

$A, b$ determines whether or not $A x = b$ is consistent. $A$ solely determines the structure of the set of all solutions.

Matrices

Let $A \in M_{m \times n}$ . Write $A = [a_{1} \dots a_{n}]$ with each $a_{i} \in R^{m}$ .

Column Space of $A$ , written as $Col (A)$ , is the $Span {a_{1}, \dots, a_{n}}$ .

A = [1324] Col (A) = Span {[13], [24]}

Let $A \in M_{m \times n} (F)$ and $b \in F^{m}$ . The system of linear equations $A x = b$ is consistent if and only if $b \in Col (A)$ .

Proof:

$(⟹)$

Assume $A x = b$ is consistent.

$\exists y = y_{1} ⋮ v y_{n}$ such that $A y = b$ .

So $b = y_{1} a_{1} + \dots + y_{n} a_{n}$ . So $b \in Col (A)$ .

$(⟸)$

Assume $b \in Col (A)$ . Thus $\exists c_{1}, \dots, c_{n} \in F$ .

Such that $c_{1} a_{1} + \dots c_{n} a_{n} = b$ .

So $A c_{1} ⋮ c_{n} = b$ .

So $c$ is a solution to $A x = b$ . So $A x = b$ is consistent.

Let $A \in M_{m \times n} (F)$ . We define the transpose of $A$ , denoted $A^{⊺}$ , by $(A^{⊺})_{ij} = (A)_{ji}$ .

Note: Rows become columns, columns become rows.

Example

$A = [142536] \to A^{⊺} = 123456$

Let $A \in M_{m \times n} (F)$ . We define the row space of $A$ , denoted by $Row (A)$ , to be the span of the transposed rows of $A$ . That is,

Row (A) = Span {(row_{1} (A))^{⊺}, (row_{2} (A))^{⊺}, \dots, (row_{m} (A))^{⊺}}

Note: Turn rows into vectors and then take their linear combinations. $Row (A) = Col (A^{⊺})$ .

Example

$A = [142536] Row (A) = Span ⎩ ⎨ ⎧ 123, 456 ⎭ ⎬ ⎫$

Let $A, B \in M_{m \times n} (F)$ . If $B$ is row equivalent to $A$ , then

Row (B) = Row (A)

EROs do not change the row space, but EROs do change the column space.

Let $A, B \in M_{m \times n} (F)$ . Prove that $A x = a_{i}$ is consistent for $1 \leq i \leq n$ .

Proof:

A e_{i} = 0 a_{1} + 0 a_{2} + \dots + 1 a_{i} + \dots + 0 a_{n} = a_{i}

$a_{i}$ is in $Col (A)$ , thus $a_{i}$ is consistent.

Let $A = [a_{1} \dots a_{n}] \in M_{m \times n} (F)$ . Then $A e_{i} = a_{i}$ for all $i = 1, \dots, n$ .

In other words, the matrix-vector multiplication of $A$ and the $i$ th standard basis vector $e_{i}$ yields the $i$ th column of $A$ . Useful result used later.

A = [142536] A 010 = [25]

Let $A, B \in M_{m \times n} (F)$ . Then

A = B ⟺ A x = B x \forall x \in F^{n}

$(⟹)$ Assume $A = B$ , then $A x = B x$ $\forall x \in F^{m}$ .

$(⟸)$ Assume $A x = B x$ $\forall x \in F^{m}$ .

So $A e_{i} = B e_{i}$ for $1 \leq i \leq n$ .

By column extraction, $A e_{i} = a_{i}$ , $B e_{i} = b_{i}$ .

So $a_{i} = b_{i}$ . So the columns of $A$ and $B$ match. So $A = B$ .

Matrix Product

Let $A \in M_{m \times n} (F)$ and $B \in M_{n \times p} (F)$ . We define the matrix product $A B = C$ to be the matrix $C \in M_{m \times p} (F)$ , constructed as follows:

C = A B = A [b_{1} b_{2} \dots b_{p}] = [A b_{1} A b_{2} \dots A b_{p}]

That is, the $j$ th column of $C$ , $c_{j}$ is obtained by multiplying the matrix $A$ by the $j$ th column of the matrix $B$ :

c_{j} = A b_{j}, \forall j = 1, \dots, p

Note: Number of columns of first matrix must match the number of rows of the second matrix.

Note: Each column of $A B$ is an element of $Col (A)$ .

A B = [10 - 3 5] [0 - 1 24] = [1 (0) + (- 3) (- 1) 0 (0) + 5 (- 1) 1 (2) + (- 3) (4) 0 (2) + 5 (4)] = [3 - 5 - 10 20]

If $A B = C$ , then to find $C_{ij}$ , take the “dot product” of $i$ th row of $A$ and $j$ th column of $B$ .

C_{ij} = k = 1 \sum n a_{ik} b_{kj}

Matrix Sum

Let $A, B \in M_{m \times n} (F)$ . We define the matrix sum $A + B = C$ to be the matrix $C \in M_{m \times n} (F)$ whose $(i, j)$ th entry is $c_{ij} = a_{ij} + b_{ij}$ , for all $i = 1, \dots, m$ and $j = 1, \dots, n$ .

The matrices must be the same size.

[1 - 2 31] + [- 1 0 52] = [0483]

Properties:

$A + B = B + A$
$A + B + C = (A + B) + C = A + (B + C)$

The $m \times n$ zero matrix is the matrix $O_{m \times n} \in M_{m \times n} (F)$ all of whose entries are $0$ .

Let $A \in M_{m \times n} (F)$ . We define the additive inverse of $A$ to be the matrix $- A$ whose $(i, j)$ th entry is $- a_{ij}$ for all $i = 1, \dots, m$ and $j = 1, \dots, n$ .

If $A \in M_{m \times n} (F)$ , then

$O + A = A + O = A$
$A + (- A) = (- A) + A = O$

If $A, B \in M_{m \times n} (F)$ , $C, D \in M_{n \times p} (F)$ and $E \in M_{p \times q} (F)$ , then

$(A + B) C = A C + BC$
$A (C + D) = A C + A D$
$(A C) E = A (CE) = A CE$

Proof of $(A + B) C = A C + BC$

[(A + B) C]_{ij} = k = 1 \sum n [A + B]_{ik} c_{kj} = k = 1 \sum n (a_{ik} + b_{ik}) c_{kj} = k = 1 \sum n (a_{ik} c_{kj} + b_{ik} + c_{kj}) = k = 1 \sum n a_{ik} c_{kj} + k = 1 \sum n b_{ik} c_{kj} = [A C]_{ij} + [BC]_{ij} = [A C + BC]_{ij}

Let $A \in M_{m \times n} (F)$ and $c \in F$ . We define the product of $c$ and $A$ to be the matrix $c A \in M_{m \times n} (F)$ whose $(i, j)$ th entry is $(c A)_{ij} = c a_{ij}$ for all $i = 1, \dots, m$ and $j = 1, \dots, n$ .

2 [1324] = [2648]

If $A, B \in M_{m \times n} (F)$ and $C \in M_{n \times k} (F)$ , then

$s (A + B) = s A + s B$
$(r + s) A = r A + s A$
$r (s A) = (rs) A$
$s (A C) = (s A) C = A (s C)$ . Thus we may write $s A C$ for any of these three quantities unambiguously.

Also, $(- 1) A = - A$ .

More properties for transpose:

$(A + B)^{⊺} = A^{⊺} + B^{⊺}$
$(s A)^{⊺} = s A^{⊺}$
$(A C)^{⊺} = C^{⊺} A^{⊺}$
$(A^{⊺})^{⊺} = A$

Proof of $(A + B)^{⊺} = A^{⊺} + B^{⊺}$ .

[(A + B)^{⊺}] = [A + B]_{ji} = a_{ji} + b_{ji} = [A^{⊺}]_{ij} + [B^{⊺}]_{ij} = [A^{⊺} + B^{⊺}]_{ij}

A matrix $A \in M_{n \times n} (F)$ where the number of rows is equal to the number of columns is called a square matrix.

Let $A \in M_{n \times n} (F)$ . We say that $A$ is upper triangular if $a_{ij} = 0$ for $i > j$ with $i = 1, \dots, n$ and $j = 1, \dots, n$ .

Square matrices in REF or RREF are upper triangular.

Let $B \in M_{n \times n} (F)$ . We say that $B$ is lower triangular if $b_{ij} = 0$ for $i < j$ with $i = 1, \dots, n$ and $j = 1, \dots, n$ .

We say that an $n \times n$ matrix $A$ is diagonal if $a_{ij} = 0$ for $i \neq = j$ with $i = 1, \dots, n$ and $j = 1, \dots, n$ . We refer to the entries $a_{ii}$ as the diagonal entries of $A$ , and denote our matrix $A$ by $A = diag (a_{11}, a_{22}, \dots, a_{nn})$ .

Diagonal matrices are both upper and lower triangular. If $A$ is upper triangular, then $A^{⊺}$ is lower triangular. If $A$ is lower triangular, then $A^{⊺}$ is upper triangular.

If $A$ is diagonal then $A^{⊺} = A$ .

The diagonal matrix $diag (1, 1, \dots, 1)$ is called the identity matrix, and is denoted by $I$ . If we wish to indicate that the size of the identity matrix is $n \times n$ , we add a subscript $n$ and write $I_{n}$ .

The identity matrix is the multiplicative identity. $A \in M_{m \times n}$ $I_{m} A = A$ and $A I_{n} = A$ .

Also, $c I_{n} = diag (c, c, \dots, c)$ .

Question

Does there exist $A \in M_{2 \times 2} (R)$ where $A \neq = O_{2}$ , such that $A^{2} = O_{2}$ .

[11 - 1 - 1] [11 - 1 - 1] = [0000]

A square matrix $P$ is called idempotent if $P^{2} = P$ .

Table of Contents

Backlinks

MATH136 Course Notes

Vectors in $R^{n}$

Spans, Lines, and Planes

Systems of Linear Equations

Matrices

Table of Contents

Backlinks

MATH136 Course Notes

Vectors in Rn

Spans, Lines, and Planes

Systems of Linear Equations

Matrices

Vectors in $R^{n}$