# Note | Linear Algebra

date

Oct 21, 2021

slug

linear-algebra

status

Published

summary

Linear algebra is also a system has input and output (function, transformation, operator). But it is amazing.

tags

Academic

Math

Study

type

Post

## Linear System

## Matrix-Vector Product

**Two perspective:**

**Inner product with row**

**Weighted sum of columns**

- The matrix A represents the system:

- the size of matrix vector should be matched

### Properties of Matrix-vector Product

and are matrices and are vectors in , and is a scalar

- is the zero vector

- is also the zero vector

**A=B?**

and are matrices. If for all in . Is it true that ？

## Has Solution or Not?

### Only unique and infinite?

#### Linear Combination

**Definition**: Given a vector set- are scalars (coefficients of linear combination)
- or "weighted sum" 😊

the linear combination of the vectors in the set

**Parallel**no solution

**Nonparallel**and are nonzero vectors, and- If and are in :
- Nonparallel has solution
- Nonparallel has solution
- If and are in :
- Nonparallel has solution

#### Span

**Definition**:- Given a vector set
is the vector set of all linear combinations of**Span S**- Span
- Vector set
- " is a generating set for " or " generates "
- One way to describe a vector set with infinite elements

- has solution or not？

Is the linear combination of the columns of ？

is in the span of the columns of ？

- Redundant vector (in my way, not the teacher's 😊)
- Given a vector set and

## How many solutions?

### Dependent and independent

**Dependent**: A set of n vectors , is linear dependent

If there exist scalars ,

**not all zero**, such that**Independent**: A set of n vectors , is linear independent

**Only if**

- Zero vector is the linear combination of any other vectors

- Any set contains zero vector would be linear dependent

- If dependent, once we
**have**solution, we**have**infinite

**Homogeneous linear equations**: always has as solution

### Rank and Nullity

- The
**rank**of a matrix is defined as the maximum number of linearly independent columns in the matrix

**Nullity**= Numbers of columns - rank- If is a matrix, columns of are independent

### ❓ Conclusion

## Solving System of Linear Equations

### Equivalent

- Equivalent: two systems of linear equations are equivalent if they have exactly the same solution set

- Elementary row operations: applying those operations will produce an equivalent one
- interchange
- scaling
- row addition

### Augmented Matrix

- Augmented matrix:

- Reduced Row Echelon Form (): a system of linear equations is easily solvable if its augmented matrix is in RREF
- Row Echelon Form:
- Each nonzero row lied above every zero row
- The leading entries are in echelon form
- Reduced: the columns containing the leading entries are standard vectors

- RREF is unique: a matrix can be transformed into multiple REF by row operation, but only on R

**Pivot positions****pivot columns**of are 1st, 3rd and 4th columns

- Free variables and basic variables / Parametric Representation

- Gaussian elimination: an algorithm for finding the reduced row echelon form of a matrix.

### What can we know from RREF?

#### RREF & Linear Combination

**Column Correspondence Theorem**:- For
- if is a linear combination of other columns of is a linear combination of the corresponding columns of with the
**same coefficients** - Or: The RREF of matrix is , and have the same solution set
- There are
**no**row correspondence theorem

- Span of columns will change but span of rows are equivalent\

- Conclusion
- Columns
- the relations between the columns are the same
- the span of the columns are different
- Rows:
- the relations between the rows are different
- the span of the rows are the same

#### RREF & Independent

- Pivot column independent

- All columns are independent every column is a pivot column Every column in RREF(a) is standard vector, therefore:
- columns are dependent

#### RREF & Rank

- Rank? Maximum number of independent columns = number of pivot column = number non-zero rows

- Matrix is
**full rank**if - If , the columns of is dependent
- In , you cannot find more than m vectors that are independent

- Basic, Free Variables & Ranks
- The number of basic variables =
- The number of basic variables = =

#### RREF & Span

- is inconsistent (no solution)

- is consistent for every
- independent vectors can span More than vectors in must be dependent

RREF of cannot have a row whose only non-zero entry is at the last column

RREF of cannot have zero row

the number of rows

- Full Rank
- the columns of are linearly independent
- has at most one solution
- All columns are pivot columns
- always have solution (at least one solution) for every row in
- Independent (square matrix): one solution
- Dependent: infinite solutions
- The columns of generate

RREF of

### Matrix Multiplication

#### Inner product

**Matrix multiplication**: given two matrices and , the-entry of is the inner product of row i of and column j of

- Linear combination of columns and rows:
- Column
**Composition**: given two functions f and g, the function is the composition- Matrix multiplication is the composition of two linear functions
- Row

#### Summation of Matrices

- Block multiplication

#### Properties

- and are square and symmetric: ;

#### Practical Issue

Order affects computation complexity!

### Matrix Inverse

#### Inverse of Matrix

- is called
**invertible**if there is a matrix - Non-singular: invertible
- Singular: not invertible
- Non-square matrix cannot be invertible
- Not all the square matrix is invertible
- Unique
- A and B are invertible nxn matrices,
- If A is invertible, is is invertible

- Solving Linear Equations:

However, this method is computationally inefficient

#### Invertible

- Let ben an matrix, is invertible
*iff.*:

**Onto ⇒ One-to-one ⇒ invertible**- The columns of span
- For every in , the system is consistent
- The rank of is n (the number of rows) = n

**One-to-one ⇒ Onto ⇒ invertible**- The columns of are linear independent
- The rank of is n (the number of columns) = n
- The nullity of is zero
- The only solution to is the zero vector
`Simplest`

: The reduced row echelon form of is

**Others:**- There exists an matrix
*s.t.* - There exists an matrix
*s.t.* - is a product of elementary matrices

#### Inverse of a Matrix

- Every Elementary Row Operation can be performed by matrix multiplication

- Elementary matrix

- How to find an elementary matrix: apply the desired elementary row operation on identity matrix

- Inverse of elementary matrix

- RREF
*v.s.*Elementary Matrix - Let be an matrix with reduced row echelon form
- There exists an invertible matrix
*s.t.*

#### Inverse of a general matrix

- matrix:

(if , is not invertible)

- Algorithm for Matrix Inversion

### Subspace v.s. Span

- The span of a vector set is a subspace
- Let
- Property 1.
- Property 2.
- Property 3.

- Null Space
- Definition: the null space of a matrix is the solution set of . It is denoted as Null A
- Null A is a subspace

- Column Space and Row Space
- Column space of a matrix is the span of its columns . It is denoted as Col A.
- Row space of a matrix is the span of its rows. It is denoted as Row A.
- Columns Space = Range: The range of a linear transformation is the same as the column space of its matrix.

- RREF
- Original Matrix A v.s. its RREF R
- Columns:
- The relations between the columns are the same.
- The span of the columns are different.
- Rows:
- The relations between the rows are changed
- The span of the rows are the same.

- Consistent
- has solution (
*consistent*) - is the linear combinations of columns of
- is in the span of the columns of
- is in Col A

- Conclusion: Subspace is Closed under addition and multiplication

## Basis

- Definition: let be a non zero subspace of . A basis for is a linearly independent generation set of
- any two independent vectors form a basis for

- The pivot columns of a matrix form a basis for its columns space

- Property

- Theorem
- A basis is the smallest generation set
- Reduction Theorem: There is a basis containing in any generation set / can be reduced to a basis for by moving some vectors
- A vector set generates must contain at least vectors
- A basis is the largest independent vector set in the subspace
- Given an independent vector set in the subspace, can be extended to a basis by adding more vectors
- Any independent vector set in contain at most vectors
- Any two bases for a subspace contain the same numbers of vectors.
- The number of vectors in a basis for a nonzero subspace is called dimension of ()
- The dimension of zero subspace is 0
- Every has dimensions

- Find a basis for : given a subspace , assume that we already know that . Suppose is a subset of with vectors.
- If is independent is basis
- If is a generation set is basis

## Column Space, Null Space, Row Space

**Rank A**- Basis: the pivot columns of form a basis for
- Dimension:

**Null A**- Solving
- s
- s
- Dimension

**Row A**- Basis: nonzero rows of RREF(A)
- Dimension

**Dimension Theorem:**

- Summary

## Coordinate System

### Outline

- Coordinate System
- Each coordinate system is a"viewpoint "for vector representation
- The same vector is represented differently in different coordinate systems
- Different vectors can have the same representation in different coordinate systems
- Changing Coordinates
- A vector set can be considered as a coordinate system for if :
- The vector set spans the the - every vector should have representation
- The vector set is independent - unique representation
- is Cartesian coordinate system
- Other System → Cartesian:
- Cartesian → Other System:

- Changing Coordinates
- Equation of ellipse
- Equation of hyperbola