 # Generalized inverse

Suppose $LaTeX: A$ is any $LaTeX: m \times n$ matrix. $LaTeX: A^+$ is a generalized inverse of $LaTeX: A$ if $LaTeX: A^+$ is $LaTeX: n \times m$ and $LaTeX: AA^+A = A$. Then, a fundamental theorem for linear equations is:

The equation, $LaTeX: Ax = b$, has a solution if and only if $LaTeX: AA^+b = b$ for any generalized inverse, $LaTeX: A^+$, in which case the solutions are of the form: $LaTeX: x = A^+b + (I - A^+A)y$   for any $LaTeX: y \in \mathbb{R}^n.$

Here is an Example.

The Moore-Penrose class additionally requires that $LaTeX: AA^+$ and $LaTeX: A^+A$ be symmetric (hermitian, if $LaTeX: A$ is complex). In particular, if $LaTeX: \mbox{rank}(A) = m$, $LaTeX: A^+ = A^T (AA^T)^{-1}$ is a Moore-Penrose inverse. Note that $LaTeX: AA^+ = I$, and $LaTeX: A^+A$ is a projection matrix for the subspace $LaTeX: \{x: x = A^T y \mbox{ for some } y \in \mathbb{R}^m\}$, since $LaTeX: x = A^T y$ implies $LaTeX: A^+Ax = A^T (AA^T)^{-1} AA^T y = A^T y = x$. Further, $LaTeX: I-A^+A$ projects into its orthogonal complement, which is the null space of $LaTeX: A$, i.e. $LaTeX: A(I-A^+A)x = (A-A)x = 0$ for any $LaTeX: x \in \mathbb{R}^n$.