Part 8 (10 points, non-coding task)
This question follows Part 7.
Denote r = \text{rank} \left( \mathbf{W} \right).
Compute the rank of \mathbf{W}^\top \mathbf{W}.
- Reasoning is required.
\color{green}{\text{### WRITE YOUR SOLUTION HERE ###}}
Let \mathbf{W} be with the following singular value decomposition:
\mathbf{W} = \sum_{i = 0}^{r-1} \mathbf{u}_i \sigma_i \mathbf{v}_i^\top .
Hence,
\begin{align*}
\mathbf{W}^\top \mathbf{W}
& = \left( \sum_{i = 0}^{r-1} \mathbf{u}_i \sigma_i \mathbf{v}_i^\top \right)^\top
\left( \sum_{j = 0}^{r-1} \mathbf{u}_j \sigma_j \mathbf{v}_j^\top \right) \\
& = \sum_{i = 0}^{r-1} \sum_{j = 0}^{r-1}
\mathbf{v}_i \sigma_i \mathbf{u}_i^\top \mathbf{u}_j \sigma_j \mathbf{v}_j ^\top \\
& = \sum_{i = 0}^{r-1} \sum_{j = 0}^{r-1}
\mathbf{v}_i \sigma_i \sigma_j \delta_{ij} \mathbf{v}_j^\top \\
& = \sum_{i = 0}^{r-1}
\mathbf{v}_i \sigma_i^2 \mathbf{v}_j^\top .
\end{align*}
This is the singular value decomposition of \mathbf{W} \mathbf{W}^\top. Therefore, the rank of this matrix is also r.
\color{red}{\text{""" END OF THIS PART """}}