2025 USA-NA-AIO Round 1, Problem 3, Part 12

AI Olympiads 2025 USA-NA-AIO Round 1
1

Part 12 (5 points, non-coding task)

In this part, you are asked to compute \nabla_{\mathbf{\beta}} \ L \left( \mathbf{\beta} \right) and express your solutions in two forms. Reasoning is not required.

  1. Write \nabla_{\mathbf{\beta}} \ L \left( \mathbf{\beta} \right) in the following summation form:

    \nabla_{\mathbf{\beta}} \ L \left( \mathbf{\beta} \right) = \sum_{n=0}^{N-1} \cdots .

  2. Denote

    \mathbf{X} = \begin{bmatrix} \mathbf{x}^{(0), \top} \\ \mathbf{x}^{(1), \top} \\ \vdots \\ \mathbf{x}^{(N-1), \top} \end{bmatrix}

    and

    \mathbf{y} = \begin{bmatrix} y^{(0)} \\ y^{(1)} \\ \vdots \\ y^{(N-1)} \end{bmatrix}

    and

    \mathbf{z} = \begin{bmatrix} \sigma \left( \mathbf{x}^{(0), \top} \mathbf{\beta} \right) \\ \sigma \left( \mathbf{x}^{(1), \top} \mathbf{\beta} \right) \\ \vdots \\ \sigma \left( \mathbf{x}^{(N-1), \top} \mathbf{\beta} \right) \end{bmatrix}

    Write \nabla_{\mathbf{\beta}} \ L \left( \mathbf{\beta} \right) in terms of \mathbf{X}, \mathbf{y}, \mathbf{z} with matrix operations (the summation symbol is not allowed).

2

\color{green}{\text{### WRITE YOUR SOLUTION HERE ###}}

We have

\begin{align*} \nabla_{\mathbf{\beta}} \ L \left( \mathbf{\beta} \right) & = \boxed{ \sum_{n = 0}^{N-1} \left( \sigma \left( \mathbf{x}^{(n), \top} \mathbf{\beta} \right) - y^{(n)} \right) \mathbf{x}^{(n)} } \\ & = \boxed{\mathbf{X}^\top \left( \mathbf{z} - \mathbf{y} \right)} . \end{align*}

\color{red}{\text{""" END OF THIS PART """}}