2025 USA-NA-AIO Round 1, Problem 3, Part 10

AI Olympiads 2025 USA-NA-AIO Round 1
1

Part 10 (5 points, non-coding task)

So far, we have finished preprocessing our dataset. We use this dataset for the purpose of doing binary classification.

In all remaining parts in this problem, we will train a logistic regression model with our preprocessed data and test its performance.

Let all training samples be indexed as 0, 1, \cdots , N-1. For the $n$th sample, denote by \mathbf{x}^{(n)} \in \Bbb R^{d \times 1} a column vector of all features and y^{(n)} \in \left\{ 0, 1 \right\} its ground-truth label.

In the logistic regression, denote by \mathbf{\beta} \in \Bbb R^{d \times 1} the learnable parameters.

Thus, our predicted label is determined according to:

y^{(n)}_{predict} = \begin{cases} 1 , & \text{with probability } \sigma \left( \mathbf{x}^{(n), \top} \mathbf{\beta} \right) \\ 0, & \text{with probability } 1 - \sigma \left( \mathbf{x}^{(n), \top} \mathbf{\beta} \right) \\ \end{cases}

where

\sigma \left( z \right) = \frac{1}{1 + e^{-z}}

is the sigmoid function.

Do the following task in this part.

To train our model, we need to solve the following optimization problem:

\min_{\mathbf{\beta}} \ L \left(\mathbf{\beta} \right) .

Write down the loss function L \left(\mathbf{\beta} \right) in the following form (reasoning is not required):

L \left(\mathbf{\beta} \right) = \sum_{n=0}^{N-1} \cdots .
2

\color{green}{\text{### WRITE YOUR SOLUTION HERE ###}}

\boxed{L \left( \mathbf{\beta} \right) = \sum_{n = 0}^{N-1} - \left( y^{(n)} \log \left( \sigma \left( \mathbf{x}^{(n), \top} \mathbf{\beta} \right) \right) + \left( 1 - y^{(n)} \right) \log \left( 1 - \sigma \left( \mathbf{x}^{(n), \top} \mathbf{\beta} \right) \right) \right) } .

\color{red}{\text{""" END OF THIS PART """}}