I am trying to implement a simple logistic regression algorithm from scratch in python (for learning purposes). Every article I've seen online so far presents the following expression for $z$ (argument of the sigmoid function):
$$z = \theta^T\cdot x$$
However when they implement it in code they always use
np.dot(X, theta) which is not the same thing. I have carefully tried to trace the dimensions of all the arrays as follows:
Dot product property: $\theta^T\cdot x = x^T\cdot\theta$
$x$ is of dimension $l\times (d+1)$ where $l$ is the number of records and $d+1$ the number of features (including the $x_0$ ones vector).
$\theta$ is of dimensions $1 \times (d+1)$ (every column in $x$ gets a weight, including the ones vector).
- $z$ is the dot product of $\theta^T$ and $x$, it will have dimensions $((d+1)\times 1)\cdot(l \times 1) = ??$ --> DOES NOT WORK!
There appears to be no way to arrange the last statement such that it remains mathematically correct and yielding an array of dimension $l\times1$ or $1\times l$ without fundamentally changing either:
- the shape of $x$
- the equation itself (as they do in the code)
So which is correct? The code or the mathematical statement? Why are they contradictory? Please help. Thanks.