0
$\begingroup$

In the "Introduction" section of the paper Support-Vector Networks, it mentioned Fisher's solution to a model of two normal distributed populations:

Fisher's solution to a model of two normal distributed populations

My questions are:

  1. How to derive equation (1)? I even doubt that it should be: $F_{sq} (x) = sign \left [ \left ( x - m_1 \right ) ^T \Sigma_1 ^{-1} \left ( x - m_1 \right ) - \left ( x - m_2 \right ) ^T \Sigma_2 ^{-1} \left ( x - m_2 \right ) + \ln { \dfrac { \left | \Sigma_1 \right |} { \left | \Sigma_2 \right | } } \right ]$
    because according to Linear discriminant analysis, the solution is: Linear discriminant analysis

  2. Why the number of free parameters in equation (1) is $\dfrac {n (n + 3)} {2}$?
    In my opinion, $m_1, m_2, \Sigma_1, \Sigma_2$ are all free parameters, because $\Sigma_1, \Sigma_2$ are symmetric matrices, so the number should be $n + n + \dfrac {n \left (n + 1 \right )}{2} + \dfrac {n \left (n + 1 \right )}{2} = n \left (n + 3 \right )$

  3. Why the number of free parameters in equation (2) is $n$?
    We can rewrite Equation (2) as $F_{sq} \left ( X \right ) = WX + b$, so $W$ and $b$ are both free parameters, then the number should be $n + 1$.

$\endgroup$
  • $\begingroup$ I think that you should split your post into different questions, so it is easier to help you. $\endgroup$ – nicoguaro Nov 26 '18 at 16:05
  • $\begingroup$ Will better answers be given on the stats stackexchange? $\endgroup$ – user14717 Dec 25 '18 at 18:02
0
$\begingroup$
  1. Everything is clearly explained in Why the logistic function? A tutorial discussion on probabilities and neural networks" Michael Jordan 1995. You see how your Equation (1) is derived. It is a must-read introductory article.
  2. To get a degenerate case $w.x+b$ from the difference of the two quadratics, we must have $\Sigma_1=\Sigma_2=\Sigma$, hence there are $m_1, m_2, \Sigma$ free parameters $n+n+n(n-1)/2=n(n+3)/2$
$\endgroup$
  • $\begingroup$ Thanks a lot, but I'm still confused: 1. The paper " Why the logistic function?" derived Equation (2) for the condition $\Sigma_1 = \Sigma_2$ , rather than quadratic equation (1). 2. paper "Support-Vector Networks" actually mentioned that the number of free parameters for Equation (2) is $n$, as is shown in the green blocks in my post. $\endgroup$ – Jiongjiong Li Nov 25 '18 at 13:42

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.