6
$\begingroup$

I often see neural networks extended to complex-values. Those networks allow complex input, complex parameters, and complex output. My understanding is that the inner products and point nonlinearities are simply extended. I can see it's advantage for signals that are naturally complex (e.g., phase/amplitude decomposition, oscillations, frequency domain processing, ...). However, computationally speaking, is it any different from converting complex signals to real values?

  1. What is the computational advantage of using the complex field compared to the real field?
  2. What is the computational advantage of complex vs real vector space?

References:

  1. Akira Hirose. Complex-Valued Neural Networks. Springer Science & Business Media. 2006, 2012
  2. Danilo P. Mandic, Vanessa Su Lee Goh.Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models. 2009
$\endgroup$
  • $\begingroup$ What is your programming language ? $\endgroup$ – Coriolis Aug 27 '16 at 16:12
  • $\begingroup$ @Coriolis I don't have a specific language in mind. Maybe Fortran, MATLAB, Julia, numpy? $\endgroup$ – Memming Aug 27 '16 at 19:07
  • 1
    $\begingroup$ This might be interesting tygert.com/ccnet.pdf $\endgroup$ – dranxo Aug 31 '16 at 19:46
  • $\begingroup$ @dranxo that's very interesting! Thanks for the pointer. $\endgroup$ – Memming Aug 31 '16 at 19:53
1
$\begingroup$

I don't really see how a complex valued Neural Network would provide anything particularly useful over a real valued Neural Network.

The whole idea of having a Neural Network that operates on complex numbers, uses complex weights, and outputs complex numbers doesn't seem any different than having a real valued Neural Network that has two times as many inputs, outputs, and weights as the complex Neural Network.

For example, if you want a network that takes some input complex number and outputs another complex number, you could represent this with a real valued Neural Network with two inputs and two outputs.

$\endgroup$
0
$\begingroup$

From a strict programming point of view, a complex number is composed of a real part and an imaginary part, that is to say, two real values. So I guess that real arithmetic applies here, except for certain operations like sqrt for a negative number for instance.

A complex number needs twice the storage of a real, because as said before, it is actually two reals. So I assume that a complex-valued neural network will require twice the storage of a real-valued neural network.

Again, since two reals are involved, it is possible that a complex-valued neural network will require twice the arithmetic operations compared to a real-valued neural network, but it may depend on the language, the compiler, some specialists in StackOverflow may give you more details.

$\endgroup$
0
$\begingroup$

I also found that book written by Akira Hirose in the library. I have the same question as well, and I found this paper: https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf

Anyway, we are not even in the "real" world. The input spaces of any neural networks implemented on modern computers are not just countable, but finite. Also the cardinality of real numbers is the same as complex numbers. So I think for the same structure NN, the real-valued NN can solve all the questions that can be solved by complex-valued NN.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.