# Why are weight changes in Oja's rule and BCM so different?

I simulated Oja's rule and BCM for a single postsynaptic neuron with two presynaptic neurons, and for 10000 inputs, where I randomly select one of $$(0,1)$$ or $$(1/2,\sqrt{3}/2)$$ as input. My learning rate was $$1/100$$ and I am using the sliding threshold for BCM. (I am following this tutorial: http://www.inf.ed.ac.uk/teaching/courses/nc/NClab8.pdf) However, both algorithms seem to converge to very different things, as pictured:

The axis are the weights from each presynaptic neuron, and the red stars are the two inputs I am feeding to the network. Blue means first steps of simulation and yellow last steps of simulation.

I know Oja's rule should extract the first principal component of the input data (which I think looks more like BCM did, so this is confusing me). But what truly puzzles me is that the weights converge to very different things! Is this supposed to happen? Why? How can I interpret the "final" weights in each case?

BCM is to be competitive between the input signals, thus finds that the vertical axis is a better separator for the two points than the horizontal axis. A very imprecise handling of the equilibrium conditions in their expectation form leads me to guess that the second weight oscillates around $$4/\sqrt3=2.31$$.
• Thank you. I have been thinking about the first part of your answer, but I don't quite get it. What does it mean better separator? I am also not sure what you mean by "equilibrium conditions in their expectation form". Also, how do you get $4/\sqrt3$? The second paragraph is clear now (after a few more experiments), thank you! Dec 7, 2021 at 19:16
• BCM is designed to highlight one input. The threshold makes it possible for the data to have two clusters, and such a clustering is the more stable situation. The rest was guessing, the average value of the vertical coordinate is $\sqrt3/4$. and the combination of $v-\theta$ and $v^2-\theta$ as factors on the right gives a tendency for $\theta$ to move towards $1$, which then makes the weight the reciprocal of the average. Dec 7, 2021 at 19:51