Confusion related to convexity of 0-1 loss function - Computational Science Stack Exchange most recent 30 from scicomp.stackexchange.com 2019-06-16T06:40:18Z https://scicomp.stackexchange.com/feeds/question/5628 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://scicomp.stackexchange.com/q/5628 1 Confusion related to convexity of 0-1 loss function user34790 https://scicomp.stackexchange.com/users/3871 2013-03-22T05:15:00Z 2013-03-22T05:45:03Z <p>I am a bit confused why the 0-1 loss function is not convex. What's wrong with it?</p> https://scicomp.stackexchange.com/questions/5628/confusion-related-to-convexity-of-0-1-loss-function/5629#5629 3 Answer by Geoff Oxberry for Confusion related to convexity of 0-1 loss function Geoff Oxberry https://scicomp.stackexchange.com/users/276 2013-03-22T05:45:03Z 2013-03-22T05:45:03Z <p>I'm not sure this is what you're looking for, but here goes:</p> <p>A zero-one loss function $L: \mathbb{R} \rightarrow \{0,1\}$ is defined as:</p> <p>\begin{align} L(x) = \left\{\begin{array}{ll} 0, &amp; \textrm{if $x \geq 0$}, \\ 1, &amp; \textrm{if $x &lt; 0.$}\end{array}\right. \end{align}</p> <p>$L$ is convex if, for all $x_1, x_2 \in \mathbb{R}$, and all $\lambda \in [0,1]$,</p> <p>\begin{align} L(\lambda x_1 + (1 - \lambda)x_2) \leq \lambda L(x_1) + (1 - \lambda) L(x_2). \end{align} </p> <p>A counterexample is: $(x_1, x_2, \lambda) = (-1, 1/2, 1/2)$.</p> <p>Then:</p> <p>\begin{align} L(\lambda x_1 + (1 - \lambda)x_2) &amp;= L(-1/4) = 1, \\ L(x_1) &amp;= 1, \\ L(x_2) &amp;= 0, \end{align}</p> <p>and $L(\lambda x_1 + (1 - \lambda)x_2) \leq \lambda L(x_1) + (1 - \lambda) L(x_2)$ does not hold, because $1 \leq 1/2$ is not true, so $L$ is not convex.</p>