I need to create a scoring model out of estimated conditional probability functions for two events, A and B. Let 0.5 be the threshold value. Ideally, the probability is in the interval $[0,0.5)$ for A and in the interval $[0.5,1]$ for B. However, my probabilities are estimated with errors. Therefore, for a certain number of events A the probabilities fall into the interval $[0.5,1]$, and for B into $[0,0.5)$. Let the minimum score min_score $=0$ ($P = 0$) and maximum score `max_score' $= 10$ ($P = 1$).

I need to map those probabilities to scores with a function that would emphasize and enlarge difference in probabilities of A and B in terms of scores (for example, a sigmoidal function). That is the higher the probability (likely event B) the higher the score, the lower the probability (likely event A), the smaller the score. Though for the intersection interval, which I can identify as $[a,b]$ the scores should be around (max_score + min_score)/2. In the interval $[t_1,t_2]$, which is symmetric about $1/2$, scores should be smoothing, while in $(0, t_1)$ and $(t_2, 1)$, the scores should be "more extreme" with a higher confidence.

The problem is $[a,b]$ can be spanned over $[0,1]$, and that effect should be smoothed. Can I apply some kind of regularization here? What function would I use and how would I establish the coefficients?

  • $\begingroup$ Can you use the log of the odds ratio? $\endgroup$ – Mike Dunlavey Aug 17 '12 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.