The simple average is commonly used to combine the predictions out of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the predictive models to get more accurate predictions?

  • $\begingroup$ There are surely an endless number of ways to combine functions, i.e., this question is very broad. Could you specify a purpose or a characterizing property that you want the solution to have? $\endgroup$ – Deathbreath Jun 30 '16 at 15:33

Bagging, Boosting, and Bayesian Model Averaging/Combination are all widely used techniques for doing this. These are discussed in many textbooks on machine learning.


Provided you have raw data you could use in this process, one could use the various different models and treat them as basis functions of sorts that you wish to merge together in a least square sense.

You could then merge the various models using a least square fit based on whatever data you have at your disposal. This is certainly different than simple averaging.

  • $\begingroup$ Thanks choward...the fitting of the least squares is depending highly on the available data...For example using data with different observations size such as 500 or 1000 is not better than 750, which is the best... so how can I make sure which size is needed for fitting the least squares optimally? $\endgroup$ – mhdella Jun 21 '16 at 5:10
  • $\begingroup$ @mhdella It should just come down to using as much data as you can. More data typically equates to a better predicting model. One thing to note is that the spread of the data does matter as well. Obviously having data largely localized to some region of the domain may not result in a global best fit since you don't have much information in other areas to help communicate what the model should be there. $\endgroup$ – spektr Jun 21 '16 at 5:18
  • $\begingroup$ Thanks again choward for your swift reply...don't you think using a non-linear fitting (regression) could be better than the least squares fitting for the combination of different models? $\endgroup$ – mhdella Jun 21 '16 at 5:23
  • 1
    $\begingroup$ @mhdella I don't see how a nonlinear regression could be useful for this. But maybe if you wrote what your various models are I could understand why you think that might work. $\endgroup$ – spektr Jun 21 '16 at 12:35
  • $\begingroup$ @choward..these various models are support vector regression for prediction the solar irradiance $\endgroup$ – mhdella Jun 22 '16 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.