2
$\begingroup$

Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data set (we can change our data set to big or small according to the model). And now what I do is keep these classifiers for various model as the node for the neural network and update their weights on the premise of who gives the correct answer. Now I am just a noob when it comes to machine learning, but can someone please settle my curiosity.

$\endgroup$
2
$\begingroup$

You can. This is called an ensemble model. For example, a linear regression between the solutions of different predictive models is a way to take a weighted average of different models. Normally, the winners of all of the machine learning contests use ensemble models since they eek out a bit more accuracy. However, they are a lot more costly to train.

$\endgroup$
  • $\begingroup$ Okay, got it. So in a way it is similar to random forest which ensembles decision tree in it. So this kind of model is linear regression the way to go or is there any other model or algo I can use like nn. $\endgroup$ – Devesh Yadav Jun 19 '17 at 18:11
  • $\begingroup$ You can use any. Linear regression (without an intercept) is just a good example because it's clear what it's doing: giving you back a weighted vector of the results from the different predictors. But you can use anything: NN, factorization machine, etc. as the ensambling technique. You can even use an ensamble of ensambling techniques, and I played around with that but it didn't seem useful/necessary. Single ensemble is likely better in most cases. Just pick one. $\endgroup$ – Chris Rackauckas Jun 19 '17 at 19:26
  • $\begingroup$ Thanks, that helped a lot. I will go through the paper you linked. $\endgroup$ – Devesh Yadav Jun 21 '17 at 16:46
  • $\begingroup$ The paper isn't too helpful since its essentially a null result. Just look for ensemble methods and you'll find a lot $\endgroup$ – Chris Rackauckas Jun 21 '17 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.