Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data set (we can change our data set to big or small according to the model). And now what I do is keep these classifiers for various model as the node for the neural network and update their weights on the premise of who gives the correct answer. Now I am just a noob when it comes to machine learning, but can someone please settle my curiosity.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
You can. This is called an ensemble model. For example, a linear regression between the solutions of different predictive models is a way to take a weighted average of different models. Normally, the winners of all of the machine learning contests use ensemble models since they eek out a bit more accuracy. However, they are a lot more costly to train.
answered Jun 18, 2017 at 0:23
-
$\begingroup$ Okay, got it. So in a way it is similar to random forest which ensembles decision tree in it. So this kind of model is linear regression the way to go or is there any other model or algo I can use like nn. $\endgroup$ Commented Jun 19, 2017 at 18:11
-
$\begingroup$ You can use any. Linear regression (without an intercept) is just a good example because it's clear what it's doing: giving you back a weighted vector of the results from the different predictors. But you can use anything: NN, factorization machine, etc. as the ensambling technique. You can even use an ensamble of ensambling techniques, and I played around with that but it didn't seem useful/necessary. Single ensemble is likely better in most cases. Just pick one. $\endgroup$ Commented Jun 19, 2017 at 19:26
-
$\begingroup$ Thanks, that helped a lot. I will go through the paper you linked. $\endgroup$ Commented Jun 21, 2017 at 16:46
-
$\begingroup$ The paper isn't too helpful since its essentially a null result. Just look for ensemble methods and you'll find a lot $\endgroup$ Commented Jun 21, 2017 at 16:49