I have built an ANN model with 5 hidden layers and 100 nodes in each layer to solve a multilabel classification problem. After the first run, I get a training accuracy of ~66% and a test set accuracy of ~55%. When I ran the code with 1000 nodes in each hidden layer,the training accuracy improves to ~98% but the test set accuracy is around ~52%. I tried dropout but it only lowered the training accuracy a bit while the wide difference between the training and test accuracy still exists. Can anyone please help?

  • 1
    $\begingroup$ Welcome to SciComp. Your question as it is now is very broad and hard to answer. Can you provide more details? $\endgroup$
    – GertVdE
    May 17, 2019 at 11:56

1 Answer 1


I think you experience the implications of overtraining. From this question and this paper:

Since a neural network with a sufficient number of neurons in the hidden layer can exactly implement an arbitrary training set, it can learn both investigated dependencies and a noise that will lower the predictive ability of the network.

So, in your case, you increased the number of nodes in the hidden layer - and now, your training set resulted in overtraining. Your neural-network now is much more directed on replicating the data in the training dataset, as opposed to "prediction of the reality". This results in the stagnation (and even reduction in quality) for the test dataset.

It is hard to give more advice without details. But I would suggest at least to:

  • play with different partitioning of the sample into training and test datasets
  • play with the model itself in terms of features
  • play with a training algorithm

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.