MNIST MLP Hyperparameter Study

• Pixel normalisation and flattening for a dense network.
• A baseline MLP, then controlled changes to width, epochs, activations, initialisation and optimiser.
• Early stopping, compared against fixed epoch counts.
• A final regularised model with its test result.

I went step by step through what changes an MLP's results on MNIST: hidden-layer width, output-layer size, number of epochs, early stopping, activation functions, weight initialisation and optimisers. Zero initialisation does not train (accuracy 0.114). Random-normal initialisation, RMSprop and SGD with momentum all reach about 0.97 validation accuracy.

The final model has three dense layers (512, 256, 128), L2 regularisation, batch normalisation, dropout, SGD with momentum and early stopping. It reaches 0.9815 test accuracy.