ENHANCING AUTO-ENCODER TRAINING SPEED USING RANDOMNESS

Abstract

This thesis investigates a strategy to accelerate the training of autoencoders withoutcompromising the latent space quality. While previous research were primarily focused on architectural innovations or large-scale hardware optimization, less attention has been given to training-time efficiency through dynamic parameter management. This work explores how sparsity, parameter freezing, and randomness can be leveraged to reduce computational cost while maintaining or improving latent space quality. We introduce and evaluate an autoencoder variant which is a dynamically sparse autoencoder implementing a prune-and-grow mechanism. This model is compared across multiple training conditions by measuring convergence epoch numbers, time, reconstruction loss, and the separability of latent representations using downstream classifiers. Additional experiments explore the effect of the different hyperparameters used in the auto-encoder and show the trade-off between parameter efficiency and performance stability. Our results show that selective parameter randomization can reduce training time by a significant margin while on small dataset preserving or even improving reconstruc- tion accuracy. In particular, the model demonstrates strong adaptability and maintains robust latent representations despite aggressive pruning schedules. Moreover, classifier performance on latent codes indicates that meaningful structure can be retained even under limited parameter budgets. This research contributes to the understanding of efficient representation learning by demonstrating that intelligent parameter management can achieve faster training with- out requiring architectural overhauls or specialized hardware. It provides a foundation for future work on scalable, energy-efficient autoencoder training and general-purpose neural network compression.