Assessing the Performance of Feedforward Neural Network Models with Random Data Split for Time Series Data: A Simulation Study

Abstract

The majority of findings and conclusions related to the application of artificial neural networks (ANNs) for time series data have been derived through non-random data-splitting procedures. In this methodology, the initial set of observations in the dataset is employed for model training, followed by a subsequent set of observations for validation and the final set of observations for testing. However, this study presents a comprehensive simulation study on assessing the performance of Feedforward neural networks (FFNN) with the random data split procedure. For this purpose, eight nonlinear models from the literature were employed to generate multiple series where they represent different features available in the time series data. The complexity of the selected models was further improved by introducing Poisson processes with different jump sizes. From each selected time series model, 30 replications were generated using distinct initial random seeds for the error term. The data were randomly partitioned, allocating 80% for training, 10% for validation, and 10% for testing purposes. The FFNN models were fitted incorporating the inputs from past observations and moving average values. Moreover, each individual fitted model was trained 30 times to obtain a statistically robust evaluation of model performance through the averaging of predictive values. The findings reveal that the FFNN models performed well with the random data split procedures for time series data with lower minimum error values. The same idea was observed by graphs fitted between the actual test values and the average of the forecasted values. The results obtained from this simulation study are important, as they provide valuable insights into the broader utilization of FFNNs with regard to the effectiveness of random data split procedures in time series forecasting.