Wind speed forecasting based on Quantile Regression Minimal Gated Memory Network and Kernel Density Estimation
Graphical abstract
Introduction
With the reduction of fossil energy and the increase in environmental problems caused by using it, wind energy has received more and more attention from all over the world as a clean renewable energy [1]. However, due to the fluctuation and randomness of wind speed, the power grid integrated into the wind power becomes unreliable [2]. Therefore, it is very important to obtain reliable and accurate wind speed predictions and quantify the uncertainty of predictions for the utilization and planning of wind power.
Wind speed prediction methods mainly include physical methods and statistical methods [3]. Physical methods usually predict wind speed from physical mechanisms through meteorological simulation, such as numeric weather prediction (NWP) [4]. Andrade et al. [4] used a Grid of NWP method to improve renewable energy forecasting, mainly including wind and solar energy. NWP method usually has high prediction accuracy but a large amount of calculation [5]. Statistical methods first construct feature inputs using historical wind speed and then select appropriate prediction method to predict wind speed [6]. Many machine learning methods are used to predict wind speed. Traditional time series methods include Autoregressive (AR), Moving Average (MA) and Auto Regressive Moving Average (ARMA) are usually used for short-term forecasting of wind speed [7]. These methods are linear and their ability for nonlinear or non-stationary time series prediction is limited. Chen and Yu integrated unscented Kalman filter into Support Vector Regression (SVR) based state-space model in order to precisely update the short-term estimation of wind speed sequence [8]. Since SVR solves the support vector through quadratic programming, when the number of samples is large, the storage and calculation of the matrix in the quadratic programming solution process consumes a large amount of machine memory and computation time. Artificial Neural Networks (ANN) is also a commonly used method for predicting wind speed since it can describe the nonlinearity of wind speed [9]. Some new proposed machine learning methods are also used to predict wind speed, such as Extreme Learning Machine (ELM) [10]. Due to the rapid development of deep learning in recent years, the performance of many traditional machine learning methods is inferior to that of deep learning methods [11]. Among deep learning methods, Recurrent Neural Networks (RNN) is suitable for solving sequence problems such as wind speed time series [12] since its network structure considers timing information. However, RNN may face long-term dependency problems when the sequence length is too long [13]. Long Short-Term Memory Network (LSTM) is proposed to solve this problem [14]. Since wind speed is time series data, LSTM has been used to predict wind speed [15].
There are two most important variants of LSTM, one is adding Peephole Connections to LSTM [16], and the other is Gated Recurrent Unit (GRU) that simplifies the gate structure of LSTM to reduce training time [17]. Greff et al tested the performance of eight variants of LSTM with three classic cases, and obtained some important conclusions [18]: (1) coupling the input and forget gates, or removing peephole connections simplified LSTM without significantly decreasing performance; (2) the forget gate and the output activation function are the most critical components of the LSTM block. The conclusions of Greff are crucial for designing a more efficient gate structure memory network with better performance. Therefore, the first problem need to solve is how to design a simplest gate structure memory network for wind speed prediction without reducing prediction accuracy. The idea of this paper is that based on Greff’s two conclusions, Minimal Gated Memory Network (MGM) is proposed, designing it with as simple structure and few weight variables as possible.
However, these wind speed prediction method mentioned above are all point prediction methods, lacking the ability to quantify forecast uncertainty [19]. A hybrid model based on shared weight LSTM and Gaussian Process Regression (GPR) is proposed for probabilistic wind speed forecasting [20], but this method has a premise of Gaussian assumptions. An ensemble of mixture density ANN is used for probabilistic wind speed forecasting, which can evaluate the uncertainties of model misspecification [21]. Quantile Regression (QR) is used to extend an existing wind power forecasting system with probabilistic forecasts since it can estimate the conditional distribution of the dependent variable [22]. In 1978, Koenker proposed the linear QR model [23]. Obviously, linear QR is difficult to solve the complex non-linear problems such as wind speed prediction. Quantile Regression Neural Network (QRNN) is combined with QR and ANN by Taylor in 2000, which not only can handle the non-linear problems but also can quantify the forecast uncertainty [24]. Recently, QRNN began to be used to obtain the conditional quantile of wind speed and estimate the probability density function (PDF) of wind speed [25]. A probabilistic wind speed forecasting approach based on Deep Belief Network (DBN) and QR is proposed to enhance the model’s ability to deal with nonlinearity and quantify the uncertainty of prediction [26]. It can be seen that QR is widely used in wind speed probability prediction. Therefore, the second problem need to solve is how to summarize a hybrid framework combined QR and any point prediction method, which can not only predict wind speed but also quantify the uncertainty of forecasting. In order to take timing information into the model and further improve the accuracy of the forecast, semi-new hybrid model combined QR and LSTM is first proposed using this framework, called QRLSTM. Similarly, another semi-new hybrid model QRGRU is also proposed. Furthermore, a brand new approach combined QR and MGM, called QRMGM, is used to perform probabilistic forecasting.
The prediction results obtained by QR, QRNN, QRLSTM, QRGRU and QRMGM are a series of conditional quantiles of wind speed, which cannot directly get the probability density function (PDF). The PDF of wind speed need to be estimated by probability density estimation methods using these conditional quantiles. Probability density estimation methods can be divided into two categories: parameter estimation methods and non-parametric estimation methods [25]. Kernel Density Estimation (KDE), a classic non-parametric estimation method, does not require a priori assumptions when estimating data distribution, which is the essential difference from the parameter estimation method [25]. Among kernel function of KDE methods, the mean square error of Epanechnikov kernel is optimal [27]. Therefore, KDE is used to estimate the PDF and Epanechnikov kernel function is used as kernel of KDE in this study.
In this study, a new method based on QRMGM and KDE is proposed to forecast wind speed probabilistic density. The main contributions are outlined as follows:
- (1)
The simplest form of the gated structure memory network, called MGM, is proposed to minimize training time without significantly reducing prediction accuracy.
- (2)
A framework of hybrid method combining QR and any point prediction models has been summarized. Furthermore, a brand new approach combined QR and MGM, called QRMGM, is used to perform probabilistic forecasting.
- (3)
Maximal Information Coefficient is used to select the feature variables while Genetic Algorithm is used to obtain optimal feature combinations, which can improve the performance of the model.
- (4)
Four wind speed prediction cases in Inner Mongolia, China are used to test four methods from five aspects: point prediction accuracy, interval prediction suitability, probability prediction comprehensive performance, forecast reliability and training time. The experimental results show that QRMGM-KDE has the ability to obtain wind speed prediction results with excellent performance on accuracy, uncertainty and reliability.
The remainder of this paper is organized as follows. In Section 2, the related methods are introduced in detail. In Section 3, the evaluation metrics are explained. In Section 4, an application of the proposed methods for wind speed probability density forecasting is presented. In Section 5, the work of this paper is summarized and the conclusions are given. The nomenclatures of this study can be seen in the Table 1.
Section snippets
Methods
In this section, Quantile Regression is first reviewed. Then the framework of a hybrid model combining QR and other point prediction models is summarized. After that, a new probabilistic forecasting model is proposed based on this framework. Finally, feature selection and combination methods are introduced to improve the performance of the model.
Method evaluation metric
In this section, evaluation metrics are explained, including point prediction metrics, interval prediction metrics, probabilistic prediction metrics and reliability metrics.
Case study
In this section, the dataset introduction is first described. Then the feature inputs are selected and the feature combination is optimized. Next, the experimental design and parameter settings are introduced. Finally, the performance of the proposed method is verified from five aspects: point prediction accuracy, interval prediction suitability, probability prediction comprehensive performance, forecast reliability and training time.
Conclusions
Wind speed prediction and its potential uncertainty is important for the planning and utilization of wind energy, which can provide insightful information to balance the risk and economic benefit. This study first introduces the existing method linear QR. Through this method, a framework of hybrid method combining QR and other point prediction model is summarized. Based on this framework, QRLSTM and QRGRU are proposed for the QRNN’s shortcoming that it cannot consider the timing information for
Declaration of Competing Interest
None declared.
Acknowledgements
This work is supported by the National Key R&D Program of China (2017YFC0405900), the National Natural Science Foundation of China (No. 91647114, 51809098, 61703199), the National Public Research Institutes for Basic R & D Operating Expenses Special Project (CKSF2017061/SZ), and special thanks are given to the anonymous reviewers and editors for their constructive comments.
References (40)
- et al.
Long Short-Term Memory Network based on Neighborhood Gates for processing complex causality in wind speed prediction
Energy Convers Manage
(2019) - et al.
Short-term wind power combined forecasting based on error forecast correction
Energy Convers Manage
(2016) - et al.
Current methods and advances in forecasting of wind power generation
Renew Energy
(2012) - et al.
Review of the use of Numerical Weather Prediction (NWP) Models for wind energy assessment
Renew Sustain Energy Rev
(2010) - et al.
A compound structure of ELM based on feature selection and parameter optimization using hybrid backtracking search algorithm for wind speed forecasting
Energy Convers Manage
(2017) - et al.
ARMA based approaches for forecasting the tuple of wind speed and direction
Appl Energy
(2011) - et al.
Short-term wind speed prediction using an unscented Kalman filter based state-space support vector regression approach
Appl Energy
(2014) - et al.
On comparing three artificial neural networks for wind speed forecasting
Appl Energy
(2010) - et al.
Multi-step ahead wind speed forecasting using a hybrid model based on two-stage decomposition technique and AdaBoost-extreme learning machine
Energy Convers Manage
(2017) - et al.
Hybrid forecasting model based on long short term memory network and deep learning neural network for wind signal
Appl Energy
(2019)
Monthly streamflow forecasting based on hidden Markov model and Gaussian Mixture Regression
J Hydrol
Wind speed prediction method using shared weight long short-term memory network and gaussian process regression
Appl Energy
Short-term wind speed and power forecasting using an ensemble of mixture density neural networks
Renew Energy
Quantile regression neural networks: Implementation in R and application to precipitation downscaling
Comput Geosci-Uk
Probability density forecasting of wind power using quantile regression neural network and kernel density estimation
Energy Convers Manage
The convergence analysis of SpikeProp algorithm with smoothing L-1/2 regularization
Neural Netw
A wind speed interval prediction system based on multi-objective optimization for machine learning method
Appl Energy
Deep neural network self-training based on unsupervised learning and dropout
Int J Fuzzy Logic Intell Syst
Improving renewable energy forecasting with a grid of numerical weather predictions
IEEE T Sustain Energy
Deep learning
Nature
Cited by (92)
A comprehensive opposition Multi-Verse Optimizer ensemble coordination constraint handling technique for hybrid hydro-thermal-wind problem
2024, Expert Systems with ApplicationsA hybrid carbon price prediction model based-combinational estimation strategies of quantile regression and long short-term memory
2023, Journal of Cleaner ProductionQuantile regression based probabilistic forecasting of renewable energy generation and building electrical load: A state of the art review
2023, Journal of Building Engineering