Sequential Data Learning, Scalable Models and Adversarial Regularization
Time: Mon 2023-06-05 14.00
Location: E32, Osquars backe 2
Video link: https://kth-se.zoom.us/j/62129883050
Doctoral student: Jin Huang , Teknisk informationsvetenskap
Opponent: Professor Geyong Min, University of Exeter
Supervisor: Ming Xiao, Teknisk informationsvetenskap
Time Series Prediction (TSP) has been used in mobile network traffic data analysis to produce predictive results for network planning and resource allocation. In the first part of this thesis, we propose a novel method of predicting mobile network traffic using neural networks based on conditional probability modeling between adjacent data windows in the time series sequence. Firstly, we develop a pre-processing method to aggregate the raw traffic log data and sample the aggregated time series to adjacent data windows, as training samples. Secondly, we use neural networks to parameterize the conditional probability between adjacent data windows and estimate the probability by training the neural networks with sampled data. The estimated conditional probability is then used to ensemble the prediction. Thirdly, we show theoretically that the prediction based on all historical data is equivalent to the prediction based on just previous data window, given the estimation of conditional probability between adjacent data windows. We also analyze computation complexity and show that seasonality will reduce the computational complexity. In the experiment, we compare the prediction performance among the models with different seasonality, sample size and number of hidden layers, and show that the proposed schemes achieve better prediction accuracy than state-of-the-art.
The Recurrent Neural Networks (RNN) with richly distributed internal states and flexible non-linear transition functions, havegradually overtaken the dynamic Bayesian networks in modeling highly structured sequential data. These data, which may come fromspeech and handwriting, often contain complex relationships between the underlying variational factors such as speakercharacteristic and the observed data. The standard RNN model has very limited randomness or variability in its structure, which comes from the output conditional probability model. To improve the variability and performance, we study the new latent variable models with novel regularization methods. The second part of this thesis will present different ways of using high level latent random variables in RNN to model the variability in the sequential data. We will explore possible ways of using adversarial methods to train a variational RNN model. Through theoretical analysis we show that, contrary to competing approaches our schemes have theoretical optimum in the model training and the symmetric objective function in the adversarial training provides better model training stability. Our approach also improves the posterior approximation in the variational inference network by a separated adversarial training step. Numerical results simulated from TIMIT speech data show that reconstruction loss and evidence lower bound converge to the same level and adversarial training loss converges in a stable course. The results also show our approach of regularization provides stability and smoothness on probability distribution distance minimization between prior and posterior of the latent variables.
In the last part of this thesis, we studies potential challenges and opportunities in intelligent road traffic sensing from the data mining and learning point of view with mobile network generated data. This part of the thesis only include qualitative analysis. Firstly, we classify the data resources available in the commercial mobile network according to different taxonomy criteria. Then, we outline the broken-down problems that fit in the framework of road traffic sensing based on mobile user network log data. We study the existing data processing and learning algorithms on extracting road traffic condition information from a large amount of mobile network log data. Finally we make suggestion on potential future work for road traffic sensing on data from mobile networks.