Analysis based on the holistic multiple time series system has been a practical and crucial topic. In this paper, we mainly study a new problem that how the data is produced underneath the multiple time series system, which means how to model time series data generating and evolving rules (here denoted as semantics). We assume that there exist a set of latent states, which are the system basis and make the system run: data generating and evolving. Thus, there are several challenges on the problem: (1) How to detect the latent states; (2) How to learn the rules based on the states; (3) What the semantics can be used for. Hence, a novel correlation field-based semantics learning method is proposed to learn the semantics. In the method, we first detect latent state assignment by comprehensively considering kinds of multiple time series characteristics, which contain tick-by-tick data, temporal ordering, relationship among multiple time series and so on. Then, the semantics are learnt by Bayesian Markov characteristic. Actually, the learned semantics could be applied into various applications, such as prediction or anomaly detection for further analysis. Thus, we propose two algorithms based on the semantics knowledge, which are applied to make next-n step prediction and detect anomalies respectively. Some experiments on real world data sets were conducted to show the efficiency of our proposed method.