Design and development of intelligent knowledge discovery system for stock exchange database

Abstract

The stock market is a complex, nonstationary, chaotic and non-linear dynamical system. Most of the existing methods suffer from drawbacks like long training times required, often hard to understand results, and inaccurate predictions. This study focuses on data mining approach for stock market prediction. The aim is to discover unknown patterns, new rules and hidden knowledge from large databases of stock index that are potentially useful and ultimately understandable for making crucial decisions related to stock market. The prototype knowledge discovery system developed in this research can produce accurate and effective information in order to facilitate economic activities. The developed prototype consists of mainly two parts: i) based on Fuzzy decision tree (FDT); and ii) based on support vector regression (SVR). In predictive FDT, aim is to combine the symbolic decision trees with approximate reasoning offered by fuzzy representation. In fuzzy reasoning method, the weights are assigned to each proposition in the antecedent part and the Certainty Factor (CF) is computed for the consequent part of each Fuzzy Production Rule (FPR). Then for stock market prediction significant weighted fuzzy production rules (WFPRs) are extracted. The predictive FDTs are tested using three data sets including Kuala Lumpur Stock Exchange (KLSE), New York Stock Exchange (NYSE) and London Stock Exchange (LSE). The results of predictive FDT method are favorably compared with those of other random walk models like Autoregression Moving Average (ARMA) and Autoregression Integrated Moving Average (ARIMA). The SVR prediction system is based on support vector machine (SVM) approach. Weighted kernel based clustering method with neighborhood constraints is incorporated in this system for getting improved prediction results. The SVM based method gives better results than backpropagation neural networks. SVM offers the advantages including: i) there is a smaller number of free parameters; ii) SVM forecasts better as it offers better generalization; iii) training SVM is faster. In essence, both the subsystems (FDT and SVR based) developed in this project are complementary to each other. As the fuzzy decision tree based system gives easily interpretable results, we mainly use it to classify past and present data records. Whereas we use the stronger aspect of the SVR based approach for prediction of future trend of the stock market, and get improved results.