Big data's tools for internet data analytics: Modelling of system dynamics

Abstract

In this paper, an application based on Apache Hadoop is deployed to gather, store and analyze the data from Internet, especially online and social media. Nowadays, this application is a common tool for media analysis. In our case, it is used to assist in the modelling of system dynamics. Basically, There are several tools that will be used, such as for file system, data crawling from the Internet, data indexing, data storage, and data analytics. The selection of technology is as the industrial trend. Surely, this is not the best approach, but as another perspective for modelling of system dynamics. A system dynamics model is developed to study the profitability of the telecommunication company and how the complaint or negative sentiment will impact to their profits. The clustering analytics is used to identify the components of the system. In continuation of the improvement process, the clustering analytics will be used not only as one time effort. It runs periodically to develop a better model of the system. Sentiment analysis tool is used as the input for one of the component, which is the complaint component. The sentiments are sourced from online and social media. Manual investigation and analytics of Internet data is required in developing the relation between the components.