Challenges

distributedDataMining (dDM) is the name of a research project that uses Internet-connected computers to perform research in the various fields of Data Analysis and Machine Learning. The project uses the Berkeley Open Infrastructure for Network Computing (BOINC) for the distribution of research related tasks to several computers. The intent of BOINC is to enable researchers to tap into the enormous processing power of personal computers around the world. If you are willing to support our research challenges please participate in the dDM-Project. During the last week, 225 project members spent 35,888 hours computational power on their 396 computers. We - the members of the scientific board - would like to thank all project members for there generous support of our research.

All dDM applications use the open source framework RapidMiner. This data mining suite provides various machine learning methods for data analysis purposes. The RapidMinder uses a comfortable plug-in mechanism to easily add new developed algorithms. This flexibility and the processing power of BOINC is an ideal foundation for scientific distributed Data Mining. The dDM project takes that opportunity and serves as a metaproject for different kind of machine learning applications. Below, you find an overview of our subprojects and the related scientific publications.

Time Series Analysis

The research area called Time Series Analysis comprises methods for analyzing time series data in order to extract meaningful statistics, rules and patterns. These rules and patterns might be used to build forecasting models that are able to predict future developments.

Stock Price Prediction (on-going)

In this case study, we try to improve time series forecasting methods. In 2006, we started the analysis of Stock Market Data by using Artificial Neural Networks [5]. The results were published as book [6]. Later on, we could improve our results by applying Support Vector Machines[7]. In 2009, we started testing additional standard machine learning algorithm to build forecast models for Dow Jones, S&P500, German Stock Index and NIKKEI index. In addition, we try to extend existing approaches and develop new forecasting methods. Thereby, we will focus on aspects of temporal pattern evolution.   read more


Social Network Analysis

In 2007, Tanja Falkowski proposed DenGraph - a density-based graph clustering algorithm. This algorithm is deployable for - among other things - Social Network Analysis. The following studies were powered by our distributedDataMining project. The results are published as a part of her PhD theses that is also available as book [1].

Temporal Dynamics of the Last.fm Music Platform (temporarily suspended)

In this case study we applied DenGraph-IO to detect and observe changes in the music listening behaviour of Last.fm users during a period of two years. The aim was to see, whether the proposed clustering technique detects meaningful communities and evolutions [2], [3].   read more

Temporal Evolution of Communities in the Enron Email Data Set (finished)

The collapse of Enron, a U.S. company honored in six consecutive years by "Fortune" as "America's Most Innovative Company", caused one of the biggest bankruptcy cases in US-history. To investigate the case, a data set of approximately 1.5 million e-mails sent or received by Enron employees was published by the Federal Energy Regulatory Commission. We've used the processing power of dDM to analyze the temporal evolution of communities extracted from these email correspondences [4].   read more


References

  1. Falkowski T. Community Analysis in Dynamic Social Networks. Goettingen: Sierke Verlag; 2009.
  2. Schlitter N, Falkowski T. Mining the Dynamics of Music Preferences from a Social Networking Site. In: Proceedings of the 2009 International Conference on Advances in Social Network Analysis and Mining. Athens: IEEE Computer Society; 2009. p. 243-8.
  3. Falkowski T, Schlitter N. Analyzing the Music Listening Behavior and its Temporal Dynamics Using Data from a Social Networking Site. Zurich; 2008.
  4. Falkowski T. Community Analysis in Dynamic Social Networks. Goettingen: Sierke Verlag; 2009.
  5. Schlitter N. A Case Study of Time Series Forecasting with Backpropagation Networks. In: Steinmüller J, Langner H, Ritter M, Zeidler J, editors. 15 Jahre Künstliche Intelligenz an der TU Chemnitz. Chemnitz: Techn. Univ. Chemnitz, Fak. für Informatik; 2008. p. 203-17. (Chemnitzer Informatik-Berichte).
  6. Schlitter N. Analyse und Prognose ökonomischer Zeitreihen: Neuronale Netze zur Aktienkursprognose. Saarbrücken: VDM Verlag Dr. Müller; 2008.
  7. Möller M, Schlitter N. Analyse und Prognose ökonomischer Zeitreihen mit Support Vector Machines. In: Steinmüller J, Langner H, Ritter M, Zeidler J, editors. 15 Jahre Künstliche Intelligenz an der Fakultät für Informatik. Chemnitz: Techn. Univ. Chemnitz, Fak. für Informatik; 2008. p. 189-201. (Chemnitzer Informatik-Berichte).