The ever-increasing flood of digital information poses new challenges to data mining and machine learning practitioners. Data sets of interest routinely reach scales that call for distributed processing architectures. In this seminar, participants will acquaint themselves with a selection of data processing tools based on the Apache Hadoop platform. In a practical part, seminar participants will work on relevant data mining problems. The Webis research group operates a large, modern high-performance compute cluster (about 1600 CPU cores, 2.5 Petabytes of disk space), which will be put to use in the course of this seminar. Students will receive training in the fundamentals of hardware and software architectures of big data cluster technologies, and learn the skills necessary to apply them. Thanks to the size of the cluster and the Webis group's expertise with big data technologies, this seminar shall provide a level of training that is currently exceptional in an academic context.

[2017-04-03] Session 1

• Kick-off meeting. [slides]

[2017-04-13, 09.15-10.45 in Bauhausstraße B11/013] Session 2

• Hadoop Tutorial (1). [slides]
• Configuration Files Summary. [txt]
• Talk discussion

[2017-04-20] Session 3

• Hadoop Tutorial (2).

[2017-04-27] Session 4

• Presentation of the data sets for the seminar topics. [slides]

[2017-08-10] Seminar Presentation.

• J. Weng, "AuthorRank"

[2017-08-31] Seminar Presentation.

• M. Wiegmann, "Feature Selection for Clickbait Detection"

[2017-09-28] Seminar Presentation.

• N. Kolyada

Seminar Paper Final Submissions

The deadline for seminar paper submissions will be determined. Submissions should consist of a single ZIP file with the following contents:

• The seminar paper in PDF format, 4-8 pages including figures and references.
• All relevant code written to produce the analyses and figures in the paper.
• A text file summarizing the layout and contents of the code folder, including instructions to compile (where applicable) and run the code.

All submissions must be handed in via email to michael.voelske[at]uni-weimar.de. The file name of the attached zip file should include the names and matriculation numbers of all group members.

• Oracle VirtualBox. [download]
  Download and install the VirtualBox platform package for your operating system (Windows/Mac/Linux).
  
  
• BigData Seminar Virtual Machine Appliance. [download]
  Download the appliance file BigData.ova (3.8GiB download) and import it into VirtualBox ("File" -> "Import Appliance")
  
  
• PuTTY SSH Client. [download]
  Install this if you're running Windows.
  
  
• Apache Hadoop 2.7.2 binary package. [download]
  (will be installed during the tutorial session)

Leskovec, Rajaraman, Ullman. Mining of Massive Datasets. Cambridge University Press, 2014. http://infolab.stanford.edu/~ullman/mmds/book.pdf

Tom White. Hadoop: The Definitive Guide, 4th Edition. O'Reilly Media, 2015. ISBN: 9781491901687.

Manning, Raghavan, Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008. http://nlp.stanford.edu/IR-book/