Meteor vs Pig

Pig, or Pig Latin, is a programming language for MapReduce specifications. Its main target (http://pig.apache.org/) is a platform for analyzing large data sets that consists of a high-level language and its the Hadoop execution engine but could also be uscompiler produces sequences of Map-Reduce programs, which are then executed ion other execution environments. Meteor is a similar system but the target is the Stratophertop of the Hadoop execution engine.¶

Meteor (http://bit.ly/QnX7T6) is a similar system, which is part of the Stratophere (http://stratosphere.eu/) stack. A Meteor program is transformed into a DAG of jobs which are run on the Nephele execution engine respectively.

Your task is to compare these systems, based on benchmarks similar to PigMix and implement them both in PIg and Meteor. The evaluation should include a comparison of the expressivness of these systems as well as a presenentation of the performance both on the high-level and the lower programming level (Map-Reduce and PACT).

The objectives of this project are the following:¶


* Get familiar with Stratosphere/PACTs
* Learn how to use Pig and Meteor
* Develop benchmark applications for Meteor, based on PigMix
* Design and execute performance comparison experiments

Supervisor: Vasiliki Kalavri

¶