Pattern and Anomaly Recognition in Industrial Scenarios

Abstract

This report presents our group’s final year project. In this project we will describe the following points: -Objective of the Project -Experiments done -Advantages and Disadvantages of using Hadoop -Our proposal The objective of our project is to use R programming and Hadoop to handle massive data processing like industrial data (structured and semi-structured). So to achieve our objective, we chose the two technologies enlisted. The first one is R programming which allows us to graphically represent the data in an efficient way. The second technology we have utilized here is Hadoop, a software framework to implement the data-intensive programs and algorithms with thousands of machines. Hadoop evenly allocates the storage, computation power and divide large jobs to separate machines with many tiny jobs. Since we are emphasizing on two relatively new technologies (R and Hadoop) to handle a large scale job, we must fully understand the characteristics of these systems. To fully comprehend and understand these two technologies, we did many experiments on testing the performance, especially the scalability of these two technologies and optimize the respective environment. Hence came the challenges, which were initially tough to tackle but were then one by one solved and the results showed up. The essence of our project deals with the aadhaar scheme and the census of India and tries to tackle some fundamental problems that are faced due to the inefficiency of the governmental systems. The data collected manually is ought to have some discrepancies, especially when done for a country like India, where population acts as a hurdle. There lies in the challenge and the right steps, if taken are bound to provide positive results to improve the data collection and analysis of our nation, which ultimately leads to better solutions for the society