twitter data analysis

Sentiment analysis is classifying the polarity of a given text at the document,sentence, or feature whether the expressed opinion in a document, a sentence or an entityfeature is positive, negative or neutral. This provides a way of sentiment analysis using Hadoop framework which will process the huge amount of data on a hadoop cluster faster in real time. 1. CREATING TWITTER APPLICATION: First of all if we want to do sentiment analysis on Twitter data we want to get Twitter data first so to get it we want to create an account in Twitter developer and create an application by clicking on the new application button provided by them.[3] After creating a new application just create the access tokens so that we no need to provide our authentication details there and also after creating application it will be having one consumer keys to access that application for getting Twitter data. The following is the figure that show clearly how the application data looks after creating the application and here it’s self we can see the consumer details and also the access token details. We want to take this keys and token details and want to set in the Flume configuration file such that we can get the required data from the Twitter in the form of tweets. 2. GETTING DATA USING FLUME: After creating an application in the Twitter developer site we want to use the consumer key and secret along with the access token and secret values. By which we can access the Twitter and we can get the information that what we want exactly here we will get everything in JSON format and this is stored in the HDFS that we have given the location where to save all the data that comes from the Twitter. The following is the configuration file that we want to use to get the Twitter data from the Twitter. 3. HIVE: Hive Query Language [HQL]: • It is a platform used to develop SQL type scripts to do MapReduce operations. • Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy. • Initially Hive was developed by Facebook, later the Apache Software Foundation took it up and developed it further as an open source under the name Apache Hive. It is used by different companies. For example, Amazon uses it in Amazon Elastic MapReduce. 4. SQOOP: Sqoop is a tool designed to transfer data between Hadoop and relational database servers. It is used to import data from relational databases such as MySQL, Oracle to Hadoop HDFS, and export from Hadoop file system to relational databases. It is provided by the Apache Software Foundat 5. SENTIMENT ANALYSIS: We retrieve the texts from the collected tweets and then assign the polarity to them using the Naïve Bayes Classifier Algorithm. Naïve Bayes Classifier Algorithm is simple module for classification. It works well text categorization. Basically it divides the text into parts of speech and assigns polarity individually and then sum up to whole. We extract these features using NLTK (Natural Language Tool Kit) package which is installed in python. We install python and some other packages which help in training the tweet parameters