Apache Flume 1.6.0 Installation on Ubuntu 14.04

posted on Nov 20th, 2016

Apache Flume

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system.

2) Apache Hadoop pre installed (How to install Hadoop on Ubuntu 14.04)

3) Apache Flume 1.6.0 software (Download Here)

Apache Flume Installation on ubuntu

Installation Steps

Step 1 - Update. Open a terminal (CTRL + ALT + T) and type the following sudo command. It is advisable to run this before installing any package, and necessary to run it to install the latest updates, even if you have not added or removed any Software Sources.

$ sudo apt-get update

Step 2 - Installing Java 7.

$ sudo apt-get install openjdk-7-jdk

Step 3 - Creating flume directory.

$ sudo mkdir /usr/local/flume

Step 4 - Change the ownership and permissions of the directory /usr/local/flume. Here 'hduser' is an Ubuntu username.

$ sudo chown -R hduser /usr/local/flume
$ sudo chmod -R 755 /usr/local/flume

Step 5 - Change the directory to /home/hduser/Desktop , In my case the downloaded apache-flume-1.6.0-bin.tar.gz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.

$ cd /home/hduser/Desktop/

Step 6 - Untar the apache-flume-1.6.0-bin.tar.gz file.

$ tar xzf apache-flume-1.6.0-bin.tar.gz

Step 7 - Move the contents of apache-flume-1.6.0-bin folder to /usr/local/flume

$ mv apache-flume-1.6.0-bin/* /usr/local/flume

Step 8 - Edit $HOME/.bashrc file by adding the flume path.

$ sudo gedit $HOME/.bashrc

$HOME/.bashrc file. Add the following lines

export FLUME_HOME=/usr/local/flume
PATH=$PATH:$FLUME_HOME/bin
export CLASSPATH=$CLASSPATH:$FLUME_HOME/lib/*:.

Step 9 - Reload your changed $HOME/.bashrc settings

$ source $HOME/.bashrc

Step 10 - Change the directory to /usr/local/flume/conf

$ cd $FLUME_HOME/conf

Step 11 - Copy the default flume-env.sh.template to flume-env.sh

$ cp flume-env.sh.template flume-env.sh

Step 12 - Edit flume-env.sh file.

$ gedit flume-env.sh

Step 13 - Add java path to flume-env.sh file. Save and Close.

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

Step 14 - Copy the default flume-conf.properties.template to flume-conf.properties

$ cp flume-conf.properties.template flume-conf.properties

Step 15 - Change the directory to /usr/local/flume/bin

$ cd $FLUME_HOME/bin

Step 16 - Verify Installation

$ ./flume-ng version

Apache Flume Installation on Ubuntu 14.04

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner

Previous Post                                                                                          Next Post

Labels : Flume Collecting twitter data   Flume NetCat Agent Configuration   Flume Moving Tomcat Logs to HDFS   Flume SeqGen Agent Configuration