How to install Graylog on CentOS 7 / RHEL 7

0
2240

Graylog is an open-source log management tool which helps you to collect, index and analyze any machine logs centrally. This guide helps you to install Graylog on CentOS 7 / RHEL 7, as well as other components.

Components:

  1. MongoDB – Being a database to store the configurations and meta information.
  2. Elasticsearch – It stores the log messages received from Graylog server and provides a facility to search them whenever required. Elasticsearch is a resource monger as it does indexing of data, so allocate more memory and use SAS or SAN disks.
  3. Graylog server – This does the parsing of logs that are coming from various inputs and provides built-in Web Interface to handle those logs.

Prerequisites:

Elasticsearch requires Java, so install either openJDK or Oracle JDK.

PS: I choose to install Oracle JDK.

Verify the Java version.

$ java -version

java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Install Elasticsearch:

Elasticsearch is one of the important components in the Graylog setup. It stores the data coming from Graylog input and displays the messages whenever a user request over the Graylog built-in web interface. Elasticseach is mainly used here to index the data/logs and provide the searching functionality when the Graylog web interface request for any information.

This guide covers basic configuration for Graylog.

Import the GPG signing key before the installation.

# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

Create an Eleasticsearch repo file to get the latest package from the official repository.

# vi /etc/yum.repos.d/elasticsearch.repo

Add below content to above file.

[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=https://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

Now, Install the latest Elasticsearch package.

# yum install elasticsearch

Reload the systemctl daemon and enable Elasticsearch to start automatically on the system startup.

# systemctl daemon-reload
# systemctl enable elasticsearch

To make Elasticsearch work with Graylog setup, modify the cluster name and set it to “graylog“. Edit the elasticsearch.yml, update it like shown below.

# vi /etc/elasticsearch/elasticsearch.yml

cluster.name: graylog
The value of cluster.name in elasticsearh.yml should match the value of elasticsearch_cluster_name in server.conf of graylog

Add below lines to avoid remote execution

script.inline: false
script.indexed: false
script.file: false

Restart the Elasticsearch.

# service elasticsearch restart

Give a minute to let the Elasticsearch get fully restarted. Elastisearch should be now listening on 9200 for processing HTTP request, use a CURL to check the response.

Cluster name should be “graylog

$ curl -X GET http://localhost:9200

{
  "name" : "Marvin Flumm",
  "cluster_name" : "graylog",
  "version" : {
    "number" : "2.3.3",
    "build_hash" : "42313423cszd6ff2c41a3df5cfa32dadcfde",
    "build_timestamp" : "2017-04-12T16:49:62Z",
    "build_snapshot" : false,
    "lucene_version" : "4.6.0"
  },
  "tagline" : "You Know, for Search"
}

Optional: Perform a health check of Elasticsearch cluster, make sure the cluster status comes as “green

$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

{
  "cluster_name" : "graylog",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Install MongoDB 3.2:

Setup a MongoDB repository is shown like below.

# vi /etc/yum.repos.d/mongodb-org-3.2.repo

Add a repository information to the above file.

[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.2.asc

Install the community edition of MongoDB using the following command.

# yum install -y mongodb-org

Start the MongoDB service and enable it on the system start-up.

# service mongod start
# chkconfig mongod on 

Install Graylog 2.0.3:

Graylog-server accepts and processes the log messages coming from the various inputs, displays data to requests that come from graylog web interface.

Download and Install graylog 2.x repository.

# rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-2.0-repository_latest.rpm

Install Graylog server using the following command.

# yum -y install graylog-server

Edit the server.conf file to begin the graylog configuration.

# vi /etc/graylog/server/server.conf

Use the following command to create a secret

$ pwgen -N 1 -s 96

fghcx3252DSFSDdf32435b3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

If you get an error like “pwgen: command not found“, install pwgen using the following command.

Make sure your system is configured with EPEL repository
# yum -y install pwgen

Place the secret like below in server.conf

password_secret = fghcx3252DSFSDdf32435b3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

Set the hash password for the root user, i.e., admin of graylog. You would need this password to login into the graylog web.

If you ever want to change/reset forgotten password of admin, you can edit/update the server.conf with hashed password.

Replace “yourpassword” with the choice of yours.

# echo -n yourpassword | sha256sum

23123123csdfvsavfsdtssdss49672c4c74e25b497770bb89b22cdeb4e951

Place the hash password.

root_password_sha2 = 23123123csdfvsavfsdtssdss49672c4c74e25b497770bb89b22cdeb4e951

You can setup email address admin user.

root_email = "emreozkan@windowslive.com"

Set timezone of root (admin) user.

root_timezone = UTC

Graylog server will try to find the Elasticsearch nodes automatically by using multicast mode. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production.

Add the following entry to graylog server.conf file, replace ipaddress with your ipaddress. You can add multiple hosts with comma separated.

elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300

Set only one master node by defining the below variable, the default setting is true.

If you add any second Graylog node, set it to false to make the node as a slave. Master node does some periodic tasks that slave nodes won’t perform.

is_master = true

Set the number of log messages to keep per index; it is recommended to have several smaller indices instead of larger ones.

elasticsearch_max_docs_per_index = 20000000

The following parameter defines to have a total number of indices; if this number is reached old index will be deleted.

elasticsearch_max_number_of_indices = 20

Shards setting rely on the number of nodes in the particular Elasticsearch cluster, if you have only one node, set it as 1.

elasticsearch_shards = 1

This the number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.

elasticsearch_replicas = 0

Install Graylog web interface:

From the Graylog version 2.x, the web interface is served directly by Graylog server, so no more Graylog web interface setup is required.

Enable Graylog web interface by editing the server.conf file.

# vi /etc/graylog/server/server.conf

Modify the entries to let Graylog Web Interface to connect to the Graylog server.

rest_listen_uri = http://your-server-ip:12900/
web_listen_uri = http://your-server-ip:9000/

Restart Graylog service.

# systemctl daemon-reload
# systemctl restart graylog-server

Make Graylog server to start automatically on system startup.

# systemctl enable graylog-server

You can check out the server startup logs; it will be useful for you to troubleshoot Graylog for any issues.

$ sudo tailf /var/log/graylog-server/server.log

If everything goes well, you should see below message in server.log file.

2016-07-01T08:21:41.538Z INFO  [ServerBootstrap] Graylog server up and running.

Firewall:

In CentOS 7 / RHEL 7, default firewall rules are set to block most of the traffic coming from the external machines.You would need to set rules for Graylog to work properly.

###  To access Graylog web interface from the externall machine

firewall-cmd --permanent --zone=public --add-port=9000/tcp

### To allow Graylog web interace to connect with Graylog server

firewall-cmd --permanent --zone=public --add-port=12900/tcp

### Optional: If you have configured any input to receive logs on port no 1514 

firewall-cmd --permanent --zone=public --add-port=1514/tcp

Reload firewalld to take an effect of the new rules.

# firewall-cmd --reload

Access Graylog web interface:

The web interface will now be listening on port 9000, point your browser to

http://ip-add-ress:9000

Login with username “admin” and the password you configured at root_password_sha2 on server.conf.

Once you logged in, you would see the getting started page.

Click on System/Overview to know the status of Graylog server.

Configure Graylog Inputs:

Graylog inputs need to be configured to receive the logs from the external source, i.e., Syslog or any logging system.
Click System –> Inputs –> select Syslog UDP and then click Launch new input. Fill with the values in the screen like below.

Once you have created the inputs, configure rsyslog or forward any system logs to your–ip-address:1514
Following screenshot shows the logs received by Graylog (Graylog console –> Search).

That’s All! you have successfully installed Graylog 2.0.3 on CentOS 7 / RHEL 7.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.