SKELK(Suricata-Kafka-Elasticsearch-Logstash-Kibana)安装

# 0.参考
[1] [Centos 6.5搭建ELK](https://blog.csdn.net/Lee_Natuo/article/details/79031518)
[2] [Network Security Monitoring with Suricata, Logz.io and the ELK Stack](https://logz.io/blog/network-security-monitoring/)
[3] [基于Centos6.5下suricata](https://www.cnblogs.com/zlslch/p/7326291.html)
[4] [Suricata Logs in Splunk and ELK](https://elatov.github.io/2016/04/suricata-logs-in-splunk-and-elk/)
[5] [ES权威指南](https://es.xiaoleilu.com/010_Intro/00_README.html)
[6] [ES+Logstash+Kibana教程](https://www.cnblogs.com/xing901022/p/4704319.html)
[7] [ELK6.0部署](https://ken.io/note/elk-deploy-guide)
[8] [ELK download](https://www.elastic.co/cn/downloads/)
[9] [logstash最佳实践](https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/contrib_plugins/kafka.html)
[10] [ELK-kafka对应关系](https://www.elastic.co/guide/en/logstash/5.6/plugins-outputs-kafka.html)

# 1.总体架构
<center>
![总体架构图](https://leanote.com/api/file/getImage?fileId=5d5aabc8ab64410bb400340d)
</center>

|主机|IP|comments|
|:--|:--|:--|
|ELK|192.168.222.145|ELK|
|kafka|192.168.222.130|kafka|
|suricata|192.168.222.144|安装suricata|

---
**NOTE:**
kafka版本要与logstash的版本相对应 elk5.xx版本对应kafka-client 0.10.0

---
# 2.ELK下载与安装

## 2.1 下载
```
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.8.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.6.8-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.6.8.tar.gz

#kafka下载
wget http://archive.apache.org/dist/kafka/0.10.0.0/kafka_2.10-0.10.0.0.tgz
```

## 2.2 环境准备
ELK不能以root账户运行

> 修改文件限制
```
# 修改系统文件
vi /etc/security/limits.conf

#增加的内容

* soft nofile 65536
* hard nofile 65536
* soft nproc 2048
* hard nproc 4096
```

> 调整进程数
```
#修改系统文件
vi /etc/security/limits.d/20-nproc.conf

#调整成以下配置
*          soft    nproc     4096
root       soft    nproc     unlimited
```

> 调整虚拟内存 & 最大并发连接数

```
#修改系统文件
vi /etc/sysctl.conf

#增加的内容
vm.max_map_count=655360
fs.file-max=655360
```

操作系统重启后生效

> JDK8安装
```
sudo yum install java-1.8.0-openjdk
```

> 创建elk专用用户

```
useradd elk
```

> 创建elk相关目录并赋权

```
#创建ELK APP目录
mkdir /usr/elk
#创建ELK 数据目录
mkdir /elk

#更改目录Owner
chown -R elk:elk /usr/elk
chown -R elk:elk /elk
```
## 2.3 安装配置
#### 2.3.1 ES
> 移动到指定目录
```
mv elasticsearch-5.6.8 /usr/elk
chown -R elk:elk /usr/elk/elasticsearch-5.6.8/
```

> 开放端口
```
#增加端口
firewall-cmd --add-port=9200/tcp --permanent
firewall-cmd --add-port=9300/tcp --permanent

#重新加载防火墙规则
firewall-cmd --reload
```
> 切换账户
```
#账号切换到 elk
su - elk
```

>数据 & 日志目录
```
创建Elasticsearch主目录
mkdir /elk/es
#创建Elasticsearch数据目录
mkdir /elk/es/data
#创建Elasticsearch日志目录
mkdir /elk/es/logs
```
> 修改配置
```
#打开目录
cd /usr/elk/elasticsearch-6.0.0

#修改配置

vi config/elasticsearch.yml
```
内容:
```
cluster.name: es 
node.name: es1
path.data: /elk/es/data
path.logs: /elk/es/logs
network.host: 192.168.222.145
http.port: 9200
transport.tcp.port: 9300
node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["192.168.222.145:9300"]
discovery.zen.minimum_master_nodes: 1
```

> 启动 & 健康检查
```
#进入elasticsearch根目录
cd /usr/elk/elasticsearch-5.6.8
#启动
./bin/elasticsearch

#查看健康状态
curl http://192.168.222.145:9200/_cluster/health
```
如果返回status=green 表示正常

#### 2.3.2 Logstash
> 移动到指定目录
```
#移动目录
mv logstash-5.6.8 /usr/elk
#赋权
chown -R elk:elk /usr/elk/logstash-5.6.8/
```

> 切换账户
```
su -elk
```

> 数据&日志目录

```
#创建Logstash主目录
mkdir /elk/logstash
#创建Logstash数据目录
mkdir /elk/logstash/data
#创建Logstash日志目录
mkdir /elk/logstash/logs
```
> 配置数据 &日志目录
```
#打开目录
cd /usr/elk/logstash-6.0.0
#修改配置
vi config/logstash.yml

#增加以下内容
path.data: /elk/logstash/data
path.logs: /elk/logstash/logs
```

> 配置kafka&es
```
input {
    kafka {
        bootstrap_servers => "192.168.222.130:9092"
        topics => ["suricata-data"]
        auto_offset_reset => "earliest"
        consumer_threads => 5  # number (optional)， default: 1
        decorate_events => true # boolean (optional)， default: false
        codec => "json"
        group_id => "test"
        }
}
filter {
}
output {
  elasticsearch {
    hosts => ["192.168.222.145:9200"]
    index => "suricata-data"
  }
  stdout {

}

}
```

#### 2.3.3 kibana部署
> 移动到指定目录
```
#移动目录
mv kibana-5.6.8-linux-x86_64 /usr/elk/kibana-5.6.8
#赋权
chown -R elk:elk /usr/elk/5.6.8/
```
> 开放端口
```
#增加端口
firewall-cmd --add-port=5601/tcp --permanent

#重新加载防火墙规则
firewall-cmd --reload
```
> 切换账户
```
#账号切换到 elk
su - elk
```

> 修改配置
```
#进入kibana-5.6.8根目录
cd /usr/elk/kibana-5.6.8
#修改配置
vi config/kibana.yml

#增加以下内容
server.port: 5601
server.host: "192.168.222.145"
elasticsearch.url: "http://192.168.222.145:9200"
```
# 3.启动
## 3.1 kafka-zookeeper
```
cd kafka_2.10-0.10.0.0
bin/zookeeper-server-start.sh config/zookeeper.properties # `start zookeeper`
bin/kafka-server-start.sh config/server.properties # `start broker`
```

## 3.2 启动suricata(注意suricata.yaml topic kafka-server配置)

```
suricata -c /etc/suricata/suricata.yaml -i ens33 -v
```
suricata.yaml配置情况：

<center>
![suricata配置](https://leanote.com/api/file/getImage?fileId=5d5aad2aab64410bb4003452)
</center>

## 3.3 启动es
```
cd /usr/elk/elasticsearch-5.6.8/
./bin/elasticsearch
```

## 3.4 启动logstash
```
cd /usr/elk/logstash-5.6.8
./bin/logstash -f config/input-output.conf
```
## 3.5 启动kibana
```
cd /usr/elk/kibana-5.6.8
./bin/kibana
```

# 4.访问网站
http://192.168.222.145:5601
<center>
![kibana](https://leanote.com/api/file/getImage?fileId=5d5aab5dab644109ba0034ab)
</center>

信息安全从业人员^_^

导航

最近发表

友情链接