elasticsearch-5.4.1集群部署

# elasticsearch-5.4.1集群部署
## 部署前准备
- jdk1.8.0_121安装包：https://www.oracle.com/java/technologies/javase/javase8-archive-downloads.html
- elasticsearch5.4.1安装包：wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.1.tar.gz
- 服务器安装：`tar`，`gzip`，`jq`

## 部署方式
### 部署jdk

```shell
# 将jdk-8u121-linux-x64.tar.gz上传至/${PATH_TO_JDK}/，如：/usr/local/
cd /${PATH_TO_JDK}/
tar xzf jdk-8u121-linux-x64.tar.gz
# 第一个EOF前必须有\
cat >> /etc/profile <<\EOF
export JAVA_HOME=/${PATH_TO_JDK}/jdk1.8.0_121
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
EOF

source /etc/profile
```

### 部署elasticsearch集群
- 集群名称：elasticsearch
- 为防止集群脑裂，最少需要三个节点:
  |    IP    | nodeName |
  | :------: | :------: |
  | 10.1.1.1 |  node-1  |
  | 10.1.1.2 |  node-2  |
  | 10.1.1.3 |  node-3  |

### 修改环境变量
```shell
cat >> /etc/profile <<\EOF
export ES_HOME=/datayes/elasticsearch-5.4.1
export ES_DATAPATH=${es数据文件夹}
export NODENAME=${实际的node名称（每个节点不同）}
export ES_CLUSTERNAME=${es集群名称（每个节点相同）}
EOF
source /etc/profile
```

### 将elasticsearch-5.4.1.tar.gz上传至/datayes/
```
cd /datayes/
tar xzf elasticsearch-5.4.1.tar.gz
```

###  添加运行用户并修改内核参数
```
useradd -d /home/elasticsearch -s /sbin/nologin elasticsearch
cat >> /etc/security/limits.conf <<EOF
elasticsearch  -  nofile  256000
EOF

# suse还需要执行下面的配置
cat >> /etc/security/limits.conf <<EOF
elasticsearch   -   memlock   256000000
EOF

cat >> /etc/sysctl.conf <<EOF
vm.max_map_count=256000000
EOF

sysctl -p
```

### 初始化es环境变量，在初次配置es时会用到

```
mkdir $ES_DATAPATH

# 使用上面的环境变量配置es集群
# 注意修改discovery.zen.ping.unicast.hosts后的集群节点ip/域名，可以指定端口如 10.1.1.1:19300

cat > /datayes/elasticsearch-5.4.1/config/elasticsearch.yml <<EOF
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: $ES_CLUSTERNAME
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: $NODENAME
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: $ES_DATAPATH
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
# 不设置为true的话建议关闭swap交换分区 `sudo swapoff -a`
bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200

#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["10.1.1.1", "10.1.1.2", "10.1.1.3"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
gateway.recover_after_nodes: 2
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
EOF
```

### 修改jvm虚拟机配置

```
export HEAPSIZE=${使用的内存大小}g
sed -e "s|^-Xms2g|-Xms$HEAPSIZE|" \
    -e "s|^-Xmx2g|-Xms$HEAPSIZE|" \
    -i.bak \
    /datayes/elasticsearch-5.4.1/config/jvm.options

setfacl -Rd -m u:elasticsearch:rwx {/datayes/elasticsearch-5.4.1,$ES_DATAPATH,/tmp}
```

### 启动elasticsearch

- 命令启动

```
  # 直接打印日志到终端
  su - elasticsearch -s /bin/bash -c '/datayes/elasticsearch-5.4.1/bin/elasticsearch'

# 后台运行
  su - elasticsearch -s /bin/bash -c '/datayes/elasticsearch-5.4.1/bin/elasticsearch -d -p /datayes/elasticsearch-5.4.1/pid'
  ```

- systemd

```
  cat > /etc/systemd/system/elasticsearch.service <<EOF
  [Unit]
  Description=elasticsearch
  After=network.target remote-fs.target

[Service]
  Type=simple
  User=elasticsearch
  Restart=on-failure
  RestartSec=5s
  LimitNOFILE=1048576
  LimitNPROC=infinity
  LimitCORE=infinity
  TasksMax=infinity
  PIDFile=/datayes/elasticsearch-5.4.1/pid
  Environment="JAVA_HOME=$JAVA_HOME"
  Environment="CLASSPATH=.:\$JAVA_HOME/lib/dt.jar:\$JAVA_HOME/lib/tools.jar"
  Environment="ES_HOME=$ES_HOME"
  Environment="PATH=$PATH:$JAVA_HOME/bin"
  ExecStart=/datayes/elasticsearch-5.4.1/bin/elasticsearch
  ExecStop=/bin/kill -s QUIT \$MAINPID

[Install]
  WantedBy=multi-user.target
  EOF

systemctl daemon-reload
  systemctl start elasticsearch

```

## 集群检查

```
[root@node-1 elasticsearch-5.4.1]#  curl -s http://127.0.0.1:9200/_cluster/health | jq
# status=green; number_of_nodes=节点数。部署就成功了
{
  "cluster_name": "elasticsearch",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 3,
  "number_of_data_nodes": 3,
  "active_primary_shards": 0,
  "active_shards": 0,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100
}

```