Dynamic Infrastructure and manage service with Consul


Consul is a distributed service mesh to create dynamic infrastructure and manage service configurations across any runtime platform. The project is hosted at Github under the Mozilla Public License, version 2.0.

Consul provides several key features:

  • Service Discover
  • Health Checking
  • Service Segmentation
  • Multi-Datacenter

It aims to shift from static infrastructure to dynamic infrastructure changes the approach to networking from host-based to service-based.

Consul has service discovery, registration and health check process that can be more useful to create service-based networking. These features allow services to scale up/down and handle failure without load balancer.

It uses LAN and WAN gossip protocols to manage membership and broadcast messages to the cluster. Also, RPC uses to allow a client to make a request of a server.

Pre-Requirements:

  • 3 Server (1 bootstrap and 2 Server)
  • Consul Binaries
  • Application user to perform installation

Step 1: Download binaries

You can access all consul configuration parameters from that link. Also, download the latest binaries from Consul page which suitable for your operating system

Step 2: Extract and manage permissions

I extracted consul binaries under “/appdata/consulv1” directory. You need to perform these steps at all servers that you will add Cluster.(1 bootstrap and 2 Server)

#mkdir /appdata/consulv1
#cd /appdata/consulv1
#unzip consul_1.4.2_linux_amd64
#chmod  u+x consul
#chown appuser:appgroup consul
#mkdir /appdata/consulv1/config
#mkdir /appdata/consulv1/data

Step 3: Create a configuration file for bootstrap and server.

It is important to set this flag for only one server in a datacenter. In a data center, there must be only one bootstrap server.

It is optional to change service ports. I only want to show that you can manage ports on Consul. If you don’t want to change default ports then remove ports from the configuration file.

--Consul Bootstrap Server configuration;

# vi /appdata/consulv1/config/bootstrap.json 
{
  "bootstrap": true,
  "server": true,
  "data_dir": "/appdata/consulv1/data",
  "encrypt": "e6c0648e561dccadad32d6a3e579b63d",
  "datacenter": "DC1",
  "log_level": "INFO",
  "enable_syslog": true,
  "disable_update_check": true,
  "ports": {
    "server": 8303,
    "http": 8501,
    "dns" : 8601,
    "serf_lan" : 8304,
    "serf_wan" : -1
  }
}

--Consul  Server configuration;

#cat /usr/lib/systemd/system/consulv1-server.service 
[Unit]
Description=Consul Agent (Server)
After=network.target

[Service]
Restart=on-failure

User=appuser
Group=appgroup

Environment=HOME=/appdata/consulv1
EnvironmentFile=/appdata/consulv1/env
WorkingDirectory=/appdata/consulv1

ExecStartPre=/bin/echo IP=`/bin/hostname --ip-address` > /appdata/consulv1/env
ExecStart=/appdata/consulv1/consul agent -ui -config-dir=/appdata/consulv1/config -client $IP -bind $IP
ExecReload=/bin/kill -USR1 $MAINPID

PermissionsStartOnly=true

[Install]
WantedBy=multi-user.target

 

Step 4: Create a service configuration file

I added the service configuration file for RHEL7-Centos and Fedora. If you have run on another platform you should create a service file to start and stop Consul.

#cat /usr/lib/systemd/system/consulv1-bootstrap.service 
[Unit]
Description=Consul Agent (BootStrap)
After=network.target

[Service]
Restart=on-failure

User=appuser
Group=appgroup

Environment=HOME=/appdata/consulv1
EnvironmentFile=/appdata/consulv1/env
WorkingDirectory=/appdata/consulv1

ExecStartPre=/bin/echo IP=`/bin/hostname --ip-address` > /appdata/consulv1/env
ExecStart=/appdata/consulv1/consul agent -ui -config-dir=/appdata/consulv1/config -client $IP -bind $IP
ExecReload=/bin/kill -USR1 $MAINPID

PermissionsStartOnly=true

[Install]
WantedBy=multi-user.target

#cat /usr/lib/systemd/system/consulv1-server.service 
[Unit]
Description=Consul Agent (Server)
After=network.target

[Service]
Restart=on-failure

User=appuser
Group=appgroup

Environment=HOME=/appdata/consulv1
EnvironmentFile=/appdata/consulv1/env
WorkingDirectory=/appdata/consulv1

ExecStartPre=/bin/echo IP=`/bin/hostname --ip-address` > /appdata/consulv1/env
ExecStart=/appdata/consulv1/consul agent -ui -config-dir=/appdata/consulv1/config -client $IP -bind $IP
ExecReload=/bin/kill -USR1 $MAINPID

PermissionsStartOnly=true

[Install]
WantedBy=multi-user.target

--Enable and  start services;

#systemctl enable  consul-bootstrap
#systemctl start  consul-bootstrap
#systemctl enable  consul-server
#systemctl start  consul-server

I added a basic python code to show how to check consul servers’ health and register an existing service to the consul.

import consul
def consul_conn_check(host):
    c = consul.Consul(host=host)
    state = False
    try:
        c.catalog.nodes()
        state = True
    except ConnectionError:
        pass
    finally:
        return state
 #print(consul_conn_check("127.0.0.1"))

def register_consul(host, port, service_id=str):
    service_address = "127.0.0.1"
    conn = consul_conn_check(host)
    if conn:
        c = consul.Consul(host=host)
        result = c.agent.service.register(
            'My_Apache_Service_80',
            service_id=service_id,
            address=service_address,
            port=port,
            check={
                "DeregisterCriticalServiceAfter": "3m",
                "http": "http://{}:{}/".format(service_address, port),
                "interval": "5s",
                "timeout": "1s"
                }
            )
        return result
    else:
        return False
print(register_consul('127.0.0.1', 80, "ApacheID_80"))

 

 

 

 

https://www.consul.io/

https://www.programcreek.com/python/example/104783/consul.Consul

Tagged In:

I'm a IT Infrastructure and Operations Architect with extensive experience and administration skills and works for Turk Telekom. I provide hardware and software support for the IT Infrastructure and Operations tasks.

205 Total Posts
Follow Me