My blog

Category: Blogs

  • A Practical Guide To HashiCorp Consul – Part 2

    This is part 2 of 2 part series on A Practical Guide to HashiCorp Consul. The previous part was primarily focused on understanding the problems that Consul solves and how it solves them. This part is focused on a practical application of Consul in a real-life example. Let’s get started.

    With most of the theory covered in the previous part, let’s move on to Consul’s practical example.

    What are we Building?

    We are going to build a Django Web Application that stores its persistent data in MongoDB. We will containerize both of them using Docker. Build and run them using Docker Compose.

    To show how our web app would scale in this context, we are going to run two instances of Django app. Also, to make this even more interesting, we will run MongoDB as a Replica Set with one primary node and two secondary nodes.

    Given we have two instances of Django app, we will need a way to balance a load among those two instances, so we are going to use Fabio, a Consul aware load-balancer, to reach Django app instances.

    This example will roughly help us simulate a real-world practical application.

     Example Application nodes and services deployed on them

    ‍

    The complete source code for this application is open-sourced and is available on GitHub – pranavcode/consul-demo.

    Note: The architecture we are discussing here is not specifically constraint with any of the technologies used to create app or data layers. This example could very well be built using a combination of Ruby on Rails and Postgres, or Node.js and MongoDB, or Laravel and MySQL.

    How Does Consul Come into the Picture?

    We are deploying both, the app and the data, layers with Docker containers. They are going to be built as services and will talk to each other over HTTP.

    Thus, we will use Consul for Service Discovery. This will allow Django servers to find MongoDB Primary node. We are going to use Consul to resolve services via Consul’s DNS interface for this example.

    Consul will also help us with the auto-configuration of Fabio as load-balancer to reach instances of our Django app.

    We are also using the health-check feature of Consul to monitor the health of each of our instances in the whole infrastructure.

    Consul provides a beautiful user interface, as part of its Web UI, out of the box to show all the services on a single dashboard. We will use it to see how our services are laid out.

    Let’s begin.

    Setup: MongoDB, Django, Consul, Fabio, and Dockerization

    We will keep this as simple and minimal as possible to the extent it fulfills our need for a demonstration.

    MongoDB

    The MongoDB setup we are targeting is in the form of MongoDB Replica Set. One primary node and two secondary nodes.

    The primary node will manage all the write operations and the oplog to maintain the sequence of writes, and replicate the data across secondaries. We are also configuring the secondaries for the read operations. You can learn more about MongoDB Replica Set on their official documentation.

    We will call our replication set as ‘consuldemo’.

    We will run MongoDB on a standard port 27017 and supply the name of the replica set on the command line using the parameter ‘–replSet’.

    As you may read from the documentation MongoDB also allows configuring replica set name via configuration file with the parameter for replication as below:

    replication:
    replSetName: "consuldemo"

    In our case, the replication set configuration that we will apply on one of the MongoDB nodes, once all the nodes are up and running is as given below:

    var config = {
    _id: "consuldemo",
    version: 1,
    members: [{
        _id: 0,
        host: "mongo_1:27017",
    }, {
        _id: 1,
        host: "mongo_2:27017",
    }, {
        _id: 2,
        host: "mongo_3:27017",
    }],
    settings: { 
        chainingAllowed: true 
    }
};
rs.initiate(config, { force: true });
rs.slaveOk();
db.getMongo().setReadPref("nearest");

    This configuration will be applied to one of the pre-defined nodes and MongoDB will decide which node will be primary and secondary.

    Note: We are not forcing the set creation with any pre-defined designations on who becomes primary and secondary to allow the dynamism in service discovery. Normally, the nodes would be defined for a specific role.

    We are allowing slave reads and reads from the nearest node as a Read Preference.

    We will start MongoDB on all nodes with the following command:

    mongod --bind_ip_all --port 27017 --dbpath /data/db --replSet "consuldemo"

    This gives us a MongoDB Replica Set with one primary instance and two secondary instances, running and ready to accept connections.

    We will discuss containerizing the MongoDB service in the latter part of this article.

    Django

    We will create a simple Django project that represents Blog application and containerizes it with Docker.

    Building the Django app from scratch is beyond the scope of this tutorial, we recommend you to refer to Django’s official documentation to get started with Django project. But, we will still go through some important aspects.

    As we need our Django app to talk to MongoDB, we will use a MongoDB connector for Django ORM, Djongo. We will set up our Django settings to use Djongo and connect with our MongoDB. Djongo is pretty straightforward in configuration.

    For a local MongoDB installation it would only take two lines of code:

    ...

DATABASES = {
    'default': {
        'ENGINE': 'djongo',
        'NAME': 'db',
    }
}

...

    In our case, as we will need to access MongoDB over another container, our config would look like this:

    ...

DATABASES = {
    'default': {
        'ENGINE': 'djongo',
        'NAME': 'db',
        'HOST': 'mongodb://mongo-primary.service.consul',
        'PORT': 27017,
    }
}

...
@velotiotech

    Details:

    • ENGINE: The database connector to use for Django ORM.
    • NAME: Name of the database.
    • HOST: Host address that has MongoDB running on it.
    • PORT: Which port is your MongoDB listening for requests.

    Djongo internally talks to PyMongo and uses MongoClient for executing queries on Mongo. We can also use other MongoDB connectors available for Django to achieve this, like, for instance, django-mongodb-engine or pymongo directly, based on our needs.

    Note: We are currently reading and writing via Django to a single MongoDB host, the primary one, but we can configure Djongo to also talk to secondary hosts for read-only operations. That is not in the scope of our discussion. You can refer to Djongo’s official documentation to achieve exactly this.

    Continuing our Django app building process, we need to define our models. As we are building a blog-like application, our models would look like this:

    from djongo import models

class Blog(models.Model):
    name = models.CharField(max_length=100)
    tagline = models.TextField()

    class Meta:
        abstract = True

class Entry(models.Model):
    blog = models.EmbeddedModelField(
        model_container=Blog,
    )
    
    headline = models.CharField(max_length=255)

    We can run a local MongoDB instance and create migrations for these models. Also, register these models into our Django Admin, like so:

    from django.contrib import admin
from.models import Entry

admin.site.register(Entry)

    We can play with the Entry model’s CRUD operations via Django Admin for this example.

    Also, to realize the Django-MongoDB connectivity we will create a custom View and Template that displays information about MongoDB setup and currently connected MongoDB host.

    Our Django views look like this:

    from django.shortcuts import render
from pymongo import MongoClient

def home(request):
    client = MongoClient("mongo-primary.service.consul")
    replica_set = client.admin.command('ismaster')

    return render(request, 'home.html', { 
        'mongo_hosts': replica_set['hosts'],
        'mongo_primary_host': replica_set['primary'],
        'mongo_connected_host': replica_set['me'],
        'mongo_is_primary': replica_set['ismaster'],
        'mongo_is_secondary': replica_set['secondary'],
    })

    Our URLs or routes configuration for the app looks like this:

    from django.urls import path
from tweetapp import views

urlpatterns = [
    path('', views.home, name='home'),
]

    And for the project – the app URLs are included like so:

    from django.contrib import admin
from django.urls import path, include
from django.conf import settings
from django.conf.urls.static import static

urlpatterns = [
    path('admin/', admin.site.urls),
    path('web', include('tweetapp.urls')),
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

    Our Django template, ‘templates/home.html’ looks like this:

    <!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
    <link href="https://fonts.googleapis.com/css?family=Armata" rel="stylesheet">

    <title>Django-Mongo-Consul</title>
</head>
<body class="bg-dark text-white p-5" style="font-family: Armata">
    <div class="p-4 border">
        <div class="m-2">
            <b>Django Database Connection</b>
        </div>
        <table class="table table-dark">
            <thead>
                <tr>
                    <th scope="col">#</th>
                    <th scope="col">Property</th>
                    <th scope="col">Value</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td>1</td>
                    <td>Mongo Hosts</td>
                    <td>{% for host in mongo_hosts %}{{ host }}<br/>{% endfor %}</td>
                </tr>
                <tr>
                    <td>2</td>
                    <td>Mongo Primary Address</td>
                    <td>{{ mongo_primary_host }}</td>
                </tr>
                <tr>
                    <td>3</td>
                    <td>Mongo Connected Address</td>
                    <td>{{ mongo_connected_host }}</td>
                </tr>
                <tr>
                    <td>4</td>
                    <td>Mongo - Is Primary?</td>
                    <td>{{ mongo_is_primary }}</td>
                </tr>
                <tr>
                    <td>5</td>
                    <td>Mongo - Is Secondary?</td>
                    <td>{{ mongo_is_secondary }}</td>
                </tr>
            </tbody>
        </table>
    </div>
    
    <script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js" integrity="sha384-UO2eT0CpHqdSJQ6hJty5KVphtPhzWj9WO1clHTMGa3JDZwrnQq4sF86dIHNDz0W1" crossorigin="anonymous"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js" integrity="sha384-JjSmVgyd0p3pXB1rRibZUAYoIIy6OrQ6VrjIEaFf/nJGzIxFDsf4x0xIM+B07jRM" crossorigin="anonymous"></script>
</body>
</html>

    To run the app we need to migrate the database first using the command below:

    python ./manage.py migrate

    And also collect all the static assets into static directory:

    python ./manage.py collectstatic --noinput

    Now run the Django app with Gunicorn, a WSGI HTTP server, as given below:

    gunicorn --bind 0.0.0.0:8000 --access-logfile - tweeter.wsgi:application

    This gives us a basic blog-like Django app that connects to MongoDB backend.

    We will discuss containerizing this Django web application in the latter part of this article.

    Consul

    We place a Consul agent on every service as part of our Consul setup.

    The Consul agent is responsible for service discovery by registering the service on the Consul cluster and also monitors the health of every service instance.

    Consul on nodes running MongoDB Replica Set

    We will discuss Consul setup in the context of MongoDB Replica Set first – as it solves an interesting problem. At any given point of time, one of the MongoDB instances can either be a Primary or a Secondary.

    The Consul agent registering and monitoring our MongoDB instance within a Replica Set has a unique mechanism – dynamically registering and deregistering MongoDB service as a Primary instance or a Secondary instance based on what Replica Set has designated it.

    We achieve this dynamism by writing and running a shell script after an interval that toggles the Consul service definition for MongoDB Primary and MongoDB Secondary on the instance node’s Consul Agent.

    The service definitions for MongoDB services are stored as JSON files on the Consul’s config directory ‘/etc/config.d’.

    Service definition for MongoDB Primary instance:

    {
    "service": {
        "name": "mongo-primary",
        "port": 27017,
        "tags": [
            "nosql",
            "database"
        ],
        "check": {
            "id": "mongo_primary_status",
            "name": "Mongo Primary Status",
            "args": ["/etc/consul.d/check_scripts/mongo_primary_check.sh"],
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    If you look closely, the service definition allows us to get a DNS entry specific to MongoDB Primary, rather than a generic MongoDB instance. This allows us to send the database writes to a specific MongoDB instance. In the case of Replica Set, the writes are maintained by MongoDB Primary.

    Thus, we are able to achieve both service discovery as well as health monitoring for Primary instance of MongoDB.

    Similarly, with a slight change the service definition for MongoDB Secondary instance goes like this:

    {
    "service": {
        "name": "mongo-secondary",
        "port": 27017,
        "tags": [
            "nosql",
            "database"
        ],
        "check": {
            "id": "mongo_secondary_status",
            "name": "Mongo Secondary Status",
            "args": ["/etc/consul.d/check_scripts/mongo_secondary_check.sh"],
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    Given all this context, can you think of the way we can dynamically switch these service definitions?

    We can identify if the given MongoDB instance is primary or not by running command `db.isMaster()` on MongoDB shell.

    The check can we drafted as a shell script as:

    #!/bin/bash

mongo_primary=$(mongo --quiet --eval 'JSON.stringify(db.isMaster())' | jq -r .ismaster 2> /dev/null)
if [[ $mongo_primary == false ]]; then
    exit 1
fi

echo "Mongo primary healthy and reachable"

    Similarly, the non-master or non-primary instances of MongoDB can also be checked against the same command, by checking a `secondary` value:

    #!/bin/bash

mongo_secondary=$(mongo --quiet --eval 'JSON.stringify(db.isMaster())' | jq -r .secondary 2> /dev/null)
if [[ $mongo_secondary == false ]]; then
    exit 1
fi

echo "Mongo secondary healthy and reachable"

    Note: We are using jq – a lightweight and flexible command-line JSON processor – to process the JSON encoded output of MongoDB shell commands.

    One way of writing a script that does this dynamic switch looks like this:

    #!/bin/bash

# Wait until Mongo starts
while [[ $(ps aux | grep [m]ongod | wc -l) -ne 1 ]]; do
    sleep 5
done

REGISTER_MASTER=0
REGISTER_SECONDARY=0

mongo_primary=$(mongo --quiet --eval 'JSON.stringify(db.isMaster())' | jq -r .ismaster 2> /dev/null)
mongo_secondary=$(mongo --quiet --eval 'JSON.stringify(db.isMaster())' | jq -r .secondary 2> /dev/null)  

if [[ $mongo_primary == false && $mongo_secondary == true ]]; then

  # Deregister as Mongo Master
  if [[ -a /etc/consul.d/check_scripts/mongo_primary_check.sh && -a /etc/consul.d/mongo_primary.json ]]; then
    rm -f /etc/consul.d/check_scripts/mongo_primary_check.sh
    rm -f /etc/consul.d/mongo_primary.json

    REGISTER_MASTER=1
  fi

  # Register as Mongo Secondary
  if [[ ! -a /etc/consul.d/check_scripts/mongo_secondary_check.sh && ! -a /etc/consul.d/mongo_secondary.json ]]; then
    cp -u /opt/checks/check_scripts/mongo_secondary_check.sh /etc/consul.d/check_scripts/
    cp -u /opt/checks/mongo_secondary.json /etc/consul.d/

    REGISTER_SECONDARY=1
  fi

else

  # Register as Mongo Master
  if [[ ! -a /etc/consul.d/check_scripts/mongo_primary_check.sh && ! -a /etc/consul.d/mongo_primary.json ]]; then
    cp -u /opt/checks/check_scripts/mongo_primary_check.sh /etc/consul.d/check_scripts/
    cp -u /opt/checks/mongo_primary.json /etc/consul.d/

    REGISTER_MASTER=2
  fi

  # Deregister as Mongo Secondary
  if [[ -a /etc/consul.d/check_scripts/mongo_secondary_check.sh && -a /etc/consul.d/mongo_secondary.json ]]; then
    rm -f /etc/consul.d/check_scripts/mongo_secondary_check.sh
    rm -f /etc/consul.d/mongo_secondary.json

    REGISTER_SECONDARY=2
  fi

fi

if [[ $REGISTER_MASTER -ne 0 && $REGISTER_SECONDARY -ne 0 ]]; then
  consul reload
fi

    Note: This is an example script, but we can be more creative and optimize the script further.

    Once we are done with our service definitions we can run the Consul agent on each MongoDB nodes. To run a agent we will use the following command:

    consul agent -bind 33.10.0.3 
    -advertise 33.10.0.3 
    -join consul_server 
    -node mongo_1 
    -dns-port 53 
    -data-dir /data 
    -config-dir /etc/consul.d 
    -enable-local-script-checks

    Here,  ‘consul_server’ represents the Consul Server running host. Similarly, we can run such agents on each of the other MongoDB instance nodes.

    Note: If we have multiple MongoDB instances running on the same host, the service definition would change to reflect the different ports used by each instance to uniquely identify, discover and monitor individual MongoDB instance.

    Consul on nodes running Django App

    For the Django application, Consul setup will be very simple. We only need to monitor Django app’s port on which Gunicorn is listening for requests.

    The Consul service definition would look like this:

    {
    "service": {
        "name": "web",
        "port": 8000,
        "tags": [
            "web",
            "application",
            "urlprefix-/web"
        ],
        "check": {
            "id": "web_app_status",
            "name": "Web Application Status",
            "tcp": "localhost:8000",
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    Once we have the Consul service definition for Django app in place, we can run the Consul agent sitting on the node Django app is running as a service. To run the Consul agent we would fire the following command:

    consul agent -bind 33.10.0.10 
    -advertise 33.10.0.10 
    -join consul_server 
    -node web_1 
    -dns-port 53 
    -data-dir /data 
    -config-dir /etc/consul.d 
    -enable-local-script-checks

    Consul Server

    We are running the Consul cluster with a dedicated Consul server node. The Consul server node can easily host, discover and monitor services running on it, exactly the same way as we did in the above sections for MongoDB and Django app.

    To run Consul in server mode and allow agents to connect to it, we will fire the following command on the node that we want to run our Consul server:

    consul agent -server 
    -bind 33.10.0.2 
    -advertise 33.10.0.2 
    -node consul_server 
    -client 0.0.0.0 
    -dns-port 53 
    -data-dir /data 
    -ui -bootstrap

    There are no services on our Consul server node for now, so there are no service definitions associated with this Consul agent configuration.

    Fabio

    We are using the power of Fabio to be auto-configurable and being Consul-aware.

    This makes our task of load-balancing the traffic to our Django app instances very easy.

    To allow Fabio to auto-detect the services via Consul, one of the ways is to add a tag or update a tag in the service definition with a prefix and a service identifier `urlprefix-/<service>`. Our Consul’s service definition for Django app would now look like this:</service>

    {
    "service": {
        "name": "web",
        "port": 8000,
        "tags": [
            "web",
            "application",
            "urlprefix-/web"
        ],
        "check": {
            "id": "web_app_status",
            "name": "Web Application Status",
            "tcp": "localhost:8000",
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    In our case, the Django app or service is the only service that will need load-balancing, thus this Consul service definition change completes the requirement on Fabio setup.

    Dockerization

    Our whole app is going to be deployed as a set of Docker containers. Let’s talk about how we are achieving it in the context of Consul.

    Dockerizing MongoDB Replica Set along with Consul Agent

    We need to run a Consul agent as described above alongside MongoDB on the same Docker container, so we will need to run a custom ENTRYPOINT on the container to allow running two processes.

    Note: This can also be achieved using Docker container level checks in Consul. So, you will be free to run a Consul agent on the host and check across service running in Docker container. Which, will essentially exec into the container to monitor the service.

    To achieve this we will use a tool similar to Foreman. It is a lifecycle management tool for physical and virtual servers – including provisioning, monitoring and configuring.

    To be precise, we will use the Golang adoption of Foreman, Goreman. It takes the configuration in the form of Heroku’sProcfile to maintain which processes to be kept alive on the host.

    In our case, the Procfile looks like this:

    # Mongo
mongo: /opt/mongo.sh

# Consul Client Agent
consul: /opt/consul.sh

# Consul Client Health Checks
consul_check: while true; do /opt/checks_toggler.sh && sleep 10; done

    The `consul_check` at the end of the Profile maintains the dynamism between both Primary and Secondary MongoDB node checks, based on who is voted for which role within MongoDB Replica Set.

    The shell scripts that are executed by the respective keys on the Procfile are as defined previously in this discussion.

    Our Dockerfile, with some additional tools for debug and diagnostics, would look like:

    FROM ubuntu:18.04

RUN apt-get update && 
    apt-get install -y 
    bash curl nano net-tools zip unzip 
    jq dnsutils iputils-ping

# Install MongoDB
RUN apt-get install -y mongodb

RUN mkdir -p /data/db
VOLUME data:/data/db

# Setup Consul and Goreman
ADD https://releases.hashicorp.com/consul/1.4.4/consul_1.4.4_linux_amd64.zip /tmp/consul.zip
RUN cd /bin && unzip /tmp/consul.zip && chmod +x /bin/consul && rm /tmp/consul.zip

ADD https://github.com/mattn/goreman/releases/download/v0.0.10/goreman_linux_amd64.zip /tmp/goreman.zip
RUN cd /bin && unzip /tmp/goreman.zip && chmod +x /bin/goreman && rm /tmp/goreman.zip

RUN mkdir -p /etc/consul.d/check_scripts
ADD ./config/mongod /etc

RUN mkdir -p /etc/checks
ADD ./config/checks /opt/checks

ADD checks_toggler.sh /opt
ADD mongo.sh /opt
ADD consul.sh /opt

ADD Procfile /root/Procfile

EXPOSE 27017

# Launch both MongoDB server and Consul
ENTRYPOINT [ "goreman" ]
CMD [ "-f", "/root/Procfile", "start" ]

    Note: We have used bare Ubuntu 18.04 image here for our purposes, but you can use official MongoDB image and adapt it to run Consul alongside MongoDB or even do Consul checks on Docker container level as mentioned in the official documentation.

    Dockerizing Django Web Application along with Consul Agent

    We also need to run a Consul agent alongside our Django App on the same Docker container as we had with MongoDB container.

    # Django
django: /web/tweeter.sh

# Consul Client Agent
consul: /opt/consul.sh

    Similarly, we will have the Dockerfile for Django Web Application as we had for our MongoDB containers.

    FROM python:3.7

RUN apt-get update && 
    apt-get install -y 
    bash curl nano net-tools zip unzip 
    jq dnsutils iputils-ping

# Python Environment Setup
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Setup Consul and Goreman
RUN mkdir -p /data/db /etc/consul.d

ADD https://releases.hashicorp.com/consul/1.4.4/consul_1.4.4_linux_amd64.zip /tmp/consul.zip
RUN cd /bin && unzip /tmp/consul.zip && chmod +x /bin/consul && rm /tmp/consul.zip

ADD https://github.com/mattn/goreman/releases/download/v0.0.10/goreman_linux_amd64.zip /tmp/goreman.zip
RUN cd /bin && unzip /tmp/goreman.zip && chmod +x /bin/goreman && rm /tmp/goreman.zip

ADD ./consul /etc/consul.d
ADD Procfile /root/Procfile

# Install pipenv
RUN pip3 install --upgrade pip
RUN pip3 install pipenv

# Setting workdir
ADD consul.sh /opt
ADD . /web
WORKDIR /web/tweeter

# Exposing appropriate ports
EXPOSE 8000/tcp

# Install dependencies
RUN pipenv install --system --deploy --ignore-pipfile

# Migrates the database, uploads staticfiles, run API server and background tasks
ENTRYPOINT [ "goreman" ]
CMD [ "-f", "/root/Procfile", "start" ]

    Dockerizing Consul Server

    We are maintaining the same flow with Consul server node to run it with custom ENTRYPOINT. It is not a requirement, but we are maintaining a consistent view of different Consul run files.

    Also, we are using Ubuntu 18.04 image for the demonstration. You can very well use Consul’s official image for this, that accepts all the custom parameters as are mentioned here.

    FROM ubuntu:18.04

RUN apt-get update && 
    apt-get install -y 
    bash curl nano net-tools zip unzip 
    jq dnsutils iputils-ping

ADD https://releases.hashicorp.com/consul/1.4.4/consul_1.4.4_linux_amd64.zip /tmp/consul.zip
RUN cd /bin && unzip /tmp/consul.zip && chmod +x /bin/consul && rm /tmp/consul.zip

# Consul ports
EXPOSE 8300 8301 8302 8400 8500

ADD consul_server.sh /opt
RUN mkdir -p /data
VOLUME /data

CMD ["/opt/consul_server.sh"]

    Docker Compose

    We are using Compose to run all our Docker containers in a desired, repeatable form.

    Our Compose file is written to denote all the aspects that we mentioned above and utilize the power of Docker Compose tool to achieve those in a seamless fashion.

    Docker Compose file would look like the one given below:

    version: "3.6"

services:

  consul_server:
    build:
      context: consul_server
      dockerfile: Dockerfile
    image: consul_server
    ports:
      - 8300:8300
      - 8301:8301
      - 8302:8302
      - 8400:8400
      - 8500:8500
    environment:
      - NODE=consul_server
      - PRIVATE_IP_ADDRESS=33.10.0.2
    networks:
      consul_network:
        ipv4_address: 33.10.0.2

  load_balancer:
    image: fabiolb/fabio
    ports:
      - 9998:9998
      - 9999:9999
    command: -registry.consul.addr="33.10.0.2:8500"
    networks:
      consul_network:
        ipv4_address: 33.10.0.100

  mongo_1:
    build:
      context: mongo
      dockerfile: Dockerfile
    image: mongo_consul
    dns:
      - 127.0.0.1
      - 8.8.8.8
      - 8.8.4.4
    environment:
      - NODE=mongo_1
      - MONGO_PORT=27017
      - PRIMARY_MONGO=33.10.0.3
      - PRIVATE_IP_ADDRESS=33.10.0.3
    restart: always
    ports:
      - 27017:27017
      - 28017:28017
    depends_on:
      - consul_server
      - mongo_2
      - mongo_3
    networks:
      consul_network:
        ipv4_address: 33.10.0.3

  mongo_2:
    build:
      context: mongo
      dockerfile: Dockerfile
    image: mongo_consul
    dns:
      - 127.0.0.1
      - 8.8.8.8
      - 8.8.4.4
    environment:
      - NODE=mongo_2
      - MONGO_PORT=27017
      - PRIMARY_MONGO=33.10.0.3
      - PRIVATE_IP_ADDRESS=33.10.0.4
    restart: always
    ports:
      - 27018:27017
      - 28018:28017
    depends_on:
      - consul_server
    networks:
      consul_network:
        ipv4_address: 33.10.0.4

  mongo_3:
    build:
      context: mongo
      dockerfile: Dockerfile
    image: mongo_consul
    dns:
      - 127.0.0.1
      - 8.8.8.8
      - 8.8.4.4
    environment:
      - NODE=mongo_3
      - MONGO_PORT=27017
      - PRIMARY_MONGO=33.10.0.3
      - PRIVATE_IP_ADDRESS=33.10.0.5
    restart: always
    ports:
      - 27019:27017
      - 28019:28017
    depends_on:
      - consul_server
    networks:
      consul_network:
        ipv4_address: 33.10.0.5

  web_1:
    build:
      context: django
      dockerfile: Dockerfile
    image: web_consul
    ports:
      - 8080:8000
    environment:
      - NODE=web_1
      - PRIMARY=1
      - LOAD_BALANCER=33.10.0.100
      - PRIVATE_IP_ADDRESS=33.10.0.10
    dns:
      - 127.0.0.1
      - 8.8.8.8
      - 8.8.4.4
    depends_on:
      - consul_server
      - mongo_1
    volumes:
      - ./django:/web
    cap_add:
      - NET_ADMIN
    networks:
      consul_network:
        ipv4_address: 33.10.0.10

  web_2:
    build:
      context: django
      dockerfile: Dockerfile
    image: web_consul
    ports:
      - 8081:8000
    environment:
      - NODE=web_2
      - LOAD_BALANCER=33.10.0.100
      - PRIVATE_IP_ADDRESS=33.10.0.11
    dns:
      - 127.0.0.1
      - 8.8.8.8
      - 8.8.4.4
    depends_on:
      - consul_server
      - mongo_1
    volumes:
      - ./django:/web
    cap_add:
      - NET_ADMIN
    networks:
      consul_network:
        ipv4_address: 33.10.0.11

networks:
  consul_network:
    driver: bridge
    ipam:
     config:
       - subnet: 33.10.0.0/16

    That brings us to the end of the whole environment setup. We can now run Docker Compose to build and run the containers.

    Service Discovery using Consul

    When all the services are up and running the Consul Web UI gives us a nice glance at our overall setup.

     Consul Web UI showing the set of services we are running and their current state

    ‍

    The MongoDB service is available for Django app to discover by virtue of Consul’s DNS interface.

    root@82857c424b15:/web/tweeter# dig @127.0.0.1 mongo-primary.service.consul

; <<>> DiG 9.10.3-P4-Debian <<>> @127.0.0.1 mongo-primary.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8369
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;mongo-primary.service.consul.	IN	A

;; ANSWER SECTION:
mongo-primary.service.consul. 0	IN	A	33.10.0.3

;; ADDITIONAL SECTION:
mongo-primary.service.consul. 0	IN	TXT	"consul-network-segment="

;; Query time: 139 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Apr 01 11:50:45 UTC 2019
;; MSG SIZE  rcvd: 109

    Django App can now connect MongoDB Primary instance and start writing data to it.

    We can use Fabio load-balancer to connect to Django App instance by auto-discovering it via Consul registry using specialized service tags and render the page with all the database connection information we are talking about.

    Our load-balancer is sitting at ‘33.10.0.100’ and ‘/web’ is configured to be routed to one of our Django application instances running behind the load-balancer.

     Fabio auto-detecting the Django Web Application end-points

    As you can see from the auto-detection and configuration of Fabio load-balancer from its UI above, it has weighted the Django Web Application end-points equally. This will help balance the request or traffic load on the Django application instances.

    When we visit our Fabio URL ‘33.10.0.100:9999’ and use the source route as ‘/web’ we are routed to one of the Django instances. So, visiting ‘33.10.0.100:9999/web’ gives us following output.

    Django Web Application renders the MongoDB connection status on the home page

    We are able to restrict Fabio to only load-balance Django app instances by only adding required tags to Consul’s service definitions of Django app services.

    This MongoDB Primary instance discovery helps Django app to do database migration and app deployment.

    One can explore Consul Web UI to see all the instances of Django web application services.

     Django Web Application services as seen on Consul’s Web UI

    ‍

    Similarly, see how MongoDB Replica Set instances are laid out.

    MongoDB Replica Set Primary service as seen on Consul’s Web UI
     MongoDB Replica Set Secondary services as seen on Consul’s Web UI

    Let’s see how Consul helps with health-checking services and discovering only the alive services.

    We will stop the current MongoDB Replica Set Primary (‘mongo_2’) container, to see what happens.

    MongoDB Primary service being swapped with one of the MongoDB Secondary instances
     MongoDB Secondary instance set is now left with only one service instance

    Consul has started failing the health-check for previous MongoDB Primary service. MongoDB Replica Set has also detected that the node is down and the re-election of Primary node needs to be done. Thus, getting us a new MongoDB Primary (‘mongo_3’) automatically.

    Our checks toggle has kicked-in and swapped the check on ‘mongo_3’ from MongoDB Secondary check to MongoDB Primary check.

    When we take a look at the view from the Django app, we see it is now connected to a new MongoDB Primary service (‘mongo_3’).

    Switching of the MongoDB Primary is also reflected in the Django Web Application

    Let’s see how this plays out when we bring back the stopped MongoDB instance.

     Failing MongoDB Primary service instance is now cleared out from service instances as it is now healthy MongoDB Secondary service instance
     Previously failed MongoDB Primary service instance is now re-adopted as MongoDB Secondary service instance as it has become healthy again

    Similarly, if we stop the service instances of Django application, Fabio would now be able to detect only a healthy instance and would only route the traffic to that instance.

     Fabio is able to auto-configure itself using Consul’s service registry and detecting alive service instances

    This is how one can use Consul’s service discovery capability to discover, monitor and health-check services.

    Service Configuration using Consul

    Currently, we are configuring Django application instances directly either from environment variables set within the containers by Docker Compose and consuming them in Django project settings or by hard-coding the configuration parameters directly.

    We can use Consul’s Key/Value store to share configuration across both the instances of Django app.

    We can use Consul’s HTTP interface to store key/value pair and retrieve them within the app using the open-source Python client for Consul, called python-consul. You may also use any other Python library that can interact with Consul’s KV store if you want.

    Let’s begin by looking at how we can set a key/value pair in Consul using its HTTP interface.

    # Flag to run Django app in debug mode
curl -X PUT -d 'True' consul_server:8500/v1/kv/web/debug

# Dynamic entries into Django app configuration 
# to denote allowed set of hosts
curl -X PUT -d 'localhost, 33.10.0.100' consul_server:8500/v1/kv/web/allowed_hosts

# Dynamic entries into Django app configuration
# to denote installed apps
curl -X PUT -d 'tweetapp' consul_server:8500/v1/kv/web/installed_apps

    Once we set the KV store we can consume it on Django app instances to configure it with these values.

    Let’s install python-consul and add it as a project dependency.

    $ pipenv shell
Launching subshell in virtual environment…
 . /home/pranav/.local/share/virtualenvs/tweeter-PYSn2zRU/bin/activate

$  . /home/pranav/.local/share/virtualenvs/tweeter-PYSn2zRU/bin/activate

(tweeter) $ pipenv install python-consul
Installing python-consul…
Adding python-consul to Pipfile's [packages]…
✔ Installation Succeeded 
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success! 
Updated Pipfile.lock (9590cc)!
Installing dependencies from Pipfile.lock (9590cc)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 14/14 — 00:00:20

    We will need to connect our app to Consul using python-consul.

    import consul

consul_client = consul.Consul(
    host='consul_server',
    port=8500,
)

    We can capture and configure our Django app accordingly using the ‘python-consul’ library.

    # Set DEBUG flag using Consul KV store
index, data = consul_client.kv.get('web/debug')
DEBUG = data.get('Value', True)

# Set ALLOWED_HOSTS dynamically using Consul KV store
ALLOWED_HOSTS = []

index, hosts = consul_client.kv.get('web/allowed_hosts')
ALLOWED_HOSTS.append(hosts.get('Value'))

# Set INSTALLED_APPS dynamically using Consul KV store
INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
]

index, apps = consul_client.kv.get('web/installed_apps')
INSTALLED_APPS += (bytes(apps.get('Value')).decode('utf-8'),)

    These key/value pair from Consul’s KV store can also be viewed and updated from its Web UI.

     Consul KV store as seen on Consul Web UI with Django app configuration parameters

    The code used as part of this guide for Consul’s service configuration section is available on ‘service-configuration’ branch of pranavcode/consul-demo project.

    That is how one can use Consul’s KV store and configure individual services in their architecture with ease.

    Service Segmentation using Consul

    As part of Consul’s Service Segmentation we are going to look at Consul Connect intentions and data center distribution.

    Connect provides service-to-service connection authorization and encryption using mutual TLS.

    To use Consul you need to enable it in the server configuration. Connect needs to be enabled across the Consul cluster for proper functioning of the cluster.

    {
    "connect": {
        "enabled": true
    }
}

    In our context, we can define that the communication is to be TLS identified and secured we will define an upstream sidecar service with a proxy on Django app for its communication with MongoDB Primary instance.

    {
    "service": {
        "name": "web",
        "port": 8000,
        "tags": [
            "web",
            "application",
            "urlprefix-/web"
        ],
        "connect": {
            "sidecar_service": {
                "proxy": {
                    "upstreams": [{
                        "destination_name": "mongo-primary",
                        "local_bind_port": 5501
                    }]
                }
            }
        },
        "check": {
            "id": "web_app_status",
            "name": "Web Application Status",
            "tcp": "localhost:8000",
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    Along with Connect configuration of sidecar proxy, we will also need to run the Connect proxy for Django app as well. This could be achieved by running the following command.

    {
    "service": {
        "name": "web",
        "port": 8000,
        "tags": [
            "web",
            "application",
            "urlprefix-/web"
        ],
        "connect": {
            "sidecar_service": {
                "proxy": {
                    "upstreams": [{
                        "destination_name": "mongo-primary",
                        "local_bind_port": 5501
                    }]
                }
            }
        },
        "check": {
            "id": "web_app_status",
            "name": "Web Application Status",
            "tcp": "localhost:8000",
            "interval": "30s",
            "timeout": "20s"
        }
    }
}

    We can add Consul Connect Intentions to create a service graph across all the services and define traffic pattern. We can create intentions as shown below:

    $ consul connect proxy -sidecar-for web

    Intentions for service graph can also be managed from Consul Web UI.

     Define access control for services via Connect and service connection restrictions

    This defines the service connection restrictions to allow or deny them to talk via Connect.

    We have also added ability on Consul agents to denote which datacenters they belong to and be accessible via one or more Consul servers in a given datacenter.

    The code used as part of this guide for Consul’s service segmentation section is available on ‘service-segmentation’ branch of velotiotech/consul-demo project.

    That is how one can use Consul’s service segmentation feature and configure service level connection access control.

    Conclusion

    Having an ability to seamlessly control the service mesh that Consul provides makes the life of an operator very easy. We hope you have learnt how Consul can be used for service discovery, configuration, and segmentation with its practical implementation.

    As usual, we hope it was an informative ride on the journey of Consul. This was the final piece of this two part series. This part tries to cover most of the aspects of Consul architecture and how it fits into your current project. In case you miss the first part, find it here.

    We will continue our endeavors with different technologies and get you the most valuable information that we possibly can in every interaction. Let’s us know what you would like to hear from us more or if you have any questions around the topic, we will be more than happy to answer those.

    References

  • Creating a Smarter, Safer World: Developing a VoIP-Enabled Audio/Video Call Mobile App for Smart Buildings

    In today’s fast-evolving world, technology has redefined how we interact with our environments. From smart homes to automated workplaces, the demand for integrated systems that offer safety, convenience and enhanced user experience is higher than ever. As part of our ongoing mission to designs and build next-gen products, we recently took on a fascinating project: developing a mobile app (Android and iOS) for audio and video calls that works seamlessly with voice entry panels supporting VoIP (Voice over Internet Protocol).

    The Client’s Vision: Safety, Accessibility and Innovation

    The client approached us and requested an app that could serve as a comprehensive communication hub between residents, visitors, and building security systems. They wanted a mobile application that would:

    • Enable smooth audio and video communication between users and VoIP-enabled voice entry panels.
    • Improve building security by allowing residents to verify visitors before granting access.
    • Increase convenience by integrating this communication into a single, easy-to-use app.
    • Allow the mobile device to provide keyless entry.
    • Should be available for both iOS and Android devices.
    • In essence, the app would become the digital key to a smarter, safer living space.

    Why VoIP? The Backbone of the Future

    Voice over Internet Protocol (VoIP) has revolutionized modern communication by allowing voice and multimedia sessions over the internet. Its use in smart buildings, particularly in voice entry panels, provides several key benefits:

    • Real-time communication: VoIP ensures fast, reliable, real-time communication between users and security systems.
    • Cost-effectiveness: Unlike traditional landlines, VoIP uses existing internet infrastructure, lowering costs and providing more flexibility for building owners.
    • Scalability: VoIP systems can easily be scaled up to accommodate new users and features as a building’s needs evolve.

    By leveraging VoIP, we were able to design an app that offers superior connectivity, reliability and security.

    Key Features of Our Audio/Video Call App

    Our team focused on creating a solution that would offer a seamless user experience while enhancing security and control over building access. Below are the key features of the app we developed:

    • Two-Way Audio/Video Communication

    The app allows residents to receive calls from visitors at building entry panels, enabling real-time audio and video communication. This allows users to verify visitors’ identities visually and audibly, adding an extra layer of security.

    • Remote Door Access

    After verifying a visitor, the user can grant or deny access to the building via the app. This feature is particularly useful for those who may not be home but still need to authorize entry, such as in the case of package deliveries or housekeepers.

    • VoIP-Based Calling

    Since the app is built around VoIP technology, users benefit from high-quality calls that bypass traditional phone networks. The app integrates with voice entry panels that support VoIP, ensuring clear communication even over long distances.

    • BLE Functionality: Turning Your Mobile Device into a Key

    One of this app’s most groundbreaking features is the integration of Bluetooth Low Energy (BLE) technology, which allows the mobile device to function as a digital key. BLE is known for its ability to facilitate short-range communication with minimal power consumption, making it an ideal solution for mobile access control.

    Instead of using traditional keycards, fobs, or physical keys, employees and residents can use their smartphones to open gates and doors. BLE communicates with the gate system within proximity, enabling automatic or one-tap entry. This improves convenience and reduces the need for physical keys, which can be lost or stolen.

    • Cloud-Based Validation: Security in the Digital Age

    To ensure that only authorized individuals can gain access, the app integrates with highly secure cloud servers for communication and validation. Every time a user attempts to open a gate or door using the app, the system verifies their credentials in real-time through cloud communication. This means that building managers or security teams can update or revoke access permissions instantly, without needing to reissue physical keys or cards.

    Cloud-based validation offers multiple security layers, including encryption and authentication, ensuring that access control is both flexible and safe. It also allows for seamless monitoring and reporting, as every access attempt is logged in the system, providing valuable data for building management and security audits.

    This feature also lets users manage communication logs, call history, and security footage from anywhere in the world. This feature is critical for building managers who want a real-time overview of all entry points.

    • Streamlined Access for Employees and Residents

    This app is designed with both employees and residents in mind. In office settings, employees can easily move through secured areas, using the app to access parking garages, secure floors, and meeting rooms. For residents of gated communities or apartment buildings, the app provides a hassle-free solution to enter gates and shared spaces like gyms or pools, without needing to fumble with keys or keycards and verifying visitors’ identities visually and audibly.

    • Customizable Alerts and Notifications

    Users can set up custom notifications for different types of events, such as when a delivery arrives or a family member enters the building. These alerts ensure that users are always aware of what’s happening around their living space.

    • Scalability

    This app can be scaled for use in various environments, from small residential buildings to large corporate complexes.

    Enhancing the Future of Smart Buildings

    This app’s development is part of a broader trend towards creating intelligent living environments where technology and security work hand-in-hand. Smart buildings equipped with VoIP-enabled voice entry panels, paired with mobile applications like ours not only enhance both security and user experience but also offer a glimpse into the future-a future where buildings are not just physical spaces, but adaptive, interactive ecosystems designed around the needs of their occupants.

    We’re proud to contribute to this vision of safer, smarter, and more livable buildings. This project is a step forward in reshaping how residents interact with their living spaces, making life more convenient while ensuring peace of mind.

    The Road Ahead

    As we continue to refine and improve our app, we are excited about the potential to incorporate even more innovative features, such as Early Media Access, AI-driven facial recognition, advanced analytics for building managers and further integrations with home automation systems like Emergency Lighting System, Apartment Intercom System, Fire Detection and Alarm System including Smoke detectors, heat detectors and carbon monoxide detectors.

    Our commitment remains strong: to create digital solutions which empower our clients and make the world a better, safer place to live in.

    Stay tuned for more updates as we continue to push the boundaries of what smart building technology can achieve!

    ‍

  • How Cross-Functional Data Collaboration Fuels AI Forecasting in Manufacturing

    Factories have always been noisy places. Not just the machines—the data too. Every department speaks a different dialect: engineering in blueprints, quality in percentages, finance in margins. Somewhere in the middle, meaning flickers for a second and disappears again.

    Now and then, though, someone manages to connect those fragments. And suddenly, forecasting stops being merely about predicting the future. It becomes more about teams listening to each other better.

    What the Big Idea Really Is

    Cross-functional data collaboration sounds grand, but in practice, it is a modest habit: people sharing what they know, even when their worlds don’t perfectly align. When that happens often enough, AI forecasting begins to find its footing.

    1. Context changes everything. Data is mute until someone explains it. When quality and operations and finance look at the same trend together, the AI model stops predicting blindly and starts learning the logic behind the numbers.
    2. Shared language breeds trust. The real progress often happens in small arguments—what counts as a defect, what counts as a delay, and so on. Agreement builds a kind of quiet reliability into the system.
    3. Speed follows understanding. Once the translation work is done, decisions move faster without anyone forcing it. Collaboration shrinks the distance between seeing and acting.
    4. Learning runs both ways. AI models learn from data, yes, but teams learn from the models too—what the system notices, what it misses, what it exaggerates.

    What Research and Experience Suggest

    Manufacturing research increasingly confirms what we experience in practice: AI forecasting only becomes truly powerful when data flows freely within the organization—across functions, not just within silos. A literature review in Applied Sciences highlights that machine learning in manufacturing gains its strategic power when production, quality, and system data are treated as interconnected (e.g., the “Four-Know” framework: what, why, when, how).

    Academic work on decentralized manufacturing also shows that sharing insights between units —for example, through knowledge distillation—improves model performance in under-informed parts of the organization. Meanwhile, quality-prediction research demonstrates that explainability techniques can prune irrelevant features from forecasting models, improving accuracy and interpretability. 

    These findings support the idea that cross-functional collaboration is not optional – it’s a pre-condition for success. When AI models are built on data that reflects operations, engineering, procurement, and quality together, forecasting becomes more accurate, more explainable, and more trusted. That kind of collaboration isn’t just technical: it’s deeply human.

    A Small Illustration to Put This Big Idea into Perspective

    Consider a mid-sized electronics manufacturer. Suddenly, the production team notices a subtle decline in yield. The quality department begins reporting higher-than-usual defect rates. Procurement, however, insists that their suppliers are stable—no obvious issues. On the surface, dashboards look fine. But when a cross-functional data task force (operations, quality, procurement, and data engineers) digs deeper, they uncover a misalignment: data from different functions is recorded in inconsistent formats and timestamps, leading to distorted aggregations.

    Together, they re-align the data – standardizing units, recalibrating timestamps, and merging datasets. Over the next few months, their AI forecasting system begins to issue early warning signals. A similar pattern to the previous yield decline appears in the forecast, but this time the team acts before the issue reaches the shop floor. The early alert isn’t magic; it’s the result of shared understanding—of functions working with the same underlying reality.

    Why This Matters for Manufacturers

    • Agility grows quietly. Teams that talk regularly move faster, even without new tools.
    • Transparency replaces suspicion. When everyone helped build the model, no one treats it like a black box.
    • Improvement loops back. Forecasts feed design; design refines process; process refines data.
    • Resilience hides in plain sight. Shop floors that promote cross-functional data sharing tend to respond better, not louder.

    Finding Rhythm in the Rough Edges

    Collaboration can be tiring. Ownership blurs. Tools overlap. Meetings multiply. There’s a temptation to smooth the friction away. But the friction is the signal. It shows that people are actually engaging, not just aligning through slide decks. In data collaboration, as in music, a little dissonance means the system is alive.

    How R Systems Helps Manufacturers Achieve More Reliable Forecasting

    At R Systems, we’ve learned that collaboration cannot be installed; it has to be engineered gently. We help manufacturers connect their data across functions—through clean integration layers, unified data models, and agentic AI systems that learn from how people already work.

    Our AI solutions don’t replace collaboration; they make it easier to sustain. The systems adapt to team rhythms, not the other way around. Forecasting then becomes a shared act between human intuition and machine precision.

    A Closing Thought – Not by Any Means a Final Word

    If perfect data is the goal, collaboration will always feel messy. But if learning is the goal, the mess starts to look like progress.

    We tend to think of AI forecasting here as only about predicting the trends of tomorrow. In fact, it’s not entirely just that. It’s largely about helping manufacturers, see clearly, the facts of today. When cross-functional teams start collaborating and sharing data, the fog around what’s next begins to clear. Talk to our experts today.

  • How AI Knowledge Engines Accelerate Data-to-Decision Cycles in Manufacturing

    Data runs through every corner of manufacturing, from shop-floor sensors to ERP systems, maintenance logs, and supplier feeds. Yet the journey from data to decision often drags, with insights stuck in silos and actions coming too late. That’s where an AI knowledge engine changes the game. It connects raw data with human expertise, reasons across sources, and turns information into timely, actionable insights. For data and AI professionals and business leaders, it’s a way to move from spotting a signal to making a confident decision and fast.

    From Manufacturing Analytics to Decision Acceleration

    Traditional manufacturing analytics has focused on reporting and visualization with dashboards, KPIs, and trend charts that describe what’s happening. But analytics alone doesn’t drive improvement. The real value lies in faster, smarter decisions: when to intervene, how to optimize, and where to allocate resources.

    An AI knowledge engine transforms that process. It absorbs structured and unstructured data from machine sensors, operator logs, quality reports, and supply chain updates and weaves them into a coherent model. Instead of isolated insights, you get connected intelligence.

    Imagine a scenario where one sensor shows a vibration anomaly while another logs a temperature spike in a different module. The engine correlates these signals, links them to past maintenance records, and infers a likely root cause, say a misaligned component creating downstream defects. The system then recommends a precise corrective action before the problem impacts production.

    That’s not just analytics. That’s decision acceleration.

    What Exactly Is an AI Knowledge Engine?

    The term may sound abstract, but the concept is straightforward. An AI knowledge engine combines data integration, semantic understanding, and reasoning to deliver contextual insights. It doesn’t just retrieve data: it learns, connects, and infers.

    Key components include:

    • Data ingestion – capturing both structured data (sensor readings, ERP data) and unstructured data (notes, images, reports).
    • Semantic modeling – using ontologies and knowledge graphs to represent how components, processes, and events relate to each other.
    • Reasoning and inference – applying AI algorithms to detect patterns, diagnose root causes, and recommend actions.
    • Continuous learning – refining models as new data and feedback come in.

    As some sources describe it, knowledge engines encode not just data but also the expertise i.e. the judgment, intuition, and decision-making logic of seasoned engineers into systems that can reason autonomously.

    In manufacturing, that means moving from descriptive (“what happened”) to prescriptive (“what should we do”) and eventually proactive (“what will happen next”).

    Why Manufacturing Needs AI Knowledge Engines

    Manufacturing is inherently complex. It’s a web of machines, materials, people, and suppliers. Decisions ripple across this network. Here’s where AI knowledge engines create tangible value:

    • Speed: They cut decision latency by automating data synthesis and surfacing insights instantly.
    • Integration: They break down silos between production, maintenance, and supply chain systems.
    • Scalability: They capture the expertise of senior engineers and make it reusable across plants and teams.
    • Predictive power: They detect hidden correlations, for instance, between supplier variability and product quality.
    • Resilience: They enable earlier interventions, reducing downtime and waste.

    Each of these outcomes ties directly to measurable KPIs like faster cycle times, higher first-pass yield, lower maintenance costs, and improved operational stability.

    Making AI Knowledge Engine Work

    Deploying an AI knowledge engine isn’t just about technology. It’s about clarity of purpose and disciplined execution.

    1. Start with the decision — Identify which decisions you want to accelerate: maintenance actions, quality interventions, or supplier responses.
    2. Map the data sources — Ensure access to both quantitative data and qualitative context, like technician notes or operator feedback.
    3. Build the semantic layer — Create a knowledge graph that reflects how your processes and assets relate.
    4. Embed into workflows — Integrate the engine’s insights into control systems, dashboards, or maintenance apps so actions follow naturally.
    5. Govern continuously — Monitor model accuracy, data quality, and explainability to build trust and avoid bias.

    When implemented well, the engine doesn’t replace human decision-makers rather, it empowers them. It brings context and foresight into everyday operations.

    A Quick Use Case

    Take a discrete manufacturing plant producing complex assemblies. Sensors monitor vibration, temperature, and throughput. Maintenance logs record interventions. Supplier data tracks material quality.

    An AI knowledge engine connects all this. When a sensor drift appears, it correlates the signal with past maintenance records and supplier deliveries. Within seconds, it flags a likely cause like a worn bearing from a recent batch and suggests a replacement schedule. The maintenance team acts before a breakdown occurs, preventing a 12-hour downtime.

    That’s data turning into decision: seamlessly.

    The Bottom Line

    Manufacturing has long chased the vision of real-time, insight-driven operations. AI knowledge engines make that vision practical. They don’t just analyze data; they understand it in context, reason over it, and translate it into decisions that matter.

    For data and AI professionals, they represent the next step in operational intelligence. For business leaders, they unlock a faster, more resilient enterprise. The gap between knowing and doing is finally closing and it’s powered by knowledge.

    How R Systems Helps Manufacturers Get There

    At R Systems, we help manufacturers bridge the gap between data and decision through AI-driven knowledge systems, analytics, and domain expertise. Our teams combine manufacturing intelligence, data engineering, and applied AI to design solutions that capture context, automate reasoning, and accelerate insights across operations.

    From predictive maintenance and process optimization to digital twins and AI-powered decision systems, we help you move beyond dashboards toward truly intelligent manufacturing.

    Turn your data into knowledge. Turn your knowledge into action. Talk to our experts today.

  • 5 Reasons to Embed FinOps Governance-as-Code in Your IaC Workflows

    Cloud adoption has changed how businesses build and scale technology. Infrastructure is now software-defined, and teams can spin up compute, storage, and networking in minutes. While this flexibility fuels innovation, it also creates a challenge: costs and compliance can easily get out of hand if not governed properly.

    That is where FinOps comes in. Traditionally, FinOps has helped organizations bring financial accountability to cloud operations. But in many cases, the discipline has been reactive. Teams analyze bills after the fact, spot anomalies, and try to course-correct. By then, the costs have already been incurred.

    With Infrastructure as Code (IaC), there is a better way. Governance-as-Code embeds FinOps principles directly into the development workflow, making cost and compliance checks proactive instead of reactive.

    Why Reactive FinOps Isn’t Enough

    Think about how DevOps transformed software delivery. Before CI/CD pipelines, testing and integration were manual, error-prone, and slow. By automating these steps, DevOps brought speed, consistency, and reliability.

    FinOps is going through a similar shift. Manual reviews and after-the-fact reporting can’t keep pace with dynamic cloud environments. Developers need to move fast, but without the right guardrails, projects risk overspending, missing compliance requirements, or violating SLAs.

    Traditional FinOps provides visibility into these problems. Governance-as-Code prevents them in the first place.

    How Governance-as-Code Works

    Governance-as-Code integrates FinOps policies into IaC tools and CI/CD pipelines. This means budgets, tagging standards, and cost limits are enforced as part of the deployment process—not afterward.

    Here are some practical examples:

    • Cost estimation with Infracost: Before a new resource is deployed, Infracost calculates its projected cost. Developers see the impact of their code changes on the cloud bill, enabling smarter choices.
    • Policy enforcement with OPA (Open Policy Agent): OPA can enforce rules such as “all resources must have tags for cost center and owner” or “deployments cannot exceed predefined budgets.” If code violates these rules, the pipeline fails, stopping the issue before it reaches production.
    • Automated anomaly detection: Pipelines can include automated checks for unusual usage patterns or unexpected cost spikes, catching issues early.

    The result: cost governance is built into the development process, without slowing down delivery.

    Benefits of FinOps Governance-as-Code

    1. Proactive Cost Control

    Instead of analyzing costs at the end of the month, teams know upfront what their changes will cost. This prevents surprises and helps organizations stay within budget.

    2. Stronger Compliance

    Tagging, access policies, and cost allocation rules are automatically enforced. This ensures clean data for reporting and audit readiness without adding manual effort.

    3. Reduced Waste

    Resources without proper governance—like orphaned storage volumes or untagged VMs—are caught at the source. This minimizes waste before it builds up.

    4. Faster Delivery

    Some worry that governance will slow down developers. The opposite is true. By automating policies, developers spend less time on manual reviews or fixes. They can move faster, with confidence that costs and compliance are under control.

    5. Better Collaboration

    Governance-as-Code creates a common framework for developers, finance teams, and operations. Instead of working in silos, everyone aligns on shared rules, improving trust and accountability.

    From Challenge to Control: A Governance-as-Code Success Story

    A global transit software leader faced exactly this challenge. With a growing cloud footprint, they struggled to enforce tagging, track anomalies, and ensure compliance with service-level agreements (SLAs). Manual checks weren’t keeping pace, and margins were under pressure.

    Working with R Systems, the company implemented a Governance-as-Code approach to FinOps. Automation enforced 100% tagging compliance, while anomaly detection was built into deployment pipelines. This meant every new resource came with full visibility, cost attribution, and compliance by design.

    The impact was clear:

    • SLAs were protected through consistent governance.
    • Margins were preserved by eliminating waste and avoiding cost surprises.
    • Delivery speed improved, since developers no longer had to worry about manual compliance checks.

    This shows how proactive FinOps discipline not only reduces risk but also gives teams the freedom to focus on innovation.

    Read the full story here- Real-Time Cloud Governance That Safeguards Margins and SLAs – R Systems

    The R Systems Approach to FinOps Governance-as-Code

    At R Systems, we see Governance-as-Code as a natural next step for FinOps. Our Cloud FinOps practice is built on three pillars:

    • Visibility and Control: Real-time insights into cloud spend, with proactive enforcement of budgets, tags, and policies.
    • Automation and Scale: Tools like Infracost, OPA, and custom governance frameworks integrated directly into CI/CD pipelines.
    • Collaboration and Enablement: Cross-functional alignment between engineering, finance, and business leaders, so FinOps is not a bottleneck but a business enabler.

    Whether you’re looking to enforce cost allocation, automate anomaly detection, or scale compliance across multi-cloud environments, R Systems brings the expertise to embed governance smoothly into your workflows.

    Explore our Cloud FinOps capabilities to learn more.

    Looking Ahead: The Future of Proactive FinOps

    Cloud environments are only getting more complex. Multi-cloud strategies, serverless architectures, and AI-driven workloads make governance even more important. Organizations that rely on manual, reactive FinOps will struggle to keep up.

    Governance-as-Code offers a way forward. It allows companies to build cost control and compliance into their infrastructure from the start, ensuring agility doesn’t come at the expense of margins.

    As many experts note, FinOps is not just about controlling spend—it’s about making the right spend. With Governance-as-Code, those right decisions happen at the speed of deployment.

    The Way Forward

    If your FinOps strategy is still reactive, it’s time to rethink. Embedding governance into IaC pipelines ensures costs are controlled, compliance is enforced, and developers can innovate without friction.

    R Systems can help you make this shift. With deep expertise in Cloud FinOps and proven success in Governance-as-Code, we enable enterprises to protect margins, accelerate delivery, and reinvest savings into growth.

    The question is not whether you can use FinOps Governance-as-Code. The question is whether you can afford to run cloud without it.

    Let’s move forward together. Start the journey — talk to our Cloud FinOps experts today.

  • The Impact of Buy Now, Pay Later Model on Consumer Behavior and the Payments Engineering Behind It

    Buy Now, Pay Later (BNPL) has moved from a niche fintech innovation to a mainstream payment method, reshaping how people shop, spend, and manage credit today. Monthly BNPL spending increased almost 21% from $201.60 in June 2024 to $243.90 in June 2025, according to Empower Personal Dashboard™ data.

    With BNPL growing, both customer expectations and spending patterns are evolving, and behind it all, payments engineering has become central. It is powering real-time credit checks, seamless checkout integrations, secure installment processing, and scalable infrastructures that ensure the Buy Now Pay Later model delivers on its promise of convenience, flexibility, and trust.

    Buy Now Pay Later model: Shifting Consumer Expectations

    The Buy Now Pay Later model is redefining what consumers demand in payments:

    • Instant Approvals: Shoppers want credit decisions in seconds, not days.
    • Transparency: Clear installment schedules and upfront costs with no hidden fees.
    • Flexibility: Options ranging from Pay-in-4 to longer repayment plans.
    • Integration: BNPL woven seamlessly into eCommerce checkouts, mobile apps, and even in-store POS systems.

    These expectations underscore why payments engineering must focus on both experience and trust.

    Payments Engineering: The Backbone of the Buy Now Pay Later Model

    For Buy Now Pay Later model to scale and remain trustworthy, payments engineering is essential. Core elements include:

    1. Real-Time Risk Assessment
      • AI-driven credit models approve or decline BNPL transactions instantly.
      • Regulatory pressure is increasing around affordability checks.
    2. Seamless Checkout Integration
      • APIs and SDKs embed the Buy Now Pay Later model directly into digital and in-store journeys.
      • UX design ensures clarity and transparency.
    3. Transaction Orchestration
      • Splitting purchases into multiple payments requires precise ledgering, routing, and reconciliation at scale.
    4. Fraud Prevention & Compliance
      • BNPL engineering integrates identity checks, AML measures, and PCI DSS compliance.
    5. Scalable Infrastructure
      • Cloud-native platforms ensure resilience and handle seasonal spikes in transaction volumes.

    Without payments engineering, the Buy Now Pay Later model could not deliver its promise of flexibility and security.

    Real-World Insights from BNPL Research

    • Checkout framing effect
      “Imagine you’re buying a $100 dress. If you see “Pay now: $100”, that’s a big number. But if the checkout shows “Pay in 4: $25 per month”, you feel the cost is more manageable—and you’re more likely to click purchase.”
    • Comparing BNPL vs. credit cards
      “Someone accustomed to paying with credit cards might see the full card bill at once, which can trigger cost awareness or even comparison-shopping. But with BNPL, because each payment is smaller and delayed, there’s less friction. BNPL users spend more under these conditions than credit‐card users do.”
    • Behavioral/psychological angle
      Use a scenario:
      “Jane wants to buy a $400 laptop. She hesitates because that’s a large hit all at once. But if the option is “4 payments of $100 with no interest,” she feels it’s more feasible, and goes ahead. The installment breakdown makes the cost feel smaller in present terms.”
      This illustrates the psychological mechanisms the study uncovers.

    Risks and Regulatory Shifts

    BNPL’s rapid adoption also comes with notable challenges:

    • Credit Reporting: Repayment histories are increasingly reported to credit bureaus, making defaults more visible and impactful.
    • Overextension: A growing number of users rely on BNPL for cash flow rather than convenience, leading to rising late payments.
    • Global Regulations: From the EU’s Consumer Credit Directive to UK affordability reforms, mandatory checks for transparency and responsible lending are reshaping the BNPL landscape.

    These shifts mean providers can no longer treat compliance and risk management as afterthoughts. This is where payments engineering takes center stage. Engineering-led approaches allow businesses to:

    • Automate credit checks, affordability assessments, and regulatory reporting
    • Design secure, scalable BNPL platforms that can adapt to global compliance requirements
    • Use AI and advanced analytics to flag high-risk behavior before it escalates
    • Ensure seamless, low-friction customer experiences while embedding compliance into the transaction flow

    To navigate these risks and regulatory shifts, providers must move beyond reactive fixes and embrace proactive, engineering-led strategies. Success depends on translating compliance requirements into technical architecture, system design, and embedded controls that scale with the business.

    At R Systems, we enable organizations to strengthen their BNPL platforms with cloud-native architectures, API-first integrations, AI-driven fraud and risk models, and compliance-by-design frameworks. In today’s market, BNPL is no longer a competitive edge, it’s a baseline expectation. With our payments engineering expertise, businesses can not only stay compliant but also lead with secure, reliable, and future-ready BNPL solutions. Talk to our Experts Now.

  • Every Millisecond Matters: How AI is Rewriting the Rules of Real-Time Transactions.

    Artificial Intelligence (AI) is reshaping the future of banking and payments. It has moved from a supporting technology to a core driver of growth and innovation. The global AI in banking and payments market is projected to reach $190.33 billion by 2030, reflecting its rapid adoption and transformative potential.

    Recent studies highlight that 86% of financial firms consider AI important to their operations, with the technology expected to unlock $340 billion in annual productivity gains. Adoption is not just theoretical, 70% of financial institutions reported AI-driven revenue growth in 2024, underscoring its tangible impact on the industry.

    This transformation is especially evident in the space of real-time transactions, where speed, security, and customer experience are non-negotiable. As real-time payments become the norm across global financial systems, the role of AI in transactions has expanded from fraud detection to personalized experiences, smarter risk scoring, and automated decision-making. By enabling instant analysis and adaptive responses, AI ensures that financial institutions can handle the demands of today’s fast-paced payment ecosystem, where every second counts, and trust is just as critical as efficiency.

    Why AI for Real-Time Transactions

    The rise of real-time payments is changing how money moves worldwide. Whether it’s peer-to-peer transfers, e-commerce checkouts, cross-border remittances, or securities trading, transactions now happen in milliseconds. This speed brings significant bottlenecks like online frauds, heightened regulatory scrutiny, compliance challenges, and the constant pressure to maintain security without disrupting the customer experience. Traditional systems often struggle to balance these demands, making AI in transactions an essential enabler of safe, efficient, and scalable payments.

    Key Roles of AI in Real-Time Payments

    1. Fraud Detection and Prevention

    AI models analyze behavioral data, device fingerprints, and transaction history in real time. Unlike static systems, they learn continuously to detect new fraud tactics, flagging suspicious activity instantly while allowing legitimate payments to proceed without friction.

    2. Smarter Risk Scoring

    Every transaction can be assigned a dynamic risk score by AI. High-risk transactions are flagged for verification, while low-risk ones move through seamlessly. This approach reduces false positives, improves approval rates, and strengthens customer trust.

    3. Personalized Customer Journeys

    AI in transactions extends beyond security into personalization. Payment platforms can recommend tailored offers, loyalty rewards, or financing options at the point of payment, enhancing both customer satisfaction and business revenue.

    4. Intelligent Automation and Compliance

    AI-powered systems streamline KYC (Know Your Customer) and AML (Anti-Money Laundering) checks, automating tasks that once caused delays. Automated dispute resolution and instant decision-making further improve operational efficiency.

    5. Performance and Scalability

    During spikes such as holiday sales or IPO launches, AI optimizes transaction routing and system performance. Predictive models forecast demand, helping payment providers ensure uptime and reliability.

    Outlook: AI as the Backbone of Real-Time Payments

    Looking ahead, the role of AI will only grow stronger as real-time payments become common universally. When we look at the future of AI in transactions, a few key trends are already starting to take shape, pointing toward a faster, smarter, and more secure payment ecosystem.

    • Explainable AI (XAI): Making AI’s decision-making transparent to regulators and customers.
    • Quantum-Resistant Security: Preparing payments infrastructure for next-gen threats.
    • Autonomous Financial Agents: AI-powered assistants conducting transactions on behalf of individuals or businesses.
    • Cross-Border Real-Time Payments: AI bridging regulatory and compliance gaps between global markets.

    Concluding

    The rise of real-time payments is transforming customer expectations, where speed and trust go hand in hand. AI in transactions is the force making this possible by detecting fraud, ensuring compliance, and keeping payments seamless and secure.

    At R Systems, we are hacking the future of real-time payments with our expertise in AI, data, and cloud engineering. By combining powerful tools and proven frameworks, we enable financial institutions to modernize faster, stay resilient, and deliver intelligent transaction experiences that inspire customer confidence today and tomorrow. Talk to our Expert Now.

  • Securing Card Transactions: The Role of Card Management Systems in Fraud Detection and Prevention

    Card fraud continues to evolve, keeping financial institutions and consumers on high alert. According to the latest predictions from the Nilson Report, global fraud losses in card payments are expected to reach $403.88 billion over the next decade. As card payment volumes surge worldwide, criminals are becoming increasingly sophisticated, ranging from bulk purchases of stolen card data to complex account takeovers and social engineering schemes.

    This isn’t a temporary spike—it’s a permanent shift in the threat landscape. Financial institutions must act with urgency or risk mounting losses and eroding customer trust. That’s where the Card Management System (CMS) comes in. More than just card issuance, a modern CMS serves as the command center for digital payment security, providing real-time authorization controls, tokenization, and integration with fraud detection systems.

    Key Card Management System Modules

    • Product & BIN management (create/configure card products)
    • Authorization rules & real-time limits (velocity, MCC, geography)
    • Tokenization & wallet provisioning connectors (device tokens, network tokens)
    • Fraud orchestration & rules engine (integration with fraud scoring services)
    • Lifecycle management (issuing, reissue, suspend, close)
    • Reporting, reconciliation & regulatory controls (PCI, AML/KYC hooks)

    How Card Management System capabilities map to fraud prevention

    1. Real-time authorization controls and dynamic rules

      A CMS enforces transaction-level rules in milliseconds, blocking suspicious activities before they result in losses. For instance, it can decline a transaction happening in two different countries within minutes or challenge an unusually high purchase with additional authentication.

      2. Tokenization & EMV payment tokens

      Tokenization ensures card numbers are never directly exposed in digital transactions. Instead, tokens tied to devices, merchants, or specific transactions reduce the usability of stolen data. EMV tokenization has become a global standard and is now a critical CMS capability.

      3. Strong Customer Authentication (SCA), 3-D Secure  

      Modern CMS platforms integrate SCA and 3-D Secure protocols, ensuring that high-risk transactions undergo step-up authentication (e.g., biometrics, OTP). Data from the European Banking Authority (EBA) confirms that SCA-protected transactions show significantly lower fraud rates compared to those without SCA.

      4. AI-Driven Fraud Detection

      Modern CMS platforms integrate with ML-driven fraud engines (in-house or third-party) to score  

      Advanced CMS platforms integrate machine learning and behavioral analytics that score transactions in real time. This reduces false positives while increasing fraud detection rates, balancing security with user experience.

      5. Issuer controls exposed to cardholder  

      Two-way controls exposed to customers via mobile apps (instant lock/unlock, merchant category blocks, spend limits, geofencing, virtual card creation) are effective first-line defenses. They reduce the window of exposure for stolen card data and strengthen user trust, and those capabilities are commonly implemented as CMS APIs.  

      6. Customer Empowerment

      Banks are increasingly exposing card control features to customers like instant lock/unlock, category-specific spending, and geo-blocking via mobile banking apps. These CMS-driven features allow cardholders to actively defend against fraud.

      Typical Card Management System architecture patterns that improve security

      • Separation of duties: Distinct services for token vault, auth/risk decisioning, and card lifecycle reduce blast radius.
      • Event-driven authorization pipeline: Use a fast, streamable pipeline to inject real-time risk signals into the CMS before authorisation responses are returned.
      • Secure, auditable key & credential management: Store keys in HSMs; use role-based access and rotate keys per policy to meet PCI and regulatory expectations.  
      • Token first, minimal PAN storage: Design systems so PANs are exchanged only at trusted boundaries and replaced with tokens in the CMS database.
      • Multi-factor flows & step-up authentication: Integrate SCA / 3-D Secure / device attestation so the CMS can require extra proof for risky transactions.

      Best Practices for Financial Institutions

      1. Adopt a token-first approach: Store PANs only in secure vaults, use tokens everywhere else.
      2. Integrate ML fraud engines: Blend rule-based controls with real-time analytics.
      3. Enable customer controls: Empower users with simple security features in mobile apps.
      4. Ensure regulatory compliance: Stay aligned with PCI DSS v4.0 and regional mandates like PSD2.
      5. Regularly update rule sets: Fraud evolves quickly, static rules are ineffective.

      Conclusion

      Card fraud is no longer a background risk, it’s a frontline battle in digital banking. Financial institutions that fail to act decisively will not only suffer financial losses but also lose customer trust, which is far harder to rebuild.

      A Card Management System is no longer just about issuing and managing cards, it is the nerve center of digital payment security. With real-time authorization controls, tokenization, integration with AI-driven fraud engines, and customer-facing controls, a modern CMS equips financial institutions to stay ahead of fraudsters.

      At R Systems, we help banks, Fintechs, and payment providers modernize their payment ecosystems with next-generation Card Management Systems. Our expertise spans:

      • Global gateway integrations
      • GenAI-driven onboarding accelerators for faster time-to-market
      • PCI-compliant mobile and web SDKs for secure checkout
      • Optimized payment routing and higher transaction success rates
      • AI-led fraud detection and orchestration to minimize risk
      • Actionable analytics unlocking additional revenue from payments data

      With proven payments engineering capabilities, R Systems enables institutions to strengthen digital payment security, reduce fraud exposure, and deliver trusted customer experiences at scale. Talk to our Experts Now.

    1. Beyond Cost Control: 3 Ways FinOps Powers Growth and Agility

      When most executives hear the term FinOps, they think about cost control. They imagine a team combing through invoices, cutting unused resources, and negotiating discounts. That is part of the story, but not the whole picture. In reality, FinOps is not just about saving money, it is about enabling growth, innovation, and agility in a cloud-driven world.

      Cloud has given organizations unprecedented flexibility to scale infrastructure and deploy new features. But that same flexibility often leads to overspending, waste, and inefficiency. A recent study suggests that up to 30% of cloud spend is wasted, often because of idle resources, lack of visibility, or poor alignment between finance and engineering. For business leaders, this isn’t just a budget concern. Every dollar wasted represents engineering time lost, product releases delayed, and innovation deferred.

      That’s where FinOps comes in.

      At its core, FinOps (short for Cloud Financial Operations) is about bringing finance, technology, and business together to make smarter decisions. It aligns spending with business impact, provides the visibility leaders need to prioritize, and frees up capital that can be reinvested in research, new capabilities, and market expansion. In other words: FinOps transforms cloud from a cost center into a growth engine.

      Why Cost Alone is the Wrong Lens

      Organizations often approach FinOps with a narrow goal: reduce the cloud bill. While cutting unnecessary spend is important, it is only the starting point. If FinOps stops there, companies miss its real value.

      Cloud waste isn’t just a financial inefficiency. It limits engineering capacity by tying up budgets in unused services. Teams hesitate to experiment with new tools because they lack clarity on budget trade-offs. Finance departments, worried about ballooning costs, become blockers instead of enablers.

      By reframing FinOps from cost-cutting to growth-enabling, leaders unlock new opportunities. Strategic savings are not about trimming fat for the sake of it rather, they are about reallocating resources to what matters most: innovation, customer experience, and market differentiation.

      How FinOps Turns Cloud Savings into Business Growth:

      1. Visibility that Powers Better Decisions

      FinOps provides transparency into cloud usage across teams, applications, and business units. This isn’t just about dashboards; it’s about understanding the link between cloud spend and business outcomes. When leaders can see which workloads drive revenue, which experiments pay off, and which services drain resources without returns, they can prioritize effectively.

      This visibility ensures that every dollar spent is an investment, not just an expense.

      2. Aligning Finance and Engineering

      In traditional IT, finance and engineering often operate at odds. Finance wants predictability, engineering wants speed. FinOps bridges the gap by creating a shared language of value. With the right governance, engineering teams gain freedom to innovate while finance gains confidence in the ROI.

      The result: finance shifts from being a gatekeeper to a trusted business partner.

      3. Reinvesting in Innovation

      Perhaps the most overlooked benefit of FinOps is the capacity it creates. Strategic cost optimization frees up capital that can be redirected into R&D, new product lines, and scaling operations. In competitive industries, this reinvestment can be the difference between leading and lagging.

      A Case in Point: Growth Through FinOps

      At R Systems, we recently worked with a leading healthcare supply chain provider that faced mounting cloud costs. The client was concerned not only about overspending, but also about delayed innovation. Their teams struggled to balance cost control with the need to modernize their supply chain systems.

      Through our Cloud Cost Governance framework, we implemented a FinOps strategy that combined cost visibility, workload optimization, and cross-team accountability. Within a year, the client cut annual cloud costs by 20%.

      But here is the real story: the savings weren’t simply pocketed. They were reinvested into innovation projects that modernized logistics operations and improved service delivery for healthcare providers nationwide. What began as a cost exercise became a growth initiative.

      This is the essence of FinOps. It is not just about efficiency rather it is about fueling transformation.

      Read the full story here- Driving Supply Chain Efficiency with Cloud Cost Governance – R Systems

      The R Systems Advantage in Cloud FinOps

      FinOps is not a one-time project. It is a continuous discipline that requires the right mix of process, culture, and technology. At R Systems, we bring this holistic view to every client engagement.

      • FinOps Cloud Cost Management: We help enterprises gain real-time visibility into spend and align costs with business outcomes.
      • FinOps Cost Optimization: Our frameworks reduce waste while ensuring teams have the resources they need to innovate.
      • FinOps as a Service: We deliver ongoing governance and automation, so FinOps practices evolve with the business.
      • Cloud Financial Management Expertise: With decades of experience in cloud engineering and enterprise IT, we design programs that balance growth with governance.

      Our approach is rooted in collaboration. We don’t just analyze numbers; we empower cross-functional teams to make informed, agile decisions. By embedding FinOps into daily operations, organizations unlock both cost savings and growth potential.

      For more on our approach, visit our Cloud FinOps page.

      Looking Ahead: FinOps as the New Normal

      The pace of digital transformation will only accelerate. Cloud adoption is no longer about “if” but “how fast” and “how smart.” In this context, FinOps will become a standard operating model for high-performing organizations.

      The companies that thrive will be those that treat FinOps not as a defensive measure, but as an offensive strategy. They will use FinOps to fund innovation, empower engineers, and turn finance into a growth partner.

      As Gurpreet Singh aptly wrote, FinOps is not about cutting costs, but about making the right costs. And as DNX Solutions reminds us, it is about moving beyond traditional cost management to create value.

      At R Systems, we believe the future of FinOps lies in this growth-oriented mindset. The organizations we work with are not just trimming expenses—they are building the capacity to innovate faster, scale smarter, and compete stronger.

      What to do next?

      If your organization views FinOps purely as a cost-cutting exercise, it’s time to rethink. The real opportunity is to harness FinOps as a growth enabler. By combining visibility, alignment, and reinvestment, you can transform your cloud strategy from reactive control to proactive innovation.

      R Systems can help you get there. Our Cloud FinOps services are designed to unlock both savings and scale, so you can invest confidently in the future.

      The question is not whether you need FinOps.

      The question is whether you will use it to cut costs, or to fuel growth.

      The choice is yours. Let’s build the future of cloud together.

      Start the journey — talk to our Cloud FinOps experts today.

    2. Shebang Your Shell Commands with GenAI using AWS Bedrock

      Generative AI (GenAI) is no longer a mystery—it’s been around for over two years now. Developers are leveraging GenAI for a wide range of tasks: writing code, handling customer queries, powering RAG pipelines for data retrieval, generating images and videos from text, and much more.

      In this blog post, we’ll integrate an AI model directly into the shell, enabling real-time translation of natural language queries into Linux shell commands—no more copying and pasting from tools like ChatGPT or Google Gemini. Even if you’re a Linux power user who knows most commands by heart, there are always moments when a specific command escapes you. We’ll use Amazon Bedrock, a fully managed serverless service, to run inferences with the model of our choice. For development and testing, we’ll start with local model hosting using Ollama and Open WebUI. Shell integration examples will cover both Zsh and Bash.

      Setting up Ollama and OpenWebUI for prompt testing

      1. Install Ollama

      curl -fsSL https://ollama.com/install.sh | sh

      2. Start Ollama service

      systemctl enable ollama && systemctl start ollama

      By default, Ollama listens on port 11434. If you’re comfortable without a user interface like ChatGPT, you can start sending prompts directly to the /api/generate endpoint using tools like curl or Postman. Alternatively, you can run a model from the shell using:

      ollama run <model_name>

      3. Install open web ui

      At this step we assume that you have python pip installed.

      pip install open-webui

      Now that Open WebUI is installed, let’s pull a model and begin prompt development. For this example, we’ll use the mistral model locally

      4. Pull mistral:7b or mistral:latest model

      ollama pull mistral:latest

      5. Start Open-WebUI server

      open-webui serve

      This starts the Open WebUI on the default port 8080. Open your favorite web browser and navigate to http://localhost:8080/. Set an initial username and password. Once configured, you’ll see an interface similar to ChatGPT. You can choose your model from the dropdown in the top-left corner.

      Testing the prompt in Open-WebUI and with API calls:

      Goal:

      • User types a natural language query
      • Model receives the input and processes it
      • Model generates a structured JSON output
      • The shell replaces the original query with the actual command

      Why Structured Output Instead of Plain Text?

      You might wonder—why not just instruct the model to return a plain shell command with strict prompting rules? During testing, we observed that even with rigid prompt instructions, the model occasionally includes explanatory text. This often happens when the command in question could be dangerous or needs caution.

      For instance, the dd command can write directly to disk at a low level. Models like Mistral or Llama may append a warning or explanation along with the command to prevent accidental misuse. Using structured JSON helps us isolate the actual command cleanly, regardless of any extra text the model may generate.

      The Prompt:

      You are a linux system administrator and devops engineer assistant used in an automated system that parses your responses as raw JSON.
STRICT RULES:
- Output MUST be only valid raw JSON. Do NOT include markdown, backticks, or formatting tags.
- NO explanations, no introductory text, and no comments.
- If no suitable command is found, output: {"command": "", "notes": "no command found", "status": "error"}
- Output must always follow this exact schema:
{
    "command": "<actual Linux command here>",
    "notes": "<if applicable, add any notes to the command>",
    "status": "success/error"
}
- Any deviation from this format will result in system error.
Respond to the following user query as per the rules above:
<Query Here>

      Let’s test it with the query:
      “start nginx container backed by alpine image”

      And here’s the structured response we get:

      {
"command": "docker run -d --name my-nginx -p 80:80 -p 443:443 -v /etc/nginx/conf.d:/etc/nginx/conf.d nginx:alpine",  
"notes": "Replace 'my-nginx' with a suitable container name.",  
"status": "success"
}

      Bingo! This is exactly the output we expect—clean, structured, and ready for direct use.

      Now that our prompt works as expected, we can test it directly via Ollama’s API.

      Assuming your payload is saved in /tmp/payload.json, you can make the API call using curl:

      {
  "model": "phi4:latest",
  "prompt": "You are a linux system administrator and devops engineer assistant used in an automated system that parses your responses as raw JSON.nSTRICT RULES:n- Output MUST be only valid raw JSON. Do NOT include markdown, backticks, or formatting tags.n- NO explanations, no introductory text, and no comments.n- If no suitable command is found, output: {"command": "", "notes": "no command found", "status": "error"}n- Output must always follow this exact schema:n{n    "command": "<actual Linux command here>",n    "notes": "<if applicable, add any notes to the command>",n    "status": "success/error"n}n- Any deviation from this format will result in system error.nRespond to the following user query as per the rules above:nstart nginx container backed by alpine image",
  "stream": false
}

      curl -d @/tmp/payload.json -H 'Content-Type: application/json' 'http://localhost:11434/api/generate'

      Note: Ensure that smart quotes (‘’) are not used in your actual command—replace them with straight quotes (”) to avoid errors in the terminal.

      This allows you to interact with the model programmatically, bypassing the UI and integrating the prompt into automated workflows or CLI tools.

      Setting up AWS Bedrock Managed Service

      Login to the AWS Console and navigate to the Bedrock service.

      Under Foundation Models, filter by Serverless models.

      Subscribe to a model that suits code generation use cases. For this blog, I’ve chosen Anthropic Claude 3.7 Sonnet, known for strong code generation capabilities.
      Alternatively, you can go with Amazon Titan or Amazon Nova models, which are more cost-effective and often produce comparable results.

      Configure Prompt Management

      1. Once subscribed, go to the left sidebar and under Builder Tools, click on Prompt Management.

      ‍

      2. Click Create prompt and give it a name—e.g., Shebang-NLP-TO-SHELL-CMD.

      3. In the next window:

      • Expand System Instructions and paste the structured prompt we tested earlier (excluding the <Query Here> placeholder).
      • In the User Message, enter {{question}} — this will act as a placeholder for the user’s natural language query.

      4. Under Generative AI Resource, select your subscribed model.

      5. Leave the randomness and diversity settings as default. You may reduce the temperature slightly to get more deterministic responses, depending on your needs.

      6. At the bottom of the screen, you should see the question variable under the Test Variables section.
      Add a sample value like: list all docker containers

      7. Click Run. You should see the structured JSON response on the right pane.

      8. If the output looks good, click Create Version to save your tested prompt.

      Setting Up a “Flow” in AWS Bedrock

      1. From the left sidebar under Builder Tools, click on Flows.

      ‍

      2. Click the Create Flow button.

      • Name your flow (e.g., ShebangShellFlow).
      • Keep the “Create and use a new service role” checkbox selected.
      • Click Create flow.

      Once created, you’ll see a flow graph with the following nodes:

      • Flow Input
      • Prompts
      • Flow Output

      Configure Nodes

      • Click on the Flow Input and Flow Output nodes.
        Note down the Node Name and Output Name (default: FlowInputNode and document, respectively).
      • Click on the Prompts node, then in the Configure tab on the left:
        • Select “Use prompt from prompt management” 
        • From the Prompt dropdown, select the one you created earlier.
        • Choose the latest Version of the prompt.
        • Click Save.
      Test the Flow

      You can now test the flow by providing a sample natural language input like:

      list all docker containers

      Finalizing the Flow

      1. Go back to the Flows list and select the flow you just created.

      2. Note down the Flow ID or ARN.

      3. Click Publish Version to create the first version of your flow.

      4. Navigate to the Aliases tab and click Create Alias:

      • Name your alias (e.g., prod or v1).
      • Choose “Use existing version to associate this alias”.
      • From the Version dropdown, select Version 1.
        Click Create alias.

      5. After it’s created, click on the new alias under the Aliases tab and note the Alias ARN—you’ll need this when calling the flow programmatically.

      Shell Integration for ZSH and BASH

      Configuring IAM Policy

      To use the Bedrock flow from your CLI, you need a minimal IAM policy as shown below:

      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Statement1",
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeFlow"
      ],
      "Resource": "<flow resource arn>"
    }
  ]
}

      Attach this policy to the IAM user whose credentials you’ll use for invoking the flow.

      Note: This guide does not cover AWS credential configuration (e.g., ~/.aws/credentials).

      Bedrock Flow API

      AWS provides a REST endpoint to invoke a Bedrock flow:

      `/flows/<flowIdentifier>/aliases/<flowAliasIdentifier>`

      You can find the official API documentation here:
      InvokeFlow API Reference

      To simplify request signing (e.g., AWS SigV4), language-specific SDKs are available. For this example, we use the AWS SDK v3 for JavaScript and the InvokeFlowCommand from the @aws-sdk/client-bedrock-agent-runtime package:

      SDK Reference:
      https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/bedrock-agent-runtime/command/InvokeFlowCommand/

      Required Parameters

      You’ll need to substitute the following values in your SDK/API calls:

      • flowIdentifier: ID or ARN of the Bedrock flow
      • flowAliasIdentifier: Alias ARN of the flow version
      • nodeName: Usually FlowInputNode
      • content.document: Natural language query
      • nodeOutputName: Usually document

      Shell Script Integration

      The Node.js script reads a natural language query from standard input (either piped or redirected) and invokes the Bedrock flow accordingly. You can find the full source code of this project in the GitHub repo:
      https://github.com/azadsagar/ai-shell-helper

      Environment Variables

      To keep the script flexible across local and cloud-based inference, the following environment variables are used:

      INFERENCE_MODE="<ollama|aws_bedrock>"

# For local inference
OLLAMA_URL="http://localhost:11434"

# For Bedrock inference
BEDROCK_FLOW_IDENTIFIER="<flow ID or ARN>"
BEDROCK_FLOW_ALIAS="<alias name or ARN>"
AWS_REGION="us-east-1"

      Set INFERENCE_MODE to ollama if you want to use a locally hosted model.

      Configure ZSH/BASH shell to perform magic – Shebang

      When you type in a Zsh shell, your input is captured in a shell variable called LBUFFER. This is a duplex variable—meaning it can be read and also written back to. Updating LBUFFER automatically updates your shell prompt in place.

      In the case of Bash, the corresponding variable is READLINE_LINE. However, unlike Zsh, you must manually update the cursor position after modifying the input. You can do this by calculating the string length using ${#READLINE_LINE} and setting the cursor accordingly. This ensures the cursor moves to the end of the updated line.

      From Natural Language to Shell Command

      Typing natural language directly in the shell and pressing Enter would usually throw a “command not found” error. Instead, we’ll map a shortcut key to a shell function that:

      • Captures the input (LBUFFER for Zsh, READLINE_LINE for Bash)
      • Sends it to a Node.js script via standard input
      • Replaces the shell line with the generated shell command

      Zsh Integration Example

      In Zsh, you must register the shell function as a Zsh widget, then bind it to a shortcut using bindkey.

      function ai-command-widget() {
  alias ai-cmd='node $HOME/ai-shell-helper/main.js'

  local input
  input="$LBUFFER"
  local cmdout
  cmdout=$(echo "$input" | ai-cmd)

  # Replace current buffer with AI-generated command
  LBUFFER="$cmdout"
}

# Register the widget
zle -N ai-command-widget

# Bind Ctrl+G to the widget
bindkey '^G' ai-command-widget

      Bash Integration Example

      In Bash, the setup is slightly different. You bind the function using the bind command and use READLINE_LINE for input and output.

      ai_command_widget() {
  local input="$READLINE_LINE"
  local cmdout
  cmdout=$(echo "$input" | node "$HOME/ai-shell-helper/main.js")

  READLINE_LINE="$cmdout"
  READLINE_POINT=${#READLINE_LINE}
}

# Bind Ctrl+G to the function
bind -x '"C-g": ai_command_widget'

      Note: Ensure that Node.js and npm are installed on your system before proceeding.

      Quick Setup

      If you’ve cloned the GitHub repo into your home directory, run the following to install dependencies and activate the integration:

      cd ~/ai-shell-helper && npm install

# For Zsh
echo "source $HOME/ai-shell-helper/zsh_int.sh" >> ~/.zshrc

# For Bash
echo "source $HOME/ai-shell-helper/bash_int.sh" >> ~/.bashrc

      Then, start a new terminal session.

      Try It Out!

      In your new shell, type a natural language query like:

      list all docker containers

      Now press Ctrl+G.
      You’ll see your input replaced with the actual command:

      docker ps -a

      And that’s the magic of Shebang Shell with GenAI!

      Demo Video: