Micro Services

Micro Services

Do you like our work......we hire!

Never miss our publications about Open Source, big data and distributed systems, low frequency of one email every two months.

Back in the days, applications were monolithic and we could use an IP address to access a service. With virtual machines (VM), multiple hosts started to appear on the same machine with multiple apps. Things were still similar with VMs and physical machines as services were still accessible from an IP.

With MicroServices, things changed and people were not prepared.

With Mesos or Kubernetes, applications are split into many services with many replicas. A bunch of services make an application. Applications may share a same IP and physical hosts and use a different port. They could also use random IP, random ports and start/stop at any time… Changes became the only constant.

Service Discovery

How can we keep our service registry updated? DNS instantiated between servers? How do we setup gateways? …

Following the Wikipedia definition, service discovery is the automatic detection of devices and services offered by these devices on a computer network. A service discovery protocol (SDP) is a network protocol that helps accomplish service discovery.

Main Requirements are:

  • Fast
  • Up to date
  • Resilient

Those are typical requirements, but it’s not enough:

  • Don’t communicate with unhealthy services
  • Subsettings, limiting the pool of potential backend tasks with which a client task interacts
  • Load distribution smarter than round robin with better optimization (for example based on host capacity, neighbours, responsibilities)
  • Minimize waste of resources
  • Protect individual tasks against overload implemented at the communication layer (not necessary in the app)


The role of the proxy (for example Nginx, Marathon-lb) is to transparently forward connections from the static service port exposing the service to the outside world into the dynamically assigned host/port of the contained service.


  • Single point of change, just update the proxy configuration
  • No in app dependencies, no need to update the app
  • Fine-grained load balancing



Custom SRV records into the DNS are used to route the traffic into the containers.


  • No hops
  • No SPOF - works with stale data
  • Protocol independent


  • In-app dependency, if we need to change anything (e.g. in load balancing), the application needs reconfiguration and eventually redeployment
  • Local load balancing, no control on load distribution, no prevention of denial of service
  • Cache invalidation

Service Mesh

A service mesh is a dedicated infrastructure layer for making service-to-service communication more secured, fast, reliable, observable and manageable.

It focuses on solving communication problems between containers. It addresses routing, rerouting with graceful degradation as services fail, and secure inter-service communication. In traditional apps, this logic is built directly into the application.

As architectures become increasingly segmented into multiple apps and multiple in-app services, moving communication logic out of the application eases the development and integration process while providing more resilient functionalities. Just as applications shouldn’t be writing their own TCP stack, they also shouldn’t be managing their own load balancing logic, or their own service discovery management, or their own retry and timeout policies. With service mesh, the application code doesn’t need to know about network topology, service discovery, load balancing and connection management logic.

It consists of two parts:

  • Sidecar
  • Controller

The sidecar is like a plug-in for a service. It is a utility container which runs next to a container with a light coupling between the two. A sidecar container is reusable and can be paired with numerous containers. It is responsible for:

  • Service discovery
  • Health checking, register/de-register
  • Routing and load balancing: when two services need to talk, they talk to the sidecar
  • Authentication and Authorization (AuthN/Z), refusing creating connections
  • Protocol abstraction, transparent TLS encryption, HTTP/1 to HTTP/2 upgrade
  • Metrics/tracing, success rates, request volumes, and latencies

The controller is the brain of the system, responsible for managing and configuring proxies to route traffic, as well as enforcing policies at runtime. It gathers the data from the sidecar, get all information and notify the sidecar about how they should behave, talk to the scheduler about what instances are running, which ones should be up or down… It also takes care of deployment.


  • No load balancing
  • No circuit breaking/repeating policy
  • No tracing

At the time of this writing, Linkerd and Istio are two open source projects which are considered matured.

Linkerd can run on Kubernetes, DC/OS, and a cluster of host machines. It is part of the Cloud Native Computing Foundation, built on top of Finagle and can use gRPC. Istio currently runs only on Kubernetes. In Istio, the sidecar container is called Data Plane (based on Envoy service proxy) and the controller is called the Plane Controller.

Share this article

Canada - Morocco - France

We are a team of Open Source enthusiasts doing consulting in Big Data, Cloud, DevOps, Data Engineering, Data Science…

We provide our customers with accurate insights on how to leverage technologies to convert their use cases to projects in production, how to reduce their costs and increase the time to market.

If you enjoy reading our publications and have an interest in what we do, contact us and we will be thrilled to cooperate with you.

Support Ukrain