DevOps

LXD: The Missing Piece

LXD stands for Linux Container Daemon. Yet another container technology. But LXD is very different. It stands apart from the pack. It is not necessarily better nor much faster nor more secure! But it resolves an issue that other containers doesn't. Many of us moved too fast from traditional Virtual Machines to Application containers because [...]

By |2018-12-28T14:21:04+00:00December 28th, 2018|Categories: Container, DevOps|Tags: , , |2 Comments

Monitoring a production Hadoop cluster with Kubernetes

Monitoring a production grade Hadoop cluster is a real challenge and needs to be constantly evolving. The software we use today is based on Nagios. Very efficient when it comes to the simplest surveillance, it is not able to meet the need for a more complex verification. In this article, we will propose an architecture [...]

Microsoft introduces Cloud Native Application Bundles

At DockerCon EU 2018 in Barcelona, Matt Butcher, Principal Engineer at Microsoft and inventor of Helm, introduced CNAB, Cloud Native Application Bundles, a packaging format for distributed applications, along with Duffle, a CLI tool to run these bundles. […]

By |2018-12-05T10:21:00+00:00December 4th, 2018|Categories: Container, DevOps|Tags: , , , |0 Comments

A CoreOS development cluster with Vagrant and VirtualBox

Following CoreOS’s instructions on how to set up a development environment in VirtualBox did not work out well for me. Here are the steps I followed to get Container Linux up and running with Vagrant. […]

By |2018-06-21T10:49:35+00:00June 20th, 2018|Categories: Container, DevOps|Tags: , , , |0 Comments

Data Lake ingestion best practices

Creating a Data Lake requires rigor and experience. Here are some good practices around data ingestion both for batch and stream architectures that we recommend and implement with our customers. […]

By |2018-06-18T09:29:50+00:00June 18th, 2018|Categories: Data Engineering, DevOps|Tags: , , , , , , , |1 Comment

Hive Metastore HA with DBTokenStore: Failed to initialize master key

This article describes my little adventure around a startup error with the Hive Metastore. It shall be reproducable with any  secure installation, meaning with Kerberos, with high availability enabled and with the storage of the delegation token in a database. The version of Hive is the 1.2 packaged inside the Hortonworks 2.4.2 distribution. Storage for [...]

By |2019-06-18T21:53:47+00:00July 21st, 2016|Categories: Big Data, DevOps|Tags: , , |0 Comments