Run Apache NiFi Cluster in Docker with SSL Enabled

Welcome to the fourth article in the series of Apache NiFi. The last article explained how to set up an Apache NiFi Docker container with a self-signed SSLcertificate. This article addresses the next pain point: how to create an Apache NiFi cluster in Docker with SSL enabled. Unlike HTTP cluster, setting up Apache NiFi cluster with SSL enabled in Docker introduces a new challenge: Hostname verification.

For added security, if HTTPS connection is enabled, Apache NiFi will verify the Hostname of requests. Therefore each request sent to Apache NiFi must have a predefined hostname. Not only the external requests, but peer-to-peer communication of NiFi nodes in a cluster also go through HTTPS and are subject to hostname verification. If the hostname provided in the HTTPS request does not match the hostname defined in the SSL certificate, NiFi will throw a javax.net.ssl.SSLPeerUnverifiedException.

If you are traditionally deploying Apache NiFi: individual servers with known IP addresses, it is easy to create certificates with those IP addresses. However, in a dynamic environment like Docker, the hostname of a container is defined at the runtime if you need flexible scaling options. Since Docker doesn't provide an option to define the hostname pattern in a scalable cluster, we have to stick to hard-coded Apache NiFi containers with predefined hostnames to create a cluster. The disadvantage of this method is that it is hard to scale up/down a cluster with hard-coded containers. Instead, you can also set up an HTTP cluster and create a load balancer with HTTPS frontend and SSL Termination between the client and NiFi UI. However, in this article, we will stick to the SSL configuration at the cluster level.

Read More

Run Apache NiFi in Docker with SSL Enabled

The last two articles in the Apache NiFi series discussed how to run Apache NiFi standalone server and NiFi cluster in Docker. However, those are far from production-ready because they are not secured. The next step in setting up a secured NiFi cluster is spinning up an Apache NiFi instance with SSL enabled in Docker. Though we are moving towards production-ready, this article will use self-signed certificates. In production, you should not use a self-signed certificate. In addition, you may also require additional safety measures like firewall and proxy.

Read More

Run Apache NiFi Cluster in Docker

The last article on Apache NiFi: Run Apache NiFi in Docker was for those who want to start playing with Apache NiFi. Though it was a good start to play with NiFi, it is far from production deployment. This article introduces the second stage of deployment: a NiFi cluster running in Docker using Docker Compose.

To begin with, you must have Docker installed in your system and also install Docker Compose as we are going to use Docker Compose to setup the Apache NiFi cluster. 

Read More

Resume Tips for Software Engineers in North America

Years ago as an international student preparing my resume to get my first job in Canada, I had a lot of questions and I did search a lot on how to make a resume that fits Canadian employer's requirements and style. Things have changed. Eventually, I got offers from some high tech companies and landed in a good job and now I am interviewing candidates who are like I was a couple of years ago. After looking at more and more resumes, I decided to share my experience here with a hope that will make someone's life better.

For whom this article is for? Well! for anyone looking for a new job in Canada. Especially if you are  new immigrant who has no idea about what Canadian employers are looking for, this article is tailored to your requirements. Though my experience is limited to Canada the companies I applied for are mostly US-based companies so I hope it can be applied anywhere in North America. This article is targeting only those who are in the software industry. I don't know how much it will overlap with other industries. The rest of the article is divided into two topics: 1. Resume Sections, 2. Resume Format. The first topic covers what to include and not to include in your resume and the second topic provides some formatting tips to make your resume get you a call from the recruiter.

Read More