I have been working in deploying large scale web applications to the cloud since 2009 when we deployed our first application to Amazon EC2. Since that time, I have been very interested in the business of cloud services. Most recently my passion has been in the area of containers and container management along the lines of Docker, Rkt, Kubernetes and Etcd. The Go language has been an enabler in this new field of application services and that is where I have focused most of my attention in the area of product development and understanding.
The interface into these containerized systems is very exciting and important in the next phase of operating system design and evolution. By taking a detailed look at libcontainer and appc one can begin to see the future of where operating systems are moving to. No longer will you need a heavy weight system to run these interfaces, but rather a lightweight software layer that sits on top of the hardware to enable this functionality.
Both Etcd and Redis are very important concepts for managing state within and among containers. Recently, I got a Redis cluster up and running and its marvelous. Salvatore, the founder of Redis, did a great job with cluster, and its nice to see it now in production finally, after more than 3 years of hard work on it. I am excited about Redis cluster. One of my main expertises is working with state machines across multiple instances. Redis cluster and Etcd are two examples of this Consensus. An implementation of the Consensus concept is Raft. I have been working intensely with the Golang implementation of Raft called Etcd. I am not a developer on the Github project but I have been studying it and using it in other projects. Redis cluster is yet another example of a very similar concept. If you review the Cluster Spec you will see the similarities between Raft and Redis Cluster.
The work I am doing involves large scale applications using Redis at the heart of its architecture. I was the first engineer hired at Spinnakr post our funding round and I was employed there until January 2015 when we decided to scale back our operations. I built out our entire Redis infrastructure and was the person who decided we need to use Redis. I architected a solution that scaled well, and is still in production today. The instances we have running on Amazon have run without failure now for over a year without any hiccups. At peak we were handling over 1 Million requests per day on about 30 Amazon instances. I designed, architected, deployed, and maintained our complete Redis Amazon infrastructure