Installing CoreOS+Kubernetes stack by yourself is not a big deal, there are plenty examples how to do that on different cloud providers, local hypervisors and bare-metal.

But the problem comes when you want to setup all these extra bits to support your cluster (CI/CD pipeline):

• git pull scripts
• docker builder
• DNS
• roll out and roll back of releases
• etc

Kubernetes by itself does not take care of it. And building and supporting all these extras is not so easy, it is really time consuming.

I'm running CoreOS clusters in production since June 2014, and I had to build all that PaaS stuff (based on fleet) by myself. It was a fun in the beginning, lots of new things got learned but in the long run it became a pain to maintain and support. Last year June Kubernetes came along but was really way too early to put it to production.

I was playing and messing with Kubernetes for a good while, which really does a good job of containers orchestration, but it is still not the completed solution. There are a few commercial solutions like Tectonic.com by CoreOS and Kismatic.com, but they are more for enterprise users.

For the smaller companies I see a very good open source option from Deis.io, they also provide commercial option (free for the preview time.) Deis.com. Deis is part of Engine Yard. Deis has all these extra bits already in place, so it really looks as a good option then.

So let's get to the point :) On July 5th, Deis v1.9 was released which has Kubernetes as an optional scheduler, so I gave a shot first running it on GCE, which was a breeze to install it and was really surprised that you do not have to know about Kubernetes at all. Just follow usual Deis release process and the rest will be taken care for you by Deis, does not matter which scheduler you use there.

As I'm a big fan of Packet.net hosted bare-metal servers, where server gets provisioned in about 5 minutes (even quicker that on AWS), so I decided give Deis shot there. Deis is based on CoreOS, so usually it needs a dedicated etcd cluster (min 3 machines), a few machines to run Deis Control Plane which performs management functions for the platform. And worker machines which run Data Plane or/and Router Mesh. I will not go deep to explaining about Deis architecture over here, they have really nice documentation, and one can read by himself.

So I spun up four machines on Packet.net:

• One was dedicated to run standalone etcd cluster and Deis Control Plane. As bare-metal server is too good to be wasted just for etcd cluster only. I know 1 machine does not give you good HA, but in some cases having a small clusters it is acceptable.
• Three machines to run Data Plane and Router Mesh

I just followed instructions how to install on bare-metal and how to isolate etcd and the Planes

• When machines were up just followed [Installation doc] (http://docs.deis.io/en/latest/installing_deis/install-platform/#install-deis-platform) and Kubernetes scheduler install
• And using Deis

So now I was able to run tests, scale my web apps with just a simple deis scale cmd=3 command and etc. Deis.io really brings Kubernetes to masses in an easy way of its usage.

Big thanks Deis.io team for such awesome system. Big thanks to Google for Kubernetes Big thanks to Packet.net for supporting my experiments and I cannot wait to have Packet.net data centres in Europe.

That is my short story of my weekend hacking Deis and bare-metal servers.