Building a PaaS with Docker, Consul and Pyth
I'm currently single-handedly building a Database-As-A-Service platform. And the problem of where and how to run database instances has been successfully solved in favor of Docker. But there still remain 2 problems: how to link containers on different hosts across the network and how to orchestrate a large pool of running instances.
You may find lots of different IaaS offerings and schedulers like Mesos and Kubernetes on the market, but all of them have a steep learning curve. Plus they don't handle stateful and non-SOA services very well.
In this talk I'll be walking you over how one can build a simple PaaS based on Docker, consul and a custom orchestration service in Python. My experience in building such system shows that it can be reproduced by the effort of one developer.
A list of topics I will cover:
- Consul as a reliable distributed storage
- Connecting containers between hosts without an overlay network
- Allocating IP addresses and maintaining their pool in Consul KV
- Correctly registering services and health checks
- How to approach writing basic orchestrator based on gevent + writing trivial admin UI for it
- Adding failover, scaling and service reallocation to the orchestrator
- Delayed tasks (backup, analytics, etc...)
- Monitoring the system (Consul + Prometheus)
Almost all code of the system, including ansible playbooks, is available on GitHub.
Konstantin Nazarov is an infrastructure engineer in Tarantool team. In the past was a system programmer and a fan of Common Lisp. Minimalist and adept of iterative approach to everything.