Genesis is an open source deployment toolchain for at-scale Cloud Foundry and BOSH deployments. Genesis lowers barriers of entry for new Cloud Foundry and BOSH operators, and provides them with solid Day 2 operations practices.
Operations teams that use Genesis see an increase in productivity, and (in our opinion) come to appreciate the Cloud Foundry platform even more.
The Genesis Topology
With Genesis, you will plan and implement a set of one or more environments, plus an operations tier.
Go ahead and click around on the diagram above to dive deeper into each component!
The operations tier acts as a control plane for all of the other environments, and includes some specialty bits of infrastructure that we don't deploy anywhere else. Here you'll find our Vault (credentials store), our Bastion Host, the global Concourse CI/CD installation, and the proto-BOSH.
In this diagram, we have one operations tier (in the middle there), and four environments of increasing importance:
- us-east-1-sandbox - A (small) environment where operations staff play.
- us-east-1-dev - A (usually quite large) production-like environment for platform consumers (i.e. developers) to test out their wild ideas.
- us-east-1-qa - A production-like environment for integration tests, quality control, and pre-production deployment of platform and applications.
- us-east-1-prod - Production. The thing that shall not go offline. Where the real money is made.
The first thing deployed in the operations tier, the
Bastion Host provides a jumping-off point for
all of the rest of the deployment. From here, armed with the
genesis program, you will stand up the rest of your
control plane; the Vault, the Concourse, and the proto-BOSH.
Before you can deploy things for real, you're going to need a place to store all of those sensitive passwords, encryption keys, X.509 certificates, CA signing keys, etc. For that, we use a small, 3-node Consul-backed Vault.
Scaling Cloud Foundry, BOSH, and the rest of our deployments across a multitude of environments eventually leads you to automate the easy (and often mundane) tasks of managing updates and keeping everything humming along smoothly.
BOSH is the backbone of the Genesis deployment story. It is single-handedly responsible for translating our requests for environments into commands for the chosen IaaS (AWS, vSphere, Azure, GCP, etc.) to carry out. The upshot: you get virtual machines, for next to no work.
You're going to want to protect the data that all these systems start creating. For that, Genesis provides SHIELD, a Data Protection solution that will schedule your backups, encrypting each of them with randomized key material, and provides a simple and easy-to-follow process for restoring systems when you do eventually lose data.
To monitor the health and well-being of all the running components of Cloud Foundry, both internal pieces like Diego cell capacity and Cloud Controller API response times, and external bits like disk availability on your blobstore solutions. Each environment gets its own Prometheus instance, chock full of helpful data visualization dashboards to keep you apprised of how things look.
With Cloud Foundry, you have the option to use an external storage system for application and package blobs. If you want, you can run your own S3-like system on-premise; we use Minio.
This is it. THE platform for your agile application development transformation. Each Genesis environment almost inevitably culminates in one of these, surrounded by other services that outfit, extend and complement.
An application without any data is not much of an application. Whether you want to store your important information in a relational SQL database like MariaDB or PostgreSQL, or if you prefer a key-value (like Redis) or an object database (i.e. MongoDB), Blacksmith has you covered.
Leveraging BOSH, Blacksmith is able to offer isolated, dedicated service instances that are less prone to disruption from noisy neighbors.
Kits: Runnable Best Practices
Systems You Can Deploy