Common software environments explained
Software deployment environments: an informal definition
While there are many definitions, I view environments to be the collection of all necessary infrastructure and software components needed for your application to work.
An environment contains your code, of course, the core piece making everything happen. But it also contains:
- The things that run your code: your application servers (Tomcat, Jetty, etc).
- The things that host the things that run your code: Docker containers, Kubernetes clusters, virtual or physical machines
- The things that hold your data: files, databases, caches
- The things that monitor and observe your application: monitoring tools (Prometheus, OpenTSDB, etc), logging tools (Logstash, Splunk, etc), alerting tools
- Utility resources: load balancers, DNS zones, networks, utility machines, security enforcement systems, backup mechanisms.
- Other software components that your application uses (or their mocks)
Local Development environment
Purpose: To give you a place to write any new code. To help you develop and test your new code in complete isolation. To test if your code compiles and runs. Does not look like the Production environment at all.
Focus: Here you focus on the new code you write. You get it to compile. You get it to start. You can unit test it.
Setup: On your machine. Usually contains a code editor from which you can run and debug your code. Libraries and packages required for building and running your code. Local versions or mocks of used storage solutions. To help with testing you can connect to remote environments (integration/staging) or remote databases/storage solutions (e.g. running in cloud or datacenter). However, usually any external resource is mocked to make local testing independent of any external factors.
Integration environment
Purpose: To give you a place to validate how your new code works together with all other services or software components used in the larger software product/ecosystem. The idea is to execute your code against running services and components, not mocks, to see how it behaves. Where you can perform integration and end-2-end tests to validate your code. Starts looking like production, but usually uses fewer resources.
Focus: Here you focus on how your new code interacts with already existing software. On validating that all contracts between software components hold. That you can use other software components or services, and that they can use you.
Setup: Deployed elsewhere. Not on your machine. A remote location from you: datacenter, cloud provider. Contains all other utility components required: load balancers, networks, virtual machines. Contains all storage solutions used. Contains all other software components or services used by your application. Automation starts to be present in terms of CI/CD pipelines to deploy and validate your code.
Staging/Pre-Prod environment
Purpose: To prepare you and your code for release to production. Aims to give you a testing environment as close as possible to production, for maximizing confidence that a new code release will provide the required correctness, quality, and performance. Looks almost exactly to production. Holds all other software components or services used by your application, deployed with stable code versions. Contains production-like data which can be used to test complex flows. Can sustain performance testing at expected production traffic levels.
Focus: Here you focus on how your new code will behave in production. You treat staging exactly like production.
Setup: Deployed on top of the same infrastructure type as production environments (cloud, datacenter, etc). If your service has deployments in multiple regions serving customers from different locations, the staging environment has at least 2 regions to help you test your application configurations and flows. For example, if you serve customers from North America from US, European customers from Ireland, and Asia/Pacific customers from Singapore, your staging environment should cover at least 2 of these regions so you can test multi-region scenarios and configurations before releasing to prod. Monitoring and alerting start to become more important as the staging environment needs to be kept as stable as production. All software components or services in a staging environment are running stable code versions. I.e., with the code version currently used in production, or the code version which will reach production shortly.
Production environment
Purpose: Provides useful functionality for your clients. Brings you money. Is the environment that your clients know and use. Is the environment where any code misbehaving can have severe financial or legal repercussions.
Focus: Here you focus on your clients and how your code behaves when used by them. Nothing is more important in production environments than the client.
Setup: Complex infrastructure both in scale and operations. Designed for high availability and high performance. Contains redundancy and failover mechanisms to enable fast recovery in case of failures of any kind, from hardware to software. Has backup and restore mechanisms to recover from even the most catastrophic failures. Usually deployed in multiple regions to ensure low latency for your clients or to abide by legal frameworks such as data security. Uses highly available monitoring and alerting mechanisms to track and notify as soon as problems occur. Has various automation mechanisms to cope with most problematic scenarios, such as autoscaling for traffic variations or traffic redirection to other regions in case of infrastructure problems. Usually closely watched by a dedicated on-call team that reacts and fixes anything that automation can’t.
Conclusion
As your code moves from your local development environment to production, the area of focus and attention grows. In dev we focus on just the new code. As we move to integration/staging, we slowly start to expand our horizon by including neighboring components. Then expand even more to include performance, multi-region setups, high availability, monitoring, and alerting. All up to production where we focus on our clients and how the code behaves for them in the real world.
Depending on requirements from your software, engineering team, or clients, you can further add more specialized environments. For example, if you have a regular release cadence, you might need to add a “Hotfix” environment acting as a second “Staging/Pre-Prod” to enable hot-fixes while in the middle of preparing a new release.
Not having any of the computing environments mentioned above brings problems. Not being able to develop locally slows down developers. Not being able to test integration with smaller components means bugs are caught much closer to production. This usually means more involved debugging flows slowing down software delivery. Not having a staging environment means many problems might surface only in production, like performance degradation. When they surface in production, problems cause the most damage to your clients and their satisfaction with your product.
My final remark is that regardless of the name or type, each deployment environment should be treated with respect and built to serve its purpose.
