Chef: Configuration Management in the Cloud
The Cloud is known for scalability, but without the right configuration management tool system, administration can become a severe bottleneck that undermines that scalability.
Consider how long it would take to configure a multi-tier application in the Cloud that required a set of relational database servers, application servers and web servers with load balancing services at each tier. Setting up such configurations manually could be prone to error. Are the IP addresses specified correctly in the database connection strings? Are each of the server firewalls properly configured? Are the correct packages installed?
In addition to the risk of error, manually setting up complex configurations can be time consuming. This is especially the case when a configuration has to be repeatedly started and shutdown. For example, you may find that you need to start dozens of servers every night to run extraction, transformation and load operations for a data warehouse. Ideally, this should be done programmatically with a script that is developed and debugged once and applied many times.
Chef, an open source configuration management product offered by Opscode under the Apache license, is designed to allow system administrators to programmatically configure Cloud-based systems and reduce the need for repeated, manual operations.
Automating Provisioning, Configuration and Integration
As the name implies, Chef uses cooking metaphors to organize the configuration and systems integration process. Recipes are series of declarations written in Ruby that describe what the state of the system should be, such as a database server configured with MySQL and a particular set of services.
Recipes make reference to resources, a level of abstraction corresponding to ingredients. Example resources include files, directories, mounts, packages, scripts and services. Recipes are collected into sets of configuration scripts known as cookbooks.
Cookbooks contain recipes to configure clients and servers as well as set parameters for different application requirements. With cookbooks and recipes, system administrators are able to consistently and efficiently deploy services in the cloud.
Chef uses the concept of a role to describe a set of services running on a single node. Typical roles include relational database servers, Web application servers, and load balancers. System administrators execute recipes to deploy servers in particular roles. Recipes help to ensure consistency across nodes which in turn reduce the likelihood of configuration related errors.
The deployment process in Chef actually consists of three sub-processes: provisioning , configuration and integration. Chef supports a variety of provisioning mechanisms, including use of operating system specific services like Kickstart on Linux, hypervisor APIs, or public cloud service APIs. Once servers have been provisioned, Chef uses recipes to configure software on the server. Chef recipes are declarative rather than procedural. That is they describe what should be configured but do not require you to specify low level commands to configure resources. Instead of specifying, for example, and apt-get command to install a package, a Chef recipe states that a particular package should be installed—Chef will determine how to install the package.
The final step in the Chef deployment is system integration. In many Cloud applications multiple servers are required to work together. For example, load balancers will need to know about the servers it will distribute jobs to and application servers will need to be configured to use a specific database backend. Chef provides the means to specify integration information so that load balancers, application servers, databases and other components will be properly configured when deployed.
In fact, Chef even supports data driven integration. A database of configuration information can be queried to discover information about the state of the configuration and perform integration steps based on that dynamic information.
Dan Sullivan is an author, systems architect, and consultant with over 20 years of IT experience with engagements in systems architecture, enterprise security, advanced analytics and business intelligence. He has worked in a broad range of industries, including financial services, manufacturing, pharmaceuticals, software development, government, retail, gas and oil production, power generation, life sciences, and education. Dan has written 16 books and numerous articles and white papers about topics ranging from data warehousing, Cloud Computing and advanced analytics to security management, collaboration, and text mining.
(Shutterstock cover image credit: Chef)