Infrastructure as Code: Is the Command Line now dead?
As a network engineer, I have had my fair share of cabling racks, puzzling over server rack mount kits and losing time (and skin) battling with a stubborn cage nut over the past 20+ years. But for the past few years, I have not had to wear my ‘steelies’ and my console cable gathers dust, as I have been working primarily with cloud technologies.
Networking in the cloud certainly differs from traditional networking. With the cloud, the routers and switches are abstracted, and their purpose is merged into the cloud fabric itself, at least from the end-user perspective. Whereas in a traditional network, the routers, switches and accompanying networking resources such as firewall and load balancer must be managed separately. Within the cloud, these can all be managed using the same interface, be that the cloud portal or the cloud’s CLI (Command Line Interface) offering. This uniformity in configuration and management opens opportunities that were not possible before, and one of them is IaC.
What is IaC?
Infrastructure as Code (IaC) is the process of creating and managing infrastructure via a text file, or files. Therefore, no matter what the resource, it can be created and managed via a simple text file(s), and software principles such as version control can then be applied to those text file(s). This concept offers tremendous advantages over manual configuration and management within the virtual cloud environment which consists of a multitude of varying resource types.
A cloud environment can host a range of resources, which can be managed either via the Cloud Provider’s (CP) portal, or a Command Line Interface (CLI). Using the portal does not provide any means of IaC, but by using the CLI, we can start to address the requirements needed for IaC. CLIs can be found from each Cloud Provider, who makes it available on Windows or Linux and via their online portal. The CLI is very efficient at running commands against a single resource, and whilst it can be scaled up to address several resources, it does not do this at the scale needed to manage a complete infrastructure as is the goal of IaC.
The aim of this article is not to compare CLIs and third-party vendor offerings. Instead, it will focus on the key requirements, benefits and challenges faced when adopting an IaC philosophy.
Key Requirements
Idempotency – There are several features all IaC products should have in common, arguably the most important is that they should be idempotent. This means any time that IaC is executed, it should not create another instance of the same resource if it is already present, nor should it delete and recreate the resource. IaC should be intelligent enough to look at the existing resources and only amend them if changed, and to create new resources only if they do not already exist. It must be able to do this consistently no matter how many times it is implemented and be able to do this at scale on any number of resources. This can be achieved by IaC checking the state of the existing infrastructure each time it is run.
Declarative Code – If the IaC language was written in an imperative manner – meaning that we write the code in the order we expect the steps to be carried out to create a resource – it can lead to a lengthy script, as well as a more error-prone one. Consider if the imperative script were to error during its run, now we would need to introduce error management into the code, adding lines of code and complexity. Instead, our IaC should be written in a declarative manner. With declarative code we only write what our infrastructure should look like, it simply says ‘I want this resource created with these parameters’, and it lets the cloud’s ‘configuration engine’ take care of how it is created. This leads to shorter code and equally as importantly a more easily readable and therefore manageable syntax.
Adopting an IaC philosophy: Top 4 Benefits and Challenges:
Benefits
1. Creating Resources with Consistency
With IaC, resources can be quickly created, amended, destroyed and re-created in a reliable, consistent manner. If all the config relating to a resource is kept in an IaC file, it will always be in a state that is ready to be deployed, as all parameters will have already been defined. If the resource is experiencing issues, then it can be deleted and a fresh instance created.
Multiple instances of the resource can be created in multiple environments, allowing development teams to create an architecture in a staging area that can easily be replicated in the production / live environment. If the live environment was to experience issues, then the resources affected can be re-created in a testing environment, allowing developers to test in isolation, thus allowing for uninterrupted operation of the live environment.
By using modules within our IaC code we can create consistent resources within our cloud estate, share modules or use modules from other IaC developers. This is widespread practice in software engineering and IaC can capitalise on this well-established practice.
2. Applying versioning to entire cloud estate
Software developers have been using version control for their releases for decades, and recent tooling such as GIT has made this process easier than ever. IaC users can take advantage of these established software best practices. With IaC we can apply versioning to our entire cloud estate, keeping a record of every deployment. This makes tracking and accountability easier as it becomes part of the deployment process itself, and we can roll back to a previously deployed version if experiencing issues with a new rollout. Using GIT, we can create a new branch to test a new feature or have multiple members of a team working on different aspects of the infrastructure at the same time, all whilst the code is safely and securely being version controlled by GIT.
3. Use of CI/CD Pipelines
Continuous Integration and Continuous Delivery (CI/CD) deals with how we can deliver our IaC files into deployment. CI/CD Pipelines can be created to effectively automate, manage and deliver IaC files. A pipeline is a series of steps written in code prepared to automate the delivery of IaC files, and as such it should adhere to the same best practices for version control that IaC itself does.
A pipeline can be linked to GIT, so an update in GIT can trigger the pipeline. This will conduct a series of steps to check the IaC files for errors, seek validation from a senior team member and/or management, and if approved deliver the code into deployment. This method ensures consistent checks and validations are carried out before each deployment is implemented.
4. Cross-Platform Integration & Configuration
IaC is offered by the big Cloud Providers:
- Microsoft Azure – Azure Resource Manager (ARM) templates and BICEP.
- Amazon – AWS have CloudFormation.
- Google – Cloud Deployment Manager
However, these providers do not offer cross-platform capabilities, as each Cloud Provider’s offering will only work within their cloud. If a multi-cloud, cross-platform IaC tool is required, the go-to tool of choice is currently Terraform. This cross-platform capability extends beyond just the big three CPs, as well as that Terraform integrates with many third parties to offer a rich suite of integration tooling.
Also, whilst IaC offers building of the cloud resources themselves, other toolsets offer Configuration Management, like Ansible which executes tasks on servers such as installing and updating software. If used in conjunction with IaC then these configuration tools should be treated with the same best practices as used for IaC.
Challenges
1. Securing Access
IaC allows us to create complex new resources and infrastructures and then deploy them with a single command, and the inverse of that is it is also possible to destroy those same resources with a single command. The popular IaC platforms have controls and warnings in place that help avoid this, but consideration needs to be given as to who has access to the IaC files. This access needs to be monitored and kept up to date as support and architecture teams change, but this is no different to securing access to a file server for instance, and those security best practices should be extended to IaC.
2. Configuration Drift
Resources can be created and managed by IaC, or via the cloud portal, but ideally not by both systems independently. A change via the portal will be picked up when IaC is run and the IaC files can be amended, but continually making changes via the portal and not updating the IaC files will lead to configuration drift, and the IaC files may need considerable effort to get back in sync. Therefore, once using IaC, it should be the primary platform for changes and maintenance to infrastructure.
It should be noted however that configuration within the resource, for example, firewall rules within a firewall, can be managed via the portal, whilst the firewall resource itself is managed via IaC, the decisions on how systems are managed should be addressed early and adhered to in order to mitigate drift.
3. Upskilling
All members of the architect and support teams will require knowledge of IaC and be trained in best practices. IaC syntax for the most part is unlike a programming language, IaC is written declaratively so it reads more like a list of resources deployed rather than programming syntax.
Engineers familiar with a scripting language such as PowerShell or Bash should be able to pick up IaC syntax relatively quickly. Engineers have been familiar with DevOps practices for a while now so version control concepts like GIT should not be a new topic either and may even already be used in other areas of the infrastructure.
4. Code Maintenance
As IaC itself is written in a language, that language itself will need to be maintained, as the IaC language evolves certain commands may become deprecated, or new ones added as new CP features are introduced and updated. The IaC code will need to be maintained to reflect these changes, else a condition known as ‘code rot’ sets in whereby the IaC code itself becomes outdated and is no longer fit for purpose.
Another area of code that will need to be maintained is the CI/CD pipeline code, as that too is subject to ‘code rot’ if not maintained.
So… is the command line now dead?
So, to answer the title of this article, is the command line now dead? No, but in this cloud era the CLI should not be used in isolation. Instead, IaC augments the traditional CLI by allowing for an entire infrastructure to:
✓ Be managed via code
✓ Add versioning which allows for rollbacks
✓ Have full accountability
✓ Self-document
✓ Have re-usability
✓ Improve security
✓ Branch off for development and testing
✓ Keep multiple environments in sync.
An organisation is now able to look at its entire infrastructure and say we are on version ‘x,’ adding a level of control and accuracy that was not possible previously whether on-premise or in the cloud.
Technologies such as IaC are a natural accompaniment and necessity to the fast-paced evolution of cloud infrastructure and the disappearing lines between the roles of traditional networking engineers, system engineers and developers in a modern workplace. DevOps is already adopted, with SecDevOps also now becoming commonplace, and IaC should be viewed as just another component of that adoption.
Amran Ahmed, CCIE #20995 and Microsoft Azure Architect, is an Azure Architect involved in all aspects of cloud technologies, providing consultancy, cloud migrations and architecting a global solution for a financial startup. Amran can draw on his 20+ years of network experience where he has designed and consulted with Public, Financial, Educational and Oil and Gas companies to bring that experience to the unique requirements of the cloud. A lifelong advocate of continual learning he is one of Systal’s key technical leadership resources in assisting Systal to develop their cloud solutions.