The more I use Terraform, the more I discover friction in the process of writing infrastructure management code. I wonder if there are tools to augment Terraform that can overcome this friction or if I should be looking at a different infrastructure code tool. The more I work with AWS, the more I view it as a collections of frameworks rather than a collection of solutions.
I find myself becoming frustrated with using Terraform. There is a fair amount of friction I encounter around the development of infrastructure code the more I use it. I believe most of this friction comes from the use of a declarative language like HCL instead of a general-purpose language.
Terrafrom is fantastic for managing infrastructure that is fairly static. It has been an absolute joy to use for setting up VPCs, networking, databases and other infrastructure that changes infrequently. It is much less fun to use Terraform to manage more dynamic resources like EC2 instances. It requires the creation of some complex external orchestration to have Terraform apply updates to resources that change frequently. It is doable, but I have to wonder if there might be a better way.
A language like HCL is very difficult to test. From my experience, it is nearly impossible to write unit tests for declarative languages. You can manage reasonably well with a combination of linting/policy checks with tools like tflint and checkov, and environment validation tests with something like Terratest or Testinfra. A tool based on a general-purpose language would allow you to write true unit and integration tests for your infrastructure code.
I don’t have an answer for either of these issues yet, but it is something I am interested in looking into more.
This is more of a passing thought, but as I gain more experience with AWS, it has become clear to me that most of the services AWS offers are really building blocks more than a complete solution. For example, if you look at something like AWS Config or GuardDuty, you notice that there isn’t built-in support for things like alerts. You are expected to build a Lambda or some other form of integration to get to a complete solution. On one hand it makes these services very flexible, on the other it makes it difficult to implement something quickly.
I wish there was more of a middle ground. Maybe a minimal solution template that would allow you to get started quickly, and then you can dig in deeper to build a more complete solution.
Time permitting, I would like to look deeper into how to implement some of the AWS security services, and maybe look into researching options for making Terraform more dynamic.