Sunday, July 7, 2019

Sync Terraform Config and .tfstate for Existing AWS Resources

Hey all!

Terraform is a great (and dominant) infrastructure automation tool. It is multi-cloud, can build all sorts of resources, and in some cases supports API calls to build resources before the native tooling from cloud providers does.

However, it's dependent on a state file that is local, and only reflects resources created by terraform, and a local configuration file to describe resources. It's not able to reach out to a cloud account and create a configuration and .tfstate file based on the existing resources that were built via another method. Or at least, it isn't able to yet. The scaffolding for this functionality exists within Terraform for the AWS cloud, and is called the "import" functionality. It's able to map a single existing resource to a single configuration block for the same resource type and fill in the info, which is of course a manual and tedious process. And imagine if you have hundreds (or thousands!) of resources. It isn't a feasible way to move forward.

Terraforming (link) is a wrapper around terraform and is able to map multiple resources at the same time to configuration blocks, as well as build .tfstate files for multiple existing resource types. Still, it's a little awkward to use - only a single resource type is able to be imported at the same time, and if a command is run against a non-existing resource type (say you don't have a batch configuration, and run a sync against the batch resource), it wipes out the existing .tfstate entirely, removing your progress.

Clearly, the tools could use some help. So I wrote some. I imagine both of these tools (terraform import & terraforming) will eventually get this same functionality. In fact, both of these tools are open source, and I'll work on adding this functionality natively to both of these tools via PRs.

However, for the time being, I'm publishing my code which permits:

  • Creating from scratch a .tfstate file for every terraforming supported resource in an AWS region
  • Creating a single (monolithic) configuration file for each existing resource in an AWS region
This code assumes you don't have an existing .tfstate file - in fact, it wipes out your existing local .tfstate file and builds a new one. So please back up your .tfstate and configuration files before running this tool

However, if your'e new to terraform and want to sync the configuration to an existing AWS region and look at the config for all the resources that exist there, this is a neat shortcut. 

Rather than post the code here and update each time I (or you! The code is open source) update it, I'll post a link to the public github repo. 

I hope it's useful to you. Please add any corrections, comments, and feature additions you'd like via pull requests. And if you know how to update the terraform or terraforming source code to add these functionalities and make my code obsolete, please do so! That would be the best case scenario. 

Thanks all. Good luck out there! 

No comments:

Post a Comment