[ AWS Transit VPC ]: AWS CloudFormation Template and Lambda Functions

Estimated Reading Time: 5 minutes

Word Count: 1,298

Welcome to a new addition to our blog series “Transit VPC inside AWS using Juniper vSRX”.

This blog is part 2 of 4, in a series of 4 blog posts on the topic of Transit VPCs. In this blog, we will be covering the following topics:

  1. Introduction to Transit VPCs
  2. The business use cases of having Transit VPCs
  3. The technical use cases of having Transit VPCs
  4. The way Transit VPCs are configured
  5. Other similar Transit VPC solutions out there
  6. Why we chose to use Juniper vSRX as our hub firewall
  7. How we pieced it all together

In the last blog post, we talked about what is Transit VPC, and at a high level how it was pieced together. In this post, we’ll dive into the topics of automation aspects to this solution of Transit VPCs, i.e. CloudFormation and Lambda inside AWS.

CloudFormation is a service inside AWS that provides its users with tools needed to orchestrate and manage artifacts and infrastructure to run applications inside the AWS construct. With CloudFormation, you could deploy custom EC2 instances, deploy a Virtual Private Cloud network inside AWS, or create a complete stack of EC2 instance running inside a VPC and also create custom cron job based poller functions using Lambda, which is another AWS provided service.

CloudFormation has two components – Templates and Stacks. Let’s talk about templates first. A template is a (JavaScript Object Notation) JSON based plain-text human-readable file with parameters defined in a hierarchical manner using regular English-language words or phrases. The JSON file is a non-scripted declarative file, which defines the resources needed for the application to come up.

Example: Inside a JSON file, we can define an AMI ID to be used to launch instances per AWS Region, or define which artifacts (like subnets, Elastic IPs, etc) should associate themselves with which particular VPC or EC2 instance.

Based on the parameters defined in the CloudFormation template, while deploying, it asks the user to input the user-defined fields. In reference to our main topic of discussion that is Transit VPC, the CloudFormation template asks the user to define and input the VPC CIDR, the four subnets to be created under the Transit VPC and other necessary parameters like the tag Key and Value to be used to check all the Virtual Private Gateways (VGW) by the VGW Poller Lambda function.

The second component of CloudFormation is a stack. CloudFormation stack is a completed running version of the artifacts successfully created using a CloudFormation Template. The stack (based on how the CloudFormation template has been designed) can be a single standalone EC2 instance, or a complete collection of a Transit VPC with four subnets:  a). two in different availability zones each in necessary security zones and b). two unique vSRX instances inside the Transit VPC.

To create a stack, the user has to point the CloudFormation service to use a predefined CloudFormation template by uploading it to an S3 bucket accessible by the service. The CloudFormation service then takes the uploaded template and creates the resources and artifacts required, creating a running environment by maintaining a data flow and managing dependencies where needed. An example of an explicit dependency in our use case would be to have a new vSRX instance created having a second Elastic Network Interface (ENI)  associated with the instance before attaching an Elastic IP (EIP) address to the two ENIs of the vSRX instance.

A major advantage of using CloudFormation for automated deployments is that it allows users to automate service provisioning steps in a simple way using JSON, thus eliminating the need of relying on someone with strong Python (or similar) scripting skill set. Another advantage of using CloudFormation is the ability of an auto-rollback in case the stack deployment were to fail at any stage due to dependency or AWS resource limit issues – thus deleting all the resources it created so far from the start till the point of failure.

Moving on to discussing Lambda, it is another service in AWS’s arsenal. AWS Lambda is an event-driven, serverless computing platform inside AWS. Lambda is a compute service, which runs code in response to events and automatically manages the compute resources required by that code – thus providing scalability and flexibility of using resources as needed.

Compared to running EC2 instances solely for polling another service and pushing appropriate code at the trigger of an event, the use of Lambda saves the user on the cost as the metering of billing for Lambda, at each iteration, runs in increments of 100 milliseconds. Whereas running an EC2 instance is a fixed hourly, monthly or yearly cost based on the instance type, irrespective of the usage pattern on the instance.

Other AWS resources like S3 buckets, DynamoDB tables, or Amazon SNS notifications can be used to define and program Lambda functions. When the resource changes, AWS Lambda executes the user-defined function and automatically scales and manages the compute resources based on the compute requests/iterations being processed.

A Lambda compute resource is invoked by writing a Lambda function that is event-triggered. A Lambda function can be written in Java, Node.js, C#, and Python code. For our use case, three functions were built using Lambda:

  1. Solution Helper – This function helps the CF template with the creation, modification and deletion of the complete stack and its resources.
  2. Virtual Private Gateway (VGW) Poller – This function is cron job based, which checks all the VGWs inside the user’s AWS account for the presence of the correct predefined Tags (Key / Value combination) for declaring the VGW to be part of a Spoke VPC. Thus the VGW Poller helps in creating VPN Connections from the Spoke VGW towards the Transit VPC’s vSRX instances.
  3. Juniper Configurator – This function would be triggered by the VGW Poller’s creation of VPN Connection towards the vSRX instances. The creation of VPN Connections places a VPN connection specific file in the S3 bucket. This file is read by the Juniper Configurator function and the parameters for the VPN Connection (for IPSec VPN and BGP) are then converted to Junos CLI set commands and pushed to the vSRX instances.

All three Lambda functions listed above were created using the CloudFormation template used to create the stack, with the polling interval of the VGW Poller Lambda function being declared in the form of a cron job inside CloudWatch rules. That is how various AWS services are interlinked for our use case.

In the next upcoming blog post, we’ll be covering the following topics:

  1. Various use cases for Transit VPC
  2. The configuration of Transit VPC using CloudFormation
  3. The configuration of the vSRX firewall inside the Transit VPC
  4. Bringing up the connectivity towards the Spoke VPC Virtual Private Gateways

To continue to Blog 3, click here

If you want to find our development contributions to the open source community, visit Serro’s GitHub Repository: https://github.com/serrollc

About the Author

Shreyans is a Solutions Engineer at Serro since early 2014. He has a Master of Science in Electrical Engineering from San Jose State University.  His experience includes enterprise, data center and service provider routing, switching and security solutions across multiple vendors – Juniper Networks, Cisco, Palo Alto Networks, Brocade and Huawei; as well as cloud computing solutions like Amazon Web Services and OpenStack. In his free time, Shreyans takes pictures of landscapes around the Bay Area, with the Golden Gate Bridge being his muse. He can be found on social media on LinkedIn, and on Instagram.