Tutorial 15: Create and Use Modules

Learning Objectives

Understand modules and their benefits in Terraform
Learn to create reusable modules
Practice using local and remote modules
Implement module versioning and best practices
Design modules for different use cases

What are Terraform Modules?

Modules are containers for multiple resources that are used together. They provide a way to organize and reuse Terraform configurations. Every Terraform configuration has at least one module, called the root module.

Benefits of Modules

Reusability: Write once, use multiple times
Organization: Group related resources logically
Abstraction: Hide complexity behind simple interfaces
Standardization: Enforce organizational standards
Testing: Test components in isolation
Versioning: Manage changes over time

Module Types

Local Modules: Stored in the same repository
Remote Modules: Stored in external repositories
Published Modules: Available in Terraform Registry
Private Modules: Hosted in private registries

Creating Your First Module

Module Structure

modules/
└── web-server/
    ├── main.tf          # Main resource definitions
    ├── variables.tf     # Input variables
    ├── outputs.tf       # Output values
    ├── versions.tf      # Provider requirements
    └── README.md        # Documentation

Example: Web Server Module

modules/web-server/variables.tf

variable "name" {
  description = "Name prefix for resources"
  type        = string
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t2.micro"
  
  validation {
    condition = contains([
      "t2.micro", "t2.small", "t2.medium",
      "t3.micro", "t3.small", "t3.medium"
    ], var.instance_type)
    error_message = "Instance type must be a valid t2 or t3 type."
  }
}

variable "ami_id" {
  description = "AMI ID for the instance"
  type        = string
  default     = null
}

variable "subnet_id" {
  description = "Subnet ID where instance will be created"
  type        = string
}

variable "vpc_id" {
  description = "VPC ID for security group"
  type        = string
}

variable "allowed_cidr_blocks" {
  description = "CIDR blocks allowed to access the web server"
  type        = list(string)
  default     = ["0.0.0.0/0"]
}

variable "enable_monitoring" {
  description = "Enable detailed monitoring"
  type        = bool
  default     = false
}

variable "tags" {
  description = "Additional tags"
  type        = map(string)
  default     = {}
}

modules/web-server/main.tf

# Data source for AMI if not provided
data "aws_ami" "amazon_linux" {
  count       = var.ami_id == null ? 1 : 0
  most_recent = true
  owners      = ["amazon"]
  
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
  
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# Local values for common computations
locals {
  ami_id = var.ami_id != null ? var.ami_id : data.aws_ami.amazon_linux[0].id
  
  common_tags = merge(
    {
      Name       = var.name
      Module     = "web-server"
      ManagedBy  = "terraform"
    },
    var.tags
  )
}

# Security group for web server
resource "aws_security_group" "web" {
  name_prefix = "${var.name}-web-"
  description = "Security group for ${var.name} web server"
  vpc_id      = var.vpc_id
  
  # HTTP access
  ingress {
    description = "HTTP"
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = var.allowed_cidr_blocks
  }
  
  # HTTPS access
  ingress {
    description = "HTTPS"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = var.allowed_cidr_blocks
  }
  
  # All outbound traffic
  egress {
    description = "All outbound traffic"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  
  tags = merge(local.common_tags, {
    Name = "${var.name}-web-sg"
  })
  
  lifecycle {
    create_before_destroy = true
  }
}

# EC2 instance
resource "aws_instance" "web" {
  ami                    = local.ami_id
  instance_type          = var.instance_type
  subnet_id              = var.subnet_id
  vpc_security_group_ids = [aws_security_group.web.id]
  monitoring             = var.enable_monitoring
  
  user_data = base64encode(templatefile("${path.module}/user-data.sh", {
    server_name = var.name
  }))
  
  tags = merge(local.common_tags, {
    Name = "${var.name}-web-server"
  })
  
  lifecycle {
    create_before_destroy = true
  }
}

# Elastic IP (optional)
resource "aws_eip" "web" {
  instance   = aws_instance.web.id
  domain     = "vpc"
  depends_on = [aws_instance.web]
  
  tags = merge(local.common_tags, {
    Name = "${var.name}-web-eip"
  })
}

modules/web-server/user-data.sh

#!/bin/bash
yum update -y
yum install -y httpd

# Start and enable Apache
systemctl start httpd
systemctl enable httpd

# Create a simple index page
cat > /var/www/html/index.html << EOF
<!DOCTYPE html>
<html>
<head>
    <title>Welcome to ${server_name}</title>
</head>
<body>
    <h1>Hello from ${server_name}!</h1>
    <p>Server started at: $(date)</p>
    <p>Instance ID: $(curl -s http://169.254.169.254/latest/meta-data/instance-id)</p>
</body>
</html>
EOF

# Configure Apache
echo "ServerTokens Prod" >> /etc/httpd/conf/httpd.conf
echo "ServerSignature Off" >> /etc/httpd/conf/httpd.conf

# Restart Apache to apply changes
systemctl restart httpd

modules/web-server/outputs.tf

output "instance_id" {
  description = "ID of the EC2 instance"
  value       = aws_instance.web.id
}

output "instance_arn" {
  description = "ARN of the EC2 instance"
  value       = aws_instance.web.arn
}

output "public_ip" {
  description = "Public IP address of the instance"
  value       = aws_eip.web.public_ip
}

output "private_ip" {
  description = "Private IP address of the instance"
  value       = aws_instance.web.private_ip
}

output "security_group_id" {
  description = "ID of the security group"
  value       = aws_security_group.web.id
}

output "website_url" {
  description = "URL to access the website"
  value       = "http://${aws_eip.web.public_ip}"
}

output "instance_details" {
  description = "Complete instance information"
  value = {
    id          = aws_instance.web.id
    arn         = aws_instance.web.arn
    public_ip   = aws_eip.web.public_ip
    private_ip  = aws_instance.web.private_ip
    instance_type = aws_instance.web.instance_type
    ami         = aws_instance.web.ami
    az          = aws_instance.web.availability_zone
  }
}

modules/web-server/versions.tf

terraform {
  required_version = ">= 1.0"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0"
    }
  }
}

modules/web-server/README.md

# Web Server Module

This module creates a simple web server with the following resources:
- EC2 instance running Amazon Linux 2
- Security group with HTTP/HTTPS access
- Elastic IP for static public IP

## Usage

```hcl
module "web_server" {
  source = "./modules/web-server"
  
  name               = "my-web-server"
  instance_type      = "t2.micro"
  subnet_id          = "subnet-12345678"
  vpc_id             = "vpc-12345678"
  allowed_cidr_blocks = ["10.0.0.0/8"]
  
  tags = {
    Environment = "development"
    Project     = "my-project"
  }
}

Requirements

Name	Version
terraform	>= 1.0
aws	>= 5.0

Inputs

Name	Description	Type	Default	Required
name	Name prefix for resources	`string`	n/a	yes
instance_type	EC2 instance type	`string`	`"t2.micro"`	no
subnet_id	Subnet ID where instance will be created	`string`	n/a	yes
vpc_id	VPC ID for security group	`string`	n/a	yes

Outputs

Name	Description
instance_id	ID of the EC2 instance
public_ip	Public IP address of the instance
website_url	URL to access the website


## Using Modules

### Using a Local Module
```hcl
# main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

# Data sources
data "aws_vpc" "default" {
  default = true
}

data "aws_subnets" "default" {
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.default.id]
  }
}

# Use the web server module
module "web_server_dev" {
  source = "./modules/web-server"
  
  name                = "dev-web"
  instance_type       = "t2.micro"
  subnet_id           = data.aws_subnets.default.ids[0]
  vpc_id              = data.aws_vpc.default.id
  allowed_cidr_blocks = ["0.0.0.0/0"]
  enable_monitoring   = false
  
  tags = {
    Environment = "development"
    Project     = "web-app"
    Owner       = "dev-team"
  }
}

module "web_server_prod" {
  source = "./modules/web-server"
  
  name                = "prod-web"
  instance_type       = "t3.medium"
  subnet_id           = data.aws_subnets.default.ids[1]
  vpc_id              = data.aws_vpc.default.id
  allowed_cidr_blocks = ["10.0.0.0/16"]
  enable_monitoring   = true
  
  tags = {
    Environment = "production"
    Project     = "web-app"
    Owner       = "ops-team"
  }
}

# Outputs from modules
output "dev_server_url" {
  description = "Development server URL"
  value       = module.web_server_dev.website_url
}

output "prod_server_url" {
  description = "Production server URL"
  value       = module.web_server_prod.website_url
}

output "all_server_details" {
  description = "All server details"
  value = {
    dev  = module.web_server_dev.instance_details
    prod = module.web_server_prod.instance_details
  }
}

Module Initialization and Deployment

# Initialize Terraform (downloads modules)
terraform init

# Plan the deployment
terraform plan

# Apply the configuration
terraform apply

# Check outputs
terraform output dev_server_url
terraform output prod_server_url

Advanced Module Patterns

Module with Count

# Multiple instances of the same module
module "web_servers" {
  count  = 3
  source = "./modules/web-server"
  
  name                = "web-${count.index + 1}"
  instance_type       = "t2.micro"
  subnet_id           = data.aws_subnets.default.ids[count.index % length(data.aws_subnets.default.ids)]
  vpc_id              = data.aws_vpc.default.id
  allowed_cidr_blocks = ["0.0.0.0/0"]
  
  tags = {
    Environment = "development"
    Index       = count.index
  }
}

output "web_server_urls" {
  value = [for server in module.web_servers : server.website_url]
}

Module with For-Each

# Different configurations for different environments
locals {
  environments = {
    dev = {
      instance_type = "t2.micro"
      monitoring    = false
    }
    staging = {
      instance_type = "t2.small"
      monitoring    = true
    }
    prod = {
      instance_type = "t3.medium"
      monitoring    = true
    }
  }
}

module "web_servers" {
  for_each = local.environments
  source   = "./modules/web-server"
  
  name                = "${each.key}-web"
  instance_type       = each.value.instance_type
  subnet_id           = data.aws_subnets.default.ids[0]
  vpc_id              = data.aws_vpc.default.id
  allowed_cidr_blocks = ["0.0.0.0/0"]
  enable_monitoring   = each.value.monitoring
  
  tags = {
    Environment = each.key
  }
}

output "environment_urls" {
  value = {
    for env, server in module.web_servers : env => server.website_url
  }
}

Conditional Module Usage

variable "create_web_server" {
  description = "Whether to create web server"
  type        = bool
  default     = true
}

module "web_server" {
  count  = var.create_web_server ? 1 : 0
  source = "./modules/web-server"
  
  name      = "conditional-web"
  subnet_id = data.aws_subnets.default.ids[0]
  vpc_id    = data.aws_vpc.default.id
}

output "web_server_url" {
  value = var.create_web_server ? module.web_server[0].website_url : "Not created"
}

Complex Module Example: Three-Tier Application

Module Structure

modules/
└── three-tier-app/
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── versions.tf
    ├── modules/
    │   ├── networking/
    │   ├── web-tier/
    │   ├── app-tier/
    │   └── data-tier/
    └── README.md

modules/three-tier-app/main.tf

# Local values for common configurations
locals {
  common_tags = merge(
    {
      Application = var.application_name
      Environment = var.environment
      ManagedBy   = "terraform"
    },
    var.tags
  )
  
  name_prefix = "${var.application_name}-${var.environment}"
}

# Networking module
module "networking" {
  source = "./modules/networking"
  
  name_prefix         = local.name_prefix
  vpc_cidr           = var.vpc_cidr
  availability_zones = var.availability_zones
  
  tags = local.common_tags
}

# Web tier module
module "web_tier" {
  source = "./modules/web-tier"
  
  name_prefix    = local.name_prefix
  vpc_id         = module.networking.vpc_id
  subnet_ids     = module.networking.public_subnet_ids
  instance_type  = var.web_instance_type
  instance_count = var.web_instance_count
  
  tags = local.common_tags
  
  depends_on = [module.networking]
}

# Application tier module
module "app_tier" {
  source = "./modules/app-tier"
  
  name_prefix         = local.name_prefix
  vpc_id              = module.networking.vpc_id
  subnet_ids          = module.networking.private_subnet_ids
  web_security_group_id = module.web_tier.security_group_id
  instance_type       = var.app_instance_type
  instance_count      = var.app_instance_count
  
  tags = local.common_tags
  
  depends_on = [module.web_tier]
}

# Data tier module
module "data_tier" {
  source = "./modules/data-tier"
  
  name_prefix           = local.name_prefix
  vpc_id                = module.networking.vpc_id
  subnet_ids            = module.networking.database_subnet_ids
  app_security_group_id = module.app_tier.security_group_id
  db_instance_class     = var.db_instance_class
  
  tags = local.common_tags
  
  depends_on = [module.app_tier]
}

modules/three-tier-app/variables.tf

variable "application_name" {
  description = "Name of the application"
  type        = string
}

variable "environment" {
  description = "Environment (dev, staging, prod)"
  type        = string
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "availability_zones" {
  description = "List of availability zones"
  type        = list(string)
}

variable "web_instance_type" {
  description = "Instance type for web tier"
  type        = string
  default     = "t3.micro"
}

variable "web_instance_count" {
  description = "Number of web tier instances"
  type        = number
  default     = 2
}

variable "app_instance_type" {
  description = "Instance type for app tier"
  type        = string
  default     = "t3.small"
}

variable "app_instance_count" {
  description = "Number of app tier instances"
  type        = number
  default     = 2
}

variable "db_instance_class" {
  description = "RDS instance class"
  type        = string
  default     = "db.t3.micro"
}

variable "tags" {
  description = "Additional tags"
  type        = map(string)
  default     = {}
}

Using the Three-Tier Module

# main.tf
module "my_application" {
  source = "./modules/three-tier-app"
  
  application_name = "ecommerce"
  environment      = "production"
  
  vpc_cidr           = "10.0.0.0/16"
  availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
  
  web_instance_type  = "t3.medium"
  web_instance_count = 3
  
  app_instance_type  = "t3.large"
  app_instance_count = 4
  
  db_instance_class = "db.r5.large"
  
  tags = {
    Project = "ecommerce-platform"
    Owner   = "platform-team"
    Cost    = "production-workload"
  }
}

output "application_load_balancer_dns" {
  value = module.my_application.load_balancer_dns
}

output "database_endpoint" {
  value     = module.my_application.database_endpoint
  sensitive = true
}

Remote Modules

Using Git-Based Modules

# Using module from Git repository
module "vpc" {
  source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=v3.19.0"
  
  name = "my-vpc"
  cidr = "10.0.0.0/16"
  
  azs             = ["us-west-2a", "us-west-2b", "us-west-2c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
  
  enable_nat_gateway = true
  enable_vpn_gateway = true
  
  tags = {
    Terraform = "true"
    Environment = "dev"
  }
}

Using Terraform Registry Modules

# Using module from Terraform Registry
module "security_group" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "4.17.1"
  
  name        = "web-server-sg"
  description = "Security group for web servers"
  vpc_id      = module.vpc.vpc_id
  
  ingress_with_cidr_blocks = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      description = "HTTP"
      cidr_blocks = "0.0.0.0/0"
    },
    {
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      description = "HTTPS"
      cidr_blocks = "0.0.0.0/0"
    }
  ]
}

Module Versioning Strategies

# Specific version (recommended for production)
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "3.19.0"
  # ... configuration
}

# Version constraints
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 3.19"  # Allow patch updates
  # ... configuration
}

# Git branch or tag
module "vpc" {
  source = "git::https://github.com/example/terraform-aws-vpc.git?ref=v1.2.3"
  # ... configuration
}

Module Testing

Basic Module Testing

# test/module_test.go
package test

import (
    "testing"
    
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestWebServerModule(t *testing.T) {
    terraformOptions := &terraform.Options{
        TerraformDir: "../examples/simple",
        VarFiles:     []string{"test.tfvars"},
    }
    
    defer terraform.Destroy(t, terraformOptions)
    terraform.InitAndApply(t, terraformOptions)
    
    // Get output values
    instanceID := terraform.Output(t, terraformOptions, "instance_id")
    publicIP := terraform.Output(t, terraformOptions, "public_ip")
    
    // Assertions
    assert.NotEmpty(t, instanceID)
    assert.NotEmpty(t, publicIP)
}

Module Validation Script

#!/bin/bash
# validate-module.sh

set -e

MODULE_DIR=${1:-"./modules/web-server"}

echo "Validating module: $MODULE_DIR"

# Check required files
required_files=("main.tf" "variables.tf" "outputs.tf" "README.md")
for file in "${required_files[@]}"; do
    if [[ ! -f "$MODULE_DIR/$file" ]]; then
        echo "Error: Missing required file: $file"
        exit 1
    fi
done

# Validate Terraform syntax
cd "$MODULE_DIR"
terraform init
terraform validate

# Check formatting
terraform fmt -check

# Generate documentation
terraform-docs markdown . > README_generated.md

echo "Module validation completed successfully"

Module Best Practices

1. Module Design Principles

# Good module design
# - Single responsibility
# - Minimal required inputs
# - Sensible defaults
# - Clear outputs
# - Good documentation

variable "name" {
  description = "Name for the resources"
  type        = string
  # Required - no default
}

variable "instance_type" {
  description = "EC2 instance type"
  type        = string
  default     = "t3.micro"  # Sensible default
}

variable "tags" {
  description = "Additional tags"
  type        = map(string)
  default     = {}  # Optional with empty default
}

2. Input Validation

variable "environment" {
  description = "Environment name"
  type        = string
  
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

variable "instance_count" {
  description = "Number of instances"
  type        = number
  default     = 1
  
  validation {
    condition     = var.instance_count >= 1 && var.instance_count <= 10
    error_message = "Instance count must be between 1 and 10."
  }
}

3. Output Organization

# Provide both individual and grouped outputs
output "instance_id" {
  description = "ID of the EC2 instance"
  value       = aws_instance.web.id
}

output "instance_details" {
  description = "Complete instance information"
  value = {
    id         = aws_instance.web.id
    arn        = aws_instance.web.arn
    public_ip  = aws_instance.web.public_ip
    private_ip = aws_instance.web.private_ip
  }
}

4. Resource Naming

# Consistent naming with name_prefix
resource "aws_instance" "web" {
  # ... configuration
  
  tags = {
    Name = "${var.name_prefix}-web-server"
  }
}

resource "aws_security_group" "web" {
  name_prefix = "${var.name_prefix}-web-"
  # ... configuration
}

5. Documentation

# Module Documentation Template

## Description
Brief description of what the module does.

## Usage
```hcl
module "example" {
  source = "./modules/example"
  
  name = "my-example"
  # ... other required variables
}

Requirements

Name	Version
terraform	>= 1.0
aws	>= 5.0

Providers

Name	Version
aws	>= 5.0

Resources

Name	Type
aws_instance.web	resource
aws_security_group.web	resource

Inputs

Name	Description	Type	Default	Required

Outputs

Name	Description


## Key Takeaways

- Modules provide reusability and organization for Terraform configurations
- Start with simple modules and gradually add complexity
- Use clear input validation and sensible defaults
- Provide comprehensive outputs for flexibility
- Document modules thoroughly with examples
- Version modules for production use
- Test modules with different configurations
- Follow consistent naming and tagging conventions
- Design modules with single responsibility principle
- Use local modules for organization, remote modules for sharing

## Next Steps

1. Complete the Configuration Management tutorials (16-25)
2. Learn about module testing and validation
3. Explore the Terraform Registry for community modules
4. Practice creating modules for your specific use cases

## Additional Resources
- [Module Documentation](https://terraform.io/docs/language/modules/)
- [Terraform Registry](https://registry.terraform.io/)
- [Module Best Practices](https://terraform.io/docs/language/modules/develop/)
- [Terratest for Testing](https://terratest.gruntwork.io/)