AskLearn
Loading...
← Back to Terraform Course
AdvancedConfiguration

Count and For Each

Resource iteration patterns

Tutorial 19: Resource Meta-Arguments (count, for_each)

Learning Objectives

By the end of this tutorial, you will be able to:

  • Use the count meta-argument for creating multiple resource instances
  • Implement for_each for more flexible resource iteration
  • Choose between count and for_each appropriately
  • Handle complex scenarios with nested loops and transformations
  • Manage resource lifecycle when using meta-arguments

Prerequisites

  • Understanding of Terraform resources and variables
  • Knowledge of HCL expressions and functions
  • Completed Tutorial 17: Local Values and Computed Values

Introduction

Meta-arguments are special arguments available for all resource types that change the behavior of resources. The count and for_each meta-arguments allow you to create multiple instances of resources dynamically, enabling scalable and maintainable infrastructure configurations.

The count Meta-Argument

Basic Count Usage

# variables.tf
variable "instance_count" {
  description = "Number of instances to create"
  type        = number
  default     = 3
}

variable "availability_zones" {
  description = "List of availability zones"
  type        = list(string)
  default     = ["us-west-2a", "us-west-2b", "us-west-2c"]
}

# main.tf
resource "aws_instance" "web" {
  count = var.instance_count
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  
  # Use count.index to differentiate instances
  availability_zone = var.availability_zones[count.index % length(var.availability_zones)]
  
  tags = {
    Name = "web-server-${count.index + 1}"
    Index = count.index
  }
}

# Create subnets using count
resource "aws_subnet" "public" {
  count = length(var.availability_zones)
  
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet("10.0.0.0/16", 8, count.index + 1)
  availability_zone = var.availability_zones[count.index]
  
  map_public_ip_on_launch = true
  
  tags = {
    Name = "public-subnet-${count.index + 1}"
    AZ   = var.availability_zones[count.index]
  }
}

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"] # Canonical
  
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }
}

Conditional Count

# variables.tf
variable "create_monitoring" {
  description = "Whether to create monitoring resources"
  type        = bool
  default     = false
}

variable "environment" {
  description = "Environment name"
  type        = string
}

# main.tf
resource "aws_cloudwatch_log_group" "app_logs" {
  count = var.create_monitoring ? 1 : 0
  
  name              = "/aws/ec2/${var.environment}"
  retention_in_days = 14
  
  tags = {
    Environment = var.environment
  }
}

resource "aws_cloudwatch_dashboard" "monitoring" {
  count = var.create_monitoring && var.environment == "prod" ? 1 : 0
  
  dashboard_name = "${var.environment}-monitoring"
  
  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/EC2", "CPUUtilization"],
          ]
          period = 300
          stat   = "Average"
          region = "us-west-2"
          title  = "EC2 CPU Utilization"
        }
      }
    ]
  })
}

Working with Count and Lists

# variables.tf
variable "user_names" {
  description = "List of IAM user names to create"
  type        = list(string)
  default     = ["alice", "bob", "charlie"]
}

variable "bucket_names" {
  description = "List of S3 bucket names"
  type        = list(string)
  default     = ["app-data", "app-logs", "app-backups"]
}

# main.tf
resource "aws_iam_user" "users" {
  count = length(var.user_names)
  
  name = var.user_names[count.index]
  
  tags = {
    Name = var.user_names[count.index]
    Type = "application-user"
  }
}

resource "aws_iam_access_key" "user_keys" {
  count = length(aws_iam_user.users)
  
  user = aws_iam_user.users[count.index].name
}

resource "aws_s3_bucket" "app_buckets" {
  count = length(var.bucket_names)
  
  bucket = "${var.bucket_names[count.index]}-${random_id.bucket_suffix.hex}"
  
  tags = {
    Name        = var.bucket_names[count.index]
    Environment = var.environment
  }
}

resource "random_id" "bucket_suffix" {
  byte_length = 4
}

# outputs.tf
output "user_arns" {
  description = "ARNs of created IAM users"
  value       = aws_iam_user.users[*].arn
}

output "bucket_names" {
  description = "Names of created S3 buckets"
  value       = aws_s3_bucket.app_buckets[*].id
}

The for_each Meta-Argument

Basic for_each with Sets

# variables.tf
variable "availability_zones" {
  description = "Set of availability zones"
  type        = set(string)
  default     = ["us-west-2a", "us-west-2b", "us-west-2c"]
}

variable "user_roles" {
  description = "Set of user roles to create"
  type        = set(string)
  default     = ["developer", "tester", "admin"]
}

# main.tf
resource "aws_subnet" "public" {
  for_each = var.availability_zones
  
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet("10.0.0.0/16", 8, index(tolist(var.availability_zones), each.key) + 1)
  availability_zone = each.value
  
  map_public_ip_on_launch = true
  
  tags = {
    Name = "public-subnet-${each.key}"
    AZ   = each.value
  }
}

resource "aws_iam_role" "app_roles" {
  for_each = var.user_roles
  
  name = "${var.project_name}-${each.value}-role"
  
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })
  
  tags = {
    Name = "${each.value}-role"
    Type = each.value
  }
}

for_each with Maps

# variables.tf
variable "environments" {
  description = "Environment configurations"
  type = map(object({
    instance_type = string
    min_size     = number
    max_size     = number
    vpc_cidr     = string
  }))
  default = {
    dev = {
      instance_type = "t3.micro"
      min_size     = 1
      max_size     = 2
      vpc_cidr     = "10.0.0.0/16"
    }
    staging = {
      instance_type = "t3.small"
      min_size     = 1
      max_size     = 3
      vpc_cidr     = "10.1.0.0/16"
    }
    prod = {
      instance_type = "t3.medium"
      min_size     = 2
      max_size     = 10
      vpc_cidr     = "10.2.0.0/16"
    }
  }
}

# main.tf
resource "aws_vpc" "environment" {
  for_each = var.environments
  
  cidr_block           = each.value.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true
  
  tags = {
    Name        = "${var.project_name}-${each.key}-vpc"
    Environment = each.key
  }
}

resource "aws_launch_template" "environment" {
  for_each = var.environments
  
  name_prefix   = "${var.project_name}-${each.key}-"
  image_id      = data.aws_ami.ubuntu.id
  instance_type = each.value.instance_type
  
  vpc_security_group_ids = [aws_security_group.environment[each.key].id]
  
  tag_specifications {
    resource_type = "instance"
    tags = {
      Name        = "${var.project_name}-${each.key}-instance"
      Environment = each.key
    }
  }
}

resource "aws_autoscaling_group" "environment" {
  for_each = var.environments
  
  name                = "${var.project_name}-${each.key}-asg"
  vpc_zone_identifier = [aws_subnet.environment[each.key].id]
  target_group_arns   = [aws_lb_target_group.environment[each.key].arn]
  health_check_type   = "ELB"
  
  min_size         = each.value.min_size
  max_size         = each.value.max_size
  desired_capacity = each.value.min_size
  
  launch_template {
    id      = aws_launch_template.environment[each.key].id
    version = "$Latest"
  }
  
  tag {
    key                 = "Name"
    value               = "${var.project_name}-${each.key}-asg"
    propagate_at_launch = false
  }
  
  tag {
    key                 = "Environment"
    value               = each.key
    propagate_at_launch = true
  }
}

Complex for_each Scenarios

# variables.tf
variable "applications" {
  description = "Application configurations"
  type = map(object({
    port         = number
    protocol     = string
    health_path  = string
    environments = set(string)
    replicas     = map(number) # replicas per environment
  }))
  default = {
    frontend = {
      port         = 80
      protocol     = "HTTP"
      health_path  = "/health"
      environments = ["dev", "staging", "prod"]
      replicas     = {
        dev     = 1
        staging = 2
        prod    = 3
      }
    }
    backend = {
      port         = 8080
      protocol     = "HTTP"
      health_path  = "/api/health"
      environments = ["staging", "prod"]
      replicas     = {
        staging = 2
        prod    = 5
      }
    }
  }
}

# main.tf
locals {
  # Flatten applications and environments for iteration
  app_environment_pairs = flatten([
    for app_name, app_config in var.applications : [
      for env in app_config.environments : {
        app         = app_name
        environment = env
        key         = "${app_name}-${env}"
        config      = app_config
        replicas    = app_config.replicas[env]
      }
    ]
  ])
  
  # Convert to map for for_each
  app_environments = {
    for pair in local.app_environment_pairs :
    pair.key => pair
  }
}

resource "aws_lb_target_group" "app_environment" {
  for_each = local.app_environments
  
  name     = "${each.value.app}-${each.value.environment}-tg"
  port     = each.value.config.port
  protocol = each.value.config.protocol
  vpc_id   = aws_vpc.environment[each.value.environment].id
  
  health_check {
    enabled             = true
    healthy_threshold   = 2
    unhealthy_threshold = 2
    timeout             = 5
    interval            = 30
    path                = each.value.config.health_path
    matcher             = "200"
  }
  
  tags = {
    Name        = "${each.value.app}-${each.value.environment}"
    Application = each.value.app
    Environment = each.value.environment
  }
}

resource "aws_instance" "app_environment" {
  for_each = {
    for pair in flatten([
      for app_env_key, app_env in local.app_environments : [
        for i in range(app_env.replicas) : {
          key = "${app_env_key}-${i}"
          app = app_env.app
          environment = app_env.environment
          config = app_env.config
          index = i
        }
      ]
    ]) : pair.key => pair
  }
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  subnet_id     = aws_subnet.environment[each.value.environment].id
  
  vpc_security_group_ids = [aws_security_group.environment[each.value.environment].id]
  
  user_data = templatefile("${path.module}/user_data.sh", {
    app_name = each.value.app
    app_port = each.value.config.port
  })
  
  tags = {
    Name        = "${each.value.app}-${each.value.environment}-${each.value.index + 1}"
    Application = each.value.app
    Environment = each.value.environment
    Instance    = each.value.index + 1
  }
}

Advanced Patterns

Dynamic Resource Creation Based on Conditions

# variables.tf
variable "features" {
  description = "Feature flags"
  type = object({
    enable_monitoring = bool
    enable_backup    = bool
    enable_logging   = bool
    enable_cdn       = bool
  })
  default = {
    enable_monitoring = true
    enable_backup    = false
    enable_logging   = true
    enable_cdn       = false
  }
}

variable "backup_schedules" {
  description = "Backup schedule configurations"
  type = map(object({
    cron_expression = string
    retention_days  = number
  }))
  default = {
    daily = {
      cron_expression = "cron(0 2 * * ? *)"
      retention_days  = 7
    }
    weekly = {
      cron_expression = "cron(0 3 ? * SUN *)"
      retention_days  = 30
    }
  }
}

# main.tf
locals {
  # Create conditional sets for for_each
  monitoring_resources = var.features.enable_monitoring ? toset(["cloudwatch", "logs"]) : toset([])
  backup_schedules = var.features.enable_backup ? var.backup_schedules : {}
  cdn_origins = var.features.enable_cdn ? toset(["api", "assets"]) : toset([])
}

resource "aws_cloudwatch_log_group" "feature_logs" {
  for_each = local.monitoring_resources
  
  name              = "/aws/${var.project_name}/${each.key}"
  retention_in_days = 14
  
  tags = {
    Name    = "${var.project_name}-${each.key}-logs"
    Feature = "monitoring"
  }
}

resource "aws_backup_plan" "scheduled" {
  for_each = local.backup_schedules
  
  name = "${var.project_name}-${each.key}-backup"
  
  rule {
    rule_name         = "${each.key}-backup-rule"
    target_vault_name = aws_backup_vault.main.name
    schedule          = each.value.cron_expression
    
    lifecycle {
      delete_after = each.value.retention_days
    }
    
    recovery_point_tags = {
      Schedule = each.key
      Project  = var.project_name
    }
  }
  
  tags = {
    Name     = "${var.project_name}-${each.key}-backup-plan"
    Schedule = each.key
  }
}

resource "aws_cloudfront_distribution" "cdn" {
  for_each = local.cdn_origins
  
  origin {
    domain_name = aws_lb.main.dns_name
    origin_id   = "${each.key}-origin"
    origin_path = each.key == "api" ? "/api" : "/assets"
    
    custom_origin_config {
      http_port              = 80
      https_port             = 443
      origin_protocol_policy = "http-only"
      origin_ssl_protocols   = ["TLSv1.2"]
    }
  }
  
  enabled = true
  
  default_cache_behavior {
    allowed_methods        = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]
    cached_methods         = ["GET", "HEAD"]
    target_origin_id       = "${each.key}-origin"
    compress               = true
    viewer_protocol_policy = "redirect-to-https"
    
    forwarded_values {
      query_string = false
      cookies {
        forward = "none"
      }
    }
  }
  
  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }
  
  viewer_certificate {
    cloudfront_default_certificate = true
  }
  
  tags = {
    Name   = "${var.project_name}-${each.key}-cdn"
    Origin = each.key
  }
}

Nested for_each with Complex Transformations

# variables.tf
variable "microservices" {
  description = "Microservice configurations"
  type = map(object({
    image           = string
    port           = number
    cpu_request    = string
    memory_request = string
    environments   = map(object({
      replicas        = number
      env_vars       = map(string)
      resource_limits = object({
        cpu    = string
        memory = string
      })
    }))
  }))
  default = {
    user_service = {
      image           = "user-service:latest"
      port           = 8080
      cpu_request    = "100m"
      memory_request = "128Mi"
      environments = {
        dev = {
          replicas = 1
          env_vars = {
            LOG_LEVEL = "DEBUG"
            DB_HOST   = "dev-db"
          }
          resource_limits = {
            cpu    = "200m"
            memory = "256Mi"
          }
        }
        prod = {
          replicas = 3
          env_vars = {
            LOG_LEVEL = "INFO"
            DB_HOST   = "prod-db"
          }
          resource_limits = {
            cpu    = "500m"
            memory = "512Mi"
          }
        }
      }
    }
    order_service = {
      image           = "order-service:latest"
      port           = 8081
      cpu_request    = "150m"
      memory_request = "256Mi"
      environments = {
        dev = {
          replicas = 1
          env_vars = {
            LOG_LEVEL = "DEBUG"
          }
          resource_limits = {
            cpu    = "300m"
            memory = "512Mi"
          }
        }
        prod = {
          replicas = 5
          env_vars = {
            LOG_LEVEL = "WARN"
          }
          resource_limits = {
            cpu    = "1000m"
            memory = "1Gi"
          }
        }
      }
    }
  }
}

# main.tf
locals {
  # Flatten microservices and environments
  service_deployments = flatten([
    for service_name, service_config in var.microservices : [
      for env_name, env_config in service_config.environments : {
        key              = "${service_name}-${env_name}"
        service_name     = service_name
        environment      = env_name
        service_config   = service_config
        env_config      = env_config
        full_image_name = "${service_config.image}-${env_name}"
      }
    ]
  ])
  
  # Convert to map for for_each
  deployments = {
    for deployment in local.service_deployments :
    deployment.key => deployment
  }
  
  # Create load balancer target groups per service per environment
  service_target_groups = {
    for deployment in local.service_deployments :
    deployment.key => {
      name        = "${deployment.service_name}-${deployment.environment}"
      port        = deployment.service_config.port
      service     = deployment.service_name
      environment = deployment.environment
    }
  }
}

# Create ECS service definitions
resource "aws_ecs_service" "microservice" {
  for_each = local.deployments
  
  name            = "${each.value.service_name}-${each.value.environment}"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.microservice[each.key].arn
  desired_count   = each.value.env_config.replicas
  
  load_balancer {
    target_group_arn = aws_lb_target_group.microservice[each.key].arn
    container_name   = each.value.service_name
    container_port   = each.value.service_config.port
  }
  
  network_configuration {
    subnets         = [aws_subnet.private[each.value.environment].id]
    security_groups = [aws_security_group.microservice[each.value.environment].id]
  }
  
  tags = {
    Name        = "${each.value.service_name}-${each.value.environment}"
    Service     = each.value.service_name
    Environment = each.value.environment
  }
}

# Create task definitions
resource "aws_ecs_task_definition" "microservice" {
  for_each = local.deployments
  
  family                   = "${each.value.service_name}-${each.value.environment}"
  requires_compatibilities = ["FARGATE"]
  network_mode            = "awsvpc"
  cpu                     = each.value.env_config.resource_limits.cpu
  memory                  = each.value.env_config.resource_limits.memory
  
  container_definitions = jsonencode([
    {
      name  = each.value.service_name
      image = each.value.full_image_name
      
      portMappings = [
        {
          containerPort = each.value.service_config.port
          hostPort      = each.value.service_config.port
        }
      ]
      
      environment = [
        for key, value in each.value.env_config.env_vars : {
          name  = key
          value = value
        }
      ]
      
      resources = {
        requests = {
          cpu    = each.value.service_config.cpu_request
          memory = each.value.service_config.memory_request
        }
        limits = {
          cpu    = each.value.env_config.resource_limits.cpu
          memory = each.value.env_config.resource_limits.memory
        }
      }
      
      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-group         = aws_cloudwatch_log_group.microservice[each.key].name
          awslogs-region        = data.aws_region.current.name
          awslogs-stream-prefix = "ecs"
        }
      }
    }
  ])
  
  tags = {
    Name        = "${each.value.service_name}-${each.value.environment}"
    Service     = each.value.service_name
    Environment = each.value.environment
  }
}

# Create target groups
resource "aws_lb_target_group" "microservice" {
  for_each = local.service_target_groups
  
  name     = each.value.name
  port     = each.value.port
  protocol = "HTTP"
  vpc_id   = aws_vpc.main.id
  
  health_check {
    enabled             = true
    healthy_threshold   = 2
    unhealthy_threshold = 2
    timeout             = 5
    interval            = 30
    path                = "/health"
    matcher             = "200"
  }
  
  tags = {
    Name        = each.value.name
    Service     = each.value.service
    Environment = each.value.environment
  }
}

# Create log groups
resource "aws_cloudwatch_log_group" "microservice" {
  for_each = local.deployments
  
  name              = "/ecs/${each.value.service_name}-${each.value.environment}"
  retention_in_days = each.value.environment == "prod" ? 30 : 7
  
  tags = {
    Name        = "${each.value.service_name}-${each.value.environment}-logs"
    Service     = each.value.service_name
    Environment = each.value.environment
  }
}

data "aws_region" "current" {}

count vs for_each: When to Use Which

Use count when:

  • Creating a simple number of identical resources
  • The number of resources is determined by a numeric variable
  • Order matters and you need to reference by index
  • Working with lists where position is important

Use for_each when:

  • Creating resources based on a map or set
  • Each resource instance needs different configuration
  • You want to add/remove resources without affecting others
  • Working with complex data structures

Comparison Example

# Using count (good for simple scenarios)
resource "aws_instance" "web_count" {
  count = var.instance_count
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
  
  tags = {
    Name = "web-${count.index + 1}"
  }
}

# Using for_each (better for complex scenarios)
resource "aws_instance" "web_foreach" {
  for_each = var.instance_configs  # map of configurations
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = each.value.instance_type
  
  tags = merge(each.value.tags, {
    Name = "${each.key}-instance"
  })
}

# Variable for for_each approach
variable "instance_configs" {
  type = map(object({
    instance_type = string
    tags         = map(string)
  }))
  default = {
    web1 = {
      instance_type = "t3.micro"
      tags = {
        Role = "web"
        Tier = "frontend"
      }
    }
    web2 = {
      instance_type = "t3.small"
      tags = {
        Role = "web"
        Tier = "frontend"
      }
    }
    api = {
      instance_type = "t3.medium"
      tags = {
        Role = "api"
        Tier = "backend"
      }
    }
  }
}

Best Practices

1. Resource Naming and Tagging

# Good: Consistent naming with meta-arguments
resource "aws_instance" "app" {
  for_each = var.environments
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = each.value.instance_type
  
  tags = merge(var.common_tags, {
    Name        = "${var.project_name}-${each.key}-app"
    Environment = each.key
    ManagedBy   = "terraform"
  })
}

2. Output Organization

# outputs.tf
output "instance_details" {
  description = "Details of all created instances"
  value = {
    for env, instance in aws_instance.app :
    env => {
      id         = instance.id
      public_ip  = instance.public_ip
      private_ip = instance.private_ip
    }
  }
}

output "instance_ids" {
  description = "List of all instance IDs"
  value = [for instance in aws_instance.app : instance.id]
}

3. Handling Dependencies

# Explicit dependencies with meta-arguments
resource "aws_security_group" "app" {
  for_each = var.environments
  
  name_prefix = "${var.project_name}-${each.key}-"
  vpc_id      = aws_vpc.environment[each.key].id
  
  # Security group rules depend on this resource existing
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_instance" "app" {
  for_each = var.environments
  
  ami           = data.aws_ami.ubuntu.id
  instance_type = each.value.instance_type
  
  vpc_security_group_ids = [aws_security_group.app[each.key].id]
  
  # Explicit dependency ensures proper creation order
  depends_on = [aws_security_group.app]
}

Common Pitfalls and Solutions

1. Changing count to for_each

# This will cause resource recreation
# Before:
resource "aws_instance" "web" {
  count = 3
  # ...
}

# After:
resource "aws_instance" "web" {
  for_each = toset(["web1", "web2", "web3"])
  # ...
}

# Solution: Use terraform state mv to migrate
# terraform state mv 'aws_instance.web[0]' 'aws_instance.web["web1"]'

2. Dynamic Values in count/for_each

# Bad: count/for_each cannot be determined until apply
resource "aws_instance" "web" {
  count = length(data.aws_availability_zones.available.names)  # Works
  # ...
}

resource "aws_instance" "bad" {
  count = aws_autoscaling_group.web.desired_capacity  # Doesn't work
  # ...
}

# Good: Use known values or locals
locals {
  instance_count = var.dynamic_scaling ? var.max_instances : var.min_instances
}

resource "aws_instance" "good" {
  count = local.instance_count
  # ...
}

Key Takeaways

  1. Choose Appropriately: Use count for simple scenarios, for_each for complex configurations
  2. Plan for Changes: Consider how adding/removing resources will affect your infrastructure
  3. Use Locals: Leverage local values for complex transformations before meta-arguments
  4. Consistent Naming: Establish clear naming conventions for resources created with meta-arguments
  5. Understand Dependencies: Be aware of how meta-arguments affect resource dependencies
  6. State Management: Plan for potential state migrations when changing between count and for_each

Next Steps

  • Tutorial 20: Learn about additional meta-arguments (depends_on, lifecycle)
  • Practice converting between count and for_each approaches
  • Experiment with complex nested iterations
  • Review the Terraform Meta-Arguments Documentation

Additional Resources

PreviousOutput DependenciesNextDependencies and Lifecycle