The Case for Terraform Modules: Scaling Your Infrastructure Organization

Infrastructure teams working with Terraform or OpenTofu inevitably encounter scaling challenges as their deployments grow. What begins as straightforward resource definitions evolves into complex configurations replicated across environments, applications, and teams. The initial solution typically involves copying and modifying Terraform code for each new service or environment, an approach which will start to accumulate technical debt as new services and applications are added.

Why Use Terraform Modules?

Terraform modules address this challenge by transforming infrastructure patterns into version-controlled, reusable components. Rather than duplicating configuration blocks, engineering teams can encapsulate common patterns in modules that can be reused with different input variables. This approach shifts infrastructure development from manual replication to systematic component reuse.

When a team needs to make a change to a load balancer configuration, or update security policies, they modify the relevant module once. Services using that module can then adopt these improvements through controlled version updates.

The impact of modular infrastructure extends beyond code organization. Teams using modules effectively find their development patterns shift from reactive copying to proactive design. Security policies, compliance requirements, and infrastructure optimizations become standardized components rather than isolated configurations. This standardization reduces configuration drift while helping to maintain infrastructure consistency at scale.

Local or External Modules: What's the Right Approach?

The path from local to external modules typically mirrors an organization's infrastructure maturity and scale. Teams often start with local modules during early development, placing reusable configurations in their infrastructure repository's modules directory. This approach makes perfect sense when one or two engineers are rapidly iterating on infrastructure; changes are immediate, testing is straightforward, and version control happens naturally alongside the main configuration.

Local modules live within your infrastructure repository's structure, typically in a modules directory with each module maintaining its own files for resources, variables, and outputs. Changes to these modules track alongside your root configuration in the same commit history, making it simple to correlate module updates with infrastructure changes:

module "app_networking" {
  source = "./modules/app_networking"
 
  vpc_cidr     = "10.0.0.0/16"
  environment  = "production"
}

As organizations mature their Infrastructure as Code (IaC) practices, they typically move modules to dedicated Git repositories. This shift occurs when teams need to share infrastructure patterns across multiple projects or when they need proper versioning to manage changes. Git-hosted modules support specific version pinning through commits or tags, enabling control of exactly which module version each configuration uses:

module "web_platform" {
  source = "git::https://example.com/infrastructure.git//modules/web-platform?ref=v1.2.0"
}

For larger repositories that contain multiple related modules, subdirectory-level organization can be employed. The double-slash syntax in module sources enables referencing these subdirectories, while keeping related modules versioned together:

locals {
  infra_repo = "git::https://example.com/infrastructure.git//modules"
  version    = "?ref=v1.2.0"
}

module "networking" {
  source = "${local.infra_repo}/networking${local.version}"
}

When infrastructure complexity and scale reaches a certain threshold, organizations typically invest in a TACOS (Terraform/OpenTofu Automation and Collaboration Software) platform like Terrateam, that provides a variety of automation features, as well as a hosted instance of a private module registry.

Registry-hosted modules introduce a more structured approach to versioning and distribution:

module "web_platform" {
  source  = "private-registry.example.com/infrastructure/web-platform"
  version = "~> 1.2.0"
}

Throughout development, engineers will inevitably face the decision about whether to build custom modules or leverage existing ones from the public, community registries like Terraform or OpenTofu. These registries offer thousands of community-maintained modules for common infrastructure patterns, from VPC configurations to database deployments. So should you ever have to re-invent the wheel?

From a security perspective, internally created modules will generally be the more secure option, especially for critical infrastructure. External modules require careful review of their implementation details: a seemingly innocuous compute module may have a malicious AMI image hardcoded into the configuration. Beyond just critical security issues, externally-sourced modules will inevitably have some degree of Swiss-army knife configuration; they have to be built to support as many use-cases as possible. Introducing an external module may provide the desired infrastructure, but at the cost of stale or unused configuration and additional technical debt.

Version management differs significantly between internal and external modules. With internal modules, teams control the entire update cycle. External modules might introduce breaking changes or dependency updates that conflict with your infrastructure. Some teams address this by forking and internally maintaining external modules, though this creates its own maintenance burden.

Going from local modules to external also means that authentication and authorization logic has to be implemented. Git-hosted modules typically rely on SSH keys or access tokens, requiring credential/secret management. Private registries add another layer of authentication, often integrating with organizational SSO while requiring service credentials for CI/CD systems.

For most enterprises, module usage progresses through these stages as their infrastructure matures:

1. Local modules serve early development when teams are small and actively iterating on infrastructure patterns.
2. Git-hosted modules emerge as organizations mature their Infrastructure as Code practices and need to share modules across teams.
3. Private registry modules through TACOS platforms become necessary when infrastructure complexity and scale demand better automation, governance and distribution mechanisms.

Handling Secrets in Modules: The Infisical Approach

As teams scale their Terraform deployments, managing secrets and credentials quickly becomes a pain point. Hardcoding credentials directly in Terraform code is obviously problematic from a security perspective, but even using environment variables or .tfvars files creates operational challenges as infrastructure grows more complex. Infrastructure teams eventually face the challenging question: how do our reusable modules securely access the credentials they need?

Infisical integrates with Terraform to tackle this challenge by providing secure secrets management that works seamlessly with modular infrastructure. With Terraform v1.10+, you can use ephemeral resources to ensure secrets never persist in state files:

terraform {
  required_providers {
    infisical = {
      source  = "infisical/infisical"
      version = ">= 0.4.0"
    }
  }
  required_version = ">= 1.10.0"
}

module "database" {
  source = "./modules/database"
  
  name         = "customer-db"
  engine       = "postgres"
  instance_type = "db.t3.medium"
  
  # Reference ephemeral secrets from Infisical
  credentials = {
    username = local.db_credentials.username
    password = local.db_credentials.password
  }
}

# Fetch secret ephemerally (never persisted in state)
ephemeral "infisical_secret" "db_creds" {
  name         = "DB_CREDENTIALS"
  env_slug     = var.environment
  workspace_id = var.infisical_workspace_id
  folder_path  = "/database"
}

# Decode the JSON secret into usable values
locals {
  db_credentials = jsondecode(ephemeral.infisical_secret.db_creds.value)
}

This approach solves several problems simultaneously:

1. Sensitive values never enter your Terraform code or state files.
2. Credential rotations happen outside your infrastructure code, reducing the need for redeployments when security policies require password changes.
3. Your modules remain flexible and reusable, as they simply use the credentials they're given without needing to know where they came from.

Read also: How to fetch secrets from Infisical with Terraform using both traditional data sources and ephemeral resources

The pattern works particularly well for database modules, where you're often dealing with both admin credentials during provisioning and application credentials for service connectivity. Rather than managing two sets of credentials through variables and outputs, both can be handled through Infisical, reducing complexity in your module interfaces.

Conclusion

At some point, local modules won't be enough. As infrastructure scales, teams need a way to manage module versions, roll out updates safely, and enforce consistency across environments. That's where structured module management and automation come in. The next post will cover how to organize Terraform modules, handle versioning, and avoid the pitfalls of scaling infrastructure code.