Pub/Sub Topics & Subscriptions

gcp-pubsub

Pub/Sub topics with schemas, push/pull/BigQuery subscriptions, dead-letter queues and retry policies preconfigured. The parts hand-rolled configs always miss are wired in: a DLQ topic plus a retention subscription (without one, dead letters are silently dropped), and the Pub/Sub service-agent IAM grants (publisher on the DLQ, subscriber on each source subscription) without which dead-lettering simply does not fire. Works with Terraform and OpenTofu (>= 1.6), Google provider >= 7.0, < 8.0.

What you get per module call:

• One topic, optionally schema-enforced (AVRO / protobuf) and CMEK-encrypted
• Any mix of pull, push (with OIDC auth) and BigQuery subscriptions
• Dead-letter topic + 7-day retention subscription + service-agent IAM, on by default for every subscription (opt out per subscription)
• Exponential retry policy (10s–600s) instead of the instant-retry default
• Publisher/subscriber IAM grants as plain member lists

Usage

module "orders" {
  source = "./gcp-pubsub"

  project_id = "my-project"
  topic_name = "orders"

  subscriptions = {
    worker = {} # pull, DLQ after 5 attempts, sane retry backoff

    webhook = {
      type                            = "push"
      push_endpoint                   = "https://worker.example.com/pubsub"
      push_oidc_service_account_email = "[email protected]"
    }

    warehouse = {
      type           = "bigquery"
      bigquery_table = "my-project.analytics.orders_raw"
      dead_letter    = false
    }
  }

  publisher_members = ["serviceAccount:[email protected]"]

  labels = { environment = "prod" }
}

Subscription resource names are prefixed with the topic (orders-worker, …) so multiple module instances never collide. BigQuery subscriptions additionally need the Pub/Sub service agent granted BigQuery write access on the target dataset — that grant is dataset-scoped and stays outside this module.

Requirements

Requirement	Version
Terraform / OpenTofu	>= 1.6
hashicorp/google	>= 7.0, < 8.0

Inputs

Name	Type	Default	Description
project_id	string	—	GCP project ID (required)
topic_name	string	—	Topic name; prefixes subscriptions + DLQ (required)
schema	object	null	{ name, type (AVRO/PROTOCOL_BUFFER), definition, encoding } — changing the definition replaces the topic
subscriptions	map(object)	one pull default	Per key: type (pull/push/bigquery), push endpoint + OIDC, BigQuery table options, ack deadline, retention, filter, ordering, exactly-once, dead_letter (default true), max_delivery_attempts (5–100), retry backoff, subscriber_members
create_dead_letter_subscription	bool	true	Retention pull subscription on the DLQ topic
publisher_members	list(string)	[]	Members granted roles/pubsub.publisher on the topic
kms_key_name	string	null	CMEK key for topic + DLQ topic
message_retention_duration	string	null	Topic-level retention (e.g. "86600s")
labels	map(string)	{}	Labels on all topics/subscriptions

Outputs

Name	Description
topic_id / topic_name	The topic
schema_id	Schema ID (null when unset)
subscription_ids / subscription_names	Map keyed by subscription key
dead_letter_topic_id	DLQ topic ID (null when unused)
dead_letter_subscription_id	DLQ retention subscription ID

Verification

Static-validated (fmt, validate, tflint). Live apply/destroy testing pending cloud sandbox availability — see catalog status.

License

Commercial — IaC Bazaar EULA. © IaC Bazaar. Original work (not derived from a third-party module).