A Dataflow Job on Private Workers that Drains on Destroy
on_delete defaults to cancel, which discards every element in flight on a streaming pipeline the moment the job is destroyed or replaced; workers get public IPs by default; and the default worker identity is the Compute Engine default account. Drain on delete, private workers with Private Google Access, a dedicated service account with the default refused, and a customer-managed key.
Verification
Static-verifiedPassed: validated and lint-clean (provider-schema-validated for AWS/Azure/GCP; Terraform-language lint elsewhere).
Conformance
- Static validation (fmt · validate · tflint)
- Security scan clean (Checkov)
- Plan tests (mocked: validation rules · outputs)
Provenance
- SHA-256 checksum
- Signature (pending)
Functional
- Live test pending (no cloud run yet)
Last verified 2026-09-14 · how we verify
Use it from the registry
terraform · opentofumodule "dataflow" {
source = "www.iac-bazaar.com/iac-bazaar/gcp-dataflow/gcp"
version = "1.0.0"
}Needs a registry token from /account/tokens. The module itself is free; the account is what identifies you. Full setup: registry docs.
Inputs & outputs
Create a free account to read this module's contract
The declared contract - every input name, type, default and description, plus every output - is shown to signed-in accounts, not to anonymous visitors.
A free account sees the contract of every module in the catalogue. There is no subscription and nothing to buy - the modules are free to download, and they run under Vizier.
Documentation
gcp-dataflow
A Dataflow job on private workers, as a named account, that drains rather
than drops on destroy. Works with Terraform and OpenTofu (>= 1.6),
google provider >= 6.0, < 7.0.
on_delete defaults to cancel, which discards every element in flight
on a streaming pipeline. drain here; cancel needs
accept_cancel_on_delete.
Workers get public IPs by default. Private here, with a subnetwork
that has Private Google Access; public needs accept_public_workers.
The default worker account is the Compute Engine default. A dedicated
service_account_email is required and the default refused.
A customer-managed key is expected for disks and state; Google's needs
accept_google_managed_key.
Verification
Static validation runs tofu fmt, init, validate, tflint and checkov.
This module has not yet had a live test, so it is published as statically
validated with its live test pending and does not carry the live-tested mark.
Usage code & full reference need an account
The complete copy-paste usage, the full input/output reference, and operational notes are free with an account - shown here and bundled in the download. Sign in and this section fills in.
- Usage
Related modules
gcp-cloud-deploy
require_approval defaults to false on every target, so a release rolls into production the moment somebody promotes it with nobody signing off; the verify flag defaults off, so skaffold verify has never run. Treats the last stage as production and refuses a pipeline whose last target skips approval, verifies after every rollout, and refuses the Compute default service account as the runner.
gcp-cloud-build
A trigger with no service account runs every step - including code from the pull request under test - as the broadest identity in the project; a trigger that deploys on push deploys whatever lands; the shared default pool has external IPs. A dedicated runner with the defaults refused, a private worker pool with no external addresses, and approval required on any trigger marked as deploying.
gcp-composer
Managed Apache Airflow on Cloud Composer 2 with small-by-default sizing, worker autoscaling pinned for predictable cost, and an opt-in private environment posture.
gcp-workflows
A Cloud Workflows workflow that runs as a dedicated least-privilege service account instead of the broad Compute Engine default, with inline YAML, deletion protection, and call logging.
gcp-dataproc
A single-node Dataproc cluster (1 master, 0 workers), the cheapest managed Spark/Hadoop cluster that still applies and destroys cleanly, with internal-only IPs and deletion protection on.