An HDInsight Spark Cluster Behind a Private Link with No Storage Key in State
An HDInsight Spark cluster in your virtual network, reached over a private link, on ADLS Gen2 through a user-assigned identity so no storage key sits in state, with encryption in transit and TLS 1.2 on the gateway. There is no scale-to-zero: nodes_at_rest says what bills when the cluster idles. An external Hive metastore makes the cluster disposable; the public gateway is accepted by name.
Verification
Static-verifiedPassed: validated and lint-clean (provider-schema-validated for AWS/Azure/GCP; Terraform-language lint elsewhere).
Conformance
- Static validation (fmt · validate · tflint)
- No applicable security policies for this provider
- Plan tests (mocked: validation rules · outputs)
Provenance
- SHA-256 checksum
- Signature (pending)
Functional
- Live test pending (no cloud run yet)
Last verified 2026-09-14 · how we verify
Use it from the registry
terraform · opentofumodule "hdinsight_spark" {
source = "www.iac-bazaar.com/iac-bazaar/azure-hdinsight-spark/azure"
version = "1.0.0"
}Needs a registry token from /account/tokens. The module itself is free; the account is what identifies you. Full setup: registry docs.
Inputs & outputs
Create a free account to read this module's contract
The declared contract - every input name, type, default and description, plus every output - is shown to signed-in accounts, not to anonymous visitors.
A free account sees the contract of every module in the catalogue. There is no subscription and nothing to buy - the modules are free to download, and they run under Vizier.
Documentation
azure-hdinsight-spark
An HDInsight Spark cluster in your VNet behind a private link, on ADLS
Gen2 through a managed identity, encrypted in transit. Works with
Terraform and OpenTofu (>= 1.6), azurerm provider >= 4.0, < 5.0.
A cluster bills by the hour from the minute it exists; there is no
scale-to-zero, and nodes_at_rest says what runs idle.
Storage keys in state are avoided: Gen2 through a user-assigned identity, no account key.
Public by default; private link here, the public gateway by name.
Verification
Static validation runs tofu fmt, init, validate, tflint and checkov.
This module has not yet had a live test, so it is published as statically
validated with its live test pending and does not carry the live-tested mark.
Usage code & full reference need an account
The complete copy-paste usage, the full input/output reference, and operational notes are free with an account - shown here and bundled in the download. Sign in and this section fills in.
- Usage
Related modules
azure-databricks
The default deployment puts the clusters in a managed network you cannot see with a public IP per node, and the workspace URL - notebooks, jobs, tokens - is reachable from the internet. VNet injection into your subnets with no public IPs, the workspace endpoint off the internet, and one Key Vault key wired to all four encryption settings including the DBFS root, a separate resource.
azure-data-explorer
The Dev(No SLA) SKUs say it in the name and are what a proof of concept becomes production on; public network access is on by default; disk and double encryption are off by default and set only at creation. Standard SKU with two instances across zones, private endpoints, both encryption layers on, a customer-managed key, and purge enabled because it is the only way to honour an erasure request.
azure-sql-managed-instance
The public data endpoint turns a private database into one listening on the internet on port 3342; SQL logins put an administrator password in state when Entra-only authentication would remove them entirely; and zone redundancy is off by default. Private, Entra-only with SQL authentication accepted by name, Business Critical across zones, TLS 1.2, and geo-zone-redundant backups.
azure-search-service
One replica - the default - is excluded from the availability SLA; the admin key, on by default, reads and writes every index and is revoked only by regenerating it for everyone; and the query endpoint is public by default. Three replicas, Entra ID only, private endpoint, 0.0.0.0/0 refused in the allow list, and optional enforcement that refuses an index without a customer-managed key.
azure-synapse
A Synapse workspace with a dedicated SQL pool and no SQL login: Entra-only authentication with a group as admin, a managed virtual network with data exfiltration protection (neither can be turned on later), public endpoints off, extended auditing and threat detection on the workspace and the pool, and vulnerability scans when you name a container. The pool bills by the hour while online.
azure-redis-cache
Azure Cache for Redis done cheap by default - the Basic C0 tier with TLS 1.2 minimum and the non-SSL port disabled - scaling cleanly up to Standard and Premium via precondition-guarded inputs.