Terraform Azure Log Analytics Data Export

Where workspace data goes, and what it costs to keep it: continuous export rules to storage or Event Hubs, and per table plan and retention.

---

Overview

Two resources, one question: what leaves the workspace, and how long does the rest stay. Data export is the supported route into an external SIEM or a tamper-protected archive, and table plans are the largest cost lever a workspace has. Neither is reached by terraform-azurerm-log-analytics-workspace, which sets plan and retention on CUSTOM tables only, so the high volume built-in tables sit on defaults forever.

• One id in, resource group derived. A data export rule is an ARM child of the workspace, so its resource group cannot differ. The module derives it from workspace_id rather than asking for a name that must agree with an id.
• enabled defaults to TRUE, against the AzureRM provider's own default of false. A rule you took the trouble to declare is a rule you want running, and a silently disabled export that reports success is a bad default for something whose whole job is to move data.
• Platform limits are enforced at plan time. Ten rules per workspace. A storage account may back only one rule. Rule names 4 to 63 characters. Valid destination kinds. Basic tables reject an explicit retention because the platform fixes it at 30 days. Total retention must be a permitted step and must not fall below analytics retention.
• event_hub_names tells the consumer what to subscribe to. Exporting to a NAMESPACE auto-creates one hub per table, named am-<TableName>. That name is what a Splunk add-on needs, and it is otherwise discovered by looking in the portal after the fact.

The check that matters most

A table on the Basic plan cannot be read by a scheduled alert rule. Move a table that a Sentinel analytics rule queries onto Basic and that detection stops matching: no error, no failed run, and nothing in the rule's own history explaining why it went quiet. The module warns about every Basic table for that reason.

Because a legitimate Basic table would then warn on every plan forever, the warning is silenced by naming the table in basic_plan_acknowledged. That is deliberate: the acknowledgement lives in code, gets reviewed in the pull request that introduces it, and turns a permanent warning into a decision somebody made once and signed. The same pattern as the Trivy waivers below.

Two further advisory checks: more than ten tables exported to an Event Hubs namespace (Basic and Standard cap at ten hubs, so the excess never starts), and analytics retention below 31 days (which saves nothing, because 31 days are included in the ingestion price, and only shortens the window an analyst can query).

Known limits

• Region is not checkable. Destinations must be in the same region as the workspace, and no resource id reveals a region, so a mismatch fails at apply rather than at plan.
• Premium storage accounts are not supported by data export, and the tier is likewise invisible in an id.
• The Auxiliary plan is not exposed by the AzureRM provider, so a table destined for Auxiliary has to be set outside this module for now.

Usage

module "data_export" {
  source  = "libre-devops/log-analytics-data-export/azurerm"
  version = "~> 1.0"

  workspace_id = module.law.workspace_ids["log-ldo-uks-prd-001"]

  data_export_rules = {
    "siem-forward" = {
      destination_resource_id = azurerm_eventhub.siem.id
      table_names             = ["SecurityEvent", "SigninLogs", "AzureActivity"]
    }
  }

  table_plans = {
    SecurityEvent = {
      retention_in_days       = 180
      total_retention_in_days = 2556
    }
  }
}

Examples

• examples/minimal - one table's retention extended, no destinations needed.
• examples/complete - both destination kinds and both table plans, with the storage account and Event Hubs namespace they need, applying and destroying in one run.

Developing

Local work needs PowerShell 7+ and just, because the recipes wrap the LibreDevOpsHelpers PowerShell module (the same engine the libre-devops/terraform-azure action runs in CI). Install just with brew install just, or uv tool add rust-just then uv run just <recipe>.

Run just to list recipes: just update-ldo-pwsh (install or force-update LibreDevOpsHelpers from PSGallery), just validate, just scan (Trivy only), just pwsh-analyze (PSScriptAnalyzer only), just plan, just apply, just destroy, just e2e, just test, and just docs (the plan/apply/destroy recipes mirror the action, including the storage firewall dance; just e2e applies an example then always destroys it, defaulting to minimal, so nothing is left running). Releasing is also just: just increment-release [patch|minor|major] bumps, tags, and publishes a GitHub release, and the Terraform Registry picks up the tag.

The test suite is plan-time and offline: terraform init -backend=false && terraform test needs no credentials. Note that terraform test treats a check block failure as a run failure, so the advisory checks cannot be asserted from a passing run; they are exercised by hand against a deliberately broken configuration when they change.

Security scan exceptions

This module is scanned with Trivy; HIGH and CRITICAL findings fail the build. Any waiver is a deliberate, reviewed decision, never a way to quiet a finding that should be fixed. Waivers live in .trivyignore.yaml (the machine-applied source of truth, passed to Trivy with --ignorefile) and are mirrored in the table below so the reason is auditable.

Trivy ID	Resource	Finding	Justification
None

To add an exception: add an entry to .trivyignore.yaml (id, optional paths to scope it, and a statement recording why), then add a matching row here. Where the finding is out of this module's scope, point the justification at the Libre DevOps module that does address it (for example the private-endpoint module). Both the file and this table are reviewed in the pull request.

Reference

The Requirements, Providers, Inputs, Outputs, and Resources below are generated by terraform-docs.

Requirements

Name	Version
terraform	>= 1.9.0, < 2.0.0
azurerm	>= 4.0.0, < 5.0.0

Providers

Name	Version
azurerm	>= 4.0.0, < 5.0.0

Modules

No modules.

Resources

Name	Type
azurerm_log_analytics_data_export_rule.this	resource
azurerm_log_analytics_workspace_table.this	resource

Inputs

Name	Description	Type	Default	Required
basic_plan_acknowledged	Tables you have confirmed no scheduled alert rule reads, and which may therefore sit on the Basic plan without warning. A Basic table cannot be queried by a scheduled alert rule, so moving a table onto Basic stops any Sentinel analytics rule that reads it from ever matching again, with no error and nothing in the rule's own run history explaining why. The module warns about every Basic table for that reason. Naming a table here silences the warning for it. That is deliberate: the acknowledgement lives in code, gets reviewed in the pull request that introduces it, and turns a warning that would otherwise fire on every plan forever into a decision somebody made once and signed.	set(string)	[]	no
data_export_rules	Continuous data export rules, keyed by rule name. Each streams the named tables out of the workspace as the data arrives, to a storage account or Event Hubs. This is the supported path into an external SIEM (Splunk through the Add-on for Microsoft Cloud Services reads the Event Hub), into a tamper-protected archive (a storage account with an immutability policy), or into a cheap long-term store. enabled defaults to TRUE here, which differs from the AzureRM provider's own default of false. A rule you took the trouble to declare is a rule you want running, and a silently disabled export that reports success is a bad default for something whose whole job is to move data. The platform limits are enforced at plan time where they can be: - Ten active rules per workspace, each covering as many tables as you like. - A storage account may back only one rule in the workspace. - Destinations must be in the same region as the workspace, which no id reveals, so it is on you. A cross-region destination fails at apply. - Premium storage accounts are not supported. Exporting to an Event Hubs NAMESPACE creates one event hub per table, named am-. Basic and Standard namespaces cap at ten event hubs, so more than ten tables to a namespace needs either several namespaces or a single named event hub as the destination instead. A check block reports this.	map(object({ destination_resource_id = string table_names = set(string) enabled = optional(bool, true) }))	{}	no
table_plans	Per table plan and retention, keyed by table name. This is the largest cost lever a workspace has, and it is the one nothing else in the library reaches: the workspace module sets plan and retention on CUSTOM tables only, so the high volume built-in tables sit on defaults forever. This resource does NOT create or delete tables. It changes settings on tables that already exist, and removing an entry reverts that table to the workspace default retention rather than leaving it as you last set it. plan is Analytics or Basic. The AzureRM provider does not yet expose the Auxiliary plan, so a table destined for Auxiliary has to be set outside this module for now. - Analytics: queryable normally, retention 4 to 730 days. - Basic: cheaper ingestion, retention fixed at 30 days and not settable, and the query surface is restricted. Critically, a Basic table cannot be used by a scheduled alert rule, so moving a table a Sentinel analytics rule reads onto Basic breaks that detection silently. A check block reports every table this call puts on Basic for exactly that reason. retention_in_days is analytics retention. Below 31 days it saves nothing, because 31 days are included in the ingestion price; a check block reports that too. total_retention_in_days adds cheap long-term retention on top, reachable only by search job. It accepts 4 to 730, or one of the yearly steps up to 4383 (12 years).	map(object({ plan = optional(string, "Analytics") retention_in_days = optional(number) total_retention_in_days = optional(number) }))	{}	no
workspace_id	Resource id of the Log Analytics workspace. Both the export rules and the table settings hang off it, and the export rules' resource group is derived from this id rather than asked for separately, because a data export rule is an ARM child of the workspace and cannot live anywhere else.	string	n/a	yes

Outputs

Name	Description
basic_plan_tables	Tables this module puts on the Basic plan, sorted. Worth asserting against in a downstream check, because a Basic table cannot be read by a scheduled alert rule.
data_export_rule_export_ids	Map of rule name to the export_rule_id Azure allocates, which is the identifier that appears in the destination's own diagnostics rather than the ARM id.
data_export_rule_ids	Map of rule name to its resource id.
data_export_rules	The full data export rule objects, keyed by rule name.
event_hub_names	The event hub Azure auto-creates for each table exported to an Event Hubs namespace, keyed by rule name. Azure names them am-, and a consumer such as a Splunk add-on needs these exact names to subscribe.
exported_tables	Every table covered by an enabled export rule, deduplicated and sorted. The answer to 'is this table leaving the workspace', which is otherwise a read of every rule.
resource_group_name	The resource group derived from workspace_id, which is where the export rules are created.
table_plan_ids	Map of table name to its resource id.
table_plans	The full table setting objects, keyed by table name.