Security Compliance Monitor

Challenge

Email authentication reports (DMARC and TLS-RPT) arrive by email, and in most organisations that is where they stay — never archived, never analysed, never watched. Authentication failures and compliance drift go unnoticed until something bounces.

Approach

A scheduled collector pulls the reports from the mailboxes through Microsoft Graph, copes with attachments that arrive without usable filenames, and hands them to a queue. A C# processor on an Azure Functions Flex Consumption plan, with no always-ready instances, runs when work arrives and costs nothing while it waits. It began as a container on Azure Container Apps, scaled from zero by KEDA, and moved to the Functions host once the processing below had been made safe to interrupt. Processing is idempotent: each message is claimed atomically so two runs can never process it twice, and any run interrupted part-way is recovered before new work begins, so no report is lost. Reports are archived to blob storage and their results written to the first-party analytics database, where an analysis layer identifies findings and tracks compliance trends that surface in the internal dashboard and the daily digest. On the container host, deployed images were pinned by digest so a restart brought back exactly what was built. Failures and missing runs raise alerts.

Outcome

• Every report archived and analysed automatically — no manual collection
• Authentication failures and compliance gaps raised as alerts, with trends visible over time
• An audit trail of what was collected, when and why — the evidence a regulator would ask for
• No idle compute between runs, and compliance data queryable alongside session and conversion records
• Running cost fell from £26.83 in the pipeline's last full month on the container-based host to £2.21 in September 2026, its first full month on a consumption plan holding no always-ready instances — the same work, without paying for idle compute. Figures are Azure Cost Management actuals for the dedicated resource group.
• The same work either side of the move: 199 reports processed in August 2026 and 181 reports in September, so the cost per report fell from about 13.5p to about 1.2p
• 678 reports processed for my own domains between 19 June and 15 October 2026, raising 485 findings

Where judgement mattered

• Scale-to-zero compute for bursty work instead of an always-on service, and moving host only after the processing could survive being interrupted
• Atomic claiming and startup recovery, so "at least once" never becomes "twice" or "never"
• Writing results to the shared first-party store rather than a silo, so they can be joined with everything else
• Digest-pinned deployments on the container host — reproducible images, no floating tags