GRC Engineer
Posts
⚙️ Signal vs. Noise: The Mental Model That Transforms GRC Effectiveness

⚙️ Signal vs. Noise: The Mental Model That Transforms GRC Effectiveness

Why your green compliance dashboards are hiding real security gaps, and the simple So What? test that reveals what actually matters

Ayoub Fandi
July 24, 2025

Your GRC program is drowning in data but starving for insight.

You're tracking compliance percentages, control implementation rates, finding counts, and risk scores. Your dashboards are green. Your metrics look impressive. Your audit evidence fills terabytes of storage.

But if you're honest about it, most of what you're measuring is noise masquerading as signal.

The Signal vs. Noise mental model, borrowed from information theory and popularised in financial markets, provides a powerful framework for distinguishing meaningful security indicators from compliance theatre metrics.

It's time to stop measuring everything and start measuring what matters.

IN PARTNERSHIP WITH

Test your Snowflake data security skills and win prizes!

Created by Varonis Threat Labs, this CTF experience is built to help defenders understand how threat actors can exploit data in Snowflake environments.

You’re a white-hat agent hired by Glacier Corp., who just received an anonymous tip that hackers plan to exploit PII from the company’s Snowflake database.

Time is melting away. Every minute is closer to a breach.

Why It Matters 🔍

The why: the core problem this solves and why you should care

Most GRC programs suffer from what I call "metric inflation", the false belief that tracking more data automatically leads to better security outcomes.

This creates dangerous blind spots:

300 half-implemented controls aren't better than 30 rock-solid ones
500 vendor assessments don't beat truly understanding your critical suppliers in-depth
1000 risks you won’t manage don't help more than focusing on 10 which are actionable

Your CSPM flags 1000 issues daily, your SIEM sends alerts every minute, your EDR catches everything that moves but your GRC program is still struggling to automatically capture proof that MFA exists.

The technical debt compounds: more tools mean more alerts, more alerts mean more noise, more noise means less actual improvement. You end up with a GRC program that has all the modern tooling but none of the maturity to use it effectively.

The signal is what actually reduces risk. Everything else is just documentation.

// GRC Metrics API: Now with built-in bullshit detection
class GRCMetricsAPI {
    constructor() {
        this.noiseFilter = new BullshitDetector();
        this.signalAmplifier = new MeaningfulMetrics();
        this.executiveDashboard = new ActuallyUsefulReporting();
        this.auditTheater = new ComplianceKabuki(); // Legacy support only
    }
    
    // Applies the "So What?" test to every metric
    async processMetric(metricName, metricValue) {
        const soWhatResult = this.soWhatTest(metricName, metricValue);
        
        if (soWhatResult === "nobody_cares") {
            return this.auditTheater.generateGreenDashboard(metricValue);
        }
        
        if (soWhatResult === "drives_actual_decisions") {
            return this.signalAmplifier.amplify(metricValue);
        }
        
        // Default fallback for 80% of current GRC metrics
        return this.noiseFilter.quarantine(metricName, "measures_activity_not_outcomes");
    }
    
    // Revolutionary feature: Metrics that predict future security posture
    soWhatTest(metric, value) {
        if (metric.includes("percentage") || metric.includes("completion_rate")) {
            return "nobody_cares";
        }
        
        if (metric.includes("time_to_remediation") || metric.includes("prevented_incidents")) {
            return "drives_actual_decisions";
        }
        
        return "probably_noise_but_lets_pretend_its_important";
    }
}

Strategic Framework 🧩

The what: The conceptual approach broken down into 3 main principles

Distinguish Between Lagging and Leading Indicators

Noise: Compliance percentages, finding counts, control implementation rates

Signal: Time to remediation, control effectiveness metrics, attack surface reduction

Most GRC metrics are lagging indicators. They tell you what happened, not what's about to happen. Leading indicators predict future security posture and guide preventive action. Instead of reporting "98% of controls implemented," think more along the lines of "average time from vulnerability discovery to patch deployment decreased from 30 to 15 days."

Source: Lean Compliance

Focus on Outcome Metrics, Not Activity Metrics

Noise: Number of risk assessments completed, policies updated, training sessions conducted

Signal: Reduction in successful phishing attempts, decrease in privileged access violations, improvement in incident response times

Activity metrics measure effort; outcome metrics measure impact. The goal is to maximise security improvements. Track behaviours that change, vulnerabilities that disappear, and attacks that are mitigated, not meetings attended and documents produced.

Apply the "So What?" Test

For every metric you currently track, ask:

"If this number changed by 50%, would our actual security posture be different?"

If the answer is no, you're measuring noise. If leadership would make different resource allocation decisions based on the metric, it's signal. This simple test eliminates most compliance theatre while preserving indicators that drive real security investment.

Execution Blueprint 🛠️

The how: 3 practical steps to put this strategy into action at your organisation

Step 1: Audit Your Current Metrics

List every metric in your GRC dashboards and quarterly reports. Apply the "So What?" test to each one. Create two columns: Signal (metrics that predict or measure actual security improvement) and Noise (metrics that measure compliance activity). You'll likely find 80% falls into the noise category.

Metric Name	Current Value	"So What?" Test Result	Classification	Why?
Controls Implemented	98%	Would 50% change affect security? No.	🔴 NOISE	Measures activity, not effectiveness
Risk Assessments Completed	247/250	Would 50% change affect security? No.	🔴 NOISE	Counts documents, not risk reduction
Policy Training Completion	100%	Would 50% change affect security? No.	🔴 NOISE	Measures attendance, not behaviour change
Audit Findings Closed	95%	Would 50% change affect security? No.	🔴 NOISE	Tracks paperwork, not actual fixes
Vendor Assessments Done	156/160	Would 50% change affect security? No.	🔴 NOISE	Counts questionnaires, not supplier risk
Framework Coverage	87% SOC2, 92% ISO27001	Would 50% change affect security? No.	🔴 NOISE	Maps to standards, not threats
Policy Updates Completed	24/24	Would 50% change affect security? No.	🔴 NOISE	Document versioning ≠ security
Mean Time to Remediation	23 days (critical findings)	Would 50% change affect security? YES	🟢 SIGNAL	Directly impacts attack window
Controls That Prevented Events	12 incidents blocked this quarter	Would 50% change affect security? YES	🟢 SIGNAL	Proves actual effectiveness
Privileged Access Violations	Down 67% from last quarter	Would 50% change affect security? YES	🟢 SIGNAL	Measures real behaviour change
Attack Surface Reduction	34% decrease in exposed services	Would 50% change affect security? YES	🟢 SIGNAL	Quantifies actual risk reduction
Security Debt Velocity	15 critical items resolved/month	Would 50% change affect security? YES	🟢 SIGNAL	Tracks improvement rate

Step 2: Identify Your Critical Signals

Focus on 3-5 metrics that directly correlate with security outcomes in your environment:

Mean Time to Remediation for critical findings
Control Effectiveness Rate (controls that prevented actual security events)
Security Debt Velocity (rate of technical security improvement)

These should align with your organisation's actual risk profile, not generic industry frameworks.

Step 3: Build Signal-Focused Reporting

Replace your existing executive dashboard with signal-focused metrics. Instead of "95% compliance achieved," report "Reduced attack surface by 40% through privilege access improvements." Instead of "All policies updated," report "Zero successful phishing attempts in Q3 following security awareness program improvements."

Did you enjoy this week's entry?

Content Queue 📖

The learn: This week's resource to dive deeper on the topic

For this week, reading back on the first entry of the GRC Engineer will get you the right foundations to build a signal-based GRC program

Lots of companies are applying these principles to run world-class programs and you’ll learn more about those very very soon ;)

⚙️ From Silos to Systems: GRC Architecture

How to Design Information Flows, Establish Unified Language, and Create Team Interfaces Before Automating GRC

grcengineer.com/p/from-silos-to-systems-grc-architecture-ccbc2434d13b365f

That’s all for this week’s issue, folks!

If you enjoyed it, you might also enjoy:

My spicier takes on LinkedIn [/in/ayoubfandi]
Listening to the GRC Engineering Podcast

See you next week!

Reply

or to participate.