Data Quality Monitoring

Background

The GridX Enterprise Rate Platform supports revenue-grade analyses that require high quality data to meet client business objectives. GridX measures and validates essential data elements to ensure a high level of data quality is maintained throughout the lifetime of the software service.

Overview

From Data Ingestion through Analysis, GridX provides four distinct capabilities for measuring data quality.

Meter Gaps - measures the interval data quality during ingestion by identifying and storing all meter gaps
Filter - applies configurable tags to customers based on common data issues such as missing billing history or insufficient history. The Filter also provides high level interval usage quality statistics per customer.
Calibration - also known as shadow billing, Calibration compares the GridX calculated bill with the Utility bill. This high fidelity quality assessment ensures GridX digital models have the correct logic and all data inputs are the same as the utility source system. Calibration also compares the GridX usage with the Utility billed usage.
Alerts - performed during analysis, Alerts are configurable business insights that can also warn users of data issues such as usage and billing anomalies.

Meter Gaps

TBD

Filter

The Filter prepares all data for a given analysis by identifying the Analysis Period for each customer and assigning tags that define whether the customer should be analyzed or categorized as an exception.

Filter Tags

By default, only customers that are tagged with OneActiveSA will be eligible for Analysis; however, this is configurable to include other tags such as InactiveSA. Customers with exception tags (i.e. NoContractScope, BillCycleNotContinuous) will not be analyzed but their exception statuses can be retrieved through dashboards, reports and APIs.

Filter Tag	Description	Recommended Actions
OneActiveSA	The contract passes all configured tags	N/A The contract will be processed in the analytics workflows
InactiveSA	The contract is not active for the given analysis date	Verify this is correct in the client CIS If analysis of inactive customers is desired, update the Filter configuration
InactiveSAOver180Days	The contract has not been active within 180 days of the analysis date	Verify this is correct in the client CIS If analysis of inactive customers is desired, update the Filter configuration
NoContractScope	The rate code for the contract is not mapped. This tag will be applied to contracts that are not in the Digital Rate Modeling scope.	Check if the rate code is in-scope If the rate code is in-scope, update the rate code mapping configuration (`rate_code` reftable) in the Tariff Modeling UI to map the rate code to the appropriate contractroot.
InsufficientBillingCycles	The contract’s billing history does not pass the threshold for the required bill cycle count.	Verify this is correct in the client CIS If history is missing, resend the Customer Billing Data to GridX If analysis is desired, lower the bill cycle count threshold
NoBillingCycleFound	There are no billing cycles for the contract for the provided configuration	Resend the Customer Billing Data to GridX Rerun the analysis using Calendar Bill Cycles configuration
BillCycleNotContinuous	The billing history for the contract is not continuous. This generally caused by one or more missing billing cycles. This tag can also occur when there’s ≥ 1 day gap between bill cycles.	Check the billing history for missing bill cycles Send the missing Customer Billing Data to GridX Check the billing history for gaps been the end of one bill and the start of the next
HasOverlappingBillCycles	The billing history for the contract includes bills that overlap, meaning 1 or more days are present with 2 bill cycles	Compare the overlapping bills with bill start/end dates in the client CIS
NoEligibleBillCycle	The bill cycles for the contract are not available or do not have interval data	Verify the billing history exists in the client CIS Resend the Customer Billing Data to GridX Verify the interval data exists in the client MDMS Resend the Interval Data to GridX
NoBillingCycleGroupID	There are no billing cycles for the contract for when running analysis using system bill cycles	Verify that the contract has the correct billing cycle attribute Resend the attribute for billing cycle Verify that the billing_cycle table contains complete billing cycle data Rerun the analysis using Customer Bill Cycles or Calendar Bill Cycles configuration
NoValidAnalysisDuration	No valid analysis duration can be derived. For example, the derived analysis start date equals or is after the analysis end date.	Validate the GridX billing history with the client CIS Send the corrected Customer Billing Data to GridX
Orphan	The relationship between the Contract ID and Meter Account ID cannot be found	Verify the relationship exists in the client CIS Resend the Customer Data to GridX
MultiActiveSP	The Contract ID is associated with more than one Meter Account ID that are both active during the analysis period.	Verify the correct relationship in the client CIS Resend the Customer Data to GridX
MultiActiveSA	The Meter Account ID associated with Contract ID is also associated with another active Contract ID	Verify the correct relationship in the client CIS Resend the Customer Data to GridX
UnexpectedError	The contract has experienced an unmapped or unknown exception	File a support ticket with GridX

Calibration

For additional detail on Calibration, please reference: Digital Rate Calibration

The Calibration results inform Data Quality in the following ways.

Calibration Capability	Description
Total Bill	Calibration compares the GridX calculated bill total with the utility bill total generated by the CIS. If the difference between the two bills is outside of the SLA tolerance and there are no data issues, the bill has failed calibration.
Total Usage	Calibration compares the GridX calculated usage with the utility billed usage as printed on the customer bill. If the difference between the two usages is greater than a configurable threshold, an alert (“U”) may be generated.
Line Items	Calibration compares all GridX calculated bill line items with the utility bill line items generated by the CIS. This capabilities can identify which line items do not match to inform potential data and model issues.
Calibration Tags	Leveraging the Calibration Line Item capability, Calibration Tags can attach metadata to the Calibration results for known data issues. These tags can be used to exclude from the SLA match rate and create exports for client data teams to investigate.

Alerts

{link}