Full Observability Check For Meshed Networks: A New Feature

Power Grid Model, specifically within the power-grid-model category, is undergoing exciting developments! This article dives deep into the proposed feature of a full observability check for meshed networks, even without relying on voltage phasor sensors. This is a significant step towards providing clear and deterministic feedback before state estimation runs, ensuring consistency and reliability.

Background & Motivation: Observability in Meshed Networks

Previously, Power Grid Model introduced enhanced observability logic for radial grids, including the recognition of voltage phasor sensors. This advancement ensured that non-observable cases consistently triggered NotObservableError rather than surfacing cryptic numerical factorization errors like SparseMatrixError. Now, the focus shifts to extending this full observability check to meshed networks. This enhancement is crucial for providing clear and deterministic feedback before state estimation algorithms (like ILSE/NRSE) are executed. This proactive approach saves time and resources by identifying potential issues upfront.

Think of it this way, guys: imagine trying to solve a puzzle with missing pieces. You might spend hours trying different combinations, only to realize it's impossible to complete. Similarly, in power grid modeling, if the system isn't fully observable, the state estimation process can fail or produce inaccurate results. This feature aims to ensure all the pieces are in place before we start solving the puzzle. The goal is to make non‑observable cases fail consistently with NotObservableError, instead of surfacing numerical factorization errors (e.g., SparseMatrixError). Extending a full observability check to meshed networks is the next step to give clear, deterministic feedback before state estimation runs, independent of the chosen algorithm (ILSE/NRSE).

This feature request introduces a graph‑based, measurement‑aware algorithm specifically designed for meshed topologies. It primarily relies on edge measurements (branch current/power sensors) while optionally incorporating node measurements. This approach aligns perfectly with the existing sensor semantics in PGM, such as current sensors measuring magnitude and the distinction between edge and node power. The algorithm explicitly excludes voltage phasor sensors for this initial implementation. The inclusion of voltage phasor sensors will be addressed in a subsequent phase. This makes the system more robust and efficient, especially for complex grid configurations. So, the exciting part of this is the algorithm excludes the presence of voltage phasor sensors, the addition of which will be one of the next steps.

Key Policy: To maintain a stable reference point, we adhere to the rule that at least one voltage sensor without phasor (i.e., magnitude‑only) must be present within the network. This provides a baseline, irrespective of the presence or absence of voltage phasors. This requirement is enforced and validated within the meshed-grid check.

Scope: Implementing a Full Observability Checker

The scope of this feature is to implement a comprehensive full observability checker tailored for meshed networks. This checker will operate based on the following principles:

• It will treat edge measurements (current/power sensors on branches) as paramount constraints. These measurements will be used to establish coverage across both loops and tree structures within the network. Think of it as tracing the flow of electricity through the grid using the data from these sensors. The system will ensure that these edge measurements are fully utilized to determine the observability of the network.
• The system will seamlessly incorporate node measurements (net injections) as well as voltage magnitude sensors. These measurements will serve as crucial anchors, providing essential reference points for the observability analysis. They act like the cornerstones of a building, providing stability and grounding for the entire structure. The voltage magnitude sensors provide a key reference point for anchoring the system's state.
• Crucially, this checker will not rely on voltage phasor sensors. If these sensors are detected within a meshed network, the observability check will be aborted, and the system will proceed directly to calculation. This design decision streamlines the process and avoids potential conflicts or ambiguities in the analysis.
• The checker is designed to provide deterministic, user‑friendly results. It will clearly indicate whether the network is Observable or raise a NotObservableError, ensuring that users receive actionable feedback before initiating the estimator. This is a huge leap forward from the previous behavior where numerical factorization errors might surface unexpectedly. This clear and consistent feedback mechanism empowers users to quickly identify and resolve observability issues, leading to more efficient and reliable simulations.

In essence, this scope defines a robust and practical solution for observability analysis in meshed networks, prioritizing accuracy, efficiency, and user experience. We're talking about a tool that will significantly improve the reliability of power grid simulations and analysis. Basically, it produces deterministic, user‑facing results (Observable vs. NotObservableError) prior to running the estimator, consistent with recent radial‑grid behavior.

Proposed Algorithm: A Three-Part Procedure

The proposed algorithm, as detailed in the workout document "Full Observability Check in Meshed Networks with edge measurement.pdf", follows a structured three-part approach. This approach is designed for efficiency and clarity, ensuring a robust and reliable observability assessment.

1. Build a Connectivity/Measurement Matrix (m): The first step involves constructing a matrix that represents the network's connectivity and the available measurements. This matrix, denoted as 'm', serves as the foundation for the observability analysis. For each edge (i,j): If the branch has an edge measurement (current/power), set m[i][j] = m[j][i] = -1; else set to 0. For each node i: If the node has a nodal measurement (e.g., net injection) or voltage magnitude, set m[i][i] = 1; else 0. The matrix effectively encodes the relationships between nodes and branches, along with the presence and type of measurements.
2. Run an Observability Traversal: With the connectivity/measurement matrix in place, the algorithm performs an observability traversal. This traversal systematically explores the network, leveraging the information encoded in the matrix to identify observable regions. It's like a detective piecing together clues to solve a mystery, following the connections and measurements to understand the system's observability. Initialize found edges list and visited nodes set. Choose a start node (first node with m[i][i] != 0, else the first zero‑diagonal node).Iteratively: Prefer unconsumed edge measurements (m[i][j] == -1) to move from node i to j, marking them consumed (-1 -> 0), and mark nodes visited. If no edge measurement is available, use a node measurement at j (m[j][j] == 1) to progress and mark it consumed. If stuck, backtrack along previously used edges that are still marked traversable (m[u][v] == 1), switching the traversal direction to continue the search.
3. Decide Full Coverage: The final step involves evaluating the results of the observability traversal to determine whether the entire network is observable. This decision is based on whether all necessary measurements have been utilized and all relevant nodes have been visited. Essentially, it checks if the detective has successfully solved the mystery and accounted for all the pieces of evidence. The algorithm determines full coverage based on exhausted measurements and visited nodes. observable iff all diagonal entries m[k][k] are 0 at the end.

Key Steps Explained:

1. Building the Connectivity/Measurement Matrix: This matrix is the cornerstone of the algorithm. It acts as a map, guiding the observability traversal. The matrix, often represented as 'm', captures the network's topology and the available measurements in a structured format.
   • Edge Measurements: For each branch (edge) in the network, the algorithm checks for the presence of current or power sensors. If an edge measurement exists, the corresponding entries in the matrix (m[i][j] and m[j][i]) are set to -1. This signifies a strong connection and a valuable source of information for the observability check. Otherwise, these entries are set to 0, indicating the absence of an edge measurement.
   • Node Measurements: The algorithm also considers measurements at nodes (junction points) within the network. This includes net injections and voltage magnitude sensors. If a node has a nodal measurement, the diagonal entry in the matrix (m[i][i]) is set to 1. This indicates a direct measurement at that node, providing a local reference point. Nodes without nodal measurements have their diagonal entries set to 0.
2. Observability Traversal: Exploring the Network: The core of the algorithm is the observability traversal. This process systematically explores the network, leveraging the connectivity/measurement matrix to determine which parts of the grid are observable.
   • Initialization: The traversal begins by initializing a list to track found edges (connections) and a set to keep track of visited nodes. The algorithm then selects a starting node. This is typically the first node with a non-zero diagonal entry in the matrix (m[i][i] != 0), indicating a nodal measurement. If no such node exists, the traversal starts from the first node in the network.
   • Iteration and Measurement Consumption: The traversal proceeds iteratively, moving from node to node and consuming measurements along the way. The algorithm prioritizes unconsumed edge measurements (m[i][j] == -1). These measurements provide the strongest connections and are crucial for spanning the network. When an edge measurement is used to move from node 'i' to node 'j', the corresponding entry in the matrix is marked as consumed (-1 is changed to 0). Both nodes 'i' and 'j' are added to the set of visited nodes.
   • Nodal Measurement Utilization: If no edge measurement is available to move from a node, the algorithm considers nodal measurements at neighboring nodes (m[j][j] == 1). Using a nodal measurement allows the traversal to progress and mark the corresponding node as visited. The matrix entry is updated to reflect the consumption of the measurement.
   • Backtracking: In some cases, the traversal may encounter a dead end, where no further progress can be made. To overcome this, the algorithm employs a backtracking mechanism. It retraces its steps along previously used edges that are still marked as traversable (m[u][v] == 1). By switching the traversal direction, the algorithm can explore alternative paths and potentially uncover previously unvisited regions of the network.
3. Decision: Determining Full Coverage: The final stage of the algorithm involves evaluating the results of the traversal to determine whether the entire network is observable. The key criterion is whether all diagonal entries in the connectivity/measurement matrix (m[k][k]) are 0 at the end of the traversal. If all diagonal entries are 0, it indicates that all nodal measurements have been utilized and the entire network is observable. If any diagonal entries remain non-zero, it signifies that there are unobservable regions within the network.

In essence, this three-part algorithm provides a systematic and robust approach to assessing the observability of meshed networks, laying the foundation for reliable state estimation and grid management. This systematic approach ensures that the observability check is thorough and accurate.

Pseudo Code:

The pseudo code provided in the original document clearly outlines the steps of the algorithm. Let's break it down for clarity:

Algorithm 1: Full Observability (Meshed)
Input: Graph G=(V,E)
1: m ← BuildConnectivityMatrix(G)
2: return CheckObservability (G, m)

This top-level algorithm takes a graph G as input and calls two sub-algorithms: BuildConnectivityMatrix and CheckObservability. The first algorithm constructs the measurement matrix, and the second algorithm performs the traversal and determines observability.

Algorithm 2: BuildConnectivityMatrix
Input: Connected graph g=(v,e)
Output: v×v matrix m
1: init m with NULL
2: for each edge (i,j) in e:
3:   if edge (i,j) has branch measurement:
4:       m[i][j] ← -1; m[j][i] ← -1
5:   else:
6:       m[i][j] ← 0;  m[j][i] ← 0
7: for each vertex i in v:
8:   if vertex i has nodal measurement:
9:       m[i][i] ← 1
10:  else:
11:      m[i][i] ← 0
12: return m

BuildConnectivityMatrix initializes the matrix m and then iterates through each edge and vertex in the graph. It populates the matrix based on the presence of branch and nodal measurements, using the -1, 0, and 1 conventions described earlier.

Algorithm 3: CheckObservability 
Input: (Sub)graph g, matrix m
Output: True if observable
1: E_found ← []; V_visited ← {}
2: i ← (first node with m[i][i]≠0) or first node
3: while True:
4:   found ← False
5:   for each j where m[i][j] ≠ NULL:
6:     if m[i][j] = -1:           # branch measurement
7:        add (i,j) to E_found; add i,j to V_visited
8:        m[i][j] ← 0; m[j][i] ← 0
9:        i ← j; found ← True; break
10:    else if m[j][j] = 1:       # nodal measurement
11:       add (i,j) to E_found; add i,j to V_visited
12:       m[j][j] ← 0; m[j][i] ← 1
13:       i ← j; found ← True; break
14:  if not found:
15:     backtrack ← False
16:     for (u,v) in E_found where m[u][v] = 1:
17:        if u = i and v ≠ i: i ← v; backtrack ← True; break
18:        else if v = i and u ≠ i: i ← u; backtrack ← True; break
19:     if not backtrack: break
20: return (∀k: m[k][k] = 0)

CheckObservability is the heart of the algorithm. It initializes the E_found (edges found) and V_visited (vertices visited) sets. It then enters a while loop that continues until no further progress can be made. The algorithm iterates through neighbors, prioritizing branch measurements and then nodal measurements. If no new nodes can be reached, it attempts to backtrack. Finally, it returns True if all diagonal elements of m are 0, indicating full observability.

Sensor Semantics & Assumptions: Defining the Rules

To ensure the algorithm functions correctly and provides meaningful results, certain assumptions and definitions regarding sensor semantics are crucial. These guidelines clarify how different types of sensors are interpreted and utilized within the observability check.

• Edge Measurements: Edge measurements encompass data from sensors located on branches, specifically current or power sensors. For the purposes of the observability checker, these two types of sensors are treated equivalently. Think of them as providing similar information about the flow of electricity along a specific path.
• Node Measurements: Node measurements include net injections and voltage magnitude sensors. These sensors provide information about the conditions at specific points within the network. Voltage magnitude sensors play a critical role in anchoring the observability analysis.
• Requirement: A fundamental requirement for this algorithm is the presence of at least one voltage magnitude sensor within the network. This sensor provides a crucial reference point for determining the overall observability of the system. It's like having a benchmark against which other measurements can be compared.
• Compatibility: The algorithm is designed to handle scenarios where voltage phasor sensors are present in the network. However, in the current implementation, the presence of voltage phasor sensors in a meshed network will trigger a bypass of the observability check. The system will proceed directly to state estimation calculation without raising any exception. This behavior ensures that the algorithm remains robust and avoids potential conflicts with phasor sensor data in this initial phase. This means Voltage phasor sensors, when present in a meshed network, must not break the check; they will trigger the ignoring behavior of this feature: i.e., pass to SE calculation.

These clearly defined sensor semantics and assumptions are essential for the accurate and reliable operation of the observability check algorithm. They establish a common understanding of how sensor data is interpreted and used, ensuring consistency and predictability in the results.

Error Semantics: Handling the Unexpected

In any complex system, it's essential to define how errors and unexpected situations are handled. Clear error semantics ensure that users receive informative feedback and can take appropriate action. In the context of this observability checker, the following error semantics apply:

• Non-Observability: If the checker definitively determines that the network is non-observable, it will raise a NotObservableError. This error will be accompanied by an actionable message, providing specific guidance to the user. For example, the message might identify unvisited subgraphs or highlight areas where edge coverage is missing. This detailed feedback empowers users to quickly diagnose and resolve the underlying observability issues. So, raise NotObservableError with an actionable message (e.g., identify unvisited subgraphs or missing edge coverage).
• Estimator Failure: It's important to distinguish between non-observability and numerical issues that may arise during the state estimation process. If the checker passes, indicating that the network should be observable, but the estimator subsequently fails numerically, this does not necessarily imply non-observability. The failure could be due to other factors, such as ill-conditioned matrices or convergence problems. In such cases, the system should not attribute the failure to observability issues. It's like saying, just because the map says you can get there, doesn't mean the road is always smooth.
• Meshed Networks and Phasor Voltage Sensors: As mentioned earlier, the presence of meshed network and phasor voltage sensor will trigger a bypass of the observability check. The system will proceed directly to state estimation calculation without raising any exception. This behavior is intentional and ensures compatibility with existing workflows. The aim is to pass to state estimation calculation without raising any exception.

By clearly defining these error semantics, the system provides a robust and informative framework for handling potential issues related to observability and state estimation. This clarity is essential for users to effectively troubleshoot problems and ensure the reliability of their simulations.

Documentation: Guiding the User

Comprehensive documentation is critical for any new feature, ensuring that users understand how to use it effectively. For this full observability checker, the following documentation updates are planned:

• Meshed Networks Subsection: A dedicated subsection will be added under the "State Estimation → Observability" section of the documentation. This section will provide detailed information about the new checker, its capabilities, and its limitations. It will serve as the primary resource for users seeking to understand and utilize this feature.
• Sensor Role Clarification: The documentation will clearly define the roles of different sensor types (edge measurements, node measurements, voltage magnitude sensors) in the observability check process. This will help users understand how the algorithm utilizes sensor data and how the presence or absence of certain sensors can impact the results. This ensures that everyone understands how the sensors influence the observability assessment.
• Minimal Voltage Magnitude Requirement: The documentation will explicitly state the requirement for at least one voltage magnitude sensor in the network. This will prevent confusion and ensure that users are aware of this essential prerequisite for running the checker. This point will be emphasized to avoid user errors.
• Phasor Incompatibility: The documentation will clearly explain the current incompatibility with voltage phasor sensors in meshed networks. It will detail the behavior of the system when phasor sensors are detected and the rationale behind bypassing the observability check in such cases. This transparency will help users avoid unexpected behavior and plan their simulations accordingly. The key is to clarify sensor roles, minimal voltage magnitude requirement, and phasor incompatibility.

These documentation updates will ensure that users have the information they need to effectively utilize the full observability checker and understand its implications for their power grid simulations.

Next Steps: Moving Forward

The development of this full observability checker is an ongoing process. The next steps include:

• Implementation and Testing: The core algorithm will be implemented and rigorously tested to ensure its accuracy and reliability. This phase will involve developing code, creating test cases, and validating the results against known scenarios. Rigorous testing is essential to ensure the checker functions correctly under various conditions.
• Documentation Updates: The documentation updates outlined above will be implemented to provide users with comprehensive guidance on using the new feature. The documentation will be a living document, updated as the feature evolves.
• Examples and Use Cases: Practical examples and use cases will be developed to illustrate how the checker can be applied in real-world scenarios. These examples will help users understand the benefits of the checker and how it can improve their power grid analysis. Real-world examples make the feature more accessible and understandable.

These next steps will pave the way for the widespread adoption and utilization of this valuable new feature in Power Grid Model. The team will implement algorithm and tests and update docs and examples.