The Grid Architecture Stack Pack
From math to methodology: how Grid Architecture does what TOGAF can't.
A block diagram is not an architecture and drawing some boxes and lines does not make one an architect. Creating good architectures for grid scale systems is a complex and difficult task, which is why the discipline of Grid Architecture exists. A system architecture consist of three types of elements: structure, black box components, and externally visible characteristics. Many people just want to focus on the components and think bottom-up, whereas to create a good architecture it is necessary to think systemically and top-down, with a strong focus on structure.
Components are easy - structure is hard.
Enterprise IT Architecture and TOGAF
We have lived through and thankfully moved beyond the era of Smart Grid in the US, where Enterprise IT (Information Technology) was the go-to approach for merging digital technology with electric distribution systems on a wide scale. In practice, Smart Grid was mostly about advanced meter infrastructure and ended with a lot of benefit promises unfulfilled for structural reasons (which is not to say the meters did not work; they do). </soapbox>
So where do Enterprise IT architectural structures come from? Essentially, from two sources: “design patterns” and IT architect improvisation. Design patterns are structures that have been created empirically and are in widespread use in IT systems. Some examples include client-server (two-tier), pub-sub, presentation-application-data (three-tier), pipe-filter, etc. These design patterns are not supported by any underlying fundamental principles but do have extensive application histories and thus are widely accepted for use in small-scale IT solutions.
The other source is ad hoc structure contrivance by individual IT architects. Since Enterprise IT lacks fundamental structural principles, it is not surprising that different IT architects arrive at very different solutions for the same problem (a widely recognized phenomenon). In this sense, Enterprise IT architecture is more art than science.
But wait - doesn’t Enterprise IT Architecture methodology solve this problem? Most Enterprise IT architects use TOGAF, perhaps with augmentations such as Agile, microservices, and the like. TOGAF is just loaded to the gills with processes, “best practices,” charts, techniques, forms, recommendations, etc., but lacks one thing: any rigorous means to actually synthesize an architecture. TOGAF, for all its methods, tools, and certifications, is essentially just an elaborate documentation scheme (not just my observation, many Enterprise IT architects say this). It gives you lots of ways to record what you did but provides no fundamental principles from which to make structural decisions.
TOGAF doesn’t help you figure out what to do.
As a side note, Enterprise IT Architecture relies heavily upon use cases to document architectural requirements. The history of Smart Grid ca. 2003-2015 has shown that this is unworkable for grid scale architecture problems and ULS complexity theory1 has shown why that is.
When I ask Enterprise IT architects what the largest scale structure concept they know of is, they reply with SoS (System of Systems). However, SoS is just a label for networked IT system aggregations. It is pretty much devoid of any structural principles or organized theory; even fundamentals such as basic frame of reference, terminology, methodologies, and rudimentary problem definitions are lacking. Without actual architectures, SoS interoperability is being treated in the same way as it has been in the US electric power grid arena, which is to say, poorly.2
Grid Architecture
Yes, the discipline of Grid Architecture has systematic processes and charts and techniques, but…
But it has something much stronger than mere documentation processes. It uses formal mathematical and logical foundations for structure wherever possible and it actually gives you rigorous means to figure out what to do to solve architecture problems. You can derive structures from fundamentals without the artistic caprice. Now don’t get alarmed - it is not as if you have to set up mathematical equations and solve them. The core principle here is more subtle and yet more powerful than that:
Mathematics can induce structure.
This is the fundamental theorem of Grid Architecture. It is the key to being able to reason about Ultra-Large-Scale systems.
As Figure 1 below shows, this approach draws upon an extensive set of disciplines and considerable domain knowledge. The good news is that the math and related principles inform practical rules, models, tools, codes, and structure generators that grid architects can use directly, instead of solving equations. Grid Architecture provides the Universal Translator from the math to the methodology.
Check out the Grid Architecture stack pack:
The foundation is extensive, comprising three stacks - Disciplines, Domain Knowledge, and Tools. The domain knowledge can be supplied by SMEs but it is helpful for the architects to know as much of it as possible. The tools come from many sources and it is notable that the MBSE/SysML tools provide more than simple architecture documentation; they provide operational simulation and the ability to validate architectures. These three stacks support the Methodology stack.
Oh, and we get a bonus: note the box in Figure 1 labeled “Derivative Solutions.” Some grid problems are surfaced or clarified once the architecture is understood. Then, discipline knowledge can be used in the context of the architecture to get solutions to these problems that may previously have been unrecognized or intractable. Examples: using graph theory to evaluate data transportability over a distribution grid communication network for all possible use cases without enumerating them, using embedded storage networks to manage power flow volatility propagation, using analytic transformations to structure data flows, or developing regulations and tariffs. Grid problem identifications and solutions such as these arise from having a full systemic view of the grid, something which is core to the Grid Architecture discipline.
So yeah, developing large scale architecture for power grids is a lot more than just drawing boxes and lines.
Feiler et al, Ultra-Large-Scale Systems: The Software Challenge of the Future, Carnegie Mellon, Software Engineering Institute, 2006.
In the disciplines of systems architecture and software architecture, it is well understood that architecture controls interfaces. One would never know that from observing the history of electric power system interoperability in the US.