Sunday, 3 May 2020

Common Software Architectural Patterns

The goal of Solution Architecture is to:
Have a common understanding of how a technical solution shall be reached, the diagrams are useful too have a communicable roadmap and ensure it is complete.  Later the diagrams are used to ensure all relevant parties have a clear unambiguous shared understanding of the IT solution.

The main tools to communicate the architectural solution design are diagrams & documents that utilise common previously used and understood patterns to ensure a safe, scalable, stable, performant, and maintainable solution.  "4+1 view, which includes the scenario, logical, physical, process, and development views of the architecture", source.

Below are patterns and thoughts that I have come across and used to solve building high quality solutions.

3/N Tier Architecture/Layered:
1) Presentation/UI layer
2) Business Logic
3) Data Layer/Data source
Here are a couple of possible example over the years you could of used
ASP > C++ Com > SQL Server 2000
ASP.NET (Web Forms) > C# Web Service (XML/SOAP) > SQL Server 2008
ASP.NET C# > C# Business Object Layer > SQL Server 2008
KO > MVC > SQL 2012
Angular 3 > C# Web API (swagger contract) > SQL 2016
REACT.JS > Node.JS > Amazon Redshift
UI > Azure Functions/Serverless > SQL Azure
Flutter > C# Web API .NET Core 3 (swagger/OpenAPI) published on Azure App Service > SQL Azure/Cosmos

API's:  Over the years we have seen many different API's at a high level:  
  • Proprietary formatted API's > 
  • XML with SOAP coming out of XML based API's > 
  • REST/JSON (other popular formats are: RAML, GraphQL > 
  • Event Driven API's may be the next big jump.

Thoughts:  As time has progressed, scaling each of these layers has become easier.  For instance Azure SQL has replication and high Availability and scalability automatically built in.  No need to think about load balancing in depth.  Plug and play and ask for more if you need it.
Microsoft SQL Server use to be a single server, then came replication, clustering, Always-on-availability, scaling greatly improved performance.
Middle Tier or Business layer use to be a singleton pattern - go thru a single server for business logic, slowly load balancing improved and caching become better.  Nowadays merely ramp on on you cloud provider.

Sharded Architecture: Application is broken into many distinct units/shards.  Each shard lives in total isolation from the other shards.  Think SOA or Microservice architectures often use this approach.  "SOA is focused on application service reusability while Microservices are more focused on decoupling".  

The problem with tight coupling multiple services are:
  1. Complexity - Difficult to change code and know the effects.  Also services need to be deployed together to test changes.
  2. Resilience - Service goes down, the whole suite goes down.
  3. Scalability - Can be an issue as the slowest component becomes the bottleneck.  
For instance build a complete application to handle ordering and a separate system that handles inventory.  So both could be in different data stores so let's say orders are on CosmosDB and Inventory is on Azure SQL.  Some of inventory data is static in nature so I decide to use App Caching (Redis).  Both the data sources site on independent server-less infrastructure.  So if you see inventory has an issue, merely scale it.  The front end store would seamless connect to both the separate.  "Sharding" databases/horizontal partitioning is a similar concept but only at the database level.  Sharding can be highly scaleable, allow for leveraging and reusing existing services, can be flexible as it grows.  Watch out for 2 Phase Commit (2PC/Segas/Distribute transactions)

Thoughts Pros: 
Great to reuse existing services instead of creating yourself. e.g. App Insights on Azure.
Great for high availability.
Increased latency - you may need to go to various systems in sequential order.
Need keys to manage e.g. clientId for this decouple architecture type, this architecture can also become complex especially if you need to expand a shard to do something it doesn't do today.
Data aggregation and ETL can become complex and have time delays.

Event-driven architecture: Client sends a request that includes a response for the server to contact when the event happens.  So if asking a server to do a complex calculation, the client could keep polling a long running operation until the server has the answer or use an Event-drive architecture to can you pls calculate and when you are done, send the response to me at...  Types of Event-driven architectures are: WebHooks, WekSockets, ESB (pub-sub), Server Sent Events (SSE).
Only run when an event happens.  They are loosely coupled.  In Azure it generally covers: Functions, Logic Apps, Event Grid (event broker) and APIM.  Easy to connect using Power Platform Connectors.
  • Client/Service sends a broadcast event
  • Consumers listens for events to see if they want to use the event
Hexagonal Architecture,
Command Query Responsibility Segregation (CQRS) - pattern/method for querying and inserting data are different./seperated.  This is a performance and scaling pattern.
Domain Driven Design (DDD) - Design software inline with business requirements.  The structure and language of the code must match the business domain.  DDD Diagrams help create a share understanding of the problem space/domain to aid with conversation and further understanding within the team. 
RACI Diagram - visual diagram showing the functional role of each person on a team or service.  Useful for seeing who is responsible for what part of a service or their role within a team.
Event Sourcing Pattern or AMQP
Competing Consumer Pattern – Multiple consumers are ready to process messages off the queue.
Priority Queue pattern -Messages have a priority and are ordered for processing based on priority.
Queue-based load leveling
Throttling pattern
Retry pattern
Circuit breaker pattern
The Twelve-Factor App methodology is a methodology for building software-as-a-service (SaaS) applications.

Key Design Decision (KDD) Document helps outline why decisions where made.  This is also often called Architectural Decisions Document or Template.
RAID Log - 

Streaming/MessageBus: Kafka, IoT,
Azure Messaging Service is made of of 6 products:
1. Service Bus - Normal ESB.  Messages are put into the queue and 1 or more apps can dirrectly connect or subscripbe to topics.
2. Relay Service - Useful for SOA when you have infra on prem.  Exposes cloud based endpoints to your on-prem. data sources.
3. Event Grid - HTTP event routing for real time notifications.
4. Event Hub - IoT ingestion, highly scalable.
5. Storage Queues - point-to-point messaging, very cheap and simple but very little functionality.
6. Notification Hub - 

Azure Durable Functions - Azure Functions are easy to create logic but are not good at long running or varying length duration functions.  To get around the timeout limits there are a couple of patterns for Functions making them better at handling long running operations.  The most common patterns are: Asyn HTTP API's (Trigger a a function using HTTP, set off other functions and the client waits for an answer by polling a separate function for the result), Function Chaining (Execute functions sequentially once the last function completes), and Fan out/Fan-in (first function call multiple functions that run in parallel) 

Lambda: great for large data architectures.  Has a batch vs streaming concept.  Each transaction pushed into a queue/stream (Kafka/Azure Queues/Azure Event Grid) and large data can be stored for later batch processing.

"Onion Architecture is based on the inversion of control principle. Onion Architecture is comprised of multiple concentric layers interfacing each other towards the core that represents the domain. The architecture does not depend on the data layer as in classic multi-tier architectures, but on the actual domain models."

Distributed Application Runtime, Dapr: Video

SAST/DAST: are application security testing methodologies used to find vulnerabilities in applications.  Another threat modeling approach is STRIDE.

OpenAPI vs GraphQL
OpenAPI specification (previously known as the Swagger specification) is my default for an API, this allows for a known RESTful API that anyone with access can use.   Open API has set contracts that returned defined objects which is great, you can work with the API like a database with simple CRUD operations as defined by the specification.  The issue is that the returned objects are fixed in structure so you may need 2 or more queries to get the data you are looking for.  Alternatively, GrapghQL allows the developer to ask for the data exactly as the want it.
Open API example:
/api/user/{2} returns the user object  // Get the user object for user 2
/api/users/{2}/orders/10  // Returns the last 10 orders for the user
GraphQL example:
Post a single HTTP request.
query {
 User(id: "") {
    orders(last: 10 {
You can see that for complex changing systems, GraphQL is potentially a better choice.  I also like the idea of using HASURA for ORM using GraphQL against PostgreSQL (hopefully SQL Server and others).

More Info:


Post a Comment