evolutionary-architecture/evolutionary-architecture-by-example

Navigate the complex landscape of .NET software architecture with our step-by-step, story-like guide. Unpack the interplay between modular monoliths, microservices, domain-driven design, and various architectural patterns. Go beyond the one-size-fits-all solutions and understand how to blend these approaches based on your unique needs.

Evolutionary Architecture

Table of Contents

• Objective
  • Problem
  • Proposed Solution
• Domain
  • Overview
  • Subdomains
  • Potential Patterns
• Chapters
  • Chapter 1: Initial Architecture: Focus On Simplicity
  • Chapter 2: Modules Separation: Focus On Maintainability
  • Chapter 3: Microservice Extraction: Focus On Growth
  • Chapter 4: Applying Tactical Domain-Driven Design: Focus On Complexity
• Repository Structure
  • Overview
  • How To Navigate?
• Libraries and Platforms Used
• Videos 🎥
  • Webinar from Architecture Weekly 🇬🇧
  • Presentation at Programistok 2023 Conference 🇵🇱
  • Evolutionary Architecture Visualized Through NDepend 🇬🇧
• Authors
• 💬 Join our Community
• ⭐ Say thanks

Objective

Problem

There are many repositories that describe software and solution architecture in .NET. As there is no single definition of software architecture, they combine several topics:

• Deployment strategies like Modular Monolith and Microservices

• Domain-Driven Design

• Clean, Onion, Hexagonal Architecture

• Infrastructure

as if each of the above were the only possible solution. And that is the problem - it is very rare that any single material shows you a concrete decision path or mix of the above approaches.

Often the examples you find are either too trivial or too complex. Sometimes they are harmful because they misunderstand different concepts. As a result, it is very difficult to take just one and follow some patterns for your own application.

It depends - one of the most overused phrases. Well, it always depends on something, but there are always heuristics to help you decide.

In summary, if you are looking for a journey - not just a single article describing a solution - look no further. It starts here, in this repo.

Proposed Solution

Our solution should be read like a story and is divided into 4 chapters:

Chapter 1: Initial Architecture: Focus On Simplicity

Chapter 2: Modules Separation: Focus On Maintainability

Chapter 3: Microservice Extraction: Focus On Growth

Chapter 4: Applying Tactical Domain-Driven Design: Focus On Complexity

In each chapter we describe the evolution of the architecture of a selected domain:

• Starting with a simple solution architecture that meets early production needs, where business processes are separated in namespaces and communication is handled by an in-memory queue

• Moving to module separation with the structure of multiple projects associated with each module (including CQRS)

• Continue with the single module extraction to a microservice and add the message queue component

• Tackling the complexity of one of the modules using tactical Domain-Driven Design

We apply everything we have learned over many years of software development - our aim is to help you avoid making the same mistakes we have. This way you can be sure that your architecture will not be too trivial for too long, or too complex from the start. Treat it as a guide, something you can refer to at any time.

What we focus on:

• Selected business domain analysis

• Split the domain into subdomains (Core and Supportive)

• Choice of architectural pattern

• Evolution from monolithic to modular

• Evolution to a mix of modular monolith and microservices

• Applying Domain-Driven Design to a Core module

• Backend (.NET) with minimal API

• Loose coupling

• Architecture decision log

• Good coding practices

What we do not focus on:

• Frontend (you can use React, Vue, Angular, Svelte or anything else)

• Logging (you can use Serilog)

• Contract testing (you can use Pact Net)

so that you get the gist of what we have to share with you. Additionally, static code analysis is enabled in all chapters to help us to keep the code base as clean as possible. It is strongly suggested to use it as well in your production code.

Note	Keep in mind that these are our suggestions. In the end you will have to decide for yourself which chapters fit your needs or combine them into one solution.

Domain

Overview

The chosen domain for analysis and implementation is a Fitness Studio. It is an area that most of people have an idea of how it works. You can:

• Get an offer

• Request for a contract

• Sign the contract

• Receive a pass (to be able to enter the fitness studio)

• Attend to fitness classes

and many more.

However, in order to identify the above processes, you usually need to analyse the domain with Domain Experts and break it down into smaller pieces called Subdomains. Otherwise, there is a high risk of falling into a big ball of mud (or distributed mud) where everything is tightly coupled.

At first glance, the domain seems small and problems of oversimplification of initial assumptions usually arise from this interpretation. There are dozens of processes related to expired passes and renewals, discount policies, VIP access, negative cases (e.g. rejection of the access). We often tend to underestimate the problem.

Important

Remember that what we want to show you in this repository is how to handle a domain split into a few example subdomains, and how to prepare building blocks that will allow you to easily extend them to cover all processes. We are not able to cover the entire Fitness Studio domain, as that would probably take us a year (or more) of work. Do not worry - we try to show ideas here that are complex enough to be applied to your application.

Subdomains

Now that you understand which business domain we are focusing on, it is time to break it down into smaller pieces called Subdomains.

There are many ways to do this. Our 3 favourites are:

• Domain Storytelling

• Event Storming

• Story Storming

In general, the idea is to find processes by discovering the flow. Based on different heuristics you are able to define the smaller blocks.

After a round of analysis, we decided to choose the following areas for implementation to show you the idea of Evolutionary Architecture:

As you can imagine, each subdomain covers a lot of different processes. Again, due to time constraints, we cannot focus on every single aspect. Therefore, we have chosen the following actions for each subdomain:

There are 6 different processes in 4 different subdomains. There is a problem here - these subdomains do not communicate with each other and this is one of the most common problems we have in our applications. So we decided to complicate things a bit and add some communication:

There are 2 triggers:

• when the contract is signed by you (a customer), then the new pass is registered to allow you to enter the fitness studio

• when the pass expires, then the new offer is prepared (which will be sent to you as a customer)

This way we are almost ready to start the implementation.

One more thing to mention - in our example, each subdomain is a separate Bounded Context.

Important

It is worth mentioning that there may be a situation where multiple subdomains create a single Bounded Context. An example of this in Fitness Studio domain can be: Assessments, Progress Tracking and Virtual Coaching that creates 1 Bounded Context called Personalised Training.

Potential Patterns

Before you decide to start coding, it is worth to look at your analysis and division one more time and check the complexity of each (it will be mainly defined by the amount of processes and its business rules/policies). Let’s take a look at below examples.

Passes

There is no business logic:

• in the pass registration process, it is only informed that the contract has been signed

• In the pass expiry process, it is only informed that the pass expiry date has been reached.

In addition, the potential for new business rules to be applied to the above processes or other actions is rather low. As it looks like a perfect candidate for CRUD operations, we want to mark it as a candidate to become an Active Record pattern.

Note	Active Record is an pattern that rationalises the persistence layer in an application. It encapsulates the idea that a database record is an object in the application, with properties that map to the columns of the database table and the behaviour (domain logic) of that object.

Offers

The story here is similar to Passes. There is no business logic, only the fact that the pass is expiring.

Contracts

This is the place where the fun begins. There are 3 business rules:

• in the process of contract preparation, it is only allowed if the customer is an adult AND smaller than the maximum height allowed (210 cm)

• in the contract signing process, it is only allowed if it is signed within 30 days of the contract being created, otherwise the contract has to be created from scratch

In addition, the potential for new business rules being applied to the above processes or other actions is quite high. Here the warning bell should go off - this has a really high potential to become more and more complex, so it might be a good candidate for a Domain Model.

Note

Domain Model is a widely used pattern in software engineering that encapsulates the concepts and behaviours of a particular problem domain. This representation is designed to mimic the structure and functionality of the real-world system. The domain model pattern is particularly well known for its ability to handle complex business logic by providing a rich, object-oriented representation of the problem domain.

Reports

This case is really simple. The only thing that we want to achieve is to get the information about new passes that have been registered in each month.

There is no business logic and there is also no need to have an object representation of the data retrieved. This is a perfect candidate for a Transaction Script.

Note

Transaction Script is a pattern commonly used in software engineering that organises business logic into procedures, where each procedure handles a single request from the presentation. Each transaction script is a series of procedural steps that represent a sequence of tasks performed as part of a transaction, similar to a script in a play.

Summary

After deeper thinking about our subdomains, we decided for following patterns that will be applied in one of 4 chapters:

Chapters

In the beginning of every greenfield project we need to make a lot of decisions and we lack of knowledge. It is called The Project Paradox:

Quite often we are biased by conferences, meetups, friends and colleagues. As a result we decide for too complex architecture.

This means starting with:

• microservices (where we do not yet know the traffic, scale and other factors)

• orchestrators

• data streaming

• NoSQL

• cache

and many more. In the end, we have a lot of problems of our own making, and the barrier to entry for any team member is extremely high. In fact, after release, we do not know if we need this or that block. We are also not optimised from a cost perspective and it is very difficult to find bottlenecks.

Another problem is choosing an architecture that is too trivial for too long (this happens less often than "overcomplicated"). This means that we just add code to a monolith, new features flood our codebase and then it becomes a big ball of mud.

What we want to show you in our story is the evolutionary approach that will tackle most of the applications you work with.

Chapter 1: Initial Architecture: Focus On Simplicity

In this chapter we will show you how to start your solution architecture. We start with modularisation from day one of the application, but modules are only separated by namespaces (there is only one project for the production code called Fitnet). Each process that occurs in each module is sliced vertically - all the code is covered in each process namespace. This gives us several advantages:

• better productivity - when we start a new design, we are not distracted by creating namespaces, renaming, moving things around

• all the code for each process is in just one namespace, so there is no need to look around in folders like Controllers, Entities, Commands, Queries etc. Everything is just in e.g. SignContract

• deleting or extracting the process is simple - you just drop or extract a namespace

Modules communicate with the in-memory queue.

Interactive diagram

Chapter 2: Modules Separation: Focus On Maintainability

This chapter focuses on the second step you can take in your application. After some time, you will find that your assumptions about modules were wrong - they grow fast, the business logic becomes more complex. You may have to decide to use a different type of database (e.g. key-value). Or the other way around - something you thought was going to be complex is actually quite simple and there is not a lot of business logic involved. Also, the team has grown and it is quite difficult to work on one project - lots of conflict and merging hell.

With this in mind, you can now start thinking to split your single Fitnet project into several ones:

• for one module it will be just Fitnet.Reports - there is only a transaction script, no business logic

• for another, it will be Fitnet.Passes.Api, Fitnet.Passes.DataAccess to build around the active record

• for the complex one it will be Fitnet.Contracts.Api, Fitnet.Contracts.Application, Fitnet.Contracts.Core, Fitnet.Contracts.Infrastructure

and so on. The modules still communicate with the in-memory queue (alternatives described in the chapter’s own README).

Chapter 3: Microservice Extraction: Focus On Growth

Over time, you may need to extract a microservice from one of your application modules. In this chapter, you will identify the most common disintegrators (decision drivers for extracting a microservice).

We will also add a message queue component to replace the in-memory queue. This way, we will improve the exchange of messages between the modules themselves and the microservice.

Here you will learn about various concepts that are important from a microservices architecture perspective.

Interactive diagram

Chapter 4: Applying Tactical Domain-Driven Design: Focus On Complexity

At the end of the story, we want to show you that it is possible to evolve to a domain model for one of your modules at some point.

You do not have to start from scratch if you do not know your business domain. And you do not need to apply all the concepts of Domain-Driven Design to get good results.

Interesting fact - if you have done a proper analysis of your business domain in the beginning and have already broken it down into different subdomains (and combined them into e.g. a module), then you are almost done from a strategic Domain-Driven Design perspective. Now you just need to make some adjustments and do some tactical DDD. Cool? Absolutely cool!

Here we focus mainly on tactical DDD and describe

• value objects

• entities

• aggregates

You will also find some tips for the next steps.

Repository Structure

Overview

We are trying to keep this repository as simple as possible, so that you can read it like a book. It contains chapters, where each chapter is an extension of the previous one, read like a story.

How To Navigate?

In the root folder of this repository you will find only this README, the assets (images & diagrams) used in it, and 4 folders containing the content for each chapter.

In each folder you will find the same root solution, but expanded:

1. Chapter 1 - Initial state of the application

2. Chapter 2 - Extraction into separate projects

3. Chapter 3 - Extracting a module to a microservice and using the message queue

4. Chapter 4 - Applying Tactical Domain-Driven Design to one of the modules

Another important thing to note is that the detailed description of each chapter is in its own README folder. There you will find the information on how to run the solution, what solution structure is used, and other important things that are only relevant to that particular chapter.

We are not repeating the information covered in the previous chapter, we are just extending it with the decision we have made.

Important

If you want to get the most out of this repository, we recommend that you read the first chapter, understand the code and description, and then navigate to another folder. Of course you can go through the folders in your own way but you might miss some concepts.

Libraries and Platforms Used

The entire application is developed using C# and .NET 8.

In each chapter we use libraries and platforms that simplify the development process (no need to create them yourself). We try to keep it to a minimum. The certain disadvantage is that we do not have full control over it (trade-off we accept). Here is the list of the most important ones:

Application:

• Docker

• Fluent Validation

• MediatR

• Dapper

• Entity Framework

• Npgsql

• SonarAnalyzer

Testing:

• xunit

• NSubstitute

• Verify

• Bogus

• Fluent Assertions

• Test Containers

Videos 🎥

You can learn the essentials of Evolutionary Architecture from these videos:

Webinar from Architecture Weekly 🇬🇧

In this webinar, Maciej "MJ" Jedrzejewski gives a detailed talk on Evolutionary Architecture. You can watch the recorded webinar here.

Presentation at Programistok 2023 Conference 🇵🇱

This is a recorded presentation from the Programistok 2023 Conference, where Evolutionary Architecture was extensively explained. You can watch it on YouTube here.

Evolutionary Architecture Visualized Through NDepend 🇬🇧

Explore Evolutionary Architecture visualized through NDepend, featuring comprehensive dependency and code analysis in the form of an interview led by Ferry de Boer with Kamil. Watch it on YouTube here.

Authors

Maciej Jedrzejewski	Kamil Baczek
Software architect, tech lead and facilitator of modern software development practices that allow shortening the feedback loop in every area of a lifecycle e.g. trunk-based development, short-living branches, vertical slices, canary releases, CI/CD, and more. Blog Linkedin YouTube	.NET Engineer, Software Architect who empowers teams to build better software through solid software architecture, utilising techniques such as Event Storming, Domain Driven Design and various architecture styles and design patterns. Blog Linkedin Github

Milestone	Date	Done
Repository Premiere 🚀	2023.10	✅
Migration to .NET 8 🔥	2023.11	✅
Architecture Tests 🔧	2023.12
Chapter 4: Focus on Complexity 🧠	2024.03
Fitness Functions ⚙️	2024.05

💬 Join our Community

Join the "Evolutionary Architecture Community" on Discord (https://discord.gg/BGxYkHFCCF) to engage with fellow architects and enthusiasts who share a fervor for pushing boundaries and crafting high-quality software systems. Whether you have questions, suggestions, or feedback for our repository, we’re excited to hear from you and collaborate towards continuous improvement.

⭐ Say thanks

Feel free to give a ⭐ to this repository if you like it. Your support is greatly appreciated!

evolutionary-architecture-by-example's Issues

MassTransit ITestingHarness and OutBoxPattern

Issue Description:

Problem:

The MassTransit library provides the ITestingHarness interface, which allows for testing and verification of published events. However, when the OutBoxPattern is configured, the behavior of publishing events changes. Instead of immediately publishing the event, the service sends the event in the background after loading it from the database.

Manual Testing:

Manually testing the scenario has confirmed that this behavior is functioning correctly.

Test To Fix

Docker file for chapter 1 references dotnet 7 rather that dotnet 8

Describe the bug
Docker file for chapter 1 references dotnet 7 rather that dotnet 8

To Reproduce
Steps to reproduce the behavior:

1. run docker-compose build in chapter 1 src directory

Expected behavior
Builds without issue.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
-Windows 11

Smartphone (please complete the following information):

• N/A

Additional context
N/A.

implement fluent validation for all queries and commands

implement fluent validation for all queries and commands

Add to docs link to Polish Presentation from Programistok about Evolutionary Architecture

Domain Error Flow is controlled by exceptions that has impact on performance and maintability - We can switch to discriminated union results

Domain Error Flow is controlled by exceptions that has impact on performance and maintability - We can switch to discriminated union results

Create solution template for evolutionary architecture

Is your feature request related to a problem? Please describe.
Some people want to start theirs projects using evolutionary architecture example.
This is not developer friendly.

Describe the solution you'd like
Create a solution template for evolutionary architecture.

Adoption of .NET 8 Native IDateTimeProvider Support in Place of ISystemClock

Is your feature request related to a problem? Please describe.
The current implementation relies on ISystemClock for managing time-related operations. However, with the introduction of native time support in .NET 8, there's an opportunity to enhance the system by utilizing the new native time capabilities.

Describe the solution you'd like
Update the codebase to replace the usage of ISystemClock with the native time support introduced in .NET 8. This will involve refactoring the relevant sections of the code to leverage the new time functionalities provided by the .NET 8 framework.

Describe alternatives you've considered
An alternative could be to continue using ISystemClock, but considering that .NET 8 provides native time support, it makes sense to migrate to the native solution for improved performance and consistency across the codebase.

Additional context
The adoption of native time in .NET 8 can bring alignment with the latest advancements in the .NET framework. It's essential to test and validate the changes thoroughly to ensure compatibility and reliability across different scenarios.

Request, Command Mapping Why? Missing Documentation

Discuss about using Dapr in Chapter 3 to not slow down development during transition from monolith to Microservices

Is your feature request addressing a specific issue? Please elaborate.

Some individuals have suggested that the next stage of evolution could involve the incorporation of Dapr. This would enable us to maintain our focus on the business aspects rather than getting entangled in Microservices infrastructure management.

Please provide further details on the proposed solution.

I would like to initiate a discussion on this matter.

Add information about how tests can evolve

Is your feature request related to a problem? Please describe.
At the moment we focus mainly on solution architecture. It would be great to include description about testing strategy and how it can grow.

Describe the solution you'd like
Each chapter should contain a section that shows possible testing strategies.

Describe alternatives you've considered
No alternatives considered.

Additional context
No additional context.

Tests using fake event bus can't dected class scopes misconfiguration like scoped in singleton

Is your feature request related to a problem? Please describe.
Tests using fake event bus can't detected class scopes misconfiguration like scoped in singleton

Describe the solution you'd like
There should be check or test which detect such situations

Add information about how infrastructure can evolve

Is your feature request related to a problem? Please describe.
At the moment we focus mainly on solution architecture. It would be great to include description about infrastructure and how it can grow.

Describe the solution you'd like
Each chapter should contain a section that shows possible infrastructure solutions.

Describe alternatives you've considered
No alternatives considered.

Additional context
No additional context.

Ice panel documentation

Is your feature request related to a problem? Please describe.

For chapter 1 and 2 component architecture looks similar only implementation details were changed but for chapter 3. Deployment and components were changed.

Describe the solution you'd like

Chapter 3 should have ice panel diagrams and linked

Transition IEnumerable<object[]> Test Classes to TheoryData in Xunit

Is your feature request related to a problem? Please describe.
The current test classes utilize IEnumerable<object[]> for parameterized tests. Transitioning to TheoryData in Xunit can enhance code readability and maintainability.

Describe the solution you'd like
Update existing test classes that use IEnumerable<object[]> to leverage Xunit's TheoryData for parameterized testing. This transition will provide a more structured and intuitive approach to writing and maintaining test cases.

SignContract.Events.ContractSignedEvent Event Handler Exception

Describe the bug

There is an exception during using the app

System.Collections.Generic.IEnumerable1[MediatR.INotificationHandler1[EvolutionaryArchitecture.Fitnet.Contracts.SignContract.Events.ContractSignedEvent]]' from root provider because it requires scoped service 'EvolutionaryArchitecture.Fitnet.Passes.Data.Database.PassesPersistence

Continuous architecture evolution direction assessment - description and definition fitness functions

Our repository has become a valuable resource for people seeking to understand how architectural evolution takes place. As we move forward, it's crucial to establish a clear framework for monitoring this evolution and ensuring alignment with our project's objectives.

Documentation of Architecture Evolution:

We should document the process of architectural evolution within our repository to provide transparency and insight into our decision-making. This documentation will help contributors and stakeholders understand how our architecture evolves over time.

Introducing Fitness Functions:

To continuously assess the direction of architectural evolution, we propose the use of fitness functions. Fitness functions are tools that enable us to quantitatively evaluate the fitness or suitability of our architecture concerning specific requirements. These functions will help us measure how well our architecture aligns with our evolving needs.

Add additional links to chapters in main readme

Is your feature request related to a problem? Please describe.
At main readme there is no links to chapters where chapters are described.

Describe the solution you'd like
Add missing links to enhance navigation

Outbox pattern implementation

Is your feature request addressing a specific issue? Please elaborate.

In Chapter 3 of our project, we have introduced asynchronous communication through an external message broker, which is a critical component of our system. However, the absence of the Outbox Pattern in this chapter could potentially lead to data inconsistency and reliability issues when dealing with asynchronous messaging.

Please provide further details on the proposed solution.

It has been suggested to incorporate the outbox pattern in Chapter 3.

Simplify Module Activation Process

Is your feature request related to a problem? Please describe.
The current implementation for enabling modules relies on feature flags, which adds complexity to the system.

Describe the solution you'd like
I would like to simplify the process by directly reading module enablement from the configuration.

Describe alternatives you've considered
One alternative is to continue using feature flags for module enablement, but this may add unnecessary complexity. Another alternative could be to explore different methods of module activation.

Additional context
No additional context at this time.

Broken links in Chapter 1 readme

Description
Chapter-1-initial-architecture/README.adoc contains two broken links to code in this repository.

For example:

You can check how does it look like here.

Solution
In the links path meaboutsoftware/evolutionary-architecture needs to re replaced with evolutionary-architecture/evolutionary-architecture-by-example

Example how to do observability?

Is your feature request related to a problem? Please describe.
It is frequent question How to implement observability? It is mentioned that observability is important from start of project. How to Do it?

My proposition is add this part as example