In many modern-day applications, a modular approach to the architecture is an extremely useful tool in the toolbox. The ability to dynamically add or remove parts of the application to restrict or extend available functionality lets you build smaller, less complex components that can be put together to form a greater whole.

The concept of a plugin has been around for a long time, with the first known instances of modular architectures in applications turning up in the 1970s - most notably, the EDT text editor on Unisys VS/9, which allowed execution of completely separate programs with access to the text buffer. Today, plugins are more or less ubiquitous - from browsers, games, and text editors to entire operating systems and backend services.

It's no secret that I love EF Core and C# - I find it an incredibly easy system to use and get into, and it's very intuitive with how it maps your code model to a database schema. It does, however, come with its drawbacks. Primarily, I've recently had some issues with trying to apply a modular architecture to a fairly large application (a Discord bot, if you're interested) that has numerous logically separate modules with only some crosstalk and intermodular dependencies. At present, the modules are all in one assembly and one model, but their design does lend itself well to a decomposition into smaller assemblies with isolated models.

EF Core, however, defaults to a design that considers your contexts as modelling your database in its entirety, with no logical or clear separation between them. It is possible to have multiple contexts, of course, but each only considers the model tree that is included in it - having a set of bounded contexts with little or no overlap and a single complete context that manages the migrations of all entities is a common approach.

This has several obvious drawbacks:

  • Code duplication (entities appear in two or more cal ontexts at minimum)
  • No true separation (entities are typically all in the same schema, with exceptions)
  • No ability to have separate migration paths for the bounded contexts

These three things in combination makes for unfortunate tight coupling between database entities that, strictly speaking, doesn't need it.

Each of these problems can thankfully be solved! We'll attach each in turn, starting with separation.

Fixing logical entity separation

This problem is pretty straightforward, and while not all database providers support the solution, it's generally available in most database providers (SQLite being a notable exception, which makes it an inappropriate provider in this particular article. It has its own time and place, but right now it's not its time to shine).

Our end goal is to be able to separate and modularize our database, allowing us to have clearly defined boundaries between unrelated or loosely related groups of entities. A tried-and-true solution to this are Schemas. Schemas, at their core, are groupings of related tables, sequences, procedures, etc. which reside in a named location in the database (the closest analogy would be a folder in a filesystem). Schemas exist independently of each other, and have no relations other than the ones you define yourself. Perfect!

Fortunately for us, EF Core has schema support by default, and it's very simple to apply.

[Table("Posts", Schema = "Core")]
public class Post
{
    public long ID { get; set; }
    public float Rating { get; set; }
}

In the above example, we have a very simple entity with a rating value. We've applied the Table attribute to the entity, and manually specified the table name and, more importantly, the schema to place the entity in. When we now include this entity in any context and generate a migration that includes it, the Core schema will be created, and the Posts table will be created inside that schema.

If we add another entity in another schema, that entity will likewise be placed in its own schema. This, however, doesn't prevent us (and EF Core!) from creating and using navigation properties that cross a schema boundary.

[Table("UserFavourites", Schema = "FavouritesModule")]
public class UserFavourite
{
    public long ID { get; set; }
    
    public virtual User User { get; set; }
    public virtual Post Favourite { get; set; }
}

These two entities (and the User entity, which I'll let you imagine the layout and location of) can coexist in the same database, using the same provider, but in different schemas and contexts - keeping their tables, sequences, and procedures clearly separated. If we want to get rid of the FavouritesModule for some reason in the future, we now have everything that belongs to that module under one roof, and we don't have to think twice about which entities belong to what.

Alright, great - we now have data separation. Easy enough, and for people with experience in database design, certainly nothing new. It's time to bring EF out to shine.

Separate modules, separate contexts

The concept of a bounded context is, as mentioned before, nothing new. However, it typically involves having a single "god" context that knows about the whole model and manages migrations. We, however, are on a quest for glory and have no time for such copypasting.

Now that the data is separated by schemas, we can make life a little easier on ourselves and separate the contexts as well. We won't implement a complete "god" context, because we want to maintain separation. Instead, we'll start by making a schema-aware base class for our contexts.

public abstract class SchemaAwareDbContext : DbContext
{
    private readonly string _schema;

    protected SchemaAwareDbContext(string schema)
    {
        _schema = schema;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        base.OnModelCreating(modelBuilder);
        modelBuilder.HasDefaultSchema(_schema);
    }
}

So far, it's simple enough - we'll provide the schema we'll be using, and inform EF that that is the default schema for entities in this context's model. While this isn't strictly speaking neccesary (since we declare the schema name on each entity already), it's going to make our lives a lot easier later on.

Next, let's set up some simple contexts for clarity's sake.

public class CoreContext : SchemaAwareDbContext
{
    public DbSet<User> Users { get; set; }
    public DbSet<Post> Posts { get; set; }

    public CoreContext() : base("Core")
    {
    }
}
public class FavouritesModuleContext : SchemaAwareDbContext
{
    public DbSet<UserFavourite> Favourites { get; set; }

    public FavouritesModuleContext() : base("FavouritesModule")
    {
    }
}

If you've been coding along, and you now try to generate migrations, you'll notice a few things. Firstly, you'll need to specify the context to generate migrations for manually, and second, when you generate migrations for FavouritesModuleContext, it's going to try and create tables in the Core schema. This is completely natural (albeit unwanted), since at this point, we haven't taught EF how to decide what to include and what to ignore in each context's model. Our end goal is, of course, to not have entities from other contexts in the migration for any one context.

Implementing separate migration paths

So, we have separation in the database, and we have separation in the code. We've only established the relationships we need, and we have no duplicated code. However, a few problems remain.

  • Migrations still contain entities from other schemas when generated for a schema-aware context
  • EF Core's migration history table is still global for the entire database, and any migrations applied (independently of the schema they're in) are logged in the same table

Fortunately, Chris Morgan from Scotland has provided us with an elegant solution to this problem: a custom migration differ. In short, by leveraging the ability we have to replace EF Core's own system services, we can override the way migrations are computed when comparing two model trees. His code, available on EF Core's GitHub page, takes the schema of an entity into account when diffing model trees, and excludes any entity that we don't want to consider part of our relevant model.

I've modified his code somewhat, and this is the end result. Let's take a look.

public class SchemaAwareMigrationsModelDiffer : MigrationsModelDiffer
{
    public SchemaAwareMigrationsModelDiffer
    (
        IRelationalTypeMappingSource typeMappingSource,
        IMigrationsAnnotationProvider migrationsAnnotations,
        IChangeDetector changeDetector,
        StateManagerDependencies stateManagerDependencies,
        CommandBatchPreparerDependencies commandBatchPreparerDependencies
    )
        : base
        (
            typeMappingSource,
            migrationsAnnotations,
            changeDetector,
            stateManagerDependencies,
            commandBatchPreparerDependencies
        )
    {
    }

    public override bool HasDifferences(IModel source, IModel target)
        => Diff(source, target, new SchemaAwareDiffContext(source, target)).Any();

    public override IReadOnlyList<MigrationOperation> GetDifferences
    (
        IModel source,
        IModel target
    )
    {
        var diffContext = new SchemaAwareDiffContext(source, target);
        return Sort(Diff(source, target, diffContext), diffContext);
    }

    private static ReferentialAction ToReferentialAction(DeleteBehavior deleteBehavior)
        => deleteBehavior == DeleteBehavior.Cascade
            ? ReferentialAction.Cascade
            : deleteBehavior == DeleteBehavior.SetNull
                ? ReferentialAction.SetNull
                : ReferentialAction.Restrict;


    protected override IEnumerable<MigrationOperation> Diff(
        IEnumerable<IForeignKey> source,
        IEnumerable<IForeignKey> target,
        DiffContext diffContext)
    {
        return DiffCollection(
            source,
            target,
            diffContext,
            Diff,
            Add,
            Remove,
            (s, t, c) =>
            {
                if (s.Relational().Name != t.Relational().Name)
                {
                    return false;
                }

                if (!s.Properties.Select(p => p.Relational().ColumnName).SequenceEqual(
                    t.Properties.Select(p => c.FindSource(p)?.Relational().ColumnName)))
                {
                    return false;
                }

                var schemaToInclude = ((SchemaAwareDiffContext) diffContext).Source.Relational().DefaultSchema;

                if (c.FindSourceTable(s.PrincipalEntityType).Schema == schemaToInclude &&
                    c.FindSourceTable(s.PrincipalEntityType) !=
                    c.FindSource(c.FindTargetTable(t.PrincipalEntityType)))
                {
                    return false;
                }

                if (t.PrincipalKey.Properties.Select(p => c.FindSource(p)?.Relational().ColumnName)
                        .First() != null && !s.PrincipalKey.Properties
                        .Select(p => p.Relational().ColumnName).SequenceEqual(
                            t.PrincipalKey.Properties.Select(p =>
                                c.FindSource(p)?.Relational().ColumnName)))
                {
                    return false;
                }

                if (ToReferentialAction(s.DeleteBehavior) != ToReferentialAction(t.DeleteBehavior))
                {
                    return false;
                }

                return !HasDifferences(MigrationsAnnotations.For(s), MigrationsAnnotations.For(t));
            }
        );
    }

    protected class SchemaAwareDiffContext : DiffContext
    {
        public IModel Source { get; }
        public IModel Target { get; }

        public SchemaAwareDiffContext(IModel source, IModel target)
            : base(source, target)
        {
            Source = source;
            Target = target;
        }

        public override IEnumerable<TableMapping> GetSourceTables()
        {
            var schemaToInclude = Source.Relational().DefaultSchema;
            var tables = base.GetSourceTables();

            return tables.Where(x => x.Schema == schemaToInclude);
        }

        public override IEnumerable<TableMapping> GetTargetTables()
        {
            var schemaToInclude = Target.Relational().DefaultSchema;
            var tables = base.GetTargetTables();

            return tables.Where(x => x.Schema == schemaToInclude);
        }
    }
}

This is a bit of a chunk, so let's break it down. Firstly, we have a class deriving from MigrationsModelDiffer, which is the main mechanism EF Core uses when determining changes between the code models you create. What we want to do is filter out any entity that appears in the model tree that has a schema differing from the one we're generating a migration for, and only include entities from our own model partition.

The class performs pretty much everything the default differ does, with two key differences. One, when comparing source tables, we assert that the source entity's table is actually in our schema.

var schemaToInclude = ((SchemaAwareDiffContext) diffContext).Source.Relational().DefaultSchema;

if (c.FindSourceTable(s.PrincipalEntityType).Schema == schemaToInclude &&
    c.FindSourceTable(s.PrincipalEntityType) != c.FindSource(c.FindTargetTable(t.PrincipalEntityType)))
{
    return false;
}

Two, we've provided our own diffing context, which only returns source and target tables that are in our schema.

protected class SchemaAwareDiffContext : DiffContext
{
    public IModel Source { get; }
    public IModel Target { get; }

    public SchemaAwareDiffContext(IModel source, IModel target)
        : base(source, target)
    {
        Source = source;
        Target = target;
    }

    public override IEnumerable<TableMapping> GetSourceTables()
    {
        var schemaToInclude = Source.Relational().DefaultSchema;
        var tables = base.GetSourceTables();

        return tables.Where(x => x.Schema == schemaToInclude);
    }

    public override IEnumerable<TableMapping> GetTargetTables()
    {
        var schemaToInclude = Target.Relational().DefaultSchema;
        var tables = base.GetTargetTables();

        return tables.Where(x => x.Schema == schemaToInclude);
    }
}

Additionally, we provide the source and target models to anyone that has the context so that they can inspect and use the schema themselves. Notice the usage of .DefaultSchema on the IModel.Relational() view - this is the reason we specify the default schema directly on our SchemaAwareDbContext.

In combination, these changes cleanly filter out entities that don't belong to our contextually relevant schema, and form migrations that only include the entities within our model partitions.

To enable this new differ for our contexts, we can replace the standard diffing service when configuring the context options. I do it directly in the context, but you can implement it in whatever way fits your use case best.

public abstract class SchemaAwareDbContext : DbContext
{
    // ...
    
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.ReplaceService<IMigrationsModelDiffer, SchemaAwareMigrationsModelDiffer>();
    }
}

Success!

There's one final piece of the puzzle - the migrations history table. Unfortunately, applied migrations still log themselves in this one common table, and while that may be desirable in some instances, we'd prefer to have truly independent migration paths with no overlap.

Separating migration history

Fortunately for us, most database providers enable us to override the name and schema of the migrations table when we select the provider (UseNpgsql, UseSqlite, UseSqlServer, etc). I'm using Npgsql in this example; adapt this to your database provider.

public abstract class SchemaAwareDbContext : DbContext
{
    // ...
    
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseNpgsql
        (
            "your connection string",
            b => b.MigrationsHistoryTable(HistoryRepository.DefaultTableName + _schema)
        );
    
        optionsBuilder.ReplaceService<IMigrationsModelDiffer, SchemaAwareMigrationsModelDiffer>();
    }
}

In the example above, you might notice that I don't specify the schema for the table, and instead append it to the table name. This is unfortunately an issue with Npgsql, wherein it assumes the migrations table is in the public schema and doesn't support changing it. If you were to pass the schema as the second argument, you'd be able to generate migrations successfully, but the table wouldn't be created automatically and you'd get a failure when updating the database. As a workaround, the history tables are instead given schema-unique names. I haven't tested specifying the schema with other providers, so your mileage may vary - I would, of course, prefer to have the table in the same schema as the entities it tracks, but hey.

And that's more or less it. An implementation of the above achieves all of our initial goals:

  • Conscise code (entities appear once in the contexts that manage their model partitions)
  • Data separation (unrelated entities reside in separate schemas, while allowing cross-schema navigation properties)
  • Separate migration paths (each model partition has its own migration history, completely separate from other schemas)

With our new superpowers in hand, modular EF Core applications are finally easy to build, easy to manage, and easy to maintain. Get building!

Final thoughts

This implementation was whipped up late at night, and I'm sure there are many improvements that could be made. We did trade some duplication for this implementation since ToReferentialAction in the differ is a private method, and the GitHub issue mentions this. No PR has been made yet, but I hope someone gets around to it so we can shave it down a little more.

Furthermore, Npgsql could do with supporting migration tables in different schemas. As far as I could determine, it's down to them simply searching all schemas for the first table matching the configured name. Sticking the history tables in the public schema is an okay workaround, but still a little annoying.

In any case, I've got a Discord bot to decouple and decompose into separate assemblies. Good luck with your projects :)