andyfrench.info

The problem

I’ve recently been looking at a problem with some software that was written to work in a cluster. This particular software service runs background jobs against a SQL Server database and in order to support fail-over scenarios the software was written to work in a cluster. Only one service instance (the master) was actually doing any work at any given time with the other instances (the slaves) providing redundancy in the case of a failure. In other words, one instance would be nominated as the master and would take responsibility for running the background jobs. If the master crashed or became unavailable one of the other instances in the cluster would take over as master.

From now on I’m going to continue to use the term service instance to describe a software comonent that participates in a cluster. Each service instance is probably a separate process.

The problem was that the mechanism used to elect and monitor the master was based on UDP broadcast, and broadcast is something that can be problematic in cloud-based environments such as AWS. Given there was a need to migrate this service to the cloud this was a significant issue.

At a high level the election algorithm being used by the cluster was for service instances to use UDP broadcast to exchange messages between themselves to agree which instance would be the master. Once the master had been nominated it took over the work of running the background jobs. The other service instances would then periodically poll the master to check that it was still alive. The first instance to find the master to be unavailable would claim the master role, take over the responsibility of running the jobs and broadcast the change in master.

The use of UDP broadcast in this context was useful because it meant that service instances didn’t need to know about each other. To use more direct addressing it would be necessary to store the addresses of all instances in the cluster in some form of registry or configuration. Configuration management across multiple environments is itself a challenge so reducing the amount of configuration can be an advantage.

However, in this case the use of UDP broadcast was an issue that needed to be addressed to facilitate a move to the cloud. This provided a good opportunity to review clustering election patterns and approaches to writing clustered software in general to see what options are available.

Note: There are alternatives to creating writing software that behave as a cluster natively (e.g. ZooKeeper). This article does not deal with these alternative approaches but focuses on the creation of natively clustered software.

Reasons for clustering

There are typically 2 reasons for writing software that supports clustering:

• Failover – to prevent outages it would be advantageous to build in redundancy so that if one service instance crashes there’s another available to take up the slack. Note that in this case it isn’t necessary for all instances to be doing useful work. Some may be on stand-by, available to take over if the primary fails but not doing anything while the primary is active.
• Performance – to facilitate greater application performance running separate software instances (probably on separate servers) may be advantageous. In this case work can be distributed between instances and processed in parallel.

 

Of course, these two aspects are not mutually exclusive; a cluster may support both high availability and distributed processing.

Characteristics of clustered software

Typically when running software as a cluster one instance will be nominated as the coordinator (leader or master). Note that this instance does not have to perform the work itself, it may choose to delegate the work to one of the other instances in the cluster. Alternatively – such as in our example above – the coordinator may perform the work itself exclusively.

This is somewhat analogous to server clustering which can be either symmetrical or asymmetrical. In the symmetrical case every server in the cluster is performing useful work. To distribute work between the servers in the cluster a load balancer is required. In the case of a software cluster it’s the instance elected as the coordinator that’s probably performing this task.

In the asymmetrical case only one server will be active with the other server instances in the cluster being passive. A passive instance will only be activated in the event of a failure of the primary. In the case of a software cluster the coordinator would be the active instance with other instances being passive.

Whichever basic topology is chosen it will be necessary for the software cluster to elect a coordinator when the cluster starts. It will also be necessary for the cluster to recognise when a coordinator has crashed or become unavailable and for this to trigger the election of a new coordinator.

When designing a system like this care should be taken to avoid the coordinator becoming a bottleneck itself. There are also other considerations. For example, in auto-scaling scenarios what happens if the coordinator is shut down as a result of downsizing the infrastructure?

Election patterns

How do software clusters go about managing the election of a coordinator? Below is a discussion of 3 possible approaches:

• Distributed mutex – use a shared mutex is made available to all service instances and is used to manage which instance is the coordinator. Essentially, all service instances race to grab the mutex. The first to succeed becomes the coordinator.
• Bully algorithm – use messaging between instances in the cluster to elect the coordinator. The election is based on some unique property of each instance (e.g. a process identifier). The process with the highest value ‘wins’. The winning instance bullies the other instances into submission by keeping the mutex and claiming the coordinator role.
• Ring algorithm – use messaging between instances in the cluster to elect the coordinator. Service instances are ordered (either physically or logically) so each instance knows its successors. Ordering in the ring is significant with election messages being passed around the ring to figure out which one is ‘at the top’. That instance is elected the coordinator.

 

More detailed descriptions of the approaches are provided below. As you’d expect each has its pros and cons.

Distributed mutex

A mutex “ensures that multiple processes that share resources do not attempt to share the same resource at the same time”. In this case the ‘resource’ is really a role – that of coordinator - that one service instance adopts.

Using a distributed mutex has the advantage that it works in situations where there is no natural leader (e.g. no suitable process identifier which would be required for the Bully Algorithm). Under some circumstances (e.g. when the coordinator is the only instance performing any work) the service instances need not know about each other either; the shared mutex is the only thing an instance needs to know about. In cases where the coordinator needs to distribute work amongst the other instances in the cluster then the coordinator must be able to contact – and therefore know about – the other instances.

The algorithm essentially follows this process:

1. Service instances race to get a lease over a distributed mutext (e.g. a database object).
2. The first instance to get the mutex is elected as the coordinator. Other instances are prevented from becoming the coordinator because they are blocked from getting a lease on the mutex.
3. The coordinator performs the task of coordingating the distribution of work (or executing it itself depending on requirements).
4. The lease must be set to expire after a period of time and the coordinator must periodically renew the lease. If the coordinator crashes or becomes unavailable it won’t be able to renew the lease on the mutext which will eventually become available again.
5. All service instances periodically check the mutex to see if the lease has expired. If a service instance finds the lease on the mutex to be available it attempts to secure the lease. If it succeeds the instance becomes the new coordinator.

 

Note that the mutext becomes a potential single point of failure so consideration should be given to a scenario where unavailability of the mutex can prevent the cluster from electing a coordinator.

Another characteristic of using a shared mutex in this way is that election of the leader is non-deterministic. Any service instance in the cluster could take on the role of coordinator.

A good explanation of the shared mutex approach can be found in this article from MSDN.

Bully algorithm

There are some assumptions for the Bully Algorithm:

• Each instance in the cluster has a unique identifier which must be an ordinal. This could be a process number or even a network address but whatever it is we should be able to order instances in the cluster using this identifier.
• Each instance knows the identifiers of the other instances that should be participating in the cluster (some may be dead for whatever reason).
• Service instances don’t know which ones are available and which are not.
• Service instances must be able to send messages to each other.

 

The basis of the Bully Algorithm is the service instance with the highest identifier will be the coordinator. The algorithm provides a mechanism for service instances to discover which of them has the highest identifier and for that instance to bully the others into submission by claiming the coordinator role. It follows this basic process:

1. A service instance sends an ELECTION message to all instances with identifiers greater than its own and awaits responses.
2. If no service instances respond the originator can conclude it has the highest identifier and is therefore safe to assume the role of coordinator. The instance sends a COORDINATOR message to all other instances announcing the fact. Other instances will then start to periodically check that the coordinator is still available. If it isn’t, the instance that finds the coordinator unavailable will start a new election (back to step 1).
3. Any service instance receiving an ELECTION message and having an identifier greater than the originator will respond with an OK message indicating it’s available.
4. If in response to an ELECTION message the originator receives an OK response back it knows there’s at least one service instance with a higher identifier than itself. The following then happens:
   1. The original service instance abandons the election (because it knows there’s at least one process with a higher identifier than itself).
   2. Any instances that responded to the ELECTION message with OK now issue ELECTION messages themselves (they start at step 1) and the process repeats until the service with the highest identifier has been elected.

A nice description of the process can be found in this article.

Ring algorithm

As with the Bully Algorithm there are some basic assumptions for the Ring Alorithm.

• The service instances are ordered in some way.
• Each service instance uses the ordering to know who its successor is (in fact it needs to know about all the instances in the ring, as we will see below).

 

The Ring Algorithm basically works like this:

1. All service instances monitor the coordinator.
2. If any service instance finds the coordinator is not available it sends an ELECTION message to its successor. If the successor is not available the message is sent to the next instance in the ring until an active one is found.
3. Each service instance that receives the ELECTION message adds its identifier to the message and passes it on as in step 2.
4. Eventually the message gets back to the originating process instance which recognises the fact because its own identifier is in the list. It examines the list of active instances and finds the one with the highest identifier. The instance then issues a COORDINATOR message informing all the instances in the ring which one is now coordinator (the one with the highest identifier).
5. The service instance with the highest identifier has now been elected as the coordinator and processing resumes.

 

Note that multiple instances could recognise that the coordinator is unavailable resulting in multiple ELECTION and COORDINATOR messages being sent around the ring. This doesn’t matter, the result is the same.

Other things to look at

A NuGet package is available for a light-weight non-intrusive leader election library for .Net called NanoCluster. Source code is available on GitHub here:

https://github.com/ruslander/NanoCluster

It’s a small project and doesn’t seem to have been used a great deal but might provide some ideas.

References

• Cloud Networking: IP Broadcasting and Multicasting in Amazon EC2 - https://www.ravellosystems.com/blog/cloud-networking-ip-broadcasting-multicasting-amazon-ec2/
• Amazon VPC FAQs - https://aws.amazon.com/vpc/faqs/#R4
• Leader Election Pattern - https://msdn.microsoft.com/en-gb/library/dn568104.aspx
• Apache ZooKeeper - https://zookeeper.apache.org/

Read More

I think the answer to this question is “most of the time”, at least that’s the case judging by a lot of the code I’ve seen lately.

For example, during a recent recruitment campaign we’ve been setting a fairly open technical test for the candidates, a test that required them to write some code to solve a problem. A number of candidates chose to expose data using something called a ‘repository’ but in almost every case the so-called repository didn’t look like a repository to me at all.

Does the following look like an interface for a repository to you?

public interface ISeatRepository
{
    IEnumerable<Seat> GetAllSeats();

    IEnumerable<Seat> GetSeatsForVenue(Venue venue);

    IEnumerable<Seat> GetSeatsForCustomer(Customer customer);
}

This is typical of the kind of thing I have been seeing.

My concern is that the term ‘repository’ is being applied to anything that exposes data with no real understanding of the original pattern. This is at best misleading and in most cases demonstrates a lack of understanding of the pattern and/or a laziness to use a more appropriate architecture.

Defining the Repository Pattern

Martin Fowler’s web site describes a repository in the following way:

“A Repository mediates between the domain and data mapping layers, acting like an in-memory domain object collection. Client objects construct query specifications declaratively and submit them to Repository for satisfaction. Objects can be added to and removed from the Repository, as they can from a simple collection of objects, and the mapping code encapsulated by the Repository will carry out the appropriate operations behind the scenes.” [1]

I like this definition. The take-away point is that the repository “Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.” Note, ‘a collection-like interface’.

To me the pattern is quite clear: the repository should look like a collection. For example:

public interface IRepository<T>
{
    void Add(T instance);
    void Remove(T instance);
    IEnumerable<T>
    FindAll();
    T Find(int id);
    IEnumerable<T>
    Find(Expression<Func<T, bool>> predicate);
}

There are many alternative approaches to repositories but the one above serves well enough as an example.

One of the key drivers behind the repository pattern is that it isolates domain objects from details of the database access code. This can be a benefit or a drawback depending on your requirements. If for example you need tight control over the SQL then a repository may not be the best way to go; the pattern is designed to hide that sort of detail from client code.

Eric Evans in his book Domain-Driven Design, Tackling Complexity in the Heart of Software talks extensively about repositories. In DDD the repository is a key concept to be used in combination with Aggregates or Aggregate Roots. In fact, repositories should be limited to aggregate roots – you don’t have repositories for every domain type in your system! A discussion of those concepts is beyond the scope of this post but I’d encourage the user to investigate further.

Evans provides this definition:

“For each type of object that needs global access, create an object that can provide the illusion of an in-memory collection of all objects of that type. Set up access through a well-known global interface. Provide methods to add and remove objects, which will encapsulate the actual insertion or removal of data in the data store. Provide methods that select objects based on some criteria and return fully instantiated objects or collections of objects whose attribute values meet the criteria, thereby encapsulating the actual storage and query technology.” [2]

If you are interested Vaughn Vernon goes in to some detail about repositories, including collection-oriented repositories, in his book Implementing Domain-Driven Design.

Repositories - how and when

Use repositories if you want to present clients with a simple model for obtaining persistent objects and managing their life cycle. Be aware that using this pattern will provide a bland interface to the data – that’s the whole point. If you need to write optimised SQL then this probably isn’t the way to go.

In my experience Object Relational Mappers (ORMs) like Entity Framework much less successful but still possible.

Note that if you use repositories you will probably need an alternative method of managing transactionality. The Unit of Work pattern is commonly used in combination with repositories to solve this problem. Rob Connery has an interesting article on why Repositories On Top UnitOfWork Are Not a Good Idea but I think a good part of the issues here are caused by Entity Framework and the need to pass DbContext objects around.

One category of data access technology I don’t think is a good fit with repositories are the micro-ORMs (e.g. PetaPoco, Dapper). These tend to have APIs that are quite close to the SQL and just don’t have a comfortable fit with repositories. If you use these technologies and your data access API doesn’t have a collection-like interface then don’t call your data access classes repositories!

An alternative

An alternative that I like is to use is to implement Commands and Queries. Having seen the potential benefits of Command Query Responsibility Segregation (CQRS) originally put forward by Udi Dahaan the basic concept of separating operations that modify data (Commands) and operations that fetch data (Queries) seemed very attractive. Now CQRS is quite an involved architecture but it’s really an extension of Command Query Separation, a principle that’s been around for years.

Note that:

• Queries: Return a result and do not change the observable state of the system (are free of side effects).
• Commands: Change the state of a system but do not return a value.

Jimmy Bogard discusses a similar approach in his post Favor query objects over repositories.

By providing some simple interfaces commands and queries can be easily defined. Firstly queries:

/// <summary>
/// This interface encapsulates queries - read-only operations against a backing store.
/// </summary>
/// <typeparam name="TParameters">The query parameters to use</typeparam>
/// <typeparam name="TResult">The type of result to return</typeparam>
public interface IQuery<TParameters, TResult>
{
    /// <summary>
    /// Execute the query against the backing store.
    /// </summary>
    /// <param name="parameters">The query parameters to use</param>
    /// <returns>The result of the query</returns>
    TResult Execute(TParameters parameters);
}

/// <summary>
/// This interface encapsulates queries - read-only operations against a backing store.
/// </summary>
/// <typeparam name="TResult">The type of result to return</typeparam>
public interface IQuery<TResult>
{
    /// <summary>
    /// Execute the query against the backing store.
    /// </summary>
    /// <returns>The result of the query</returns>
    TResult Execute();
}

And now commands:

/// <summary>
/// An interface for commands. Commands write to a backing store.
/// </summary>
/// <remarks>
/// Commands do not return values because it is expected that the operation will succeed. If it fails an exception will be thrown.
/// </remarks>
public interface ICommand
{
    /// <summary>
    /// Execute the command.
    /// </summary>
    void Execute();
}

/// <summary>
/// An interface for commands with parameters. Commands write to a backing store.
/// </summary>
/// <remarks>
/// Commands do not return values because it is expected that the operation will succeed. If it fails an exception will be thrown.
/// You should create classes to represent you <typeparamref name="TParameters"/>, simple POCOs with properties for the
/// individual parameters to be passed into the <see cref="Execute"/> method.
/// </remarks>
/// <typeparam name="TParameters">The type of parameters to use</typeparam>
public interface ICommand<TParameters>
{
    /// <summary>
    /// Execute the command using the given parameters.
    /// </summary>
    /// <param name="parameters">The parameters to use</param>
    void Execute(TParameters parameters);
}

/// <summary>
/// An interface for commands with parameters. Commands write to a backing store.
/// </summary>
/// <remarks>
/// Commands do not return values because it is expected that the operation will succeed. If it fails an exception will be thrown.
/// You should create classes to represent your <typeparamref name="TParameters"/>, simple POCOs with properties for the
/// individual parameters to be passed into the <see cref="Execute"/> method.
/// </remarks>
/// <typeparam name="TParameters">The type of parameters to use</typeparam>
/// <typeparam name="TResult">The type of result to return</typeparam>
public interface ICommand<TParameters, TResult>
{
    /// <summary>
    /// Execute the command using the given parameters.
    /// </summary>
    /// <param name="parameters">The parameters to use</param>
    /// <returns>The result</returns>
    TResult Execute(TParameters parameters);
}

Interfaces such as these allow transactionality to be implemented on the Execute() methods. For example a base class could be created that automatically has transactions applied to an abstract Execute() method using an attribute (I have done this using Spring.Net which provides a TransactionAttribute).

A simple command might look like this:

/// <summary>
/// A command for saving a <see cref="User"/>.
/// </summary>
public class SaveUserCommand : ICommand<User>
{
    /// <summary>
    /// Execute the command using the given parameters.
    /// </summary>
    /// <param name="user">The user to save</param>
    public void Execute(User user)
    {
        // Save the user using your technology of choice...
    }
}

One thing I have noticed about this approach is that you quickly build up a library of commands and queries which can become difficult to manage and error prone. They all start to look the same. A case of buyer beware.

Wrapping up

If you are writing a data access API and it doesn’t have a collection-like interface then don’t call it a repository. Don’t use the name ‘repository’ as a crutch for poor, lazy design. There are alternatives.

References

[1] Repository, by Edward Hieatt and Rob Mee, http://martinfowler.com/eaaCatalog/repository.html
[2] Evans, Domain-Driven Design, Tackling Complexity in the Heart of Software, p.151-152

Read More

Some time ago I blogged about Breaking dependencies on specific DI containers. That post was concerned with resolving a WCF service’s dependencies using an IInstanceProvider and how to break a tight decency on Unity. I chose to use the interface provided by the Common Service Locator library on CodePlex. But there is still a problem, and one that didn’t occur to me as I was writing the code because I was so wrapped up in the details of WCF.

In short, the use of a Service Locator is considered by some to be an anti-pattern. Having thought things over I think I have to agree.

The problem with a Service Locator is that it hides dependencies in your code making them difficult to figure out and potentially leads to errors that only manifest themselves at runtime. If you use a Service Locator your code may compile but hide the fact that the Service Locator has been incorrectly configured. At runtime, when your code makes use of the Service Locator to resolve a dependency that hasn’t been configured, an error will occur. The compiler wasn’t able to help. Moreover, by using a Service Locator you can’t see from the API exactly what dependencies a given class may have.

At least one proposed solution to this problem is to use Constructor Injection to explicitly pass all of a classes dependencies in when it is instantiated. I like the logic of this solution. You can see from the constructor signature exactly what dependencies a class has and if one is missing you’ll get a compiler error.

Let’s look at an example. Imagine some code like this:

class Program
{
    static void Main(string[] args)
    {
        var needsDependencies = new NeedsDependencies(new ServiceLocator());
        needsDependencies.CallMe();
    }
}

Can you tell from the above code what dependencies the NeedsDependencies class actually has? How can you tell if the ServiceLocator has been configured correctly without running the code and seeing if it fails? In fact you can only see the dependencies by looking at the internal code of NeedsDependencies:

public class NeedsDependencies
{
    private IServiceLocator _serviceLocator;

    public NeedsDependencies(IServiceLocator serviceLocator)
    {
        _serviceLocator = serviceLocator;
    }

    public void CallMe()
    {
        var dependency = (Dependency)_serviceLocator.GetInstance(typeof(Dependency));
        dependency.CallMe();
    }
}

From the source code we can see that the NeedsDependencies class actually needs an instance of the Dependency class. If we didn’t have access to the source code (e.g. we were using a library provided by a 3rd party) we’d be be none the wiser and would only see that the class needs a Service Locator.

If we remove the Service Locator and replace it with the actual dependencies we can see exactly what a class needs up front:

public class NeedsDependencies
{
    private Dependency _dependency;

    public NeedsDependencies(Dependency dependency)
    {
        _dependency = dependency;
    }

    public void CallMe()
    {
        _dependency.CallMe();
    }
}

To call this version we’d do something like this:

class Program
{
    static void Main(string[] args)
    {
        var NeedsDependencies = new NeedsDependencies(new Dependency());
        needsDependencies.CallMe();
    }
}

We can see right away that NeedsDependencies there is a dependency on the Dependency class – because the constructor makes this clear - and if we fail to provide one the compiler will complain.

It is important to note that we can still use IoC to resolve dependencies and we can even use a Service Locator. The point is we shouldn’t pass the Service Locator around but should restrict its use to one place in the application, usually at at start-up when dependencies are being resolved.

References

• Service Locator is an Anti-Pattern, Mark Seemann.
• Inversion of Control Containers and the Dependency Injection pattern, Martin Fowler

Read More

Health warning – The code featured in this post hasn’t been tested. These are just my musings on the subject of the Unit of Work pattern.

The Unit of Work pattern has been around for quite a while. I first encountered this pattern when I adopted NHibernate as the basis of my data access code. In NHibernate the Session object represents a Unit of Work. I’ve been investigating the Microsoft Entity Framework again - the latest code first option in particular - and I’ve noticed that in many examples the Unit of Work pattern is being implemented explicitly. This is particularly prevalent when the Repository pattern is being used.

So, what is the Unit of Work pattern?

Martin Fowler defines Unit of Work in the following terms:

"Maintains a list of objects affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems.

…When you're pulling data in and out of a database, it's important to keep track of what you've changed; otherwise, that data won't be written back into the database. Similarly you have to insert new objects you create and remove any objects you delete…

…A Unit of Work keeps track of everything you do during a business transaction that can affect the database. When you're done, it figures out everything that needs to be done to alter the database as a result of your work." *

Folwer goes on to suggest an interface for the Unit of Work pattern. In C# – and taking advantage of Generics - the interface might look something like this:

public interface IUnitOfWork<T>
{
    void RegisterNew(T instance);
    void RegisterDirty(T instance);
    void RegisterClean(T instance);
    void RegisterDeleted(T instance);
    void Commit();
    void Rollback();
}

For my purposes I think I can drop the RegisterClean(T instance) and Rollback() methods leaving:

public interface IUnitOfWork<T>
{
    void RegisterNew(T instance); // for tracking new instances
    void RegisterDirty(T instance); // for tracking updated instances 
    void RegisterDeleted(T instance); // for tracking deleted instances
    void Commit(); // to flush registered changes to the backing store
}

Many people seem to remove the Rollback() method and simply choose to not commit changes when something goes wrong. If you need to work with transactions you might choose differently.

Once you have an interface for your Unit of Work you need to decide what is going to implement it. I have observed 2 basic approaches to this:

1. Create a Unit of Work class that implements the IUnitOfWork<T> interface and pass this to a Repository. In this case the IUnitOfWork<T> acts as a wrapper or adapter for the NHibernate session or EF context etc.
2. Have your repository implement IUnitOfWork<T>. In this case the Repository is the Unit of Work.

I do not like the second option because I think it blurs the distinction between the 2 patterns. When we look at things from the point of view of separation of concerns or single responsibility the first option looks better to me.

So, given our IUnitOfWork<T> interface, lets imagine an NHibernate implementation of the Unit of Work as well as a Repository:

public class NHibernateUnitOfWork<T> : IUnitOfWork<T>
{
    private ISession _session;

    public NHibernateUnitOfWork(ISession session)
    {
        _session = session;
    }

    public void RegisterNew(T instance)
    {
        _session.Save(instance);
    }

    public void RegisterDirty(T instance)
    {
        _session.Update(instance);
    }

    public void RegisterDeleted(T instance)
    {
        _session.Delete(instance);
    }

    public void Commit()
    {
        _session.Flush();
    }
}

public interface IRepository<T>
{
    void Add(T instance);
    void Remove(T instance);
    void Update(T instance);
}

public class Repository<T> : IRepository<T>
{
    private IUnitOfWork<T> _unitOfWork;

    public Repository(IUnitOfWork<T> unitOfWork)
    {
        _unitOfWork = unitOfWork;
    }

    public void Add(T instance)
    {
        _unitOfWork.RegisterNew(instance);
    }

    public void Remove(T instance)
    {
        _unitOfWork.RegisterDeleted(instance);
    }

    public void Update(T instance)
    {
        _unitOfWork.RegisterDirty(instance);
    }
}

public class User
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

To make use of the Unit of Work we might do something like this:

//Configure NHibernate
var configuration = new Configuration();
configuration.Configure();

ISessionFactory sessionFactory = configuration.BuildSessionFactory();
ISession session = sessionFactory.OpenSession();

// Create and use a Unit of Work
IUnitOfWork<User> unitOfWork = new NHibernateUnitOfWork<User>(session);
IRepository<User> repository = new Repository<User>(unitOfWork);

var user = new User {FirstName = "Andy", LastName = "French"};

repository.Add(user);
// Maybe do other things with the repository here...
            
unitOfWork.Commit();

Note that in this example we are simply delegating to the NHibernate session which already represents a Unit of Work. Rather than do this some people like to keep lists of the affected entities in the Unit of Work class and then process the lists as a batch in the Commit() method. Perhaps such an implementation would look like this:

public class NHibernateUnitOfWork<T> : IUnitOfWork<T>
{
    private ISession _session;
    private IList<T> _added = new List<T>();
    private IList<T> _deleted = new List<T>();
    private IList<T> _updated = new List<T>();

    public NHibernateUnitOfWork(ISession session)
    {
        _session = session;
    }

    public void RegisterNew(T instance)
    {
        _added.Add(instance);
    }

    public void RegisterDirty(T instance)
    {
        _updated.Add(instance);
    }

    public void RegisterDeleted(T instance)
    {
        _deleted.Add(instance);
    }

    public void Commit()
    {
        using (ITransaction transaction = _session.BeginTransaction())
        {
            ProcessAdded();
            ProcessUpdated();
            ProcessDeleted();

            transaction.Commit();
        }
    }

    private void ProcessAdded()
    {
        foreach(var instance in _added)
        {
            _session.Save(instance);
        }
    }

    private void ProcessUpdated()
    {
        foreach (var instance in _updated)
        {
            _session.Update(instance);
        }
    }

    private void ProcessDeleted()
    {
        foreach (var instance in _deleted)
        {
            _session.Delete(instance);
        }
    }
}

Note that in this version we’ve been able to use NHibernate transaction support. Choose your poison.

References

* Unit of Work, Martin Fowler, http://martinfowler.com/eaaCatalog/unitOfWork.html

Read More

If you listen to the .Net Rocks! podcast you may have heard the episode where Udi Dahan Clarifies CQRS. CQRS? What’s that all about then?
Health warning: I’m learning here. Errors and misconceptions probable. Jump to References below to go to more authoritative resources.
It looks like CQRS started with Bertrand Meyer as part of his work on the Eiffel programming language. At that time Meyer referred to the principle as Command-Query Separation (CQS). That principle states:

“…that every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer. More formally, methods should return a value only if they are referentially transparent and hence possess no side effects.” - Wikipedia

The implications of this principle are that if you have a return value you cannot mutate state. It also implies that if you mutate state your return type must be void.
CQRS extends CQS by effectively mandating that when we interact with data stores objects are assigned to one of two ‘types’: Commands and Queries. This division of responsibility extends into the rest of the architecture. You may have separate services dedicated to either commands (e.g. create, update and delete operations) or queries (read-only operations). It can even extend to the level of the data store – you may choose to have separate stores for read-only queries and for commands.
Why is this separation important? Well, it recognises that the two sides have very different requirements:

	Command	Query
Data Store	Store normalised data for transactional updates etc.	Store denormalised data for fast and convenient querying (e.g. minimise the number of joins required).
Scalability	Scalability may be less important for commands. For example many web systems are skewed towards more frequent read-only operations.	Scalability may be very important, especially in web systems. We may want to use caching here.

CQRS recognises that rather than having one unified system where create, read, update and delete operations are treated as being the same it may be better to have two complimentary systems working side by side: one for read-only query operations and another for command operations. The following diagram is somewhat over-simplified but…

Features of the Command side

• Our ‘data access layer’ for command operations becomes behavioural as opposed to data-centric.
• Our data transfer objects don’t need to expose internal state – we create a DTO and fire it off with a command.
• We don’t need to process commands immediately – they can be queued.
• We might not even require an ‘always on’ connection to the data store.
• We might use events to manage interaction between components in the command system.

Features of the Query side (the Thin Read Layer)

• We can create an object model that is optimised for display purposes (e.g. view models).
• We can create a separate data store that is optimised to meet the needs of the display object model (e.g. data can be denormalised to fit the requirements of specific views).
• We can optimise data access to prevent round trips to the data layer so that all the data required by a view is returned in one operation.
• We can optimise read-only queries in isolation from the requirements of update operations (this might be compromised in a non-CQRS system where, for example, an ORM is used for all data access operations).
• We can bypass direct interaction with the data store and use cached data (although we should be explicit about this when we do it – let the user know how fresh the data is).
• We can use different levels of caching for different parts of the application (perhaps one screen requires data to be fresher than another).

How do we keep the query data store in sync?

This is where events come in to the picture and I think I’ll leave that for another post!

References

• CQRS Introduction
• Udi Dahan – CQRS Clarified
• Rinat Abdullin - CQRS Starting Page

Read More

Warning: Before going any further you probably want to have a look at Service locator anti-pattern. The code featured in this post uses Service Locator but the pattern is regarded by some to be an anti-pattern and as such should be avoided.

I’ve been working on a WCF service that uses Unity for dependency resolution. Everything has been working but I’ve been unhappy about a tight dependency on Unity itself that I have introduced in my code. I recalled that there is a service locator library knocking around that defines an interface that IoCs can adopt.

“The Common Service Locator library contains a shared interface for service location which application and framework developers can reference. The library provides an abstraction over IoC containers and service locators. Using the library allows an application to indirectly access the capabilities without relying on hard references. The hope is that using this library, third-party applications and frameworks can begin to leverage IoC/Service Location without tying themselves down to a specific implementation.” - http://commonservicelocator.codeplex.com/

The Common Service Locator library provides a simple interface for service location:

public interface IServiceLocator : IServiceProvider
{
    object GetInstance(Type serviceType);
    object GetInstance(Type serviceType, string key);
    IEnumerable<object> GetAllInstances(Type serviceType);
    TService GetInstance<TService>();
    TService GetInstance<TService>(string key);
    IEnumerable<TService> GetAllInstances<TService>();
}

It turns out that the library is supported by Unity (as well as a bunch of other IoC implementations). The Common Service Locator library site links to an adapter for Unity but peeking around I found the UnityServiceLocator class in the Microsoft.Practices.Unity assembly.

I have now been able to replace all references to IUnityContainer with IServiceLocator (i.e. breaking the tight dependency on Unity). In the case of the WCF service all I needed to do was create a ServiceHostFactory implementation that passes an instance of UnityServiceLocator around rather than an instance of UnityContainer.

public class UnityServiceLocatorServiceHostFactory : ServiceHostFactory
{
    protected override ServiceHost CreateServiceHost(Type serviceType, Uri[] baseAddresses)
    {
        var unityContainer = new UnityContainer();
        unityContainer.LoadConfiguration();
        var unityServiceLocator = new UnityServiceLocator(unityContainer);
        return new ServiceLocatorServiceHost(serviceType, unityServiceLocator, baseAddresses);
    }  
}

The UnityServiceLocatorServiceHostFactory is the only class that has a tight dependency on Unity and can be farmed off into a separate Unity assembly. All other classes, including the service host implementation, only need to deal with IServiceLocator:

public class ServiceLocatorServiceHost : ServiceHost
{
    private IServiceLocator _serviceLocator;
        
    public ServiceLocatorServiceHost(IServiceLocator serviceLocator) : base()
    {
        _serviceLocator = serviceLocator;
    }

    public ServiceLocatorServiceHost(Type serviceType, IServiceLocator serviceLocator, params Uri[] baseAddresses)
        : base(serviceType, baseAddresses)
    {
        _serviceLocator = serviceLocator;
    }

    protected override void OnOpening()
    {
        if (Description.Behaviors.Find<ServiceLocatorServiceBehaviour>() == null)
        {
            Description.Behaviors.Add(new ServiceLocatorServiceBehaviour(_serviceLocator));
        }

        base.OnOpening();
    }
}

The ServiceLocatorServiceBehavior adds a service locator instance provider to the endpoint dispatcher, something like this:

public class ServiceLocatorServiceBehavior : IServiceBehavior
{   
    private readonly IServiceLocator _serviceLocator;
	
    public ServiceLocatorServiceBehavior(IServiceLocator serviceLocator)
    {
        _serviceLocator = serviceLocator;
    }

    public void Validate(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
        // Nothing to see here. Move along...
    }

    public void AddBindingParameters(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase, Collection<ServiceEndpoint> endpoints, BindingParameterCollection bindingParameters)
    {
        // Nothing to see here. Move along...
    }

    public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
        foreach (ChannelDispatcher channelDispatcher in serviceHostBase.ChannelDispatchers)
        {
            foreach (EndpointDispatcher endpointDispatcher in channelDispatcher.Endpoints)
            {
                string contractName = endpointDispatcher.ContractName;
                ServiceEndpoint serviceEndpoint = serviceDescription.Endpoints.FirstOrDefault(e => e.Contract.Name == contractName);
                endpointDispatcher.DispatchRuntime.InstanceProvider = new ServiceLocatorInstanceProvider(_serviceLocator, serviceEndpoint.Contract.ContractType);
            }
        }
    }
}

And finally the ServiceLocatorInstanceProvider uses the service locator to resolve dependencies, something like this:

public class ServiceLocatorInstanceProvider : IInstanceProvider
{
    private readonly IServiceLocator _serviceLocator;
    private readonly Type _contractType;

    public ServiceLocatorInstanceProvider(IServiceLocator serviceLocator, Type contractType)
    {
        this._serviceLocator = serviceLocator;
        this._contractType = contractType;
    }

    public object GetInstance(InstanceContext instanceContext)
    {
        return GetInstance(instanceContext, null);
    }

    public object GetInstance(InstanceContext instanceContext, Message message)
    {
        return _serviceLocator.GetInstance(_contractType);
    }

    public void ReleaseInstance(InstanceContext instanceContext, object instance)
    {
            
    }
}

Read More

A pattern that is showing up frequently in .Net development at the moment is Service Locator. It has appeared in the Composite Application Library (Prism - intended for use in Silverlight and WPF applications) as well as ASP.Net MVC. Service Locator is often seen going hand in hand with DI and IoC and one useful implementation of Service Locator might be to decouple an application from specific IoC containers (i.e. use the Service Locator pattern on top of IoC).

In essence, Service Locator is a simple pattern:

“Provide a global point of access to a service without coupling users to the concrete class
that implements it.” *

Martin Fowler describes the pattern in the following terms:

“The basic idea behind a service locator is to have an object that knows how to get hold of all of the services that an application might need.” **

The Composite Application Library documentation offers a number of drawbacks to Service Locator:

“The Service Locator pattern has the following liabilities:

• There are more solution elements to manage.
• You have to write additional code to add service references to the locator before your objects use it.
• Your classes have an extra dependency on the service locator.
• The source code has added complexity; this makes the source code more difficult to understand.” ***

In short the criticism is that Service Locator adds complexity to the application. Martin Fowler also notes that use of Service Locator can lead to testability issues:

“I've often heard the complaint that these kinds of service locators are a bad thing because they aren't testable because you can't substitute implementations for them. Certainly you can design them badly to get into this kind of trouble, but you don't have to.”

Common Service Locator

Many frameworks (including MVC 3) are now working to the Common Service Locator library. This is described in the following terms:

“The Common Service Locator library contains a shared interface for service location which application and framework developers can reference. The library provides an abstraction over IoC containers and service locators. Using the library allows an application to indirectly access the capabilities without relying on hard references.”

A number of IoC frameworks already provide implementations of the core interface (IServiceLocator) including Spring.Net, Unity, and Castle Windsor. The IServiceLocator interface requires that an IoC container provide the following methods:

TService GetInstance<TService>();
object GetInstance(Type serviceType, string key);
TService GetInstance<TService>(string key);
IEnumerable<object> GetAllInstances(Type serviceType);
IEnumerable<TService> GetAllInstances<TService>();

This is interesting but one has to ask if it’s an abstraction too far.

Update: If you are considering using the Service Locator pattern you should be aware that it is regarded by many as an anti-pattern. See Service locator anti-pattern.

* Service Locator, Game Programming Patterns / Communicating Patterns
** Using a Service Locator, Martin Fowler
*** Service Locator

Read More

Having spent some time looking at several presentation technologies recently I thought I should take stock of a number of design patterns that are frequently referenced.

Silverlight and WPF are often associated with the Model-View-ViewModel (MVVM) pattern (see frameworks such as Prism, MVVM Light, Caliburn etc). ASP.Net MVC of course utilises Model-View-Controller. In the past I have tried to overcome some of the short-comings of ASP.Net Web Forms using variations of the Model-View-Presenter (MVP) pattern.

Each of these patterns tries to address a number of problems:

1. Where and how do we maintain state in the UI?
2. Where does business logic live in the application and how is it invoked?
3. How do we keep the UI synchronised with changes to the data and between UI elements?
4. How can we ensure good separation of concerns and create loosely coupled testable code?

So the question is, how do they compare?

Note: There are variations on the theme here so there may be alternatives especially with MVP (e.g. Passive View, Supervising Controller, Front Controller).

MVC	MVP	MVVM
Input is routed to the Controller. The Controller decides which View to render and builds the Model which is bound to the View. A Controller can choose to render one of many Views. The View has no knowledge of its Controller. Business logic exists in the Controller. MVC is useful when state can’t be maintained between user input requests (e.g. over HTTP – a stateless protocol).	Input is routed to the View. The Presenter updates the View usually in response to events raised by the View. State is effectively stored in the View. Business logic exists in the Presenter.	Input is routed to the View. The View knows about nothing but the ViewModel. The ViewModel knows about nothing but the Model. The View gets its data from the ViewModel, not the Model itself. This is generally accomplished by data binding.  State and business logic exist in the ViewModel. The ViewModel can be thought of as an abstract representation of the UI. MVVM is useful when state can be maintained between UI requests (e.g. Silverlight, WPF).

Read More

Silverlight and WPF development have lead to the uptake of a number of development patterns many of which were first introduced by Martin Fowler or which are variations of them. This post contains notes about the various patterns that are appearing in frameworks like Caliburn.Micro or Prism.

These patterns are put together to form a presentation framework.

Firstly, there is the Application Controller:

“A centralized point for handling screen navigation and the flow of an application.” – http://martinfowler.com/eaaCatalog/applicationController.html (Martin Fowler)

The Application Controller is often comprised of a number of elements such as:

• Screen(s)
• Screen Factories
• Screen Factory Registry
• Screen Conductor
• Screen Collection
• Subject

Screen

• The class that handles coordinates the Model, View and ViewModel.
• Coordinates the marrying of a View to a ViewModel (the View doesn’t know about the ViewModel, the ViewModel doesn’t know about View).
• Implements the IScreen interface.

IScreen

• An interface that all Screens will implement.
• Requires methods such as CanLeave() and CleanUp().

Screen Factory

• There is a one-to-one relationship between Screen Factories and Screens.
• Screen Factories implement the IScreenFactory interface
• The Screen Factories are registered with the presentation framework (i.e. the Screen Factory Registry singleton class).
• Screen Factories are used by the Screen Conductor to instantiate screens.

IScreenFactory

• Requires a CreateScreen() method.

Screen Factory Registry

• The Screen Factory Registry contains a registry of all of the screen factories.
• Modules can register their Screen Factory classes with the Screen Factory Registry.
• The Screen Factory Registry has methods including GetFactory(), HasFactory(), and a Screen Factory Dictionary

Screen Conductor

• The Screen Conductor coordinates the creation and destruction of screens.
• The Screen Conductor makes use of the Screen Factory Registry which it uses to create Screens as necessary.
• Screen creation includes screen visibility, screen location, loading Subjects into screens, etc.
• The Screen Conductor has a Screen Collection; a collection of all activated screen instances.
• The Screen Conductor is often a singleton.

Screen Collection

• The Screen Conductor has a Screen Collection which contains all of the activated screen instances.
• The Screen Collection is maintained by the Screen Conductor.

Subject

• The Subject is analogous to the Model; it’s what is required by the screen to fulfil its purpose.

 

After the Application Controller there is the Application Shell.

Application Shell

• The Application Shell is analogous to the main form in the application.
• The responsibility of the shell is to hold the main components of the user interface (e.g. menus, ribbons, panels, docking managers, etc.) for the screens that get activated later. 

 

There sometimes are other elements that are combined with the above as part of the Presentation Framework.

Bootstrapper

• The Bootstrapper is responsible for instantiating elements like the Application Shell and the IoC container (should one be used).
• The Bootstrapper can also be used to activate services used by the application (e.g. caching service).

Read More

MVVM is gaining considerable traction in the WPF and Silverlight communities. The MVVM pattern can be said to be a specialisation of Fowler’s Presentation Model pattern.

The essence of a Presentation Model is of a fully self-contained class that represents all the data and behavior of the UI window, but without any of the controls used to render that UI on the screen. A view then simply projects the state of the presentation model onto the glass.

There is also some similarity with the Model-View-Presenter (MVP) pattern, with the ViewModel being roughly analogous to the Presenter.  However, unlike MVP the ViewModel doesn’t need a reference to the View – the view uses data binding to be made aware of changes. The ViewModel and not the View performs all modifications made to the model data.

Some features of MVVM:

• View classes are unaware of Model classes
• ViewModel and Model classes are unaware View classes (the ViewModel mustn’t assume types of rendering in the View  - e.g. that a button exists)
• Model classes are unaware of ViewModel and View classes
• ViewModel classes are aware of Model classes
• View classes are aware of ViewModel classes

To summarise:

	Aware of View	Aware of Model	Aware of ViewModel
View	N/A	No	Yes
Model	No	N/A	No
ViewModel	No	Yes	N/A

Data binding facilitates loose coupling between the View and the ViewModel. It also supports a standardised input validation model.

The data is typically implemented as properties on the ViewModel. The View consumes that data via data binding. Application logic is typically implemented as methods on the ViewModel that the View can invoke through commanding.

A possible weakness with MVVM in Silverlight applications is the lack of support for Commands.

The following are the differences between Silverlight and WPF regarding commanding:

• Routed commands are not available in Silverlight. However, the ICommand interface is available in Silverlight, allowing developers to create their own custom commands. The Composite Application Library provides the DelegateCommand and CompositeCommand classes to simplify command implementation. For more information, see the Commands technical concept.
• In WPF, controls can be hooked up to commands through their Command property. By doing this, developers can declaratively associate controls and commands. This also means a control can interact with a command so that it can invoke the command and have its enabled status automatically updated. Silverlight controls do not currently support the Command property. To help work around this issue, the Composite Application Library provides an attached property-based extensibility mechanism that allows commands and controls to be declaratively associated with each other. For more information, see "Using Commands in Silverlight Commands" in the Commands technical concept.
• There is no input gesture or input binding support in Silverlight.

http://msdn.microsoft.com/en-us/library/dd458872.aspx

ViewModel classes are easy to test, taking key logic out of the dreaded code behind files.

I need to check out a few things:

• http://www.codeplex.com/CompositeWPF/
• http://development-guides.silverbaylabs.org/Video/Silverlight-Prism
• http://projects.nikhilk.net/SilverlightFX – maybe a bit old now
• http://caliburn.codeplex.com/
• http://mvvmlight.codeplex.com/

See also http://andysonlinenotes.blogspot.com/2010/06/stuff-i-should-be-interested-in.html

Read More