Sunday, 31 May 2015

Creating a JSON service using TopShelf and Nancy

I love TopShelf. I first encountered it when I was looking at MassTransit. Both MassTransit and TopShelf originated from the same developers (Chris Patterson and Dru Sellers). The NServiceBus host is also built with TopShelf.

What TopShelf allows you to do is to write a service as a console application – which is great for development and testing – but to deploy it later as a Windows service. You do this by simply by passing in some command line parameters to the console application. Fantastic.

“Topshelf is a framework for hosting services written using the .NET framework. The creation of services is simplified, allowing developers to create a simple console application that can be installed as a service using Topshelf. The reason for this is simple: It is far easier to debug a console application than a service. And once the application is tested and ready for production, Topshelf makes it easy to install the application as a service.” – TopShelf Overview

I have been encouraged to look at Nancy by a colleague (thanks Matt!) who has used it on a number of projects. It’s been on my radar for a while but somehow I’ve never got round to looking at it. As it happens for a small side project I’m working on I need to expose some JSON data from a service. How opportune!

“Nancy is a lightweight, low-ceremony, framework for building HTTP based services on .Net and Mono. The goal of the framework is to stay out of the way as much as possible and provide a super-duper-happy-path to all interactions.” – Introduction (Nancy documentation)

Visual Studio and NuGet packages

Firstly I created a Visual Studio solution and a console application project. Once that was done I installed a number of NuGet packages:

  • TopShelf
  • Ninject (for dependency injection)
  • Ninject.Web.Common (so we can scope Ninject bindings to an HTTP request)
  • TopShelf.Ninject (which provides extensions to TopShelf so it can painlessly use Ninject)
  • Nancy
  • Nancy.Hosting.Self (so Nancy can be hosted inside the service)

I also added a few other packages for logging, configuration and JSON serialisation but they aren’t core to the task at hand so I’ll ignore them for now.



Defining the service

The first thing I did was to define a service contract - an interface - for the service that would be hosted by TopShelf.

namespace Andy.French.Region.Growing.Venue.Service
{
    /// <summary>
    /// Classes implementing this interface provide methods for getting
    /// or manipulating venue data.
    /// </summary>
    public interface IVenueService
    {
        /// <summary>
        /// Starts the service.
        /// </summary>
        void Start();

        /// <summary>
        /// Stops the service.
        /// </summary>
        void Stop();
    }
}
The Start() and Stop() methods align with the TopShelf service configurator that allows you to assign actions when a service is started and stopped. More on that later.

The implementation of the interface would be the service to be hosted by TopShelf and which would spin up the Nancy endpoint. Let’s see what that looked like.

namespace Andy.French.Region.Growing.Venue.Service
{
    using System;
    using Andy.French.Configuration.Service;
    using Andy.French.Logging;
    using global::Nancy.Hosting.Self;

    /// <summary>
    /// The venue service.
    /// </summary>
    public class VenueService : IVenueService
    {
        /// <summary>The logger factory.</summary>
        private readonly ILoggerFactory loggerFactory;

        /// <summary>The logger.</summary>
        private ILogger logger;

        /// <summary>The nancy host.</summary>
        private NancyHost nancyHost;

        /// <summary>The host URL.</summary>
        private string hostUrl;

        /// <summary>The JSON file path.</summary>
        private string jsonFilePath;

        /// <summary>
        /// Initialises a new instance of the <see cref="VenueService"/> class.
        /// </summary>
        /// <param name="loggerFactory">The logger factory.</param>
        /// <param name="configurationService">The configuration service.</param>
        public VenueService(ILoggerFactory loggerFactory, IConfigurationService configurationService)
        {
            this.loggerFactory = loggerFactory;
            this.hostUrl = configurationService.GetString("host.url");
        }

        /// <summary>
        /// Gets the logger.
        /// </summary>
        /// <value>The logger.</value>
        public ILogger Logger
        {
            get
            {
                if (this.logger == null)
                {
                    this.logger = this.loggerFactory.CreateInstance(typeof(VenueService));
                }

                return this.logger;
            }
        }

        /// <summary>
        /// Starts the service. In this case it starts the Nancy endpoint.
        /// </summary>
        public void Start()
        {
            this.Logger.Information("The venue service is starting.");

            this.nancyHost = new NancyHost(new Uri(this.hostUrl));
            this.nancyHost.Start();
        }

        /// <summary>
        /// Stops the service. In this case it stops and disposes of the Nancy host.
        /// </summary>
        public void Stop()
        {
            this.Logger.Information("The venue service is stopping.");
            this.nancyHost.Stop();
            this.nancyHost.Dispose();
        }
    }
}
The interesting stuff appears around line 63 in the Start() method. Here I create a Nancy host and start it. The URL passed in (the hostUrl) is loaded from configuration but its actually something like http://localhost:1234. In the Stop() method I stop the Nancy host and dispose of it.

Basically, this is all the code it took to get the Nancy host up-and-running but there was a little bit of plumbing required to set up routes and handlers.

The Nancy module

At this point I had a Nancy host but that was only part of the story. I also needed to define a route and a handler for an HTTP verb (e.g. GET). It’s actually very easy to do this in Nancy and is accomplished using a Nancy module.

namespace Andy.French.Region.Growing.Venue.Service.Nancy
{
    using System.Collections.Generic;
    using System.IO;
    using System.Threading.Tasks;

    using Andy.French.Configuration.Service;
    using Andy.French.Region.Growing.Domain;

    using global::Nancy;

    using Newtonsoft.Json;

    /// <summary>
    /// A Nancy module for the venues service.
    /// </summary>
    public class VenuesModule : NancyModule
    {
        /// <summary>The venues.</summary>
        private List<Venue> venues;

        /// <summary>The JSON file path.</summary>
        private string jsonFilePath;

        /// <summary>
        /// Initialises a new instance of the <see cref="VenuesModule"/> class.
        /// </summary>
        /// <param name="configurationService">The configuration service.</param>
        public VenuesModule(IConfigurationService configurationService)
        {
            this.jsonFilePath = configurationService.GetString("json.file.path");

            this.Get["/venues", true] = async (x, ct) =>
            {
                await this.LoadVenues();
                return this.Response.AsJson(this.venues);
            };
        }

        /// <summary>
        /// Loads the venues.
        /// </summary>
        /// <returns>The task to load the venues.</returns>
        private async Task LoadVenues()
        {
            await Task.Factory.StartNew(
                () =>
                    {
                        this.venues = JsonConvert.DeserializeObject<List<Domain.Venue>>(File.ReadAllText(this.jsonFilePath));
                    });
        }
    }
}

You can see that I created a class that extended NancyModule and defined a route and a handler in the constructor (line 33). This basically says that we are expecting GET requests to arrive at ‘/venues’ and defines what we want to do with them. In my na├»ve service I simply load a JSON file and return the contents in the response as JSON (line 36).

Dependency injection

You’ll notice that the Nancy module and the Venue service both have dependencies passed in on their constructors (e.g. a logger factory or a configuration service). To get these resolved I needed to plug in Ninject. That was done with a simple Ninject module:

namespace Andy.French.Region.Growing.Venue.Service.Ninject
{
    using Andy.French.Configuration.Service;
    using Andy.French.Logging;
    using Andy.French.Logging.Log4Net;

    using global::Ninject.Modules;
    using global::Ninject.Web.Common;

    /// <summary>
    /// A <c>Ninject</c> module.
    /// </summary>
    public class VenueServiceModule : NinjectModule
    {
        /// <summary>
        /// Loads the module into the kernel.
        /// </summary>
        public override void Load()
        {
            this.Bind<ILoggerFactory>().To<LoggerFactory>().InRequestScope();
            this.Bind<IVenueService>().To<VenueService>().InRequestScope();
            this.Bind<IConfigurationService>().To<AppSettingsConfigurationService>().InRequestScope();
        }
    }
}

That was all there was to that.

Wiring up TopShelf

All that remained was to bring everything together with TopShelf. That was done in the main program method:

namespace Andy.French.Region.Growing.Venue.Service
{
    using Andy.French.Region.Growing.Venue.Service.Ninject;

    using Topshelf;
    using Topshelf.Ninject;

    /// <summary>
    /// The main application program.
    /// </summary>
    public class Program
    {
        /// <summary>
        /// Defines the entry point of the application.
        /// </summary>
        public static void Main()
        {
            HostFactory.Run(hostConfigurator =>
            {
                hostConfigurator.UseNinject(new VenueServiceModule());

                hostConfigurator.Service<IVenueService>(serviceConfigurator =>
                {
                    serviceConfigurator.ConstructUsingNinject();
                    serviceConfigurator.WhenStarted(service => service.Start());
                    serviceConfigurator.WhenStopped(service => service.Stop());
                });

                hostConfigurator.RunAsLocalSystem();
                hostConfigurator.SetDescription("The venue service provides venue data.");
                hostConfigurator.SetDisplayName("Region Growing Venue Service");
                hostConfigurator.SetServiceName("VenueService"); 
            });
        }
    }
}

Trying it out

Aside from some log4Net configuration that was pretty much it. Hitting F5 in Visual Studio resulted in the console application being launched and the service starting up. You can see the INFO message indicating the Start() method of the service has been called and therefore the Nancy host has been started.




Using the Fiddler composer I fired a request at the service.



This resulted in JSON being returned (albeit a small incomplete example at this stage). The content type was correctly set to application/json.



Hitting Ctrl-C caused TopShelf to call the Stop() method of the service which stops and disposes of the Nancy host.




That’s it! All too easy.

Sunday, 24 May 2015

How to setup a TeamCity build server with Azure

I thought it would be interesting to see what it would take to setup a Continuous Integration (CI) build server with Azure infrastructure. No doubt there are alternative ways of doing it but this approach certainly got me to a point where had a CI server installed and ready to go. In this case I used TeamCity.

Throughout this post I’ll be using the updated Azure portal.

Create a virtual network (optional)

The first thing I did was create a Virtual Network (VN). Why? Well, this would allow backend virtual machines to communicate with each other should that be necessary. If you’re unsure if you need one this article might help.

To set up a VN click the NEW icon and choose Networking > Virtual Network.

SNAGHTML6d27dde
To create a VN enter a name and click ADDRESS SPACE. This section is required. In the example above I’ve added an Address space CIDR block of 10.0.0.0/16. I’ve then created a subnet with a CIDR block of 10.0.1.0/24. I’ll be adding other resources such as a Virtual Machine (VM) to this subnet.

You might be interested in this if you’re new to Azure VN: http://azure.microsoft.com/en-gb/documentation/services/virtual-network/

Create storage account (optional)

The next thing I did was to create a Storage Account. This would be used for the virtual disks etc. required by the VM I created later. You can omit this step in which case a storage account will be created for you when you create the virtual machine.

SNAGHTML6d1ea58

To add a new storage account click the NEW icon and go to Data + Storage > Storage. It’s quite annoying but when adding the name for your storage account you’re restricted to letters and numbers only. If like me you use a naming convention that involved hyphens you’re hosed.

Create SQL server database (optional)

I chose to use a SQL Server database for TeamCity. Of course there are other options including the default HSQLDB database engine that ships with TeamCity. The problem with the default database is that it can crash and lose your data as well as other problems:

“The internal database may crash and lose all your data (e.g. on out of disk space condition). Also, internal database can become extremely slow on large data sets (say, database storage files over 200Mb). Please also note that our support does not cover any performance or database data loss issues if you are using the internal database.
In short, do not EVER use internal HSQLDB database for production TeamCity instances.” - Setting up an External Database

SQL Server it is. To add a new SQL Server database click the NEW icon and go to Data + Storage > SQL Database. You may need to add a new SQL Server instance or you can use an existing one.


SNAGHTML6deb84c

 

It’s worth having a look at the pricing information for SQL Server in Azure. I chose S0 for this installation that comes out at about £9.16 per month at the time of writing.


SNAGHTML6e17f3c

 

You can get the connection details for your server from the settings but if you try to connect straightaway using SQL Server Management Studio (SSMS) from your local machine you will be disappointed. Before you can connect you need to punch a hole in the firewall.

Select your SQL Server from the browse list in the Azure portal and go to Settings > Firewall. Add a rule with a name and an IP address range. I didn’t add a range because I just wanted my desktop machine to be able to connect - so the start and end IP addresses were the same.


SNAGHTML6e87ad5

 

Don’t forget to save! I did first time around.

SNAGHTML6e93a9b

 

At this point I was able to connect to the newly created SQL Server using SSMS and was able to create a blank database ready to be used by TeamCity. I also added a login and a database user, something like this:

CREATE LOGIN teamcity 
    WITH PASSWORD = 'password_goes_here' 
GO

CREATE USER teamcity
    FOR LOGIN teamcity
GO

Create a Virtual Machine

So far I had a virtual network, a storage account and a SQL Server complete with blank database. Now it was time to create the Virtual Machine (VM) ready to install TeamCity.
To create a new VM click the NEW icon and go to Compute > Windows Server 2012 R2 Datacenter. Fill in the host name, user name and password. The next step is to choose your pricing plan. For this example I chose D1 but you can always check out the pricing guide.


SNAGHTML701ba07

After choosing the price band etc. it’s time to set up the networking side of things. Remember the VN I setup right at the beginning? Well, this is where it came in to play. By selecting the OTIONAL CONFIGURATION I was able to set the Virtual Network and assign the VM to a subnet under NETWORK.


SNAGHTML70fe672

I was also able to assign the storage account that I created earlier on too. This time under the STORAGE ACCOUNT in the optional configuration.

So, at this point I had a Virtual Machine on my Virtual Network and using my storage account.Time to logon to the Virtual Machine.

Logon to the Virtual Machine and install TeamCity

Logging on the the VM was quite easy. Just click on Connect when viewing the VM properties.

 
SNAGHTML7159d18
HINT: I had a few issues with connecting this was from the new portal. I was able to connect by switching to the old portal and doing it from there.

Once I had logged in to the new VM the first job was to download the TeamCity installation package an run it. For this exercise I installed to the default locations but you might consider setting up a separate disk in your VM. Note that I configured TeamCity to run on port 80. More on this later.

Once the basic installation was complete I ran the browser-based UI which starts the setup of TeamCity. When it came to the database I selected the SQL Server database type and entered the details I prepared earlier on my Azure-based SQL Server.

One thing to remember is you need to install the SQL Server JDBC driver by copying the jar into the JDBC directory.


SNAGHTML78e711a

Once the setup was complete I could check with SSMS back on my local machine and see the tables etc. had been created in the SQL Server database.


SNAGHTML798609e
At this point TeamCity was installed and the UI was viewable locally on the VM on port 80. However to make the TeamCity UI visible from my local development machine there were a couple more steps to take.

Firstly, in the VM needed to add some firewall rules added to allow inbound and outbound traffic on port 80. To do this I opened the Server Manager and selected Tools > Windows Firewall with Advanced Security.


SNAGHTML7946da2

Then it was a case of adding the new rules. A picture is worth a thousand words so here’s a few thousand words worth showing the basic steps and settings for the inbound rule (the outbound rule is much the same):


image


image


image


image


image


image


With the firewall rules completed in the VM there was one thing left to do: add a firewall rule to the VM in the Azure portal. This involved the following:

  • Select the VM in the browse list in the Azure portal.
  • Click Settings.
  • Click Endpoints.
  • Click Add and create a new endpoint for the TCP protocol on port 80. 


SNAGHTML79a4ea3

With that done I was able to view the TeamCity UI over the web from my local development machine.

Job’s a good ‘un.