Where Patterns go to Die

At every point in my career in software there has been a current trend in architecture. The trend has usually changed every couple of years, far faster than there have been significant changes in the underlying languages. This has often struck me as significant, as (ignoring the growing areas of machine learning and AI) the basic problems we’re trying to solve have not changed hugely over the years. So why was it ok in the 1990’s to build a monolith? Why is it no longer a good idea to install an Enterprise Service Bus? What changed?

I’ve often heard people talk about past patterns with phrases like “back when we thought this was a good idea”, but I’m hesitant to believe that so many people could have been wrong about the same thing at the same time. I think there must be more to it than that.

There are always benefits

When I think about all the monoliths I’ve worked on, I can’t say that the experience has always been terrible. There are obvious benefits to a monolithic approach, such as being able to easily share code without the need for package management. They can be a lot easier to test as there are no application boundaries, just about every test you can think of could be implemented as a unit test, although the units may get rather large. Because everything is in one solution we don’t get as many network calls, so we aren’t exposed to network outages or versioning issues during releases.

What about SOA? That was huge at one point. Services focused on business processes which can be called by various applications which need that functionality, it doesn’t sound unreasonable. In fact it sounds a lot like microservices. I have worked on dozens of service oriented architectures over the last decade, none of which would have been described by their builders as SOA.

Enterprise Service Bus – almost no-one likes these any more. Yet the idea of having an integration platform which allows each application to carry on it’s day to day processes without ever having to be aware of any other application which might need to know about its data or processes is not a silly one.

How about: the Service Locator Pattern, Stored Procedures, utility classes, remote procedure calls? I’m sure if you think long enough, there will always be some other ‘good idea at the time’ that is now generally frowned upon.

But what about: microservices, serverless, cloud computing, native apps, no sql databases? Surely these things are destined to be around forever..? We got it right these times. Right?

You still have to design stuff

“How do you build a microservice?”

Is this a good question? If you can answer this question, does that mean you can implement a micro-architecture successfully?

If you know how to deploy, manage, and push code to an enterprise service bus, does that mean you can successfully implement one?

Let me ask these questions in another way:

Which business problem is solved by either micro-architecture or ESB? Because if you aren’t solving a business problem, then you aren’t successfully implementing anything.

It seems to me that an awful lot of technologists follow trends like each one is their next true religion, without ever seeing the actual truths behind them. I know for absolute certainty that every single ‘bad idea’ that has at one time been ‘the latest trend’ will fix a specific problem pretty well. It may lead to other problems if not managed correctly, but that isn’t the point – if you choose an approach, you must choose an approach which is going to work for you and continue to work.

Characteristics

These are some of the characteristics of microservices:

  • They are individually deployable.
  • They are individually testable.
  • They are individually versionable.
  • They represent a business process.
  • They own their own database and data.
  • When changing one service, it’s important to know what other services consume it and test them alongside the change.
  • The number of microservices in an enterprise can grow pretty quickly, managing them requires a high degree of automation.

These are some of the characteristics of monoliths:

  • All code is in a single solution.
  • Boundaries are defined by namespaces.
  • The entire application is redeployed each time.
  • User interfaces are built into the same application as business logic.
  • They often write to databases which are used by other applications.
  • If they become too big, the code becomes gridlocked and difficult to change.

These are some of the characteristics of enterprise service busses:

  • They can be highly available.
  • They allow for moving data in a resilient fashion.
  • Changes can be deployed without interfering with other applications.
  • They can integrate applications across LAN boundaries in a resilient fashion.
  • They can abstract away the implementation of business concerns behind facades.
  • They can quickly become an expensive dependency which can be updated only by a specific few people who understand the technology.

These are some of the characteristics of the service locator pattern:

  • It allows access to an IOC kernel from objects which haven’t necessarily been instantiated by that kernel.
  • It allows access to different implementations based on the requirements of the consuming class.
  • It isn’t immediately obvious that the pattern is in use, which can lead to confusion when working on an unfamiliar codebase.

These are some of the characteristics of a serverless approach:

  • Developers can think more about the code they’re writing and less about the platform.
  • The container running the code is generally the same in dev as test and production.
  • Some serverless implementations are reduced to the function level and are very small.
  • When they become very small, services can become harder to group and manage.
  • Building serverless can sometimes require extra technical knowledge of the specific flavour of serverless in use.

Each of these patterns, approaches, or technologies, all have benefits and down sides. There is a time and a place for each. More importantly, there are more scenarios where each of these patterns would be a mistake than where they would work well. Even where one of these could be a good idea, there’s still plenty of scope to implement it poorly.

Blind faith

I think this is what happens. I think technologists at work get pressured into delivering quickly, or have people telling them they have to work with specific technologies, or their piers laugh when they build something uncool. I think as technologists there are too many of us out there who don’t put enough consideration into whether they are solving the business problem, or whether they are just regurgitating the same stuff that was built previously because ‘that pattern worked ok for them’. I think too may people only see the label, when they should be looking at what’s behind the label.

Piling code on top of code in a situation which isn’t being watched because “it’s worked perfectly fine so far” is what leads to problems. Whether you’re building a single application, pushing into a service fabric, or programming an ESB – if you take your eye off the ball, your pattern itself will rot.

Take SOA for example, how many huge, complicated, poorly documented, misunderstood API’s are deployed which back onto a dozen databases and a range of other services? API’s getting called by all sorts of applications deployed to all sorts of locations, with calls proxied through multiple layers to find their way to this one point of functionality. At some point those API’s were small enough to be a good idea. They were consumed by a few applications deployed somewhere not too distant, and didn’t need much documentation because their functionality was well scoped. Then someone took their eye off the ball and logic which should have been implemented in new services was thrown into the existing one because ‘deploying something else would be tricky’.

This is where patterns get thrown out, as if it was inevitable that the pattern would lead to something unmanageable. Well I have news for you: the pattern doesn’t make decisions, you do.

If you solve a problem by building a service which represents a business process, doesn’t need to call other services, but has been stuck on top of a well used legacy monolithic database, then well done! Who cares that it isn’t quite a microservice? As long as you have understood the downsides to the choices you have made, and know how they are managed or mitigated in your specific circumstances, then that’s just fine. You don’t have to build from the text book every time.

By solving problems rather than implementing cool patterns, we move the focus onto what is actually important. Support the business first.

My First Release Weekend

At the time of writing this post, I am 41 years old, I’ve been in the business of writing software for over 20 years, and I have never ever experienced a release weekend. Until now.

It’s now nearly 1 pm. I’ve been here since 7 am. There are a dozen or so different applications which are being deployed today, which are highly coupled and maddeningly unresilient. For my part, I was deploying a web application and some config to a security platform. We again hit a myriad of issues which hadn’t been seen in prior environments and spent a lot of time scratching our heads. The automated deployment pipeline I built for the change takes roughly a minute do deploy everything, and yet it took us almost 3 hours to get to the point where someone could log in.

The release was immediately labelled a ‘success’ and everyone starts singing praises. As subsequent deployments of other applications start to fail.

This is not success!

Success is when the release takes the 60 seconds for the pipeline to run and it’s all working! Success isn’t having to intervene to diagnose issues in an environment no-one’s allowed access to until the release weekend! Success is knowing the release is good because the deploy status is green!

But when I look at the processes being followed, I know that this pain is going to happen. As do others, who appear to expect it and accept it, with hearty comments of ‘this is real world development’ and ‘this is just how we roll here’.

So much effort and failure thrown at releasing a fraction of the functionality which could have been out there if quality was the barrier to release, not red tape.

And yet I know I’m surrounded here by some very intelligent people, who know there are better ways to work. I can’t help wondering where and why progress is being blocked.

Scale or Fail

I’ve heard a lot of people say something like “but we don’t need huge scalability” when pushed for reason why their architecture is straight out of the 90’s. “We’re not big enough for devops” is another regular excuse. But while it’s certainly true that many enterprises don’t need to worry so much about high loads and high availability, there are some other, very real benefits to embracing early 21st century architecture principals.

Scalable architecture is simple architecture

Keep it simple, stupid! It’s harder to do than it might seem. What initially appears to be the easy solution can quickly turn into a big ball of unmanageable, tightly coupled string of dependencies where one bad line of code can affect a dozen different applications.

In order to scale easily, a system should be simple. When scaling, you could end up with dozens or even hundreds of instances, so any complexity is multiplied. Complexity is also a recipe for waste. If you scale a complex application, the chances are you’re scaling bits which simply don’t need to scale. Systems should be designed so hot functions can be scaled independently of those which are under utilised.

Simple architecture takes thought and consideration. It’s decoupled for good reason – small things are easier to keep ‘easy’ than big things. An array of small things all built with the same basic rules and standards, can be easily managed if a little effort is put in to working out an approach which works for you. Once you have a few small things all being managed in the same way, growing to lots of small things is easy, if it’s needed.

Simple architecture is also resilient, because simple things tend not to break. And even if you aren’t bothered about a few outages, it’s better to only have the outages you plan for.

Scalable architecture is decoupled

If you need to make changes in anything more than a reverse proxy in order to scale one service, then your architecture is coupled, and shows signs of in-elasticity. Other than being scalable, decoupled architecture is much easier to maintain, and keeps a much higher level of quality because it’s easier to test.

Decoupled architecture is scoped to a specific few modules which can be deployed together repeatedly as a single stack with relative ease, once automated. Outages are easy to fix, as it’s just a case of hitting the redeploy button.

Your end users will find that your decoupled architecture is much nicer to use as well. Without having to make dozens of calls to load and save data in a myriad of different applications and databases, a decoupled application would just make only one or two calls to load or save the data to a dedicated store, then raise events for other systems to handle. It’s called eventual consistency and it isn’t difficult to make work. In fact it’s almost impossible to avoid in an enterprise system, so embracing the principal wholeheartedly makes the required thought processes easier to adopt.

Scalable architecture is easier to test

If you are deploying a small, well understood, stack with very well known behaviours and endpoints, then it’s going to be no-brainer to get some decent automated tests deployed. These can be triggered from a deployment platform with every deploy. As the data store is part of the stack and you’re following micro-architecture rules, the only records in the stack come from something in the stack. So setting up test data is simply a case of calling the API’s you’re testing, which in turn tests those API’s. You don’t have to test beyond the interface, as it shouldn’t matter (functionally) how the data is stored, only that the stack functions correctly.

Scalable architecture moves quicker to market

Given small, easily managed, scalable stacks of software, adding a new feature is a doddle. Automated tests reduce the manual test overhead. Some features can get into production in a single day, even when they require changes across several systems.

Scalable architecture leads to higher quality software

Given that in a scaling situation you would want to know your new instances are going to function, you need attain a high standard of quality in what’s built. Fortunately, as it’s easier to test, quicker to deploy, and easier to understand, higher quality is something you get. Writing test first code becomes second nature, even writing integration tests up front.

Scalable architecture reduces staff turnover

It really does! If you’re building software with the same practices which have been causing headaches and failures for the last several decades, then people aren’t going to want to work for you for very long. Your best people will eventually get frustrated and go elsewhere. You could find yourself in a position where you finally realise you have to change things, but everyone with the knowledge and skills to make the change has left.

Fringe benefits

I guess what I’m trying to point out is that I haven’t ever heard a good reason for not building something which can easily scale. Building for scale helps focus solutions on good architectural practices; decoupled, simple, easily testable, micro-architectures. Are there any enterprises where these benefits are seen as undesirable? Yet, when faced with the decision of either continuing to build the same, tightly coupled, monoliths which require full weekends (or more!) just to deploy, or building something small, light weight, easily deployed, easily maintained, and ultimately scalable, there are plenty of people claiming “Only in an ideal world!” or “We aren’t that big!”.

Bonkers!

What’s Slowing Your Business?

There are lots of problems that prevent businesses from responding to market trends as quickly as they’d like. Many are not IT related, some are. I’d like to discuss a few problems that I see over and over again, and maybe present some useful solutions. As you read this, please remember that there are always exceptions. But deciding that you have one of these exceptional circumstances is always easier when starting from a sensible basic idea.

Business focused targeting.

For many kinds of work, quicker is better. For software development, quicker is better. But working faster isn’t the same thing as delivering faster.

I remember working as technical lead for a price comparison site in the UK, where once a week each department would read out a list of the things they achieved in the last week and how that had benefited the business. For many parts of the business there was a nice and easy line that could be drawn from what they did each week and a statistic of growth (even if some seemed quite contrived). But the development team was still quite inexperienced, and struggling to do CI never mind CD. For the less experienced devs, being told to “produce things quicker” had the opposite effect. Traditional stick and carrot doesn’t have the same impact on software development as on other functions, because a lot of the time what speeds up delivery seems counter intuitive.

  • Have two people working on each task (pair programming)
  • Focus on only one feature at a time
  • Write as much (or more) test code as functional code
  • Spend time discussing terminology and agreeing a ubiquitous language
  • Decouple from other systems
  • Build automated delivery pipelines

These are just a few examples of things which can be pushed out because someone wants the dev team to work faster. But in reality, having these things present is what enables a dev team to work faster.

Development teams feel a lot of pressure to deliver, because they know how good they can be. They know how quickly software can be written, but it takes mature development practices to deliver quickly and maintain quality. Without the required automation, delivering quick will almost always mean a reduction in quality and more time taken fixing bugs. Then there are the bugs created while fixing other bugs, and so on. Never mind the huge architectural spirals because not enough thought went into things at the start. In the world of software, slow and steady may lose the first round, but it sets the rest of the race up for a sure win.

Tightly coupling systems.

I can’t count how often I’ve heard someone say “We made a tactical decision to tightly couple with <insert some system>, because it will save us money in the long run.”

No.

Just no.

Please stop thinking this.

Is it impossible for highly coupled systems to be beneficial? No. Is yours one of these cases? Probably not.

There are so many hidden expenses incurred due to tightly coupled designs that it almost never makes any sense. The target system is quite often the one thing everything ends up being coupled with, because it’s probably the least flexible ‘off the shelf’ dinosaur which was sold to the business without any technical review. There are probably not many choices for how to work with it. Well the bottom line is: find a way, or get rid. Ending up with dozens of applications all tightly bound to one central monster app. Changes become a nightmare of breaking everyone else’s code. Deployments take entire weekends. License fees for the dinosaur go through the roof. Vendor lock in turns into shackles and chains. Reality breaks down. Time reverses, and mullets become cool.

Maybe I exaggerated with the mullets.

Once you start down this path, you will gradually lose whatever technical individuals you have who really ‘get’ software delivery. The people who could make a real difference to your business will gradually go somewhere their skills can make a difference. New features will not only cost you more to implement but they’ll come with added risk to other systems.

If you are building two services which have highly related functionality, ie. they’re in the same sub-domain (from a DDD perspective), then you might decide that they should be aware of each other on a conceptual level, and have some logic which spans both services and depends on both being ‘up’, and which get versioned together. This might be acceptable and might not lead to war or famine, but I’m making no promises.

It’s too hard to implement Dev Ops.

No, it isn’t.

Yes, you need at least someone who understands how to do it, but moving to a Dev Ops approach doesn’t mean implementing it across the board right away. That would be an obscene way forwards. Start with the next thing you need to build. Make it deployable, make it testable with integration tests written by the developer. Work out how to transform the configuration for different environments. Get it into production. Look at how you did it, decide what you can do better. Do it better with the next thing. Update the first thing. Learn why people use each different type of technology, and whether it’s relevant for you.

Also, it’s never too early to do Dev Ops. If you are building one ‘thing’ then it will be easier to work with if you are doing Dev Ops. If you have the full stack defined in a CI/CD pipeline and you can get all your changes tested in pre-production environments (even infra changes) then you’re winning from the start. Changes become easy.

If you have a development team who don’t want to do Dev Ops then you have a bigger problem. It’s likely that they aren’t the people who are going to make your business succeed.

Ops do routing, DBA’s do databases.

Your developers should be building the entire stack. They should be building the deployment pipeline for the entire stack. During deployment, the pipeline should configure DNS, update routing tables, configure firewalls, apply WAF rules, deploy EC2 instances, install the built application, run database migration scripts, and run tests end to end to make sure the whole lot is done correctly. Anything other than this is just throwing a problem over the fence to someone else.

The joke of the matter is that the people doing the developer’s ‘dirty work’ think this is how they support the business. When in reality, this is how they allow developers to build software that can never work in a deployed state. This is why software breaks when it gets moved to a different environment.

Ops, DBA’s, and other technology specialists should be responsible for defining the overall patterns which get implemented, and the standards which must be met. The actual work should be done by the developer. If for no other reason than the fact that when the developer needs a SQL script writing, there will never be a DBA available. The same goes for any out-of-team dependencies – they’re never available. This is one of the biggest blockers to progress in software development: waiting for other people to do their bit. It’s another form of tight coupling, building inter-dependent teams. It’s a people anti-pattern.

If you developers need help to get their heads around routing principals or database indexing, then get them in a room with your experts. Don’t get those people to do the dirty work for everyone else, that won’t scale.

BAU handle defects.

A defect found by a customer should go straight back to the team which built the software. If that team is no longer there, then whichever team was ‘given’ responsibility for that piece of software gets to fix the bug.

Development teams will go a long way to give themselves an easy life. That includes adding enough error handling, logging, and resilient design practices to make bug fixing a cinch, but only if they’re the ones who have to deal with the bugs.

Fundamental design flaws won’t get fixed unless they’re blocking the development team.

Everything else.

This isn’t an exhaustive list. Even now there are more and more things springing to mind, but if I tried to shout every one out then I’d have a book, not a blog post. The really unfortunate truth is that 90% of the time I see incredibly intelligent people at the development level being ignored by the business, by architects, even by each other, because even though a person hears someone saying ‘this is a good/bad idea’ being able to see past their own preconceptions to understand that point of view is often incredibly difficult. Technologists all too often lack the soft skills required to make themselves heard and understood. It’s up to those who have made a career from their ‘soft skills’ to recognise that and pay extra attention. A drowning person won’t usually thrash about and make a noise.

Getting FitNesse to Work

Sample code here.

Recently I’ve been looking into Specification by Example, which people keep defining to me as BDD done the right way. Specification by Example fully implemented includes the idea of an executable specification. A concept that has led me back to FitNesse having given it the cold shoulder for the last six or seven years.

I’ve always thought of FitNesse as a great idea but I struggled to see how to use it correctly when as a developer I was mainly focused on continuous delivery and Dev Ops. I didn’t see where in the development cycle it fit or how tests in a wiki could also be a part of a CD pipeline. Revisiting FitNesse with a focus on Specification by Example gave me the opportunity to work some of this out and I think I’m very likely to suggest using this tool in future projects.

Before it’s possible to talk about FitNesse in a CI environment, there are a few basics to master. I don’t want to go into a full breakdown of all functionality, but I do want to communicate enough detail to allow someone to get started. In this post I’ll concentrate on getting FitNesse working locally and using it to execute different classes of test. In a later post, I’ll cover how to make FitNesse work with CI/CD pipelines and introduce a more enterprise level approach.

This post will focus on using FitNesse with the Slim test runner against .NET code but should be relevant for a wider audience.

Installing FitNesse

Installing and running FitNesse locally is really easy and even if you’re intending to deploy to a server, getting things running locally is still important. Follow these steps:

  1. Install Java
  2. Follow these instructions to install and configure FitNesse
  3. Create a batch file to run FitNesse when you need it. My command looks like:
    java -jar .fitnesse-standalone.jar -p 9080
    

Once you’re running the FitNesse process, hit http://localhost:9080 (use whichever port you started it on) and you’ll find a lot of material to help you get to grips with things, including a full set of acceptance tests for FitNesse itself.

What FitNesse Isn’t

Before I get into talking about how I think FitNesse can be of use in agile development, I’d like to point out what it isn’t useful for.

Tracking Work

FitNesse is not a work tracking tool; it won’t replace a dedicated ticketing system such as Jira, Pivotal Tracker or TFS. Although it may be possible to see what has not yet been built by seeing what tests fail, that work cannot be assigned to an individual or team within FitNesse.

Unit Testing

FitNesse can definitely be used for unit testing and some tests executed from FitNesse will be unit tests, so this might seem a bit contradictory. When I say FitNesse isn’t for unit testing, I mean that it isn’t what a developer should be using for many unit tests. A lot of unit testing is focused on a much smaller unit than I would suggest FitNesse should be concerned with.

[TestFixture]
public class TestingSomething
{
    [Test]
    [ExpectedException(typeof(ArgumentNullException))]
    public void Constructor_NullConnectionString_ThrowsExpectedException()
    {
        var something = new Something(null);
    }
}

This test should never be executed from FitNesse. It’s a completely valid test but other than a developer, who cares? Tests like this could well be written in a TDD fashion right along with the code they’re testing. Introducing a huge layer of abstraction such as FitNesse would simply kill the developer’s flow. In any case, what would the wiki page look like?

Deploying Code

Fitnesse will execute code and code can do anything you want it to, including deploying other code. It is entirely possible to create a wiki page around each thing you want to deploy and have the test outcomes driven by successful deployments. But really, do you want to do that? Download Go or Ansible – they’re far better at this.

What FitNesse Is

OK, so now we’ve covered a few things that FitNesse is definitely not, let’s talk about how I think it can most usefully fit into the agile development process.

FitNesse closes the gap between specification and executable tests, creating an executable specification. This specification will live through the whole development process and into continued support processes. Let’s start with creating a specification.

Creating a Specification

A good FitNesse based specification should be all about behaviour. Break down the system under test into chunks of behaviour in exactly the same way as you would when writing stories for a SCRUM team. The behaviours that you identify should be added to a FitNesse wiki. Nesting pages for relating behaviours makes a lot of sense. If you’re defining how a service endpoint behaves during various exception states then a parent page of ‘Exception States’ with one child page per state could make some sense but you aren’t forced to adhere to that structure and what works for you could be completely different.

Giving a definition in pros is great for communicating a generalisation of what you’re wanting to define and gives context for someone reading the page. What pros don’t do well is define specfic examples – this is where specficiation by example can be useful. Work as a team (not just ‘3 amigos’) to generate sufficient specific examples of input and output to cover the complete behaviour you’re trying to define. The more people you have involved, the less likely you are to miss anything. Create a decision table in the wiki page for these examples. You now have the framework for your executable tests.

Different Types of Test

In any development project, there are a number of different levels of testing. Different people refer to these differently, but in general we have:

  • Unit Tests – testing a small unit of code ‘in process’ with the test runner
  • Component Tests – testing a running piece of software in isolation
  • Integration Tests – testing a running piece of software with other software in a live-like environment
  • UI Teststesting how a software’s user interface behaves, can be either a component test or integration test
  • Load Tests – testing how a running piece of software responds under load, usually an integration test
  • Manual Tests – testing things that aren’t easily quantifiable within an automated test or carrying out exploratory testing

Manual tests are by their nature not automated, so FitNesse is probably not the right tool to drive these. Also FitNesse does not natively support UI tests, so I won’t go into those here. Load tests are important but may require resources that would become expensive if run continuously, so although these might be triggered from a FitNesse page, they would probably be classified in such a way so they aren’t run constantly. Perhaps using a top level wiki page ‘Behaviour Under Load’.

In any case, load tests are a type of integration test, so we’re left with three different types of test we could automate from FitNesse. So which type of test should be used for which behaviour?

The test pyramid was conceived by Mike Cohn and has become very familiar to most software engineers. There are a few different examples floating around on the internet, this is one:

This diagram shows that ideally there should be more unit tests than component tests, and more integration tests than system tests etc. This assertion comes from trying to keep thing simple (which is a great principle to follow); some tests are really easy to write and to run whereas some tests take a long time to execute or even require someone to manually interact with the system. The preference should always be to test behaviour in the simplest way that proves success but when we come to convince ourselves of a specific piece of behaviour, a single class of test may not suffice. There could be extensive unit test coverage for a class but unless our tests also prove that class is being called, then we still don’t have our proof. So we’re left with the possibility that any given behaviour could require a mix of unit tests, component tests and integration tests to prove things are working.

Hooking up the Code

So, how do we get each of our different classes of tests to run from FitNesse? Our three primary types of test are unit, component and integration. Let’s look at unit tests first.

Unit Tests

A unit test executes directly against application code, in process with the test itself. External dependencies to the ‘unit’ are mocked using dependency injection and mocking frameworks. A unit test depends only on the code under test, calls will never be made to any external resources such as databases or file systems.

Let’s assume we’re building a support ticketing system. We might have some code which adds a comment onto a ticket.

namespace Ticketing
{
    public class CommentManager
    {
        private readonly Ticket _ticket;

        public CommentManager(Ticket ticket)
        {
            _ticket = ticket;
        }

        public void AddComment(string comment, DateTime commentedOn, string username)
        {
            _ticket.AddComment(new Comment(comment, username, commentedOn));
        }
    }
}

To test this, we can use a decision table which would be defined in FitNesse using Slim as:

|do comments get added|
|comment|username|commented on|comment was added?|
|A comment text|AUser|21-Jan-2012|true|

This will look for a class called DoCommentsGetAdded, set the properties Comment, Username and CommentedOn and call the method CommentWasAdded() from which it expects a boolean. There is only one test line in this test which tests the happy path, but others can be added. The idea should be to add enough examples to fully define the behaviour but not so many that people can’t see the wood for the trees.

We obviously have to create the class DoCommentsGetAdded and allow it to be called from FitNesse. I added the Ticketing.CommentManager to a solution called FitSharpTest, I’m now going to add another class library to that project called Ticketing.Test.Unit. I’ll add a single class called DoCommentsGetAdded.

namespace Ticketing.Test.Unit
{
    public class DoCommentsGetAdded
    {
        public string Comment { get; set; }
        public string Username { get; set; }
        public DateTime CommentedOn { get; set; }

        public bool CommentWasAdded()
        {
            var ticket = new Ticket();
            var manager = new CommentManager(ticket);
            int commentCount = ticket.Comments.Count;
            manager.AddComment(Comment, CommentedOn, Username);
            return ticket.Comments.Count == commentCount + 1;
        }
    }
}

To reference this class from FitNesse you’ll have to do three things:

  1. Install FitSharp to allow Slim to recognise .NET assemblies. If you use Java then you won’t need to do this part, FitNesse natively supports Java.
  2. Add the namespace of the class either in FitSharp’s config.xml or directly into the page using a Slim Import Table. I updated the config to the following:
    
    
      
        Ticketing.Test.Unit
      
    
    
  3. Add an !path declaration to your FitNesse Test page pointing at your test assembly.

To get your Test page working with FitSharp, the first four lines in edit mode should look like this:

!define TEST_SYSTEM {slim}
!define COMMAND_PATTERN {%m -r fitSharp.Slim.Service.Runner -c D:ProgramsFitnesseFitSharpconfig.xml %p}
!define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunner.exe}
!path C:devFitSharpTestTicketing.Test.UnitbinDebugTicketing.Test.Unit.dll

Notice the !path entry (line 4). The last step is to add your truth table to the page, so the whole page in edit mode looks like this:

!define TEST_SYSTEM {slim}
!define COMMAND_PATTERN {%m -r fitSharp.Slim.Service.Runner -c D:ProgramsFitnesseFitSharpconfig.xml %p}
!define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunner.exe}
!path C:devFitSharpTestTicketing.Test.UnitbinDebugTicketing.Test.Unit.dll

|do comments get added|
|comment|username|commented on|comment was added?|
|A comment text|AUser|21-Jan-2012|true|

Now make sure your solution is built (if you have referenced the debug assemblies then make sure you build in debug) save your changes in FitNesse and hit the ‘Test’ button at the top of the page.

 

There are of course lots of different types of tables you can use to orchestrate different types of tests. The full list is beyond the scope of this post but if you dip into the documentation under ‘Slim’ then you’ll find it quite easily.

The mechanism always remains the same, however – the diagram below outlines the general pattern.

 

 

This is not really any different to what you would see if you were to swap Slim for NUnit. You have the system driving the test (Slim or NUnit), the test assembly and the assembly under test. The difference here is that rather than executing the test within your IDE using Resharper (or another test runner) you’re executing the test from a wiki page. This is a trade off, but like I said at the beginning: not all tests should be in FitNesse, we’re only interested in behaviour which a BA might specify. There will no doubt be other unit tests executed in the usual manner.

Integration Tests

Integration tests run against a shared environment which contains running copies of all deployables and their dependencies. This environment is very like production but generally not geared up for high load or the same level of resilience.

The sequence diagram for unit tests is actually still relevant for integration tests. All we’re doing for the integration tests is making the test assembly call running endpoints instead of calling classes ‘in process’.

Let’s set up a running service for our ‘add comment’ logic and test it from FitNesse. For fun, let’s make this a Web API service running in OWIN and hosted with TopShelf. Follow these steps to get running:

  1. Add a new command line application to your solution. Call it TicketingService (I’m using .NET 4.6.1 for this).
  2. Add NuGet references so your packages file looks like this:
    
    
      
      
      
      
      
      
      
      
      
      
    
    
  3. Add a Startup class which should look like this:
    using System.Web.Http;
    using Owin;
    
    namespace TicketingService
    {
        public class Startup
        {
            public void Configuration(IAppBuilder appBuilder)
            {
                HttpConfiguration httpConfiguration = new HttpConfiguration();
                httpConfiguration.Routes.MapHttpRoute(
                       name: "DefaultApi",
                       routeTemplate: "api/{controller}/{id}",
                       defaults: new { id = RouteParameter.Optional }
                   );
                appBuilder.UseWebApi(httpConfiguration);
            }
        }
    }
    
  4. Add a Service class which looks like this:
    using System;
    using Microsoft.Owin.Hosting;
    
    namespace TicketingService
    {
        public class Service
        {
            private IDisposable _webApp;
    
            public void Start()
            {
                string baseAddress = "http://localhost:9000/";
    
                // Start OWIN host
                _webApp = WebApp.Start<Startup>(url: baseAddress);
            }
    
            public void Stop()
            {
                _webApp.Dispose();
            }
        }
    }
    
  5. Modify your Program class so it looks like this:
    using System;
    using Topshelf;
    
    namespace TicketingService
    {
        class Program
        {
            static void Main(string[] args)
            {
                HostFactory.Run(
                    c =>
                    {
    
                        c.Service<Service>(s =>
                        {
                            s.ConstructUsing(name => new Service()); 
    
                            s.WhenStarted(service => service.Start());
                            s.WhenStopped(service => service.Stop());
                        });
    
                        c.SetServiceName("TicketingService");
                        c.SetDisplayName("Ticketing Service");
                        c.SetDescription("Ticketing Service");
    
                        c.EnablePauseAndContinue();
                        c.EnableShutdown();
    
                        c.RunAsLocalSystem();
                        c.StartAutomatically();
                    });
    
                Console.ReadKey();
            }
        }
    }
    
  6. Make sure your TicketService project is referencing your Ticketing project.
  7. Add a fake back end service to provide some persistence:
    using Ticketing;
    
    namespace TicketingService
    {
        public static class BackEndService
        {
            public static Ticket Ticket { get; set; } = new Ticket();
        }
    }
    
  8. And finally, add a controller to handle the ‘add comment’ call and return a count (yes, I know this is a bit of a hack, but it’s just to demonstrate things)
    using System.Web.Http;
    using Ticketing;
    
    namespace TicketingService
    {
        public class CommentController : ApiController
        {
    
            public void Post(Comment comment)
            {
                BackEndService.Ticket.AddComment(comment);
            }
    
            public int GetCount()
            {
                return BackEndService.Ticket.Comments.Count;
            }
        }
    }
    

Right, run this service and you’ll see that you can add as many comments as you like and retrieve the count.

Going back to our sequence diagram, we have a page in FitNesse already, and now we have an assembly to test. What we need to do is create a test assembly to sit in the middle. I’m adding Ticketing.Test.Integration as a class library in my solution.

It’s important to be aware of what version of .NET FitSharp is built on. The version I’m using is built against .NET 4, so my Ticketing.Test.Integration will be a .NET 4 project.

I’ve added a class to the new project called DoCommentsGetAdded which looks like this:

using System;
using RestSharp;

namespace Ticketing.Test.Integration
{
    public class DoCommentsGetAdded
    {
        public string Comment { get; set; }
        public string Username { get; set; }
        public DateTime CommentedOn { get; set; }

        public bool CommentWasAdded()
        {
            var client = new RestClient("http://localhost:9000");
            var request = new RestRequest("api/comment", Method.POST);
            request.AddJsonBody(new {Text = Comment, Username = Username, CommentedOn = CommentedOn});
            request.AddHeader("Content-Type", "application/json");
            client.Execute(request);

            var checkRequest = new RestRequest("api/comment/count", Method.GET);
            IRestResponse checkResponse = client.Execute(checkRequest);

            return checkResponse.Content == "1";
        }
    }
}

I’ve also added RestSharp via NuGet.

There are now only two things I need to do to make my test page use the new test class.

  1. Add the namespace Ticketing.Test.Integration to FitSharp’s config file:
    
    
      
        Ticketing.Test.Unit
        Ticketing.Test.Integration
      
    
    
  2. Change the !path property in the test page to point at the right test assembly:
    !define TEST_SYSTEM {slim}
    !define COMMAND_PATTERN {%m -r fitSharp.Slim.Service.Runner -c D:ProgramsFitnesseFitSharpconfig.xml %p}
    !define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunner.exe}
    !path C:devFitSharpTestTicketing.Test.IntegrationbinDebugTicketing.Test.Integration.dll
    
    |do comments get added|
    |comment|username|commented on|comment was added?|
    |A comment text|AUser|21-Jan-2012|true|
    

Now, make sure the service is running and hit the ‘Test’ button in FitNesse.

Obviously, this test was configured to execute against our localhost, but it could just as easily have been configured to execute against a deployed environment. Also, the code in the test class isn’t exactly what I would call first class, this is just to get stuff working.

The important thing to take from this is that the pattern in the original sequence diagram still holds true, so it’s quite possible to test this behaviour as either a unit test or as an integration test, or even both.

Component Tests

A component test is like an integration test in that it requires your code to be running in order to test. Instead of your code executing in an environment with other real life services, it’s tested against stubs which can be configured to respond with specific scenarios for specific tests, something which is very difficult to do in an integration environment where multiple sets of tests may be running simultaneously.

I recently wrote a short post about using Mountebank for component testing. The same solution can be applied here.

I’ve added the following controller to my TicketingService project:

using System.Net;
using System.Web;
using System.Web.Http;
using RestSharp;
using Ticketing;

namespace TicketingService
{
    public class TicketController : ApiController
    {
        public void Post(Ticket ticket)
        {
            var client = new RestClient("http://localhost:9999");
            var request = new RestRequest("ticketservice", Method.POST);
            request.AddHeader("Content-Type", "application/json");
            request.AddJsonBody(ticket);

            IRestResponse response = client.Execute(request);

            if (response.StatusCode != HttpStatusCode.Created)
            {
                throw new HttpException(500, "Failed to create ticket");
            }
        }
    }
}

There are so many things wrong with this code, but remember this is just for an example. The functioning of this controller depends on an HTTP endpoint which is not part of this solution. It cannot be tested by an integration test without that endpoint being available. If we want to test this as a component test, then we need something to pretend to be that endpoint at http://localhost:9999/ticketservice.

To do this, install Mountebank by following these instructions:

  1. Make sure you have an up to date version of node.js installed by downloading it from here.
  2. Run the Command Prompt for Visual Studio as an administrator.
  3. Execute:
    npm install -g mountebank
    
  4. Run Mountebank by executing:
    mb
    
  5. Test it’s running by visiting http://localhost:2525.

To create an imposter which will return a 201 response when data is POSTed to /ticketservice, use the following json:

{
    "port": 9999,
    "protocol": "http",
    "stubs": [{
        "responses": [
          { "is": { "statusCode": 201 }}
        ],
        "predicates": [
              {
                  "equals": {
                      "path": "/ticketservice",
                      "method": "POST",
                      "headers": {
                          "Content-Type": "application/json"
                      }
                  }
              }
            ]
    }]
}

Our heavily hacked TicketController class will function correctly only if this imposter returns 201, anything else and it will fail.

Now, I’m going to list the code I used to get this component test executing from FitNesse from a Script Table. I’m very certain that this code is not best practice – I’m trying to show the technicality of making the service run against Mountebank in order to make the test pass.

I’ve added a new project to my solution called Ticketing.Test.Component and I updated the config.xml file with the correct namespace. I have two files in that solution, one is called imposters.js which contains the json payload for configuring Mountebank, the other is a new version of DoCommentsGetAdded.cs which looks like this:

using System;
using System.IO;
using System.Net;
using RestSharp;

namespace Ticketing.Test.Component
{
    public class DoCommentsGetAdded
    {
        private HttpStatusCode _result;

        public string Comment { get; set; }
        public string Username { get; set; }
        public DateTime CommentedOn { get; set; }

        public void Setup(string imposterFile)
        {
            var client = new RestClient("http://localhost:2525");
            var request = new RestRequest("imposters", Method.POST);
            request.AddHeader("Content-Type", "application/json");
            using (FileStream imposterJsFs = File.OpenRead(imposterFile))
            {
                using (TextReader reader = new StreamReader(imposterJsFs))
                {
                    string imposterJs = reader.ReadToEnd();
                    request.AddParameter("application/json", imposterJs, ParameterType.RequestBody);
                }
            }
            client.Execute(request);
        }

        public void AddComment()
        {
            var client = new RestClient("http://localhost:9000");
            var request = new RestRequest("api/ticket", Method.POST);
            request.AddJsonBody(new { Number = "TicketABC" });
            request.AddHeader("Content-Type", "application/json");
            IRestResponse restResponse = client.Execute(request);
            _result = restResponse.StatusCode;
        }

        public bool CommentWasAdded()
        {
            return _result != HttpStatusCode.InternalServerError;
        }

        public void TearDown(string port)
        {
            var client = new RestClient("http://localhost:2525");
            var request = new RestRequest($"imposters/{port}", Method.DELETE);
            request.AddHeader("Content-Type", "application/json");
            client.Execute(request);
        }
    }
}

I’ve updated my FitNesse Test page to the following:

!define TEST_SYSTEM {slim}
!define COMMAND_PATTERN {%m -r fitSharp.Slim.Service.Runner -c D:ProgramsFitnesseFitSharpconfig.xml %p}
!define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunner.exe}
!path C:devFitSharpTestTicketing.Test.ComponentbinDebugTicketing.Test.Component.dll

|Script:do comments get added|
|setup;|C:devFitSharpTestTicketing.Test.ComponentbinDebugimposter.js|
|add comment|
|check|comment was added|true|
|tear down;|9999|

A Script Table basically allows a number of methods to be executed in sequence and with various parameters. It also allows for ‘checks’ to be made (which are your assertions) at various stages – it looks very much like you would expect a test script to look.

In this table, we’re instantiating our DoCommentsGetAdded class, calling Setup() and passing the path to our imposters.js file, calling AddComment() to add a comment and then checking that CommentWasAdded() returns true. Then we’re calling TearDown().

Setup() and TearDown() are there specifically to configure Mountebank appropriately for the test and to destroy the Imposter afterwards. If you try to set up a new Imposter at the same port as an existing Imposter, Mountebank will throw an exception, so it’s important to clean up. Another option would have been to set up the Imposter in the DoCommentsGetAdded constructor and add a ~DoCommentsGetAdded() destructor to clean up – this would mean the second and last lines of our Script Table could be removed. I am a bit torn as to which approach I prefer, or whether a combination of both is appropriate. In any case, cleaning up is important if you want to avoid port conflicts.

Ok, so run your service, make sure Mountebank is also running and then run your test from FitNesse.

Again, this works because we have the pattern of our intermediary test assembly sat between FitNesse and the code under test. We can write pretty much whatever code we want here to call any environment we like or to just execute directly against an assembly.

Debugging

I spent a lot of time running my test code from NUnit in order to debug and the experience grated because it felt like an unnecessary step. Then I Googled to see what other people were doing and I found that by changing the test runner from:

!define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunner.exe}

to:

!define TEST_RUNNER {D:ProgramsFitnesseFitSharpRunnerW.exe}

FitSharp helpfully popped up a .NET dialog box and waited for me to click ‘Go’ before running the test. This gives an opportunity to attach the debugger.

Summary

A high level view of what has been discussed here:

  • Three types of test which can be usefully run from FitNesse: Unit, Integration and Component.
  • FitNesse is concerned with behaviour – don’t throw NUnit out just yet.
  • The basic pattern is to create an intermediary assembly to sit between FitNesse and the code under test. Use this to abstract away technical implementation.
  • Clean up Mountebank Imposters.
  • You need FitSharp to run FitNesse against .NET code.
  • Debug with FitSharp by referencing the RunnerW.exe test runner and attaching a debugger to the dialog box when it appears.

Not Quite Enterprise

In this first post on FitNesse, I’ve outlined a few different types of test which can be executed and listed code snippets and instructions which should allow someone to get FitNesse running on their machine. This is only a small part of the picture. Having separate versions of the test suite sat on everyone’s machine is not a useful solution. A developer’s laptop can’t be referenced by a CI/CD platform. FitNesse can be deployed to a server. There are strategies for getting tests to execute as part of a DevOps pipeline.

This has been quite a lengthy post and I think these topics along with versioning and multi-environment scenarios will be best tackled in a subsequent post.

I also want to take a more process oriented look at how tests get created, who should be involved and when. So maybe I have a couple of new entries to work on.

Integration Testing Behaviour with Mountebank

Developer’s machine > dev shared environment > staging environment > UAT > production.

Probably not exactly how everyone structures their delivery pipelines but probably not that far off. It allows instant feedback on whether what a developer is writing actually works with the code other developers are writing. And that’s a really good thing. Unfortunately, it misses something…

Each environment (other than the developer’s own machine) is shared with other developers who are also deploying new code at the same time. So how do you get an integration test for component A that relies on component B behaving in a custom manner (maybe even failing) to run automatically, without impacting the people who are trying to build and deploy component B?

If we were writing a unit test we would simply inject a mocked dependency. Fortunately there’s now a fantastic piece of kit available for doing exactly this but on an integration scale: enter Mountebank.

This clever piece of kit will intercept a network call for ANY protocol and respond in the way you ask it to. Transparent to your component and as easy to use as most mocking frameworks. I won’t go into detail about how to configure ‘Imposters’ as their own documentation is excellent, but suffice to say it can be easily configured in a TestFixtureSetup or similar.

So where does this fit into our pipeline? Personally, I think the flow should be:

Push code to repo > Code is pulled onto a build server > Build > Unit test > Integration test > Start deployment pipeline

The step where Mountebank comes in is obviously ‘integration testing’.

Keep in mind that installing the component and running it on the build agent is probably not a great idea, so make good use of the cloud or docker or both to spin up a temporary instance which has Mountebank already installed and running. Push your component to it, and run your integration tests. Once your tests have run then the instance can be blown away (or if constantly destroying environments gets a bit slow, maybe have them refreshing every night so they don’t get cluttered). Docker will definitely help keeping these processes efficient.

This principle of spinning up an isolated test instance can work in all kinds of situations, not just where Mountebank would be used. Calls to SQL Server can be redirected to a .mdf file for data dependent testing. Or DynamoDb tables can be generated specifically scoped to the running test.

What we end up with is the ability to test more behaviours than we can do in a shared environment where other people are trying to run their tests at the same time. Without this, our integration tests can get restricted to only very basic ‘check they talk to each other’ style tests which although have value do not cover everything we’d like.