Tuesday, January 19, 2016

Forcing all links to open in a new tab

At my current work, we have a feature called custom pages. This feature provides a bare bones CMS for administrators to make custom page posts. We then display these posts inside an iframe. The iframe contents are served over https. However, users might publish links that serve up content over http. Unfortunately what will happen if a user tries to click the link in an evergreen browser is that the browser will block the link from navigating away. This effectively causes the iframe to display no content.

We were thinking of the best way to force all links to open in a new tab. We thought we had a few options:

  1. Modify all anchor tags to have a target="_blank" attribute
  2. Use an event handler to handle all click events on anchor tags to open a new tab
It turns out there is an even easier and more elegant option, the base tag. This tag has 2 attributes: href and target. The href attribute forces all anchor tags to use the specified value as the base url for all relative links. The target attribute forces all anchor tags to use that value for navigating. It's important to note that individual anchor tags can still override the value in the base tag.

So there you have it, force all links to open in a new window with a very simple html tag.

Friday, May 22, 2015

Package Updates

I have updated a few of my open source libraries recently:

Wednesday, November 26, 2014

npm link with interdependent modules

Sometimes you need to develop on multiple interdependent node modules at the same time. When developing on a dependent module, in order to test your changes, you can do one of several things:
  1. Publish your module and run npm update
  2. Copy and paste code into your node_modules folder
  3. Use npm link to use the in progress version of your module in your other applications
Out of any of the above options only 3 really scales and doesn't leave much potential for errors. Before going any further let's review what npm link does.

Let's say you have 2 modules:
  • dependent-module
  • master-module
master-module also has a dependency on dependent-module. Running npm link inside the dependent-module directory will create a symlink from that directory to the global node_modules/dependent-module directory. Then when running npm link dependent-module in the master-module directory will add dependent-module as a symlink inside the local node_modules directory. This allows you to make changes to dependent-module and instantly have them available inside master-module.

This works great until you have a module that is used in both of your modules that also has global state. A good example is mongoose, the excellent mongodb odm. Schemas and models are defined on the global mongoose object. If each of your modules has mongoose installed, they will have 2 different mongoose objects. So if your models are defined in one module, those models won't be available in your other module. Normally this behavior doesn't happen because npm is smart enough not to install module dependencies if they have already been installed, but when using npm link this won't be the case.

In order to avoid this problem, you will need to run npm link on any susceptible modules. So in our above example if both modules need mongoose, you will need to run npm link mongoose in each affected module directory.

Something to also keep in mind is that running npm update will destroy the symlinks, meaning you need to run npm link dependent-module and npm link mongoose after your run npm update.

Friday, August 29, 2014

AngularJS custom scrollbar directive

On a recent project I needed to create a widget that had a similar look and feel across multiple platforms. This widget had a limited size and needed scrolling. When developing on my Macbook the scrollbars were working fine and everything seemed ok. As soon as we started testing it in Windows, the scrollbar width caused an issue with the rendering pushing all the content down.

The solution that we decided on was to use a custom scrollbar. After scouring the web for custom scrollbars in JavaScript I found TinyScrollbar. I liked it because it didn't use jQuery as we were using AngularJS and didn't want to bog down our app with jQuery. I decided to tweak it and port it to AngularJS as a directive.

It's available on github and via bower (bower install ng-tiny-scrollbar) and a demo can be seen here.

Wednesday, July 9, 2014

npm install ENOENT errors

I've been working on a project that is built using NodeJS. Our development environment is a bit of a hybrid. Most of the developers are developing on Macs, with one exception. Our build server is on Windows and our site is running in Windows Azure. For the most part developing NodeJS on a Mac is quite straightforward. All the tooling is readily available and tends to work quite well. Not so much on Windows. So when our build server starting to run into issues installing the requestify package, we chalked it up to something not liking Windows.

We were getting ENOENT errors where npm was complaining that it couldn't find files that it was supposed to download on the file system. Since the error was happening installing a dependency of a dependency, we thought maybe it was a long paths issue. However, we couldn't even get it to install running in the root directory. We decided to use a work around to switch to a different version of that particular dependency.

Weeks later I ran into the same issue, but this time on my Mac. I found a solution by running npm cache clean before running npm install again. I tried it on the Windows build server and it started working again.

TL;DR;
If you're seeing ENOENT errors when running npm install try running npm cache clean and try again.

Thursday, November 14, 2013

Simple TypeMock Auto-Mocking container

TDD is a great methodology for simplifying your code and focusing on delivering business value. When doing TDD, we always want to strive to make it as friction-free as possible. This means making writing tests and, more importantly, refactoring as painless as we can. One of the ways we can do that is to use an Auto-Mocking container.

A very common reason for tests to change is because we added a new dependency to our SUT and we have to modify all of our previous tests to use this new dependency. One way to avoid this is to use an object that will build your SUT and inject all appropriate dependencies as mocked objects. So your tests go from looking like this:

dependency1 = new Mock<IDependency1>();
dependency2 = new Mock<IDependency2>();
sut = new MyClass(dependency1, dependency2);

to this:

mocker = new Automocker();
sut = mocker.CreateSut<MyClass>();
dependency1 = mocker.GetMock<IDependency1>();
dependency2 = mocker.GetMock<IDependency2>();

We can now add another dependency without having to modify this test (assuming that this dependency is not used in this test).

Most of the popular mocking frameworks that are in use today (Moq, Rhino.Mocks, NSubstitute, etc) all have 1 or more Auto-Mocking frameworks that can be downloaded via NuGet. On our current project, we are using TypeMock, which is a very powerful, commercial grade, mocking tool. It can mock out almost anything and is used quite extensively in the SharePoint development world. There is no Auto-Mocking container for it, so I decided to write one.

An Auto-Mocking container needs to be able to do 2 things. Generate a SUT object with all dependencies mocked out and be able to retrieve the dependencies that were injected. TypeMock uses the following method to generate a mock object

Isolate.Fake.Instance<IDependency>();

Since it uses a static generic method, there is no way to call this method with a Type object without using reflection. So we will use reflection to call this method. We can get the MethodInfo object for this method in the following way:

TypeMockInstance = Isolate.Fake.GetType()
        .GetMethods()
        .Where(m => m.Name == "Instance")
        .Select(m => new
             {
              Method = m,
              Params = m.GetParameters(),
              Args = m.GetGenericArguments()
             })
        .Where(x => x.Params.Length == 0
           && x.Args.Length == 1)
        .Select(x => x.Method)
        .First();

and we would use it like so:

TypeMockInstance.MakeGenericMethod(type).Invoke(Isolate.Fake, null);

When creating our SUT we need to use reflection to get the types of the constructor parameters and use the method above to create them. In order to retrieve them at a later point we need to create a registry, which will simply be a Dictionary<Type, object>; In the case of multiple constructors we will pick the largest constructor. Here's what our CreateSut method looks like:

public T CreateSut<T>()
{
 var type = typeof (T);
 var constructors = type.GetConstructors(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);
 //get largest constructor
 ConstructorInfo largestConstructor = null;
 foreach (var constructor in constructors)
 {
  if (largestConstructor == null ||
   largestConstructor.GetParameters().Length < constructor.GetParameters().Length)
  {
   largestConstructor = constructor;
  }
 }
 //generate mocks and build up parameter list
 var parameters = new List<object>();
 foreach (var parameterInfo in largestConstructor.GetParameters())
 {
  if (!_typeRegistry.ContainsKey(parameterInfo.ParameterType))
  {
   _typeRegistry[parameterInfo.ParameterType] = GetMockInstance(parameterInfo.ParameterType);
  }
  parameters.Add(_typeRegistry[parameterInfo.ParameterType]);
 }
 return (T) largestConstructor.Invoke(parameters.ToArray());
}

To get our generated mock objects we can grab them from our dictionary after our SUT has been created. The full source for the Automocker can be found on Github.

Tuesday, June 18, 2013

Implementing an external service lookup cache

We have a form in our web application that takes an address. The field is an auto-complete field and as the user types their address, the form shows them matching addresses. There is a web service that is used in this organization to help with address lookup. This service takes a string (partial address) and returns a list of addresses that start with that particular string, up to a maximum of 1000 addresses. I wanted to implement a cache of this look up and it turned out to be an interesting little problem. How do we cache the addresses and more importantly, how can we figure out if we should hit the server or the cache?

An initial thought was that we could store all addresses from the service in a HashSet to avoid duplicates. We could check if there are any items in the cache that start with the search string and if so, we can return those items. Here's what this lookup class might look like.

public class AddressLookup
{
 public static AddressLookup Instance { get; private set; }

 private ISet<AddressType> Addresses
 {
  get
  {
   lock (this)
   {
    if (HttpRuntime.Cache["Addresses"] == null)
    {
     HttpRuntime.Cache["Addresses"] = new HashSet<AddressType>();
    }
    return (ISet<AddressType>)HttpRuntime.Cache["Addresses"];

   }
  }
 }

 static AddressLookup()
 {
  Instance = new AddressLookup();
 }

 public IEnumerable<AddressType> GetByPartialAddress(string searchAddress)
 {
  lock (this)
  {
   searchAddress = searchAddress.ToUpper();
   var cachedAddresses = Addresses.Where(address => address.FormattedAddress.StartsWith(searchAddress, StringComparison.InvariantCultureIgnoreCase)).ToArray();
   if (cachedAddresses.Length > 0)
   {
    return cachedAddresses;
   }

   var addresses = new AddressManager().RetrieveByPartialAddress(searchAddress);
   Addresses.UnionWith(addresses);
   return addresses;
  }
 }
}

This idea works, but consider a worst case performance issue of traversing a large collection of values, not finding the value and then making a call to the web service after all. Not to mention the issue of having an incomplete cache of values due to the web service only returning the top 1000 results. For an illustration of the issue consider a web service that only returns the top 2 results.

Total data set1, 12, 122
Uncached results for search string "1"1, 12
Uncached results for search string "12"12, 122
Cached results for search string "12" after searching string "1"12

To solve this issue I decided in addition to caching the address list, I would also cache the searches. This will fix both issues, the first by improving cache check performance, the second by returning a better result set from the cache. Below is the implementation:

public class AddressLookup
{
 public static AddressLookup Instance { get; private set; }

 private ISet<string> Searches
 {
  get
  {
   lock (this)
   {
    if (HttpRuntime.Cache["AddressSearches"] == null)
    {
     HttpRuntime.Cache["AddressSearches"] = new HashSet<string>();
    }
    return (ISet<string>)HttpRuntime.Cache["AddressSearches"];
   }
  }
 }

 private ISet<AddressType> Addresses
 {
  get
  {
   lock (this)
   {
    if (HttpRuntime.Cache["Addresses"] == null)
    {
     HttpRuntime.Cache["Addresses"] = new HashSet<AddressType>();
    }
    return (ISet<AddressType>)HttpRuntime.Cache["Addresses"];

   }
  }
 }

 static AddressLookup()
 {
  Instance = new AddressLookup();
 }

 public IEnumerable<AddressType> GetByPartialAddress(string searchAddress)
 {
  lock (this)
  {
   searchAddress = searchAddress.ToUpper();
   if (Searches.Contains(searchAddress))
   {
    return Addresses.Where(address => address.FormattedAddress.StartsWith(searchAddress, StringComparison.InvariantCultureIgnoreCase));
   }

   Searches.Add(searchAddress);
   var addresses = new AddressManager().RetrieveByPartialAddress(searchAddress);
   Addresses.UnionWith(addresses);
   return addresses;
  }
 }
}

The next potential for improvement would be to implement a rolling cache. One that only keeps the n most recent searches. I haven't taken this implementation that far, but a potential implementation might be to associate the search results on a per search basis in a dictionary, rather than keeping a global address list. Let me know how you might implement a rolling cache or if you have any improvement suggestions with my implementation.

Thursday, February 28, 2013

How I feel as a developer

Here's a blog with a bunch of animated gifs illustrating what it's like to be a developer.

Monday, October 1, 2012

Node.js, the next web development platform?

Node.js has been gaining steam over the last little while, thanks to videos like this, showing how Node.js can handle 1000 concurrent requests. The development community has grown by leaps and bounds and new packages are constantly being added to NPM. In addition to this, Microsoft has thrown its weight behind it and has provided support for Node.js development in Windows Azure and their WebMatrix IDE. The selling features sound pretty good compared to existing frameworks and architectures:
  • Better memory efficiency under high loads compared to traditional thread based concurrency model
  • No locks and as a result no dead locks
  • HTTP is a first class protocol, this means it's a good foundation for web libraries and frameworks
  • Developers can write code in one language on the client and server
In order to jump right in and start learning Node.js I decided to write an adapter for the jugglingdb ORM. This is an ORM which is bundled in the very Ruby on Rails like framework Railway.js. The adapter was written to integrate into Windows Azure Table Storage and is available in github and on npm.

From this experience I've learned several things, the most important of which is that Node.js is not ready for primetime. While it's great for doing simple endpoints, complex applications seem to be very hard to troubleshoot and debug. This is due to the use of async callbacks all over the place. While this allows high concurrency, debugging becomes a muddled mess. Here is an example of the getTable function in the TableService class, provided by Microsoft.

TableService.prototype.getTable = function (table, optionsOrCallback, callback) {
  var options = null;
  if (typeof optionsOrCallback === 'function' && !callback) {
    callback = optionsOrCallback;
  } else {
    options = optionsOrCallback;
  }

  validateTableName(table);
  validateCallback(callback);

  var webResource = WebResource.get("Tables('" + table + "')");
  webResource.addOptionalHeader(HeaderConstants.CONTENT_TYPE, 'application/atom+xml;charset="utf-8"');
  webResource.addOptionalHeader(HeaderConstants.CONTENT_LENGTH, 0);

  var processResponseCallback = function (responseObject, next) {
    responseObject.tableResponse = null;
    if (!responseObject.error) {
      responseObject.tableResponse = TableResult.parse(responseObject.response.body);
    }

    var finalCallback = function (returnObject) {
      callback(returnObject.error, returnObject.tableResponse, returnObject.response);
    };

    next(responseObject, finalCallback);
  };

  this._performRequestExtended(webResource, null, options, processResponseCallback);
};

Not only is this hard to read, it's not clear why all of these layers of callbacks are even necessary. This could just be this one library, however, it also seems that Node.js is solving a problem that isn't there for a vast majority of people. The Railways.js and Express frameworks are great, however, if the problem you are trying to solve is concurrency, you are unlikely to use a heavyweight framework. You are much more likely to build a highly specialized and tuned end point, using low level platform constructs.

That is not to say that it's a lost cause. Projects like TypeScript are seeking to make Node.js development much more scalable. Tools like Jetbrains WebStorm with the Node.js plugin, provide a slew of  functionality for JavaScript development that you may be used to in other development environments. This includes refactoring, code completion, a nodeunit test runner, and debugging

Would I use Node.js on my next project? Potentially for small endpoints that may need high concurrency and throughput. However, I am interested in seeing where the development community takes the concept of JavaScript on the server. Perhaps at some point this will become a viable alternative to the existing platforms and frameworks.

Friday, April 27, 2012

Using Sqlite to test database interaction with NHibernate

In a previous post, Stubbing Out NHibernate's ISession, I talked about creating a mock class in order to stub out QueryOver interaction in your classes. In this post I'd like to talk about another approach to take: using an in-memory Sqlite database to simulate real data interaction. The upside to using an in-memory Sqlite database, is that you can quickly test to make sure if your query will actually work as you expect it to work. There are a couple of downsides, however. One is that you're using Sqlite as opposed to whatever production grade database you will be using in deployment. The second, is that set-ups for your tests can get pretty complex, as you may potentially need to persist a lot of entities.

The first thing you'll need to do, is to download Sqlite and the appropriate .NET wrapper. These can be found here and here respectively. Next you'll need to create some sort NHibernate session manager. This class should be able to create an in-memory Sqlite database, create the schema, and open an ISession. Here's  what mine looks like

public class NHibernateSqliteSessionFactory
{
 private static readonly NHibernateSqliteSessionFactory _instance = new NHibernateSqliteSessionFactory();
 private static Configuration _configuration;
 private readonly ISessionFactory _sessionFactory;

 private NHibernateSqliteSessionFactory()
 {
  var model = AutoMap.AssemblyOf<Entity>(new GasOverBitumenConfiguration())
   .UseOverridesFromAssemblyOf<NHibernateSessionFactory>()
   .Conventions.Add(
    DefaultLazy.Always(),
    DynamicUpdate.AlwaysTrue(),
    DynamicInsert.AlwaysTrue())
   .Conventions.AddFromAssemblyOf<Entity>();

  FluentConfiguration fluentConfig = Fluently.Configure()
   .Database(
    new SQLiteConfiguration()
     .InMemory()
     .UseReflectionOptimizer()
   )
   .ProxyFactoryFactory("NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle")
   .Mappings(m =>
        {
         m.AutoMappings.Add(model);
         m.FluentMappings.AddFromAssemblyOf<Entity>();
        })
   .ExposeConfiguration(c => _configuration = c);

  _sessionFactory = fluentConfig.BuildSessionFactory();
 }

 public static NHibernateSqliteSessionFactory Instance
 {
  get { return _instance; }
 }

 public ISession OpenSession()
 {
  var session = _sessionFactory.OpenSession();
  new SchemaExport(_configuration).Execute(false, true, false, session.Connection, new StringWriter());
  session.BeginTransaction();
  return session;
 }
}

Note that we are using FluentNhibernate to create the configuration. This can be just as easily accomplished with either traditional or loquacious configuration. Furthermore the OpenSession method creates a session and sets up the database schema. When the session is closed, the in-memory database is gone.

It might be a good idea to also persist several entities in your database, such that your unit tests are not required to have large set up methods. Your test can then simply open the session, do what is being tested and verify that the result is correct. In your clean up step, it's a good idea to dispose the session.

There you have it. An easy way to set up an in-memory Sqlite database for NHibernate.

Wednesday, December 14, 2011

Stubbing out NHibernate's ISession

When I first started using NHibernate I didn't know much about design patterns. My early attempts were messy, ugly, and unstable. I finally learned about the repository pattern and started using it. Things were great, I was able to write cleaner code and was able to abstract away my use of NHiberante. The thing is, NHibernate's ISession is already an implementation of this pattern, in my opinion. So abstracting an abstraction just adds an extra layer of complexity. That's not to say that you shouldn't use the repository pattern to persist objects, but for basic reads, using the ISession directly, has a lot of benefits. (For a more thorough analysis see Ayende's blog here and here.)

There is, however, one advantage to abstracting away the ISession. It allows you to unit test your code in a much easier fashion. Since you can easily mock out your repository object, you can write unit tests for modules that use it. This is much easier than if you used the ISession directly. Whether you use the Criteria, QueryOver, or the Linq Query apis, it's almost impossible to stub out this interaction. If you do happen to stub it out using a mocking framework, your unit test Arrange step would become so messy, that your unit tests would become completely unreadable.

So what's the solution? In a recent project, we decided not to abstract away, NHibernate when doing queries. Of course, we ran up against the problem of unit testing the code. The solution was remarkable simple. Since we were using the QueryOver api, all we did was write a QueryOverStub implementation of the IQueryOver<TRoot, TSub>interface. Here's a simple implementation with unused methods not being implemented.


    public class QueryOverStub<TRoot, TSub> : IQueryOver<TRoot, TSub>
    {
        private readonly TRoot _singleOrDefault;
        private readonly IList<TRoot> _list;
        private readonly ICriteria _root = MockRepository.GenerateStub<ICriteria>();

        public QueryOverStub(IList<TRoot> list)
        {
            _list = list;
        }

        public QueryOverStub(TRoot singleOrDefault)
        {
            _singleOrDefault = singleOrDefault;
        }

        public ICriteria UnderlyingCriteria
        {
            get { return _root; }
        }

        public ICriteria RootCriteria
        {
            get { return _root; }
        }

        public IList<TRoot> List()
        {
            return _list;
        }

        public IList<U> List<U>()
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot, TRoot> ToRowCountQuery()
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot, TRoot> ToRowCountInt64Query()
        {
            throw new NotImplementedException();
        }

        public int RowCount()
        {
            return _list.Count;
        }

        public long RowCountInt64()
        {
            throw new NotImplementedException();
        }

        public TRoot SingleOrDefault()
        {
            return _singleOrDefault;
        }

        public U SingleOrDefault<U>()
        {
            throw new NotImplementedException();
        }

        public IEnumerable<TRoot> Future()
        {
            return _list;
        }

        public IEnumerable<U> Future<U>()
        {
            throw new NotImplementedException();
        }

        public IFutureValue<TRoot> FutureValue()
        {
            throw new NotImplementedException();
        }

        public IFutureValue<U> FutureValue<U>()
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot, TRoot> Clone()
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot> ClearOrders()
        {
            return this;
        }

        public IQueryOver<TRoot> Skip(int firstResult)
        {
            return this;
        }

        public IQueryOver<TRoot> Take(int maxResults)
        {
            return this;
        }

        public IQueryOver<TRoot> Cacheable()
        {
            return this;
        }

        public IQueryOver<TRoot> CacheMode(CacheMode cacheMode)
        {
            return this;
        }

        public IQueryOver<TRoot> CacheRegion(string cacheRegion)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> And(Expression<Func<TSub, bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> And(Expression<Func<bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> And(ICriterion expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> AndNot(Expression<Func<TSub, bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> AndNot(Expression<Func<bool>> expression)
        {
            return this;
        }

        public IQueryOverRestrictionBuilder<TRoot, TSub> AndRestrictionOn(Expression<Func<TSub, object>> expression)
        {
            throw new NotImplementedException();
        }

        public IQueryOverRestrictionBuilder<TRoot, TSub> AndRestrictionOn(Expression<Func<object>> expression)
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot, TSub> Where(Expression<Func<TSub, bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> Where(Expression<Func<bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> Where(ICriterion expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> WhereNot(Expression<Func<TSub, bool>> expression)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> WhereNot(Expression<Func<bool>> expression)
        {
            return this;
        }

        public IQueryOverRestrictionBuilder<TRoot, TSub> WhereRestrictionOn(Expression<Func<TSub, object>> expression)
        {
            return new IQueryOverRestrictionBuilder<TRoot, TSub>(this, "prop");
        }

        public IQueryOverRestrictionBuilder<TRoot, TSub> WhereRestrictionOn(Expression<Func<object>> expression)
        {
            return new IQueryOverRestrictionBuilder<TRoot, TSub>(this, "prop");
        }

        public IQueryOver<TRoot, TSub> Select(params Expression<Func<TRoot, object>>[] projections)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> Select(params IProjection[] projections)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> SelectList(Func<QueryOverProjectionBuilder<TRoot>, QueryOverProjectionBuilder<TRoot>> list)
        {
            return this;
        }

        public IQueryOverOrderBuilder<TRoot, TSub> OrderBy(Expression<Func<TSub, object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> OrderBy(Expression<Func<object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path, false);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> OrderBy(IProjection projection)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, projection);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> OrderByAlias(Expression<Func<object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path, true);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> ThenBy(Expression<Func<TSub, object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> ThenBy(Expression<Func<object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path, false);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> ThenBy(IProjection projection)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, projection);
        }

        public IQueryOverOrderBuilder<TRoot, TSub> ThenByAlias(Expression<Func<object>> path)
        {
            return new IQueryOverOrderBuilder<TRoot, TSub>(this, path, true);
        }

        public IQueryOver<TRoot, TSub> TransformUsing(IResultTransformer resultTransformer)
        {
            return this;
        }

        public IQueryOverFetchBuilder<TRoot, TSub> Fetch(Expression<Func<TRoot, object>> path)
        {
            return new IQueryOverFetchBuilder<TRoot, TSub>(this, path);
        }

        public IQueryOverLockBuilder<TRoot, TSub> Lock()
        {
            throw new NotImplementedException();
        }

        public IQueryOverLockBuilder<TRoot, TSub> Lock(Expression<Func<object>> alias)
        {
            throw new NotImplementedException();
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, U>> path)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<U>> path)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, U>> path, Expression<Func<U>> alias)
        {
            return new QueryOverStub<TRoot, U>(_list);
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<U>> path, Expression<Func<U>> alias)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, U>> path, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<U>> path, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, U>> path, Expression<Func<U>> alias, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<U>> path, Expression<Func<U>> alias, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, IEnumerable<U>>> path)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<IEnumerable<U>>> path)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, IEnumerable<U>>> path, Expression<Func<U>> alias)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<IEnumerable<U>>> path, Expression<Func<U>> alias)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, IEnumerable<U>>> path, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<IEnumerable<U>>> path, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<TSub, IEnumerable<U>>> path, Expression<Func<U>> alias, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, U> JoinQueryOver<U>(Expression<Func<IEnumerable<U>>> path, Expression<Func<U>> alias, JoinType joinType)
        {
            return new QueryOverStub<TRoot, U>(new List<TRoot>());
        }

        public IQueryOver<TRoot, TSub> JoinAlias(Expression<Func<TSub, object>> path, Expression<Func<object>> alias)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> JoinAlias(Expression<Func<object>> path, Expression<Func<object>> alias)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> JoinAlias(Expression<Func<TSub, object>> path, Expression<Func<object>> alias, JoinType joinType)
        {
            return this;
        }

        public IQueryOver<TRoot, TSub> JoinAlias(Expression<Func<object>> path, Expression<Func<object>> alias, JoinType joinType)
        {
            return this;
        }

        public IQueryOverSubqueryBuilder<TRoot, TSub> WithSubquery
        {
            get { return new IQueryOverSubqueryBuilder<TRoot, TSub>(this); }
        }

        public IQueryOverJoinBuilder<TRoot, TSub> Inner
        {
            get { return new IQueryOverJoinBuilder<TRoot, TSub>(this, JoinType.InnerJoin); }
        }

        public IQueryOverJoinBuilder<TRoot, TSub> Left
        {
            get { return new IQueryOverJoinBuilder<TRoot, TSub>(this, JoinType.LeftOuterJoin); }
        }

        public IQueryOverJoinBuilder<TRoot, TSub> Right
        {
            get { return new IQueryOverJoinBuilder<TRoot, TSub>(this, JoinType.RightOuterJoin); }
        }

        public IQueryOverJoinBuilder<TRoot, TSub> Full
        {
            get { return new IQueryOverJoinBuilder<TRoot, TSub>(this, JoinType.FullJoin); }
        }
    }

So you can easily specify the item or list of items you want the query to return in your test.

new QueryOverStub<Entity, Entity>(new List<Entity>())

The only thing missing is: if you're using projections or getting something like the count. In order to make this implementation work in those situations, we would just need to write implementations for the List<U> and SingleOrDefault<U> methods. In an upcoming blog post I'll talk about how to use an in memory Sqlite database as an alternative to using this stub.

Monday, November 28, 2011

Writing testable ETL processes with Rhino-ETL

On a recent project we had to integrate several external data sources into the application database. Some of these sources were csv files, while another one was an Oracle database. Our application, meanwhile, used a SQL Server database. Furthermore some of the data had to be loaded automatically, while other data had to be loaded by a business user. We tossed around several ideas for how to accomplish this: SSIS, Informatica, something custom built. We finally arrived on Ayende Rahien's (aka Oren Eini) ETL framework Rhino-ETL. It looked like it would fit the bill as it could be integrated into any .NET application and it allowed us to write unit and integration tests around it.

Unfortunately there is a dearth of information around how to use this framework. The only good piece of documentation is this great video tutorial. While a good starting point, I thought I'd write a blog to show how to get started with the framework.

Rhino ETL is based on a pipeline concept. Each process consists of a bunch of operations strung together. Each operation's output is fed into the next operation as input. The operations interact by the following method:

public interface IOperation : IDisposable
{
    ...
    IEnumerable<Row> Execute(IEnumerable<Row> rows);
    ...
}

So each operation takes in a collection of Rows and outputs a collection of Rows. A Row is basically an instance of Dictionary<object, object> with an added twist: no exceptions are thrown if a key does not exist. A Row will simply return null when a key does not exist in the dictionary. Each operation can also leverage the oft unused C# keyword of yield return. A typical operation might looks something like this:

public class MyOperation : AbstractOperation
{
    public override IEnumerable<Row> Execute(IEnumerable<Row> rows)
    {
        foreach (var row in rows)
        {
            //process the row
            yield return row;
        }
    }
}

The yield return keyword instructs the compiler to generate an iterator that, for each row, would execute whatever code you had in the foreach statement. This allows us to defer execution of each row to the point in time when it passes through the operation pipeline.

Since your operations expose the Execute method, you can easily write unit tests around each operation, with any kind of data as input. Voila, by using this framework and developing a good suite of unit tests, your ETL process has become far more robust than if you were using a tool like SSIS.

The main thing to note about Rhino-ETL is that you will have to code all of your operations. Typically your operations will inherit from the AbstractOperation class. Some other useful operations:
SqlBulkInsertOperation
Set up the schema and do a bulk insert into a table. Useful for inserting large data sets quickly and efficiently
SqlBatchOperation
Batch your sql operations to reduce server roundtrips
BranchingOperation
Send rows to multiple operation pipelines
JoinOperation
Join your rows to a result set from another operation
PartialProcessOperation
Typically used with BranchingOperations to create an operation pipeline within a branch

A process is typically created by inheriting from the EtlProcess class.

public class MyProcess : EtlProcess
{
    protected override Initialize()
    {
        Register(new MyOperation());
        //register other operations
    }
}

A final note, in order to get any output from your etl process you will need to set log4net up. A simple console output can be created by the following entry in your App.config:

<log4net debug="false">
    <appender name="console" type="log4net.Appender.ConsoleAppender, log4net">
      <threshold value="WARN" />
      <layout type="log4net.Layout.PatternLayout,log4net">
        <param name="ConversionPattern" value="%d [%t] %-5p %c - %m%n"/>
      </layout>
    </appender>
    <root>
      <level value="INFO" />
      <appender-ref ref="console"/>
    </root>
  </log4net>

and by adding the following line of code to your application startup:

log4net.Config.XmlConfigurator.Configure();

To conclude, while you do give up fancy designers and useful operations like SSIS's Fuzzy Lookup, the benefit gained from being able to write unit tests around your ETL process can be invaluable. Due to its simplicity and the fact that it can be easily integrated into other .NET applications, Rhino ETL is a very useful tool to have in your arsenal.

Wednesday, July 13, 2011

Why you should use a DVCS

Over the last several years DVCSs (Distributed Version Control Systems) have gained enormous acceptance in the development community. There are a lots of good reasons to ditch your old Subversion (or god forbid CVS) repository and switch to one of the new kids on the block like Git or Mercurial. Let's take a look at some of these reasons.

Experiment to your heart's content

I'm sure this has happened to everybody. You start developing a feature. You change a few minor things here and there (e.g. add a new property or method to an entity). Then you make a few more changes; and a few more; and a few more. All of a sudden you realize that your approach just isn't going to work. Hold on though, some of the changes you made earlier are still valid and you want to keep them. So you start trying to revert just the later changes, only to give up and revert everything back to the latest copy that was in source control. Begrudgingly you reproduce your earlier work as you curse your source control system.

If you're using a DVCS, all of that can be avoided. Since you have a source control repository right on your own machine, tracking your code. You can easily reset your code to one of the commit points along your journey to the dead end you've found yourself in. You can even save your progress in a tag or a branch, in case you want to refer to some of the code in the future.

Branch and merge with ease

A common scenario in software development is to do a product release, create a release branch, and continue development on the main line trunk. If a bug is reported in the release, you would check out the release branch, fix the bug and check in your changes. After a few of these changes you can do another release and merge your branch fixes into your main line trunk. At least that's how it's supposed to work. What happens more often is on the last step, when you try to do a merge, you find so many conflicts that it then takes the entire team a day or more to resolve them.

Since a DVCS is built around branching and merging, these operations are not only easy to do, but also painless. Most of the time you won't even need to manually merge anything as the systems are very good at figuring out what was intended. So go ahead, make as many branches as you want, merge them whenever you want, and stop trying to mangle your process with the limitations of your VCS.

Safety and redundancy

One of the great things about having multiple copies of the repository floating around among the development team and the central location is redundancy. In a traditional central version control system environment, if your your repository became corrupted, or the server hard drive failed, you would have to go find a backup and restore it, possibly losing work and holding up development.

In a distributed environment, everyone has a copy of the repository. So if the same scenario were to happen, the fix can be as simple as one of the developers pushing in the latest copy to a new shared location. Alternatively one developer's repositories can be chosen to be a central repository until the server can be rebuilt. That is not to say you shouldn't have backups in a distributed environment, but it is far less catastrophic not to have them.

The bottom line is that constantly working with the benefits of source control far outweigh the initial pain of converting to a DVCS. If you're developing in Windows, I highly recommend Mercurial, if Linux or Mac is your preferred environment, then Git is definitely the way to go. So go on and give it a try. I promise you won't go back to your old VCS.

Monday, May 30, 2011

Git with ssh on Windows

While Git is a great source control system that can bend to almost any source control workflow you might have, support on windows varies from awesome (Git Extensions, msysgit) to downright awful (using git, ssh, or http protocol). I've struggled enough with setting up an ssh server on windows to host a central repository to warrant documenting it.

First you will need to install Cygwin with the openssh and git modules enabled.

After the installation we'll need to let cygwin know about the domain accounts we want to use. For this we'll use the mkpasswd command. For domain accounts use the '-d' option.

$ mkpasswd -d -u username >> /etc/passwd

This will append the entry to the /etc/passwd file. Do this for all of the accounts you want to add. You may need to subsequently edit the /etc/passwd file and set the appropriate home directory to /home/username especially if your company likes to have a network home drive for all of their users.

Now it's time to create a new bare repository.

$ git init --bare myrepo.git
$ git config core.sharedRepository group

The second command lets git know that it should use group write permissions when creating files and directories. We'll also want to set the group permissions to special so that newly created files and folders have the same group permissions.

$ chmod -R g+ws .

Now that your repository is set up. Let's setup the ssh server.

$ ssh-host-config

The script will begin by generating host key files and config files. It will then prompt you to use privilege separation. Answer yes and again when it prompts you to create an unprivileged user.

Next the script will ask if you want to install sshd as a service. Answer yes. The next question will tell you that you need a privileged account to run the service and if you want to use the name 'cyg_server'. Either say no, or if you already have an appropriate privileged account answer yes and enter the name you want to use. If the account you entered does not have the appropriate permissions the script will warn you. I recommend letting the script create a 'cyg_server' account. When prompted enter an appropriate password that conforms to your password policies.

Finally the script will ask you to enter the values for the CYGWIN environment variable. Enter 'tty acl'. This will let you use windows security to access the file system and insert appropriate characters.

After all of that, ssh will be configured so we just need to start it.

$ cygrunsrv --start sshd

Alternatively you can go into the services list, find CYGWIN sshd and start it there.
If you start seeing errors at this point there is likely something wrong with the password you set up for the 'cyg_server' account.

As a final step you will likely want to create a symlink to your repository in the root folder. This will allow the ssh URI to be more concise and friendly.

$ ln -s /path/to/your/repo /repo

Once you have the ssh server running, it's time to connect to it. Let's push our local repository to the remote.

$ git push ssh://server/repo

If you did not set up a symlink your address will need to contain the appropriate cygwin path to your repo starting from /. The first time you connect it will prompt you to add the host key to your known_hosts file. Finally you will be prompted you to enter your password. At this point you have everything in place to access the repository remotely.

As a final step you can set up public/private keys in order to identify yourself to the ssh server. This will allow you to forgo having to enter your password. First, set up your ssh keys

$ ssh-keygen -t rsa

This will create 2 files in your ~/.ssh drive, one is your public key (id_rsa.pub) and the other is your private key (id_rsa). The process will involve you copying the contents of your public key to a /home/username/.ssh/authorized_keys file on the server. One way of doing this is to create a share to your home drive on the server that only your account can access, then you can create an authorized_keys file and copy the contents in notepad.

Git can still be a pain to work with on Windows, but hopefully this will make your life a little easier.

Update:
If you would rather set up git via http protocol. Have a look at this post if you want to go via Apache. Or for an IIS solution check this out.

Wednesday, May 4, 2011

Custom properties for IPrincipal

Sometimes you need to provide application specific values that are associated with a particular user. Let's say your application must handle role based security by using AD groups. Furthermore let's say that you wanted those group mappings to be configurable. A way to solve this would be to create your own implementation of IPrincipal. It might look something like this:

public class ApplicationPrincipal : IPrincipal
{
    private readonly IIdentity _identity;
    private readonly IPrincipal _principal;
    private readonly ISecurityConfiguration _config;

    public class ApplicationPrincipal(IIdentity identity, IPrincipal principal, ISecurityConfiguration config)
    {
        _identity = identity;
        _principal = principal;
        _config = config;
    }

    public IIdentity Identity { get { return _identity; } }

    public bool IsInRole(string role)
    {
        var windowsGroup = MapInternalRoleToWindowsGroup(role);
        return _principla.IsInRole(windowsGroup);
    }

    public string MapInternalRoleToWindowsGroup(string role)
    {
        switch (role)
        {
            case "Admin": return _config.AdminGroup;
            case "User": return _config.UserGroup;
            default: throw new SecurityException("Invalid group name: " + role);
        }
    }
}

Now that we have our own principal implementation we need a way to let the application access it. A good place to do that would be in an HttpModule that handles the PostAuthenticateRequest event.

public class ApplicationPrincipalModule : IHttpModule
{
    public void Init(HttpApplication context)
    {
        context.PostAuthenticateRequest += PostAuthenticateRequest;
    }

    public void Dispose()
    {
        context.PostAuthenticateRequest -= PostAuthenticateRequest;
    }

    private void PostAuthenticateRequest(object sender, EventArgs e)
    {
        var httpContext = HttpContext.Current;
        var principal = new ApplicationPrincipal(
                             httpContext.User.Identity, httpContext.User, 
                             new SecurityConfiguration());
        httpContext.User = principal;
    }
}

Once a user has been authenticated, by whichever method your application is set up to authenticate, this module will set the HttpContext.Current.User to your implementation of IPrincipal. This can then be referenced where needed in your application.

Monday, April 4, 2011

Getting started with BDD using NBehave

What is BDD? BDD stands for Behavior Driven Development and at its core is about implementing an application by describing its behavior from the perspective of its stakeholders. It is an agile software development methodology that started as an offshoot of TDD. However, in my opinion it is better suited to software delivery than TDD is. The reason is that TDD is great as a code design methodology, but there is a disconnect between code design and stakeholder value. When practicing TDD, developers typically produce cleaner and more maintainable code. However, that elegant code may not deliver what the stakeholders wanted.

BDD also allows non technical users, like BAs and business users to specify system behavior in a non technical, natural language. Tools like NBehave, allow the developers to verify and demonstrate that the system delivers what the stakeholders expect.

How do we write a scenario? Scenarios typically follow the Given/When/Then structure. So for example:

Scenario: User logs in
Given that I am not logged in
And I have an account
When I enter my username and password
Then I should see “Welcome Vadim”

How would we use NBehave to verify that the system does what the scenario says? First thing's first download the latest NBehave build and extract the contents of the zip file.
There are 2 versions of the DLLs, one for .Net 3.5 and one for .Net 4.0.

Create a project for your tests and add a reference to NBehave.Narrator.Framework.dll. Also add a reference to your favorite testing framework (like NUnit or xUnit) and the appropriate NBehave.Spec.<framework>.dll. Currently NBehave comes with extension methods for NUnit, xUnit, MbUnit, and MSTest. These methods are simply wrappers around Assert statements, that allow a more natural language feel when writing your code. So if you're using something like MSpec, which already has Should<DoSomething> type assertions, then you don't need to worry about this last step.

Next, create a features directory inside your project. This is where you'll store your feature and scenario files. Let's pretend we're writing a calculator application. Add a file to the features folder called calculator_turns_on.feature. Time to write our scenario.

Scenario: I turn on the calculator
Given the calculator is off
When I press the on button
Then the calculator should display 0

Let's add a folder called steps. This is where we're going to put our tests. Also create a code file and call it CalculatorTurnsOn.cs. Let's start with a skeleton test:

[ActionSteps]
public class CalculatorTurnsOn
{
    
}

Compile the solution then run the NBehave-Console like so

NBehave-Console.exe bin\Debug\NBehaveIntro.dll /sf=features\calculator_turns_on.feature

You should see all results yellow with a PENDING tag next to them. These aren't failures, but we know that we have to implement this scenario. Let's create a Calculator class.

public class Calculator
{
    public float Display { get; set; }
}

Next we'll flesh out our test.

[ActionSteps]
public class CalculatorTurnsOn
{
    private Calculator calculator;

    [Given("the calculator is off")]
    public void CalculatorOff()
    {
        calculator = null;
    }

    [When("I press the on button")]
    public void TurnCalculatorOn()
    {
        calculator = new Calculator();
    }

    [Then("the calculator should display 0")]
    public void CalculatorDisplayIs0()
    {
        calculator.Display.ShouldEqual(0);
    }
}

Compile and run the command again. Now the tests all pass.

Note how the attributes are decorated with the same sentences that were written in our scenario. That's how NBehave knows how the scenarios are mapped. You can decorate a method with multiple Given, When, and Then attributes and you can have any number of methods in a given ActionSteps class. Furthermore, you can have regular expressions or string tokens in the description as well. For example if we had the following scenario:

Scenario: I can add two numbers
Given numbers [one] and [two]
When I add them
Then the sum should be [sum]

Examples:
|one|two|sum|
|1  |2  |3  |
|2  |2  |4  |
|10 |35 |45 |

Our attribute decorations might look like:
[Given("numbers $one and $two")]
public void TwoNumbers(float one, float two)
...
[Then("the sum should be $sum")]
public void VerifySum(float sum)
...

One last note, the scenario above also contained an example table. This table allows us to write one scenario with multiple inputs and outputs to test. This saves us from having to write repetitive scenarios when warranted.

For further info see the following links:
http://nbehave.org/
http://nbehave.codeplex.com/wikipage?title=0.5.0&referringTitle=Documentation

Monday, March 21, 2011

NHibernate unit of work in MVC using Ninject (part 2)

In part 1 I talked about a problematic NHibernate Unit of Work implementation that we had in our MVC application. In part 2 I will talk about why this approach turned out to be problematic and the ultimate solution that we came up with.

If you recall, the original solution was to use an HttpModule to subscribe to the BeginRequest and EndRequest events. When we upgraded to Ninject 2.1, none of our entities would get saved. Reads all worked fine, but no CUD actions worked. The problem turned out to be due to a new component of Ninject called the OnePerRequestModule. This module subscribes to the EndRequest event and disposes any objects that were bound in request scope. The problem was that this module would dispose of our session, before we had a chance to commit the transaction and end the session ourselves. It was also registering first (which meant it was first to handle EndRequest) and there appeared to be no hook to turn it off.

The solution that we came up with was to use a filter attribute in order to encapsulate our unit of work. The first part of the refactor was to create an IUnitOfWork interface:

public interface IUnitOfWork
{
 void Begin();
 void End();
}

That's all a unit of work really should look like. Second we moved our BeginSession and EndSession code into an implementation of IUnitOfWork:

public class NHibernateUnitOfWork : IUnitOfWork, IDisposable
{
 private readonly ISession _session;

 public NHibernateUnitOfWork(ISession session)
 {
  _session = session;
 }

 public void Begin()
 {
  _session.BeginTransaction();
 }

 public void End()
 {
  if (_session.IsOpen)
  {
   CommitTransaction();
   _session.Close();
  }
  _session.Dispose();
 }

 private void CommitTransaction()
 {
  if (_session.Transaction != null && _session.Transaction.IsActive)
  {
   try
   {
    _session.Transaction.Commit();
   }
   catch (Exception)
   {
    _session.Transaction.Rollback();
    throw;
   }
  }
 }

 public void Dispose()
 {
  End();
 }
}

This separated repository logic from the unit of work implementation. The binding for this implementation was bound in request scope. Finally we created our filter attribute:

public class UnitOfWorkAttribute : FilterAttribute, IActionFilter
{
 [Inject]
 public IUnitOfWork UnitOfWork { get; set; }

 public UnitOfWorkAttribute()
 {
  Order = 0;
 }

 public void OnActionExecuting(ActionExecutingContext filterContext)
 {
  UnitOfWork.Begin();
 }

 public void OnActionExecuted(ActionExecutedContext filterContext)
 {
  UnitOfWork.End();
 }
}

The order was set to 0 to ensure that this filter was always run first. We also did not have to explicitly call End on the unit of work since it would be disposed at the end of the request and would call End itself. However, it's good practice to be explicit with these things. Finally you can apply this filter at the controller level or the action level. So you can decorate only those actions that actually perform data access.
8USTXZWHRBTH

Tuesday, March 8, 2011

NHibernate unit of work in MVC using Ninject (part 1)

We just released our third and (likely) final release of the application we've been working on for the last 11 months. During that time I've gone through two implementations of the NHibernate Unit of Work pattern. I will talk about it in two parts. The first part will talk about our first implementation, that turned out to be a little bit flawed and very incompatible with Ninject 2.1. The second one will talk about a better approach that we shipped with this final release.

I've previous blogged about NHibernate and the unit of work pattern here. The approach, I first decided to take was to create a generic Repository class. This class was not only responsible for CRUD operations and transaction management, but it also had 2 special methods on it called BeginRequest and EndRequest. Here's a simple implementation:

public class Repository : IRepository
{
    private ISession _session;
    public Repository(ISession session) { _session = session; }

    public Save<T>(T entity) where T : Entity
    {
        _session.SaveOrUpdate(entity);
    }
    public Get<T>(int id) where T : Entity
    {
        return _session.Get<T>(id);
    }
    public Delete<T>(T entity) where T : Entity
    {
        _session.Delete(entity);
    }

    public BeginTransaction() { _session.BeginTransaction(); }
    public CommitTransaction() { _session.Transaction.Commit(); }
    public RollbackTransaction() { _session.Transaction.Rollback(); }

    public BeginRequest() { BeginTransaction(); }
    public EndRequest() { CommitTransaction(); }
}

Obviously the real class was more robust and had much more error checking around transaction management, but those details aren't important for this discussion.

In addition to this Repository we also had an HttpModule that called Repository.BeginRequest() on BeginRequest and Repository.EndRequest() on EndRequest. It looked something like this:

public class DataModule : IHttpModule
{
 public void Init(HttpApplication context)
 {
  context.BeginRequest += BeginRequest;
  context.EndRequest += EndRequest;
 }

 public void Dispose()
 {
 }

 public void BeginRequest(object sender, EventArgs e)
 {
  var app = (WebApplication)sender;
  var repository = app.Kernel.Get<IRepository>().BeginRequest();
 }

 public void EndRequest(object sender, EventArgs e)
 {
  var app = (WebApplication)sender;
  var repository = app.Kernel.Get<IRepository>().EndRequest();
 }
}

Our Ninject bindings for these classes were as follows:

Bind<ISession>()
       .ToMethod(context => NHibernateSessionFactory.Instance.OpenSession())
       .InRequestScope();
Bind<IRepository>().To<Repository>().InTransientScope();

An important note is that the NHibernate session was bound in request scope, while the repository was bound in transient scope. So all instances of Repository in request scope, shared the same session. (NHibernateSessionFactory is our singleton in charge of configuring the SessionFactory and creating new sessions.)

This approach worked fine with Ninject 2.0, but suffered from several problems. First of all, we would be creating a new session and starting/commiting a new transaction on every http request. This means any request, even for content files like javascript or css, would trigger the DataModule. The second problem was, more architectural/stylistic. We had code that dealt with unit of work implementation inside a repository. We could have moved the Begin/End Request logic off to the DataModule and that would have alleviated this problem some what. However, we still would have been stuck with the first problem.

The kick in the pants for the rewrite came, when we upgraded to Ninject 2.1 and Ninject.Web.Mvc 2.1. Ninject 2.1 introduced a very eager way to ensure objects bound in Request scope didn't hang around after the EndRequest event. I'll talk about this and our implementation rewrite in part 2.

Monday, February 7, 2011

JavaScript/jQuery code optimization (or famous IE bottlenecks)

After we pulled out the Infragistics grid in favor of Telerik MVC Extensions, We've had to code several features, that were available in Infragistics out of the box, ourselves. This was partly due to our data structure not being fully supported by the Telerik grid. That's a topic for another blog post though. One of the features was the ability to select several cells and do spreadsheet like functionality, such as copy/paste and fill. We were mostly testing this out in Chrome/Firefox and with small tables. However, as typically happens, after releasing into our demo environment, our BA told us that the performance was brutal.

Surprise, surprise, it turned out that our grid page ran very slowly in IE(7) with very large tables (50 rows by 35 columns). I wound up being able to trace the bottle neck down to this chunk of code.
$('tbody td', $(this.element)).each(function() {
 if (this.parentElement.rowIndex >= selection.startRowIndex && 
 this.parentElement.rowIndex <= selection.endRowIndex &&
 this.cellIndex >= selection.startColIndex && 
 this.cellIndex <= selection.endColIndex) {
  if (columns[this.cellIndex].readonly != true) {
   $(this).addClass('selected');
  }
 }
});
This is the code for when a user has a cell selected and is either dragging the mouse or holding shift and clicking to select a block of cells. What's happening here is that we're going through all cells in the table body and seeing if it is between the first selected cell and the cell located at the mouse position. I.E. whether or not the cell is in a 'box'. The problem is that we're going through every cell in the table body, even if only one has been selected.

The first optimization was to only go through the rows/columns that were in the selected box, so to speak. This was done using the jQuery slice method. The code is below

$('tbody tr', this.element)
 .slice(selection.startRowIndex, selection.endRowIndex + 1)
 .each(function() {
  $(this).children('td')
   .slice(selection.startColIndex, selection.endColIndex + 1)
   .each(function() {
    if (!columns[this.cellIndex].readonly) {
     $(this).addClass('selected');
    }
   });
 });

The performance improved a lot when the selection box was small, but if the user selected anything bigger than a 4 row by 10 column box the script crawled again. Each row would take approximately 190 ms to run in IE7. So if the user selected 10 rows it would take almost 2 seconds to run. The problem seemed to be the readonly check. If it was taken out, the timing dropped to a fraction of a millisecond.

The problem is that Internet Explorer is very slow processing conditional statements. Doing some unscientific tests on this site I got around the following numbers:

BrowserAverage Time
Chrome 80.02 ms
Firefox 4 b100.66 ms
Internet Explorer 852 ms
Internet Explorer 794 ms

So we had to find a way to optimize that conditional statement. I finally found a jQuery call that seemed to do the trick.

$('tbody tr', this.element)
 .slice(selection.startRowIndex, selection.endRowIndex + 1)
 .each(function() {
  $(this).children('td')
   .slice(selection.startColIndex, selection.endColIndex + 1)
   .not(function() {
    return columns[this.cellIndex].readonly;
   })
   .addClass('selected');
   });

After going through the jQuery code, I'm still not 100% sure why the performance is better with the last statement. A couple of things I've noticed is that using each to do a check on every item seems to be slightly more expensive than not (or for that matter filter). Since each also does a check to make sure the callback hasn't returned false. Also addClass has a lot of overhead so it's quite inefficient to run it on elements one by one. Not sure if that's necessarily the problem either though because the 2nd code snippet ran very quickly when the readonly check was taken out.

If anyone has any ideas on why they think the 3d code snippet runs much faster than the 2nd one I would love to hear it. Until then this just another example of why it's important to optimize your jQuery code.