Category Archives: Uncategorized

MongoDb Ninjitsu: Using ObjectId as a Timestamp

Whilst reading around yesterday I stumbled upon this little gem of knowledge:

Mongo’s ObjectIds contains a Utc timestamp with 1 second resolution.

This means, that if we don’t need millisecond accuracy, we can drop all those “CreatedOn” fields from our schemas, and by doing so we win twice:

  • Storage – no need to store a seperate time stamp field means less to store on disk and less for the server to shunt around.
  • Free Index – assuming your ObjectId is your primary key, it already has an index on it by default, so not only are you saving the space of having to store a seperate time stamp, but it’s also indexed for free.

To make it a little easier, I created an extension class to convert DateTimes to and from ObjectIds. This makes it insanely simple to query for objects created before/after a certain date. Here’s the code:

/// Author	: Daniel Harman (http://www.danharman.net)
/// Date	: 26.Oct.2011
/// License : Public Domain with no warranty given. Please maintain author credit.
using System;
using System.Collections.Generic;
using System.Linq;
using MongoDB.Bson;

namespace Mmphs.Utils.MongoDb
{
	public static class DateTimeExtensions
	{
		/// <summary>
		/// Converts a DateTime to an ObjectId.
		/// n.b. missing values that would ensure uniqueness. This is only intended for time comparisons.
		/// </summary>
		/// <param name="dateTime"></param>
		/// <returns></returns>
		public static ObjectId ToObjectId(this DateTime dateTime)
		{
			var timestamp = (int)(dateTime - BsonConstants.UnixEpoch).TotalSeconds;
			return new ObjectId(timestamp, 0, 0, 0);
		}

		/// <summary>
		/// Convert an ObjectId to a DateTime.
		/// </summary>
		/// <param name="id"></param>
		/// <returns></returns>
		public static DateTime ToDateTime(this ObjectId id)
		{
			return id.CreationTime;
		}

		static readonly int DATETIME_TRUNCATE_FACTOR = 10000;

		/// <summary>
		/// Truncate the accuracy of a datetime to the same resolution as a mongo db datetime.
		/// </summary>
		/// <param name="dateTime"></param>
		/// <returns></returns>
		public static DateTime MongoTruncate(this DateTime dateTime)
		{
			long ticks = dateTime.Ticks / DATETIME_TRUNCATE_FACTOR;
			return new DateTime(ticks * DATETIME_TRUNCATE_FACTOR, dateTime.Kind);
		}
	}
}

I’ve thrown in a little bonus there too – a method to truncate a DateTime in the same way MongoDb does when it persists one. This is handy for unit tests where you want to compare an object you’ve created and persisted and then loaded back up e.g. when testing a query brings back the right record.

Here is an example of using the ObjectId to get items after a certain date:

		public IEnumerable<Drop> GetStreamActivities(ObjectId streamId, DateTime utcSince)
		{
			return _drops
				.AsQueryable()
				.Where(a => a.StreamId == streamId && a.Id >= utcSince.ToObjectId())
				.OrderBy(a => a.Id);
		}

n.b. I’m using FluentMongo, but you certainly don’t need to.

and here are the unit tests:

/// Author	: Daniel Harman (http://www.danharman.net)
/// Date	: 26.Oct.2011
/// License : Public Domain with no warranty given. Please maintain author credit.

using System;
using System.Collections.Generic;
using System.Linq;
using MbUnit.Framework;
using MongoDB.Bson;

namespace Mmphs.Utils.MongoDb.Test.DateTimeExtensions
{
	[TestFixture]
	public class DateTimeExtensionsTest
	{
		[Test]
		public void Can_Convert_DateTime_To_ObjectId()
		{
			// Arrange
			DateTime dateTime = DateTime.UtcNow;

			// Act
			ObjectId result = dateTime.ToObjectId();

			// Assert
			Assert.AreApproximatelyEqual(dateTime, result.CreationTime, TimeSpan.FromSeconds(1));
		}

		[Test]
		public void Can_Convert_ObjectId_To_DateTime()
		{
			// Arrange
			ObjectId objectId = ObjectId.GenerateNewId();
			DateTime dateTime = DateTime.UtcNow;

			// Act
			var result = objectId.ToDateTime();

			// Assert
			Assert.AreApproximatelyEqual(dateTime, result, TimeSpan.FromSeconds(1));
		}
	}
}
/// Author	: Daniel Harman (http://www.danharman.net)
/// Date	: 26.Oct.2011
/// License : Public Domain with no warranty given. Please maintain author credit.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MbUnit.Framework;
using MongoDB.Bson;
using MongoDB.Bson.Serialization;

namespace Mmphs.Utils.MongoDb.Test.DateTimeExtensions
{
	[TestFixture]
	public class When_Truncating_DateTime
	{
		[Test]
		public void Should_Match_Mongo()
		{
			// Arrage
			DateTime dt = DateTime.UtcNow;
			var asJson = dt.ToJson();

			// Act
			var asTrunc = dt.MongoTruncate();
			
			// Assert
			var fromJson = BsonSerializer.Deserialize<DateTime>(asJson);
			Assert.AreEqual(fromJson, asTrunc);
		}
	}
}

Encrypting (Hashing) Passwords for your Website

In my previous blog entry which explains how to implement a custom MembershipProvider on an ASP.Net MVC web site, I mentioned the BCrypt library in passing. Today I’d like to dig into the library a little more and show you how and why you should consider using it.

Before we get into that though, what is what is ‘hashing’ and how is it different to ‘encryption’?

Encryption is part of a two way process in which data is encoded, but must be decryptable afterwards. For password storage, this decryption step is unnecessary and is actually a security vulnerability if someone gets hold of your keys.

In contrast, hashing is a one way transformation that is very hard to reverse. So to store passwords, a site would simply hash them before saving them to a database. Then when a user logs in, the password they supply is also hashed and compared to the hashed value in the database. If they typed in the same password the hashes will match and the authentication passes.

The advantage of this, is that the web site doesn’t need to decrypt a password at any point. All it ever does is hash them which is great for security.

There are various hashing algorithms out there, and many of them are wonderfully optimised and very fast. A typical example is SHA. An optimal hasher sounds great doesn’t it?

… well it isn’t. In fact it’s a weakness with the power of modern cpus and cloud computing resources. It means that its very possible and remarkably cheap to brute force SHA with a dictionary attack, simply because it is so fast.

This is where BCrypt comes in. It is intentionally slow, so should someone try to brute force it, its simply not viable. Whereas a hacker can generate 100s of millions of SHA hashes a second, they can’t with BCrypt, and as technology marches on, you can just increase the work factor on BCrypt to keep ahead of processors.

There are a few BCrypt.Net implementations floating around, and the one I’ve used I pulled down with nuget. To use it is as simple as:

private static readonly int BCRYPT_WORK_FACTOR = 10;
string hashedPassword = BCrypt.Net.BCrypt.HashPassword(account.HashedPassword, BCRYPT_WORK_FACTOR);
bool matched = BCrypt.Net.BCrypt.Verify(password, match.HashedPassword))

Every increment in work factor doubles the type the hash takes. Some of the .net implemtations struggle to get past 31 due to a bug, but you shouldn’t need a value anything like that high for a few years at least!

10 Print “Hello World”

Hello, and welcome.

I’m a professional software developer, who has worked in the investment banking sector for the majority of my career. I started programming rather earlier than that though, taking my first tentative steps aged 8 on a BBC Micro.

I still remember going into Dixons consumer electronics store, and using the display computers to create such wonders as:

10 Print "Hello World!"
20 Goto 10

Well… we all had to start somewhere!

As a lead developer and team lead, I regularly find myself needing to teach or write up tutorials for internal distribution. Rather than share these solely with colleagues, I’ve decided to reach a wider audience hopefully helping others and benefiting from constructive feedback.

I’m interested in a variety of technologies, but at the moment I’m predominantly focussing on:

  • Architecture & Design
  • C#/.net 4.0
  • WPF, particularly MVVM patterns
  • Reactive Extensions (Rx)
  • Task Parallel Library (TPL)
  • Asp.net MVC 3
  • MongoDB
  • GeoLocation

So these are likely to be the main topics of this blog, at least initially.