CQRS flowchart

In my previous more technical post about switching serialization technique I talk about me needing to run a conversion in two steps to change 1500000 events in my database from one format to another.

That scenario by itself might not be very common (it simply shouldn’t, but I am at early stages in my CQRS exploration, hence my decision to change a pretty technical aspect ).

Put that aside, the beauty of CQRS is it allows you to do application upgrades etc, typically just putting a new version in production – WHILE USERS STILL ARE USING IT.

CQRS could very well be the end of, “oh, I need to stay at work til late when all users have logged out to upgrade the system”.  To be honest CQRS by itself is not the answer, a loosly coupled architechture is (and it could be achieved by other means than adopting CQRS), but in my point of view CQRS really drives you (the developer) to  loosly coupled architechturing.

Here is a flowchart (click on it to see full size):

By utilizing persistant queues, and of course adopting the thinking of eventual consistancy, you could very well take down the “Business Logic” process (commandhandler) while your users keeps using the client – upgrade it, and start it up again. And it will start processing commands where it left off.

Using JSon as intermediate when switching binary serialization technique

Background: For some reasons I chose certain architechture for storing binary representations of my objects

I had a base class

EventBase
Guid AggregateId
DateTime Created
int SequenceNo

My applications “events” inherited from this base class like

public class ProjectActivatedEvent : EventBase, ICustomBinarySerializable
{
public ProjectActivatedEvent(Guid aggid, bool active)
: base(aggid, DateTime.Now)
{
Active = active;
}
public bool Active { get; set; }

For storage of serialized data I used a SQL Server table (eventstore)

and serialized using Greg Youngs fast serializer (ICustomBinarySerializable) since its really fast and effective

My serialization code looked like

public void WriteDataTo(SerializationWriter writer)
{
base.WriteDataToBase(writer);
writer.Write(Active);
}

public void SetDataFrom(SerializationReader reader)
{
base.SetDataFromBase(reader);
Active = reader.ReadBoolean();
}

Now I wanted to do a major rearchitecturing of my code.

For starters, get rid of EventBase and introduce a Interface IEvent instead which all events inherits from.
Second: Use protobuf-net instead.

Basically:


[ProtoContract]
public class ProjectActivatedEvent : IEvent
{
public ProjectActivatedEvent()
{

}

public ProjectActivatedEvent(Guid aggid, bool active)
{
AggregateId = aggid;
Created = DateTime.Now;

Active = active;
}
[ProtoMember(10)] public bool Active { get; set; }

//IEvent
[ProtoMember(1)] public Guid AggregateId { get; set; }
[ProtoMember(2)] public int SequenceNo { get; set; }
[ProtoMember(3)] public DateTime Created { get; set; }
//

}

And there I was. I had two branches of my sourcecode, one the old way, one with the new way. Tests shows everything is working ok.
How can I now convert the (1500000) existing records in table eventstore to the new format?

The solution was to introduce JSon into the equation. So (using my old branch) I coded a app which loops all (old style) events, possible since the old code branch knows how to deserialize them of course,  and converts them to JSon. Storing them into a intermediate table.

Second: from the new code branch I simple coded a new app which deserialized the JSon (into my new classes), and stored them the
Protobuf way.

The beauty of JSon (de)serializer is that it simply works on field names. I did not change the names of my fields so
it was therefore possible to use it as a pretty effective intermediate storage.

It did take some time running these loops but at least it worked.

I hope to present some more articles on this (and related) matters but want to show you the the trick to make a generic
(de) serializer out of Json without the need to specify the type:

class Json
{
static JsonSerializerSettings jsonSerializerSettings =
new JsonSerializerSettings
{
DefaultValueHandling = DefaultValueHandling.Ignore,
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore,
TypeNameHandling = TypeNameHandling.Objects
};
public static string Serialize(object theObject)
{

return JsonConvert.SerializeObject(theObject, Newtonsoft.Json.Formatting.Indented, jsonSerializerSettings);
}
public static TType Deserialize(string sInData)
{
return (TType)JsonConvert.DeserializeObject(sInData, typeof(TType), jsonSerializerSettings);
}
}

The secret is TypeNameHandling.Objects. This adds a “$type:<typename>” in front of the JSon when serializing and
therefore it is able to create such an object when deserializing.
I still do have a TType, can’t remember why, since I stole thew code from an old project I made years ago, but the
point is you can send in Object or a baseclass as type.

I deserialize my data like:
Eventing.IEvent ev2 = Json.Deserialize<Eventing.IEvent>(row["data"].ToString());

And serialization was like
Eventing.EventBase b = …
string sa = Json.Serialize(b);

Why use a ServiceBus in your .NET applications

So I am currently evaluating the three major (open source) servicebuses for .NET, namely nServiceBus, Rhino Bus and MassTransit. I will (soon) come back with a post on what the outcome was (specific for me, but nonetheless).

But first I thought I’d start with explaining why I feel I need a servicebus (at all) as doing that might show some light of what it’s good for, it might help you decide if thats suitable for you as well.

First ask yourself if your application is distributed at all. Even if it isn’t (for example I have a lot of plain client/server oldschool apps which are nothing but client app and database server) – it might gain some my making it distributed. I wont get into all the reasons why such a decision could be beneficial, as thats nothing I can say something about, but in general if you centralize your applications core code your making it available for other  type of clients, like web. Also scaling (out, not up) gets easier.

It might also be that you want asynchronous commands, where you delegate actions such as “Send this email to this person” I dont have time to do it cause I am calculating this very valuable rate or whatever right now. This also gives you the effect of a clear architechture where you split different things up.

A servicebus abstracts messaging and transports

What do I mean by that? First of all, a message could be seen as a serialized command.  Say I sant to send that command “Send this email to this person”. Then a message must be delivered to that service. Typically you setup (in configfile) the different endpoints (servers) you want to talk with and do something like bus.Send(new EmailCommand(…)) .

A servicebus also abstracts HOW the messages are being delivered. Do we want to use MSMQ or ActiveMQ? Thats also configurable – and typically also possible to extend (plugin) more transports.

A servicebus helps you with client AND server development

Above we looked at how the client programming becomes really easy. A servicebus should also help you with programming the server part (in our example the email service). It should be configurable the same way as the client. It should in short handle all the transport details, and even more it should allow you to just write the business logic for each command (type of message), not much else. No plumbing code.

A servicebus decouples messaging

Fire and forget. Just like delegation of work to a collegue. We should be able to Send the message. And go back to the more important things we were doing.

A servicebus should help setup pub/sub

Apart from the fact that transports are pluggable, you would still benefit from a bus even if you have a plicy to just use a single transport, like say MSMQ – or WCF for that part. Instead of programming directly against it, when you use a servicebus you gain the benefit of some nice message distribution scenarios. The bus lets you setup and handle publisher/subscriber scenarios, where you might have multiple listeners (servers) for a certain message.

A servicebus should help setup sagas

A saga is a serie of commands. Like a workflow. I wont get into this right now, but this is also a thing you wont get out of the box from the MSMQ api.

A servicebus goes well into CQRS scheme

I am now developing all my apps the CQRS way. The benefits and reasons for that is out of scope for this article

CQRS sample app

So this summer I really try to update myself and my coding skills. Next on the list is CQRS.

I attended a Greg Young session at NDC in Oslo and really had a feeling, thats the way applications should be developed. Its been a long time since I had that feeling, I have been comfortably developing my apps basically the same way for the last 8 years or so this is not only refreshing but also a lot of fun, me being the tech nerd I am…

So for about a month or so I have been trying to get a POC CQRS app up and running. Using nServiceBus and MSMQ as transport.

The whole experience (far from finished) can be read and full source can be downloaded from the CQRS NServiceBus example site