Hello Biggy

Tuesday, February 2 2014 databases craziness biggy wtfrob

I don't know why I do these things. Inspiration hits and I think "maybe someone will find this useful". Or not... either way here's Biggy.

A File-based Document Store for .NET

I've been using NeDB (a file-based document store for Node) for a few projects and I utterly love it. Such a simple idea, so fast, so elegant and many times just what I need! I had assumed that such a thing must be around for .NET because there are about 100 different kinds of lists in C#... someone must have made one with a persistent backing store!

But I looked around and couldn't find it, so I made it (as I'll need this in the coming months).

The idea is basically this: I want to use LINQ, I like Dynamics, and I like speed. So that's it, and here's Biggy:

dynamic db = new BiggyDB();  
db.Clowns.Add(new { Name = "Goofy Toots", Age = 1002 });  

This creates a JSON store in your project root, in a new Data directory. It's just JSON, nothing special. You don't have to be all dynamic, though - you can get all Statically-static with your typed self:

class Product {  
  public String Sku { get; set; }
  public String Name { get; set; }
  public Decimal Price { get; set; }
  public DateTime CreatedAt { get; set; }

  public Product() {
    this.CreatedAt = DateTime.Now;

  public override bool Equals(object obj) {
    var p1 = (Product)obj;
    return this.Sku == p1.Sku;

//add and save to this list as above - now with even MORE INTELLISENSE
var products = new BiggyList<Product>();  
products.Add(new Product{Sku : "Steve", Name : "Monkeying with .NET", Price : 12.00M});  

These are just list operations, nothing special here aside from Save(), which flushes everything to disk. It's a single write and firehoses whatever is there. Yes, I know, a bit nuts... more of this later.

If you have a lot of writes to do at once, you can batch them and dump them asynchronously:

var products = new BiggyList<Product>();  
for (int i = 0; i < 1000; i++) {  
  var p = new Product { Sku = "SKUDDY" + i, Name = "Glurp " + i, Price = 12.00M };

//save it asynchronous!

The Github repo is here.

Events and Callbacks

Gotta do it :). There are 5 or so events you can use when working with your data (which I hope seem obvious):

This means you can subscribe to things as you like, which is helpful if you're doing async:

var products = new BiggyList<Product>();  
products.ItemAdded += ItemJustAdded;

products.Add(new Product{Sku : "Steve", Name : "Monkeying with .NET", Price : 12.00M});  

public void ItemAdded(object sender, EventArgs e){  
  var biggyArgs = (BiggyEventArgs<Product>)e;
  //you can access Item and Items on biggyArgs
  Console.WriteLine("Now we have {0} products!", biggyArgs.Items.Count);

Reading and Writing

So, by now you should be wondering why this is useful. The simple answer is that if you have a high-read application (like a blog, CMS, etc) then something like Biggy could speed things up.

Whenever you instantiate a new BiggyList it tries to read it's data from disk - this is good, and it's bad. It's good because from that point on whenever you try to query your data (using LINQ) it's an in-memory operation and you can't get much faster than that.

It's bad because this means you probably want to have a single DB instance around for the life of your app. This might be easy for some, might be repulsive to others. I'm used to doing this kind of thing with Node (all modules in Node are cached which means you always hit the same module instance).

For a blog engine, this might be a very fun thing to have - no database installs, superfast, easy to use. For a Twitter clone... not so much.

Just an Idea For Now

I'll be goofing around with this for a few months to see if it has legs but I thought it would be fun to share it with others - even if it's to tell me it sucks and to make it go away.