[update] If you’re having issue with the video at the bottom, you can check it out on youtube.

Explanation

A while back I decided I wanted to learn erlang (for a myriad of reasons, most of all its vm). I started working on grove, my first erlang project, after watching a video on Sproutcore and thick client architectures (Sproutcore part is about 14 minutes in). Part of the talk is about “micro-services” and I thought Mnesia, being relatively awesome when it comes to scaling horizontally, along with simple data delivery, would make an ideal paring for something like Sproutcore/Flash/Silverlight/Other thick client.

Grove’s purpose is to provide easy access to Mnesia (and other data stores, just need an adapter module) through both GET and POST requests to a YAWS server (it’s sits behind a yaws appmod module at this point) and a standard JSON query structure or erlang tuple. Mnesia does all the fanciness with scaling.

In the video below, you’ll see both methods, but to summarize, using a GET request will mean calling an actual erlang function in a module specified in the grove config (default is grove_custom). With this method you can do pretty much whatever you want, the example function find uses functions from the grove_mnesia module but you’re obviously not restricted to that. As an aside, I decided to move these functions to a separate module because of the dynamic nature of the calls to them makes it insecure to include them anywhere else.

Using a POST request will require passing a variable query that contains a query JSON object to a url representing the table/object you want data from (see video). Grove will turn this into an Mnesia set comprehension and give you back the results. I’m really not convinced this is the best structure for the query, but either way it should look something like this:

What you don’t see there or in the video, which is important to know, is that, if a string on either side of an operation matches a column name in the Mnesia table, it will assume you want to compare that column with the other side of the operation. To force a comparison of the actual string in that case, you can use something just like the above, replacing “atom” with “string”. This tells grove that you want it to compare it as a string and not the column, in the same way with “atom” that the query should compare with the atom derived from the JSON string. You could make a case for it being the other way around (specify “column”), but I thought that the number of cases where you would be comparing for a value that happens to be a column name exactly was small (could be wrong here).

You’ll need mochiweb, and yaws to get it working. I used mochiweb to convert JSON back and forth which means I should really use it as the server, which I plan to do. I also have a good set of eunit tests in there as well (see grove.erl for example queries in the integration tests).

So thats about it for how it works.

Right now it doesn’t write to the database it only reads, because to do that I really need to come up with an authentication scheme. I’ll let you know how that goes :P. Also, it really could use a small javascript library to provide a query object which can be to_json’d. Then a client side hacker would have an easier time building queries. Additionally, joins (don’t remember if this is the mnesia terminology for it) and more complex ordering need to be added in. Finally, it uses Mnesia to store its config settings, since that was the easiest thing to do at the time, and my understanding of ets tables is that they disappear when the thread running does (not so good for an appmod).

What I am really hoping for here is some constructive criticism from any erlang hackers about what I did wrong/right. Please let me know in the comments.

Video

Below is a short video on how it works. I didn’t narrate it since that would have required a lot of takes. Instead, I added some captions and a nice selection of my favorite music. If you don’t like it, feel free to mute.

You can watch it in HD by selecting the HD option on the bottom right of the video once you click play. It was a pain getting that set up but it does make a difference. If you don’t see that option just double click the video to watch it on youtube.

Otherwise enjoy!

Github (still need to do a readme): http://github.com/johnbender/grove/tree/master