Tail Recursion

boot args

David Weldon — Tue, 19 Jul 2011 23:24:20 +0000

Have you ever wanted your code to behave differently when in production and development? An easy way to handle this is to add a meaningful boot argument like so:
$ erl -production
Now you can use init to check for the existence of the production argument. In this example, I’ll use this to select my redirect target domain:

Domain =
    case init:get_argument(production) of
        {ok, _} -> “www.example.org”;
        error -> "localhost:8000"
    end

You can also pass in values with your arguments like so:
$ erl -dictionary_size 500
And you can read the value back out with:
> init:get_argument(dictionary_size). {ok,[["500"]]}

spell correction

David Weldon — Tue, 19 Jul 2011 05:46:33 +0000

Over the past couple of years I’ve done some NLP work in erlang including LSA, text segmentation and automatic spell correction. If you’ve ever thought about creating a spell checker, I’d recommend reading this article. Much can be improved in the implementation presented – chiefly, the brute force search of all conceivable variations of the input text. My own application required automatic correction on the order of a few milliseconds, so that approach was clearly not going to work.

I then found this article, which explains a terrific solution. In summary, if you keep your dictionary in a trie you can narrow your search to only variations which exist in the dictionary. This greatly improves the speed of the algorithm. I implemented the same idea in erlang, and ever since I’ve been doing ~1ms corrections with a dictionary of tens of thousands of words.

While most of my work is application-specific, the trie structure and search algorithm are generic so I decided to open source them here. Like all of my projects, please feel free to send suggestions and pull requests.

utf-8

David Weldon — Tue, 31 May 2011 20:08:51 +0000

I recently had a problem where I was rendering an ErlyDTL template and symbols like © and ® were not displaying right. After a little trial and error I realised that the character encoding in my HTML template was set to UTF-8, but erlang’s default encoding is ISO-latin-1 (ISO8859-1). Fortunately the unicode module provides an easy fix for this:
{ok, Html} = my_template_dtl:render(Data), unicode:characters_to_binary(Html).
And voilà, we have a UTF-8 binary.

block expressions

David Weldon — Mon, 17 Jan 2011 00:25:54 +0000

I’ve long been annoyed that list comprehensions can only accept a single expression. What if I want to print and perform some computation on each value? What I didn’t know was that expressions can be grouped together as blocks:

> [begin erlang:display(N), N*10 end || N <- lists:seq(1,3)].
1
2
3
[10,20,30]

For more details see this.

where is my priv_dir?

David Weldon — Sun, 09 Jan 2011 00:13:46 +0000

Let’s say you’re writing an application that needs to load a file from its priv directory, so you need know its path. You just call code:priv_dir/1 with the name of your application right? Maybe. The catch is that this function only succeeds if your application is in the code path. As far as I’m able to tell that leaves us with three choices:

1) Make sure your application is installed to somewhere under a directory in $ERL_LIBS. This makes sense for a shared library, but may not make sense for your web server. Furthermore, when working on the code it’s annoying to have to add your development directory to $ERL_LIBS.
2) Just write code like file:read_file("./priv/dictionary"). Unfortunately that will only work if you run it from the application directory. Fail.
3) Use the following code:
{ModPath, _} = filename:find_src(?MODULE), AppPath = filename:dirname(filename:dirname(ModPath)), filename:join(AppPath, "priv").

So far that seems to be the winning solution for me. If you have a better solution or you can think of a reason why this doesn’t work please let me know.

edate

David Weldon — Sat, 03 Jul 2010 23:22:55 +0000

Date manipulation in erlang isn’t very pleasant. The calendar module provides a lot of the raw tools but none of the convenience of something like the Date class in ruby, or the time calculation extensions in rails. I borrowed ideas from both and created edate. Check it out and let me know if I’m missing anything that would be helpful. Examples:
> Date = edate:string_to_date("7/15/2010"). {2010,7,15} > edate:end_of_month(Date). {2010,7,31} > edate:shift(Date, 52, weeks). {2011,7,14} > edate:day_of_week(Date). "thursday"

matching with append

David Weldon — Sat, 03 Jul 2010 22:35:20 +0000

I was recently reading this example, and I was totally surprised by the syntax in the is_authorized example. Apparently you can use the ++ operator to to pattern match. For example:
> "pajamas:" ++ Color = "pajamas:blue".
Color now has the value “blue”. Be aware that this trick has it’s limitations – as far as I can tell it only works with a single variable and a single constant in the order given above.

It pays to read other people’s code – you’ll never know what interesting tidbits you may find.

erlang library manager

David Weldon — Fri, 26 Feb 2010 01:55:00 +0000

If you’ve ever wanted to install and update all your favorite erlang libraries with a single command, you should check out elm.

I’ve grown tired of manually checking for updates and writing verbose install scripts for my servers. All you have to do is store your repository urls in a single file, and elm can figure out how to pull the source and build it. Check out the readme for complete instructions and a full list of examples.

erlsom easy install

David Weldon — Tue, 23 Feb 2010 17:45:39 +0000

I’ve been using erlsom on my production servers for almost a year now – it’s parsed billions of xml documents in that time without an issue. My only problem with the library is how unpleasant it is to install. The build system is overly complex, the files are all dos formatted, it compiles with warnings, it’s hosted on sourceforge, etc. I cleaned up all of these issues and added the code to github. Please see the readme for more information about what I changed.

node communication and automatic discovery on ec2

David Weldon — Sat, 20 Feb 2010 17:55:41 +0000

So you’ve signed up for ec2 and figured out how to create instances with erlang installed. Sweet. I’m going to show you how to get your erlang nodes to communicate – first if we know the IP addresses of both machines, and second if we want the nodes to automatically discover each other.

I’m assuming at this point that you have two instances: I1 and I2, which are both members of the security group S. The first thing you need to do is edit the security group settings to allow full port access between all members of S. If you are using the management console, just add a new entry, leave everything blank but add S as the source and hit save. After a refresh it will show that icmp, tcp, and udp are all open for all members of S.

Open ssh connections to I1 and I2 in two terminals. Note the private DNS names of both instances – it’s the hostname followed by “.ec2.internal” (on us-east). For example: ip-10-200-100-50.ec2.internal

On both instances:
$ erl -name test@`hostname`.ec2.internal -setcookie test
If you don’t know what these parameters do, see the distributed programming section from this doc.

Assuming the private DNS name of I2 is ip-10-200-100-50.ec2.internal, on I1 we can do this:
> net_adm:ping('test@ip-10-200-100-50.ec2.internal').
If you received a pong message, congratulations – your nodes know about each other and can now communicate! Try running nodes() on each.

Next let’s have a look at doing this in a more automated way. Leave I2 running, but quit out of the erl shell on I1 and install Eric Cestari’s ec2nodefinder. As his README points out, you will need to either set the environment variables AMAZON_ACCESS_KEY_ID and AMAZON_SECRET_ACCESS_KEY, or you will need to edit the ec2nodefinder.app file in the ebin directory. Be careful of the commas in the app file if you go that route – if the file isn’t syntactically correct, the app won’t start. I like Eric’s version of ec2nodefinder better than the original version from the dukes, because it’s easier to install and it doesn’t rely on ec2-describe-instances.

Start your node on I1 again:
erl -name test@`hostname`.ec2.internal -setcookie test
You can now do:
> ec2nodefinder:start().

> ec2nodefinder:discover().
Which, if successful, will return something like:
{ok,[{'test@ip-10-200-100-50.ec2.internal',pong}]}
Calling nodes() on either node should again show they are connected. Awesome. Now you can do all kinds of crazy stuff like have nodes which automatically become part of an mnesia cluster when they start.