Okay, so after a few drawbacks regarding a proper way to save my data, the bot is going to collect, I finally had some great progress this time.
Database madness
I started with at least trying out ETS, an erlang in-memory saving mechanism. This kind of failed at first, but, as I later found out, only because I implemented it wrong.
So then I thought, it would be better to have it in some kind of database anyway, as it would be easier to browse this data with a simple node.js application. So I thought CouchDB (don’t ask why, it was just in my head). So I went out and tried to install it on my server. This was a huge pain in the behind. After 3 hours of desperately tinkering around, finding proper documentation, and trying to set it up, I finally got something that, at least, worked on the command-line, but the init-script was totally broken. A quick chat in the #erlang channel gave me the statement, that the init-scripts are broken for long, and it’s a known cause.
Seeing too much trouble ahead of me, I struggled another hour to remove most of the traces this thing made in the first place, and switched to MongoDB. I don’t want to start a discussion about it here, had plenty of that in the #erlang chat, and I still only partially get it. I had good experiences with it using meteor.js, so I thought, why not? I installed it, which was a breeze (thank goodness), and went to install third party erlang driver (called mongrel, nope NOT the ruby server, which doesn’t make it easier to google for it properly), and tried out the documented examples. Nightmare, I tell you. The documentation is not really beginner-friendly, and the examples plain didn’t work. Searching for help on the net didn’t result in anything, so I quickly removed everything again and went frustratedly to the #erlang chat.
There people said riak might be a solution, but as I was frustrated from having installed 2 other servers, I tried to change my initial roadmap. I figured, it doesn’t really matter if the bot crashes and loses all it’s data. It can just be restarted, and start anew. It doesn’t learn anything, and it usually should be implemented to post whatever it finds.
So I went back to ETS, tinkered around with it in the erlang shell, and now, finally, figured it out.
And the code to save my word-counts, maybe not beautiful, looks like someone knows his way around erlang =)
I am proud of this:
save_word_counts(Msg) ->
Words = [ X || X <- string:tokens(Msg, ?BREAKSTRING), string:len(X) >= ?MINWORDCOUNT, string:len(X) =< ?MAXWORDCOUNT],
lists:foreach( fun(One) ->
Result = ets:lookup(words, One),
case Result of
[] -> ets:insert(words,{One,1});
_ -> [{One,LastCount}|_] = ets:lookup(words, One),
ets:delete(words, One),
ets:insert(words, {One, LastCount+1})
end
end, Words).It works, and does exactly what it’s inteded to do. Yay.
And the string function stuff took me even less time, than expected, so I immediately made up for the lost time dealing with the external servers.
Bottom line
Erlang is still fun, and I am learning a lot. The #erlang chatroom is pretty nice, and helpful, and you USUALLY find enough help with erlang-internal stuff (third party is another case though).