I was reading Tim Bray's recent blog posts (here and here) about his Topfew utility with interest, and wondered how my go-to systems programming language (Nim) stacks up compared to Go. I knocked up a quick and dirty implementation and then ran against a 900MB access_log from my own site. A minute or so later I hit CTRL+C, realising my quick and dirty implementation was (a) not quick at all, and (b) perhaps a bit too "dirty". ಠ_ಠ
Once I changed from the naive read everything into memory approach, to using Nim's streams, I ended up with something slightly more acceptable. Caveat: not really optimised (well... apart from compiling with -d:release --passC:-mcpu=native --boundChecks:off flags), lazily developed, probably still naive, but at least has border-line acceptable performance (Github).
Results on my laptop (with an approx 1.5 million line access_log):
| Elapsed | User | System | vs Go | |
|---|---|---|---|---|
| Tim's topfew | 2.68 | 3.86 | 0.41 | 1.0 |
| ls/uniq/sort | 9.10 | 1.76 | 0.26 | 3.4 |
| Nim | 3.91 | 3.65 | 0.25 | 1.5 |
1.5x isn't brilliant, but it's not bad considering the minimal amount of effort I spent on it. But that led me to wonder, what about Python performance? Another Q&D implementation later (and this time I haven't bothered to dig into the performance at all)...
| Elapsed | User | System | vs Go | |
|---|---|---|---|---|
| Tim's topfew | 2.68 | 3.86 | 0.41 | 1.0 |
| ls/uniq/sort | 9.10 | 1.76 | 0.26 | 3.4 |
| Nim | 3.91 | 3.65 | 0.25 | 1.5 |
| Python | 8.57 | 7.77 | 0.79 | 3.2 |
Not much better than the uniq/sort version, but the python version is arguably a little more readable than Nim. One interesting difference between Python and Nim -- the Python version does read the whole file into memory before tokenising...
with open(filename) as f:
for line in f.readlines():
......yet the performance was significantly better than my earlier naive Nim implementation which also read the whole file. So it's not just the right tool for the job, but the right technique for the tool.
Do I feel like dusting off the rather rusty Haskell skills though...?