useful bits, picked up and dropped off
MacPorts is useful for installing a variety of command-line utilities and programs for Mac OS X. There are others, e.g. Homebrew. After using MacPorts to update a GNU gcc installation, it is useful to select the new revision. Tips were posted to this Stack Overflow thread. Basically, it boils down to two steps:
sudo port select --list gccsudo port select --set gcc mp-gcc47Here’s a one-liner that converts jarch files to starch format, stripping the input file’s extension so that it can be replaced with a new one:
$ for i in `ls *.jarch`; do echo "${i%.*}.starch"; gchr $i | starch - > "${i%.*}.starch"; done
PolarSSL is a C-based cryptography and SSL library which has a GPL license, which makes it ideal for use with BEDOPS, where I plan to use it for quick SHA-1 hashes of metadata, so as to help validate the integrity of the archive.
I’ve been testing it out in Mac OS X 10.8 and it seems pretty straightforward. Here’s a simple project that hashes the string abc:
#include <stdlib.h>
#include <stdio.h>
#include "polarssl/config.h"
#include "polarssl/sha1.h"
int main(int argc, char **argv)
{
unsigned char output[20];
unsigned char *buf;
size_t bufLength;
size_t idx;
buf = strdup("abc");
bufLength = strlen(buf);
sha1(buf, bufLength, output);
for (idx = 0; idx < 20; idx++) {
fprintf(stdout, "%02x", output[idx]);
if ((idx + 1) % 4 == 0)
fprintf(stdout, " ");
}
fprintf(stdout, "\n");
free(buf);
return EXIT_SUCCESS;
}
To compile it:
gcc -Wall -lpolarssl sha1test.c -o sha1test
When we ask for the hash value of the string abc, we get the following result:
$ ./sha1test a9993e36 4706816a ba3e2571 7850c26c 9cd0d89d
This agrees with the value reported at NIST, which is also:
a9993e36 4706816a ba3e2571 7850c26c 9cd0d89d
Testing output against standards is useful for validation.
I plan to use a magic number in the second major release of our BEDOPS suite to uniquely identify starch-formatted archive files.
Looking at the first few bytes of the archive will help us because I plan to move the metadata to the back of the archive file, and it would be expensive to seek to the end of the file just to determine the file type. I figure a 64-byte header is enough to hold some extra data that identifies the file type and points the unstarch and starchcat tools to the right byte in the file where the metadata is kept:
This image describes what I’m planning for Starch v2. Basically, we reserve a 64-byte header at the front of the archive that contains the following four items:
After these 64 bytes, the compressed, per-chromosome streams start, and we then wrap up with the metadata at the end of the file.
Initially, the header will be the magic number and all zeros. At the conclusion of creating the archive, once all the streams are prepared and the metadata is ready to hash, parts of the header are written over with calculated values (offset and hash).
To assist with picking the right magic number, Ned Batchelder put together a Python script to generate all the hex words possible from a small subset of the English dictionary — words like deadbeef, dec0ded and 0ddba11 which can be used as magic numbers to identify a file type. Thanks, Ned! So far, I’m liking the magic word ca5cade5 as it has a nice biological flavor to it.
When I want to make BEDOPS documentation available for offline browsing, here is the command I use:
wget –no-parent –recursive –page-requisites –html-extension –convert-links -E -l 2 http://code.google.com/p/bedops
This handy wget statement fixes up the img and other URL references so that links to images and other resources load up from the local copy. So what the end user sees is almost exactly like what she would see if the documentation was retrieved through a network connection. Very nice.
WordPress replaces quotation marks with curly equivalents, which is really frustrating when trying to copy and paste code. Add remove_filter('the_content', 'wptexturize'); to the bottom of the theme’s functions.php file.
Interestingly, this doesn’t seem to affect post titles, but that’s okay with me.
I had recently updated my copy of R to 2.15.1 and ended up needing to reinstall some libraries, including rgl.
If you use this R library, it can be tricky to install with the built-in build of Mesa/OpenGL in Lion. In fact, a straightforward install.packages("rgl") just won’t work at all.
But after using MacPorts to install freeglut, I found the following R command helps build rgl successfully:
install.packages(“rgl”, repos=”http://R-Forge.R-project.org”, configure.args=”–disable-cocoa –with-gl-includes=/opt/local/include –with-gl-libs=/opt/local/lib –with-x”)
Once installed, rgl can make cool figures, like this:
