HOWTO: deploy multithreading on Matlab with the Theron C++ Actors library

Preamble: What follows is a HOWTO guide for getting Theron-2.09 multithreaded C++ mex files built on Matlab. For my general musings on Matlab concurrency and Theron, see the previous post.

Building multithreaded Matlab mex files with Theron

This was pretty straightforward on the Mac, somewhat tricky but not terrible on Linux. I don't have any working C++ compiler with Matlab on Windows thanks to absolutely abysmal support from Microsoft and Mathworks (essentially Mathworks only supports Microsoft and Microsoft's compilers are impossible to get installed correctly). But the notes below may help you get things working on Windows anyway and I added some Windows specific notes at the very bottom. Before trying to get Theron working, I encourage you to write a simple C++ mex program (maybe a single-threaded version of your planned algorithm) and get it built on Matlab first, to confirm that your system is correctly set up for mex.

I'm fairly confident most Mac users can get this working. On Linux I have only tested on 2010 and 2011 64-bit versions of Matlab, and there are reasons to fear that earlier versions may have more trouble working. Fair warning.

Simple(r) concurrent Matlab programming with Theron

Preamble: What follows are my (long-winded) musings on the current (Sep. 2011) state of concurrent programming in Matlab. For a specific guide to using the Theron C++ library for Actors concurrency in Matlab, see the following post.

Concurrent Matlab

Matlab is a standard programming environment for scientific calculations. In our lab, we use it for a lot of analysis, especially prototyping new analyses. Eventually, some of the performance critical stuff gets ported over to Java, but we still have plenty of things we need to run that are Matlab-only and need to be fast.

As personal computers add more and more cores without improving clock speeds much, there is more and more need for parallelized (AKA concurrent, multithreaded) software. Matlab is not a very popular language for supercomputing, but high-performance clusters also of course demand parallelized software. Matlab has some support for multithreading internally, so depending on which Matlab internal functions you use, you may see significant multicore use on your machine. But there is currently no support for explicitly writing multithreaded programs in vanilla Matlab.

Broken Mac OS 10.6 (and 10.7 and 10.8 and...) Java

Apple has a pretty lousy history with Java support, but this still surprised me. If you are having trouble getting programs to compile with older Java versions (e.g. for backward compatibility), you might want to take a look in this directory: /System/Library/Frameworks/JavaVM.framework/Versions

Here's what I found on my OS 10.6 system:

lrwxrwxr-x   1 root  wheel    5 Jun 29 13:37 1.3 -> 1.3.1
drwxr-xr-x   3 root  wheel  102 Feb 11  2010 1.3.1
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.4 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.4.2 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.5 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.5.0 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.6 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   10 Jun 29 13:37 1.6.0 -> CurrentJDK
lrwxrwxr-x   1 root  wheel   59 Jun 29 13:37 CurrentJDK -> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents

I kept trying to get Eclipse to compile with 1.5 and kept getting incompatible binaries. Thanks Apple! YMMV, but I found this post a good guide to getting a real Java 1.5 installation.

Mac OS 10.6.8 breaks OpenSSL dev compatibility

Update (2011-07-29): Globus Toolkit devs have told me that this is a frequent problem with Mac OS updates. Thanks Apple. So they recommend running with the --without-openssl-header-check flag if you don't want to go to the trouble of actually installing compatible headers.

If you are trying to build anything that requires OpenSSL on Mac OS 10.6 and have updated to 10.6.8, you may run into the following complaint:

checking OpenSSL header version... 9080cf (OpenSSL 0.9.8l 5 Nov 2009)
checking OpenSSL library version... 90812f (OpenSSL 0.9.8r 8 Feb 2011)
checking whether OpenSSL's headers match the library... no
configure: error: Your OpenSSL headers do not match your library. Check config.log for details.
If you are sure your installation is consistent, you can disable the check by running "./configure --without-openssl-header-check".
Also see contrib/findssl.sh for help identifying header/library mismatches.

For some reason, when Apple upgraded to OpenSSL 0.9.8r to address security issues they failed to update the headers, so now there is a version mismatch (i.e. header version is 0.9.8l but library version is 0.9.8r).

If you want to track this issue in more detail, you can find a copy of the findssl.sh script, either on your system somewhere or just find it online. If you download it online, you will probably have to hack the script to find Mac's .dylib shared library files instead of the Linux standard .so files. Alternatively, just try these commands:

locate opensslv.h
locate libcrypto.dylib

That should show you where the different versions of headers and libs are respectively. Then you can look inside the opensslv.h file in /usr/include and confirm that it is 0.9.8l instead of 0.9.8r.

In any case, the solution seems to be:

1. Get OpenSSL 0.9.8r from http://www.openssl.org/source/
2. Untar, copy the include/openssl directory to somewhere like /usr/include/openssl-0.9.8r
3. Point your build to these new headers. I decided to just blast the incompatible headers, so I moved my old /usr/include/openssl to /usr/include/openssl-0.9.8l and then symlinked openssl to openssl-0.9.8r.
4. Now it builds!

Combining USDA SR23 with glycemic index data

USDA provides a National Nutrient Database for Standard Reference currently in Version 23 (SR23), which has loads of nutritional data on a good selection of food products. On my wishlist would be cross-referencing with UPC bar codes, as well as some additional data that are currently missing, such as glycemic index and separate soluble vs. insoluble fiber values.

I did find one website that has done a pretty job with glycemic index already: nutritiondata.self.com. In addition to pulling in glycemic index data, they've also calculated glycemic load and then built a statistical model to try to infer/impute glycemic load for foods that don't have published glycemic index data. And they also have some other summary statistics that they've developed in-house such as "Fullness Factor" and an overall healthiness rating. Kudos to them, but I wanted to play with these data myself. Also I didn't see a fiber breakdown there.

One data source I found that has both glycemic load and soluble vs. insoluble fiber (at least for some entries) is the Diet History Questionnaire II (DHQ2). The DHQ is available to download as a single giant table. I plan to play with this.

Another option is to link the glycemic index data derived for the DHQ back to SR23. This appears to be possible thanks to a couple of additional tables. First, DHQ provides their glycemic index data in a table referenced against food codes from the USDA Continuing Survey of Food Intakes by Individuals (CSFII). These codes match the food codes used in the USDA Food and Nutrient Database for Dietary Studies (FNDDS). The FNDDS database includes a table that links selected FNDDS/CSFII codes back to one or more SR23 codes. (Since my main interest in glycemic index is to compare glycemic index to sugar composition, and FNDDS only has a value for total sugar not individual components, I'm not too interested in FNDDS by itself.)

I'll be exploring DHQ vs. SR23 (augmented with glycemic indices linked from FNDDS+DHQ) and will report back.

Globus Toolkit 5.0.3 on Cygwin 1.7

Cygwin patches (see earlier post) did not make it into GT5.0.3. The patch requirement is therefore the same as for GT5.0.2 (see earlier post). These patches may make it into future distributions.

I took slightly more detailed notes during the 5.0.3 build process, so you can now find more details about which packages require patches here.

DIY KitchenAid attachments

Update: (2011-03-14): So this does not quite fit perfectly into the Attachment Hub, but it is good enough that with a housing for the shaft I'm pretty sure it will work. I'll take more precise measurements and post any updates.

Update (2011-02-18): Arrived no problem from Mending Shed; Parcel Post was quite fast actually. Don't have the KitchenAid here with me right now so will not be able to test at all for a few weeks.

Our KitchenAid stand mixer is probably the fanciest piece of hardware we own other than our car. I've been thinking for a while that there really ought to be a DIY modding community around this appliance; it's a powerful, well built kitchen motor waiting to be exploited (plus planetary gearing if that is helpful). But I haven't found much so far. Here's one nice rotisserie mod I found, and that's about it.

KitchenAid makes a number of official attachments. Some of them attach to the bottom head where the planetary gearing is, while others attach to a port on the front (the "Attachment Hub") that accepts square shafted ("Power Shaft") attachments. I searched around a little and was able to find a part that I'm pretty sure will fit the attachment hub: KitchenAid 9709315, the "clutch shaft". Amazon sells it.

Note that this doesn't include the shaft housing that will allow you to lock it into place; figuring out a way to hack this together will be the first job before anyone can really use this thing for DIY projects. I'm wondering if putting something together in plastic through Ponoko or the like will get the job done. It probably requires some bearings too.

I ordered one from Mending Shed (slightly cheaper than Amazon and will ship Parcel Post), will update with progress.