I got mod_security from EPEL for my RHEL 5 setup. Apparently the binary you get from EPEL was compiled with the switch -DDISABLE_HTACCESS_CONFIG. I have been pulling my hair trying to get Apache to not throw an error 500 every time I wanted to selectively put in any mod_security directives. I ended up just throwing the directives directly into the VirtualHost or other context and reloading Apache.
Google had a hard time finding the problem, so I hope this blog post will help someone else.