CageFS and LDAP
As Jim celebrates the beautiful history of WordPress and his own personal trajectory towards both the software and regular blogging at the bava, it inspired me that I need to use my blog a bit more as a personal mind dump. I had a bit of fun with him a few days ago in fact about whether highly technical posts were "useful" to larger audiences. What's "useful" though is a moving target. You might read a few sentences here and decide it's not for you, whereas someone else may be searching the web and come across exactly what they needed from the words you wrote (I know I sure have). In an effort to get into a more regular habit of documenting things I'll likely try to hit the Publish button a bit more here on these things, starting today.
I ran into an odd bug with one of our partner schools, the University of Oklahoma, where we have an Active Directory integration to allow their campus to sign in and get an account for their Domain of One's Own platform. It had worked perfectly for well over a year and then sometime in late November I started hearing issues with some users logging in. We figured out it was only new users, which we attributed to the AD plugin we use caching credentials to allow a fallback to local logins. But we had to figure out why the connection had suddenly stopped working. We chased a lot of theories including firewall settings, plugin conflicts, out-dated plugins, plugin updates that may have broken it, new settings on the AD server that were causing a conflict. Nothing seemed to make it work, users kept getting "Operation Failed" when logging in and the test tool provided with the plugin reported a failed connection to the LDAP server.
Today I finally found the smoking gun in the form of some security software we began running on the server in November called CloudLinux. CloudLinux is great, it makes sure no single user can overwhelm a server's resources and improves overall performance and security of the server. It also comes with a lot of nice bells and whistles like smart PHP caching and the ability to run different versions of PHP, Python, and Ruby from a single shared hosting server. We enabled this on OU's server primarily to manage load issues they were having but also to handle security of accounts and one feature CloudLinux has (not on by default but I imagine from this post you can guess I enabled it) called CageFS. CageFS is a jailed shell environment to prevent a user from traversing to other areas of the server. Makes a lot of sense, especially if someone's account is hacked that they shouldn't be able to see anything outside their home folder or call server binaries not available to them.
As I found out today, CageFS conflicts with LDAP connections. I'm not sure why as the plugin uses the PHP LDAP extension to make the connection and not a server binary of any kind but as soon as I disabled CageFS for that particular user on the server and ran the test tool we had a working connection again. Security on servers is often a balanced nightmare like this where you fix a lot of things and plug up holes and then find conflicts and have to work backwards to figure out what's going on (for example it's great when a firewall blocks all the bad folks from trying to login to your site, but not so much if it accidentally blocks you). It's a constant moving target that I hope to learn more about as time goes on and I'll be sure to document what I learn along the way.