contegix: beyond managed hosting

Posts Tagged ‘security’

We’re often asked at Contegix, “Do you perform automatic upgrades of Application XYZ?”, and are answer is always emphatically, “No”. This tends to spark some debate, since we do tend to perform RHEL updates automatically. First, let’s define “automatic”, because obviously we’re not shutting down instances/servers without explicit permission from you or your team. In regards to standard RHEL updates, we inform you after the updates have passed a rigorous round of testing and have both the Redhat and Contegix internal “go-ahead” that we need to perform updates on your servers. We consider these mandatory for the reasoning of security. Redhat doesn’t push superfluous updates down the pipe to your servers. They’re generally provided for very specific means, and the number one reason is security. We can push these updates because 99% of the time, the end-user (you) won’t even notice the difference in most cases. On the rare occasion an update may have an odd effect, but I’d like to stress that the case of that happening is excruciatingly rare.

Let’s compare this to… well, -any- web application you’re running right now. First off, keeping up with what’s running on every customer’s server is a massive chore in of itself. Keeping up with that list, and checking to make sure every web application is running the newest version is just a research nightmare. Obviously, the big applications (aka our managed applications) we’re aware of, such as the Atlassian suite of applications, Wordpress, Jive’s suite of applications, and so on. Unfortunately, keeping tabs on all the various web applications we use, and their version numbers, is a bit rough, but is something we plan on tackling in the future. The real problem however exists in the following question: “Do you really want to upgrade?”

The problem is that many applications have introduced the wondrous world of plugins into their applications. Honestly, from our side of the fence, plugins create a lot of havoc. For one, they’re not always supported by the main developers of the application in question, which leaves us restricted in the level of support we can offer to a product using them. Secondly, they make application upgrades comparable to a roller coaster where the cars may or may not come unhinged from the track, sending you careening into a brick wall. That’s not to say we don’t like plugins, because we love plugins. For instance, the Wordpress Automatic Upgrade Plugin turns Wordpress upgrades into a quick 5 minute ordeal. No need to worry about asking us to upgrade your Wordpress, take backups, and hope that we catch any theme changes that need to be made in the process. Instead, a few button clicks and this plugin will complete the upgrade in no time flat, bringing you to the latest version of Wordpress. I’ve used it on my personal blog a couple times now, and it worked flawlessly. Obviously, your mileage may vary, but if nothing else it performs backups before it does anything, so if the upgrade fails, reverting back is a snap.

Why on Earth would a Wordpress upgrade fail though? Plugins. It’s the same reason we have upgrade problems with any application we work with, plugins inherently create issues for upgrade procedures because they introduce new quirks that may fail when the core application is upgraded. Depending on how integral that plugin is to your application instance, this could cause an upgrade to become a complete failure. A default instance of Confluence/JIRA/Crowd upgrades smoothly, no problems to worry about. An instance of Confluence with a bunch of plugins, theme changes, and so on, however tends to be a bit more interesting. It’s not really Confluence’s fault, in fact it’s quite likely that weird plugin you were skeptical about installing is breaking something internally, thus causing the upgrade to fail. More often than not though, Confluence upgrades can fail due to heavy edits to themes, generally via the Theme Builder plugins. This causes theme anomalies, as the Theme Builder plugin is out of date, not functioning properly, and the changes in Confluence between versions have also contributed to some issues with your themes, such as in 2.8 when the theme was prettied up quite a bit (nice job Atlassian!). All of sudden, what should have been an easy, smooth ride, is now resulting in an extra half hour of down time as we scramble to fix the problems. Then we have to come to a decision on rolling back, or progressing through the issues.

This is why we generally frown on automatic upgrades, because plugins add a significant curve ball to the mix that we can’t foresee. If keeping up with every web application is a documentation job of epic proportions, imagine trying to track compatibility of plugins, the plugins installed, and the ones not installed on all customer Confluence instances! We like to keep downtime to an absolute minimum, which is half the reason you’re with us we hope, and that’s why we avoid automatic upgrades. Instead we encourage staging instances, scheduled tasks, and taking each upgrade on a case by case basis. Do you want us to merely say “Confluence 2.8.1 is out, and we’ll be upgrading you on MM/DD/YYYY at 00:00″? We believe it’s in everyone’s best interest for you to decide when to upgrade, and to let us know. We’ll work through the process with you, check compatibility/dependency issues, and set the event up for a time that suits your needs best. If you’d like to see it staged out first, that’s fine too, we’re more than happy to setup a small staging instance for the upgrade when necessary, assuming it’s not detrimental to the overall health of the server. We want to work with you, as much as we work for you and your company. If you have any thoughts or suggestions on our upgrade procedures, feel free to drop them in the comment box!

As a large hosting provider we use a lot of different applications, and we try to keep them all as secure as possible. Unfortunately, we can only win so many battles at any given time, and we do require help from you, the customer at times to ensure your system is safe. Wordpress, as great as it might be as a blogging platform, seems to find itself getting hacked more than most applications that we host. Now, I’m not saying that Wordpress is a bad application by any means, but with it being such a large platform it draws a lot of unwanted attention.

As such, there are quite a few hackers and script kiddies out there that will try to compromise your Wordpress based website. We’re hoping that with this article we can further educate Wordpress users on how to protect their sites. Here’s a few helpful tips we can provide, some of which you can have us do, and some things we’ll recommend that you do:

5. Please, please, please don’t use a user name of ‘admin’! I know that’s the Wordpress default, and it’s just easy to use it, but what user name do you think is in every brute force attack? You guessed it, ‘admin’. We’d recommend using a unique user name for administration purposes, like ‘mark.rogers’, or ‘mrogers’. Of course, you can use your own name if you don’t like mine I suppose.

4. Remove the Wordpress version number from any headers, footers, css, etc, etc. Leaving the version number in your page source is a dead giveaway to would be vandals to dig through google to find ways to exploit your specific version. Granted this is the equivalent of leaving your lights on at home while you’re away, but if it deters someone, then consider it a victory! It’s just too easy to use the version number to find exploits for your site, as Wordpress exploits become public knowledge too often.

3.Let us put basic Apache authorization on the /wp-admin section of your blog! We’d be more than happy to do it, and it’ll make every php file in the /wp-admin path even harder to get to. Granted it can be a bit of a nuisance to double login, but not nearly as big of a nuisance as restoring from that backup you took last week right? We can also limit access to the wp-content, and wp-includes directories as well. Plus we can lock it down by IP, or user name/password combos.

2. I know it should go without saying, but please choose hard, random passwords. I know a lot of blogs, my own included, started off really small, and I never worried about getting hacked. My blog never got big, but maybe yours will! Either way, play it safe, and go with a hard password from the get go. That way if your little playground gets bigger one day, or if you land on Digg by accident, you’ll be at least somewhat prepared.

1. While the above options are great for helping secure your instance of Wordpress, there’s one piece of the puzzle that is probably the most important. That piece? Keep Wordpress up to date at all costs! There isn’t an option that can replace this critical piece, because Wordpress being the giant of blogging that it is, is constantly being updated to fix security flaws. Staying up to date is a way of staying ahead of the game, and it’s generally a ten minute ordeal that we’ll take care of for you if you’re a customer of ours! Look at it this way, if you’re running a year old version of Wordpress then you’ve given vandals a year to figure out how to hack you. Why give them that edge? Most Wordpress upgrades are painless, and you know we’ll gladly work with you to schedule it for a time that’s best for your company’s needs as well.

Hopefully this helps answer some question on how to protect yourself from would-be hackers in regards to Wordpress. The fun part is that this applies to quite a few PHP applications in a general sense. Drupal, Simple Machine Forums, and so on can all benefit from these security tips, especially security tip #1! As always customers, drop us a line at support@contegix.com with any question you might have.