Wednesday, July 14, 2010

Fixing slow Apache on localhost under Windows 7

A couple days ago, I documented my recent experience with my wireless network and how I got hacked. I briefly mentioned that I'm installing Windows 7 this time around.

Yesterday, I ran into an issue with 32-bit Apache 2.2.15 running on Windows 7 Ultimate 64-bit. This issue appears to only affect 64-bit Windows 7 and 64-bit Windows Vista web developers attempting to run 32-bit Apache on the system.

NOTE: I didn't test 64-bit Apache because that is experimental and 64-bit PHP is even more experimental. However, I doubt the results would be too different.

The issue is slow response times (anywhere from 1 to 3 seconds per request) when connecting to 'http://localhost/'. Connecting to 'http://127.0.0.1/' and 'http://[NetBIOScomputernamegoeshere]/' have fast response times. I've seen various fixes around that seem to boil down to these three:

- Disable the Windows Firewall.
- Disable IPv6 support.
- Edit the 'hosts' file.

It looks like there is a bug in IPv6 DNS mapping for localhost under Windows that causes the slowdown when used in conjunction with the Windows Firewall. If Microsoft employees are reading this - THIS IS A BUG...FIX IT! Apache doesn't seem to be the issue but it could be that 'localhost' via the firewall first tries an IPv6 address first, fails after a second or two, and then falls back to IPv4 at which point a connection to 32-bit Apache is established. Apache isn't the issue here - the Windows Firewall combined with IPv6 is. Anything 'localhost' only listening on IPv4 will have slow responses.

What worked for me was to use the IPv4 version of 'localhost' in my hosts file.  The 'hosts' file is usually located in the 'C:\Windows\System32\drivers\etc' directory on most systems. Editing the 'hosts' file was a pain in the neck. You can't simply right click or double-click to open the file (BUG! ALSO NEEDS TO BE FIXED!). You first have to fire up Notepad or your favorite editor using "Run As Administrator" (right-click) and then hunt for the file using the "File -> Open" dialog. Windows prevents editing that file normally (a good thing) but then get in the way of actually editing it (a bad thing).

Anyway, under Windows 7, the following two lines are commented out:

# 127.0.0.1 localhost
# ::1 localhost

Uncomment the first line but leave the second commented:

127.0.0.1 localhost
# ::1 localhost

Save the file. Try loading pages in Apache again. Should be back to normal speed.

Why this works?

The 'hosts' file supersedes all DNS mappings. So this "fix" merely forces the IPv4 mapping to occur when 'localhost' is requested. Uncomment the second line and Apache will slow down again.

Disabling the Windows Firewall supposedly fixes the issue but I'm pretty sure no one wants to run without a firewall. Disabling IPv6 also apparently fixes the issue but IPv6 is supposedly the future - IPv4 is what the OS falls back on without IPv6. The 'hosts' file fix is really the most desirable until Microsoft can figure out why the Windows Firewall is broken.

Hopefully this helps someone else and helps provide insight into the actual problem.

Side note: If you want to simplify future installations of Apache and you also use MySQL and PHP, you might want to try my portable version of Apache + MySQL + PHP for Windows.

Monday, July 12, 2010

My wireless network got hacked. Unremovable rootkit? New botnet tactic?

Edit (July 6, 2013): It has taken me almost three (3) years to come up with a secure WiFi solution since I first wrote and published this article. It is my professional and personal opinion that, outside of running 200 ft. of Ethernet (which I actually did), only a WPA2-Enterprise AES w/ EAP+TLS setup is secure. My hope is that you come to the realization that your own WiFi network is not secure as you read my story below.

I take security VERY seriously and violations of that security even more seriously. Hacking my personal networking infrastructure is near impossible. Or so I thought.

First, some background. I run...well, I used to run a wireless network access point. Yes, it is one of those consumer-grade, wireless network setups because I'm a cheapskate. The brand doesn't matter. Here's the critical bit of information: I ran the wireless access point with WPA-PSK (TKIP) using a completely random key of about 40 characters in length and a different SSID from the default plus MAC address filtering. Standard WPA-PSK cracking tools were theoretically never going to gain access to my personal network. I also had the benefit of being surrounded by quite a few open wireless networks. I figured those networks would be picked over mine. Plus, I don't usually turn on my laptops very often - so passive traffic sniffing would make it a nightmare to gain access. Basically, the worst possible set of conditions for anyone hacking my network - psychologically undesirable, physically undesirable, hard to crack, and I monitor my network and computers like a hawk.

I hereby declare my original article on setting up a "secure" wireless network as Dead On Arrival (DOA). It has become rather painfully clear to me that WPA-PSK is completely broken - somehow. Searching Google turns up nothing other than some basic articles utilizing mostly dictionary attacks. Someone, somewhere has figured out how to hack any secured WPA-PSK wireless access point. The only protocol left is WPA2-PSK. And I doubt it too will remain secure if it hasn't already been equally broken. Consider all wireless access points to be huge vulnerabilities in your network.

Thankfully I was sitting in front of my computer when it happened. Then again, I rarely step away from my computer when it is turned on. I noticed that I lost keyboard and application focus three times over the course of two minutes. While I was editing a Powerpoint presentation. I was curious after the first time, concerned the second time, and went to fire up Task Manager after the third time. Right before I fired up Task Manager, Internet Explorer started on its own. I don't run IE these days for various reasons - mostly because Firebug exists.

At that point, I knew I had been hacked. I glanced over my active applications I had open to memorize what was active and immediately shut down the computer. My data is critical to me and therefore must remain secure and out of the hands of would-be thieves. If the computer is off, the most the enemy would get is a few megabytes. However, the person seemed more interested in dumping additional software on my computer than seeking out my data. Probably to solidify their position on the box.

I happen to keep around a few Rescue and Live CDs for just-in-case scenarios like this. I ran the Rescue disks (F-Secure and BitDefender). As of July 2010, F-Secure has the edge in terms of ease-of-use. BitDefender seems to be lacking on the networking front - it couldn't find my Ethernet cards, so I couldn't get the latest definitions.

I also have a netbook with Ubuntu on it, which I fired up right away and started doing research while the rescue CDs did their thing to find malware on the system. I tried to figure out what I might have done in the previous 24 hours to allow for this to happen. I determined that it was impossible for it to have been me unless the MySQL developer's website was somehow compromised. Then I started thinking of the absurd...and my first thought was my wireless network. I immediately logged into the router, flew to the active wireless network connection list and, lo-and-behold, not five minutes before, someone had gotten onto my wireless network coincidentally around the time my Windows computer had been violated. I foolishly forgot to take a screenshot because my knee-jerk reaction was to shut down the wireless radio. A reasonably wise decision on some level. A screenshot would definitely help the credibility of this blog post but I was in full-on panic mode at the time. I did disconnect the Ethernet cables except the one to the Ubuntu netbook and turned the wireless radio back on for a couple minutes to see if I would get any takers but eventually decided the risk wasn't worth it and shut the radio back down for good. (I also needed to go to the bathroom at that point.) It is also important to note that I'm pretty sure they only had access to my network for a few minutes. I'd probably leave too and not return if I noticed a Linux box come online shortly after I deployed a rootkit and the wireless radio mysteriously vanish shortly after that.

The F-Secure Rescue CD picked up on a backdoor rootkit and a piece of spyware and supposedly eliminated them. BitDefender picked up on my own personally developed software - VerifyMyPC - because it has both registry and file system scanning capabilities. False positive there. Again, as of July 2010 - F-Secure has the lead in building Rescue CDs with solid detection. But apparently that isn't good enough.

I spent the next couple days simply scanning the system with software I trusted - including installing AdAware, which now appears to have a real-time component. Bleh. The rootkit apparently hid itself quite well during the time AdAware was running. I eventually uninstalled it just to see if I was in the clear. It took a couple reboots but the rootkit and all that came with it returned with a vengeance. I shut down the computer again and began my plans for reinstalling Windows.

I knew I was pretty much hosed at that point. Reinstalling an OS is the ONLY way to know that you have truly gotten rid of a rootkit.

By the way, Live CDs of Linux are awesome. I highly recommend burning a copy of the Ubuntu Live CD. It works VERY well for recovering from failed Windows installations. Or environments with rootkits installed. Plus it seems to have every device driver on the planet included, which means networking and pretty much every USB thumbdrive and external hard drive will work right away - so backing up data prior to a reinstall is really easy.

Fortunately, I keep very good backups. Microsoft SyncToy saved my butt. I also eventually started keeping all my core data on a separate hard drive for the just-in-case scenario I need to reinstall Windows. Doing this helps to lessen the painful process.

On the upside, I've been meaning to install Windows 7. So this merely forced the issue of upgrading. Also, during this reinstall, I've made the decision to use as many "portable applications" as possible. I really miss one aspect of the old DOS days where each application was centrally contained in its own directory. Any application that did anything outside of the directory it was installed to was deemed to be a bad application. Windows came along and messed up my beautifully organized world of self-contained applications. With the advent of the USB thumbdrive, some good people have taken it upon themselves to recreate a world of self-contained, well-behaved applications again under Windows. By using portable applications, I have a pretty good chance of easily installing newer versions of Windows in the future.

There are a number of applications I use extensively that I hate reinstalling: Firefox, Thunderbird, Apache, PHP, MySQL, and Visual Studio. Standalone Visual Studio is a pipe-dream. But the others are definitely doable. Firefox and Thunderbird both have "portable" versions. I use both of those more than anything else these days. And the portable versions seem to operate rather well thus far.

At any rate, I'm left wondering what exactly the purpose of dropping a rootkit onto my computer was. I'm also pretty sure that my neighbors aren't intelligent enough to use the hardware and software tools required to hack a "secure" wireless network. But you never know for certain. On top of that, there are several insecure wireless networks in range of my own network. Low-hanging fruit and all that aside, someone put forth effort to gain access to my "secured" wireless network.

Given that my neighbors aren't likely to have done thins, it leaves me with two options - rogue government operation or botnet. Let's shave paranoia with Occam's Razor - if the government really wanted to access my computer, there's not much I could do to stop them up front. Which brings me to my question: Is this the next generation of botnet tactics? Growing botnets by seeking out both insecure and secure wireless networks? I'm still not sure how my completely random WPA-PSK key was obtained but, with the processing power of a botnet at one's disposal, I can conceive that it would be possible to hack such networks. Once inside, seek out Windows boxes, expand network, wash, rinse, repeat. Each new computer on the botnet means that much more processing power, thus shortening access times to keys.

This definitely has the smell of a botnet. Botnet operators are predictable creatures of habit. When they gain access to a new machine or network, they immediately deploy a rootkit on the system (check), deploy spyware/additional tools (check), then start seeing what is on the system that is of value (probably check?), and start transferring things of value (maybe 1MB at most - if that - pretty sure they never got that far). The way the rootkit came back and something started talking again to a remote system really says "botnet" to me. In a sexy, nerdy/geeky sort of way. Interestingly, none of the anti-virus and anti-malware products truly cleaned the system up. Even the Rescue CDs missed whatever was installed, which means unknown, undetectable malware. Even if I had real-time anti-virus software installed, there was a good chance it wouldn't have detected the installation of the malware.

Prior to the installation of Windows, I wiped my BIOS and reinstalled it - I keep a copy of my current system BIOS around as well just-in-case - and also wiped the CMOS settings. It depends on the rootkit, but they occasionally inject themselves into the BIOS. If anything survived all of that, I'll be impressed but slightly annoyed.

My only conclusions are thus: All consumer wireless networks are insecure and running Windows on a wireless access point is just asking for a rootkit and botnet to get installed.

In a future blog post, I will share a solution to this problem that will allow Windows users to safely operate on a wireless access point. Assuming my idea works. Until then, turn off your wireless radio and use Ethernet cables. Data transfer is significantly faster over Ethernet anyway.