I've discovered the perfect solution to removing advertisements from 99% of all websites. Well, I've known about the solution for a long time but only last night found the last piece to the puzzle.
A little known aspect of Windows computers is the 'hosts' file. Linux people use this file all the time for various purposes just because Linux is all about editing configuration files. Gag. Anyway, I've known for some time that I can eliminate advertisements on the web by poisoning my DNS cache via a modified 'hosts' file.
At this point you are wondering about various things like, "What is DNS?" "What is a DNS cache?" "What is DNS cache poisoning?" I'm so glad that you are asking smart questions like those instead of clicking the 'back' button. DNS is short for Domain Name Server. Basically, when you type in www.yahoo.com, a query is made to your ISP's DNS server to obtain an IP address for www.yahoo.com. Usually the ISP doesn't know what address a DNS server is, so it has to look it up. Usually an ISP has an ISP above it and one above that and so on. Obviously there has to be some sort of root server. There are actually 13 root servers:
http://en.wikipedia.org/wiki/Root_nameserver
If you want to play around with lookups to learn how they operate, go to a command prompt and type in 'nslookup'. That connects you to your ISPs DNS server. You can tell nslookup to switch to the root servers by typing in 'root'. The root servers only contain information on how to find the DNS server responsible for a domain. So a root server query is only the starting point to obtaining an IP address.
IP addresses are how the Internet talks to each other. Basically a web browser and e-mail client and FTP client and so on are all sugar-coating applications that wrap up protocols that use IP addresses to identify the servers that are sitting on the Internet.
DNS caches are how the Internet keeps from getting bogged down with requests for name to IP mappings. Basically, each host in the DNS chain keeps track of the name to IP addresses it has already resolved. Of course, this introduces the problem of making sure everyone gets updates. Sometimes hosts change IP addresses, which means the old name to IP addresss mappings are no longer valid. Each DNS entry has a Time To Live (TTL) associated with it.
DNS cache poisoning is where a name is intentionally mapped to the wrong IP address. This is usually done by attackers. Modern-day attackers have used this approach to poison DNS caches to execute phishing scams. However, intentional poisoning can be useful as I will address in a moment.
The 'hosts' file is one of the first places Windows looks when it tries to resolve a DNS entry to an IP address. The 'hosts' file is a name to IP address mapping and usually only contains a single entry:
127.0.0.1 localhost
Pinging 'localhost' via the command-line 'ping' command displays:
ping localhost
Pinging MyHost [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Ping statistics for 127.0.0.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Windows looked up 'localhost' in the 'hosts' file and mapped it to 127.0.0.1. With this knowledge, it is possible to map any host name to 127.0.0.1 by editing the 'hosts' file.
Luckily, several other people already do a good job of this:
http://en.wikipedia.org/wiki/Hosts_file
(See the external links at the end - I prefer Mike's Ad Blocking host file over the others.)
However, what has held me back up to this point is that simply having a 'hosts' file is not good enough. This is because ads show up either as broken images or take forever to load. Or, for those of us who do local web development, ugly "HTTP 404" error messages from our running web server. I'm more concerned about that latter part - someone could carefully craft a website designed to exploit a localhost webserver.
The solution to this problem is to use a tool called eDexter (http://www.pyrenean.com/) in combination with a modified host file. The modified host file poisons the DNS cache and then eDexter handles the actual web request and returns an image (or bogus .js or empty .swf file). The images, unfortunately, are not completely transparent (a couple pink pixels in the middle). So, changing them to completely transparent GIF images is a good idea. eDexter is a special web server designed to handle poisoned DNS caches that point to localhost by returning an image. eDexter is, as of this writing, the best tool for the job (the other free tool is nohttpd but lacks the feature set of eDexter).
For those who run a web server locally, the solution is to move the local web server to an alternate port (e.g. port 81) or make the server start manually instead of every time the computer boots. eDexter, unfortunately, has to reside on port 80 and has no way to configure it to operate on another IP address (e.g. 127.0.0.2) The IP address range 127.0.0.* is reserved for the local computer.
This combination of a hosts file with eDexter is quite impressive and powerful. Combine that with the Google toolbar (toolbar.google.com) and every advertisement is gone. Instantly. A very nice solution to a major annoyance to software developers. Nothing is more annoying than experiencing popups and intrusive ads when hunting for a solution to a coding problem.
A little known aspect of Windows computers is the 'hosts' file. Linux people use this file all the time for various purposes just because Linux is all about editing configuration files. Gag. Anyway, I've known for some time that I can eliminate advertisements on the web by poisoning my DNS cache via a modified 'hosts' file.
At this point you are wondering about various things like, "What is DNS?" "What is a DNS cache?" "What is DNS cache poisoning?" I'm so glad that you are asking smart questions like those instead of clicking the 'back' button. DNS is short for Domain Name Server. Basically, when you type in www.yahoo.com, a query is made to your ISP's DNS server to obtain an IP address for www.yahoo.com. Usually the ISP doesn't know what address a DNS server is, so it has to look it up. Usually an ISP has an ISP above it and one above that and so on. Obviously there has to be some sort of root server. There are actually 13 root servers:
http://en.wikipedia.org/wiki/Root_nameserver
If you want to play around with lookups to learn how they operate, go to a command prompt and type in 'nslookup'. That connects you to your ISPs DNS server. You can tell nslookup to switch to the root servers by typing in 'root'. The root servers only contain information on how to find the DNS server responsible for a domain. So a root server query is only the starting point to obtaining an IP address.
IP addresses are how the Internet talks to each other. Basically a web browser and e-mail client and FTP client and so on are all sugar-coating applications that wrap up protocols that use IP addresses to identify the servers that are sitting on the Internet.
DNS caches are how the Internet keeps from getting bogged down with requests for name to IP mappings. Basically, each host in the DNS chain keeps track of the name to IP addresses it has already resolved. Of course, this introduces the problem of making sure everyone gets updates. Sometimes hosts change IP addresses, which means the old name to IP addresss mappings are no longer valid. Each DNS entry has a Time To Live (TTL) associated with it.
DNS cache poisoning is where a name is intentionally mapped to the wrong IP address. This is usually done by attackers. Modern-day attackers have used this approach to poison DNS caches to execute phishing scams. However, intentional poisoning can be useful as I will address in a moment.
The 'hosts' file is one of the first places Windows looks when it tries to resolve a DNS entry to an IP address. The 'hosts' file is a name to IP address mapping and usually only contains a single entry:
127.0.0.1 localhost
Pinging 'localhost' via the command-line 'ping' command displays:
ping localhost
Pinging MyHost [127.0.0.1] with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Ping statistics for 127.0.0.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms
Windows looked up 'localhost' in the 'hosts' file and mapped it to 127.0.0.1. With this knowledge, it is possible to map any host name to 127.0.0.1 by editing the 'hosts' file.
Luckily, several other people already do a good job of this:
http://en.wikipedia.org/wiki/Hosts_file
(See the external links at the end - I prefer Mike's Ad Blocking host file over the others.)
However, what has held me back up to this point is that simply having a 'hosts' file is not good enough. This is because ads show up either as broken images or take forever to load. Or, for those of us who do local web development, ugly "HTTP 404" error messages from our running web server. I'm more concerned about that latter part - someone could carefully craft a website designed to exploit a localhost webserver.
The solution to this problem is to use a tool called eDexter (http://www.pyrenean.com/) in combination with a modified host file. The modified host file poisons the DNS cache and then eDexter handles the actual web request and returns an image (or bogus .js or empty .swf file). The images, unfortunately, are not completely transparent (a couple pink pixels in the middle). So, changing them to completely transparent GIF images is a good idea. eDexter is a special web server designed to handle poisoned DNS caches that point to localhost by returning an image. eDexter is, as of this writing, the best tool for the job (the other free tool is nohttpd but lacks the feature set of eDexter).
For those who run a web server locally, the solution is to move the local web server to an alternate port (e.g. port 81) or make the server start manually instead of every time the computer boots. eDexter, unfortunately, has to reside on port 80 and has no way to configure it to operate on another IP address (e.g. 127.0.0.2) The IP address range 127.0.0.* is reserved for the local computer.
This combination of a hosts file with eDexter is quite impressive and powerful. Combine that with the Google toolbar (toolbar.google.com) and every advertisement is gone. Instantly. A very nice solution to a major annoyance to software developers. Nothing is more annoying than experiencing popups and intrusive ads when hunting for a solution to a coding problem.
Dear Thomas
ReplyDeleteMany thanks for eDextor.I have been having these pop up ads from past 6 months on my home pc.I installed eDextor it stopped displaying ads when I connect to internet, however, my browser window still opens with a blank page or page cannot found.how do i stop opening theses browser windowa?it's very annoying. any ideas would be much appreciated.
Thanks very much.
Sreenath
Sreenath,
ReplyDeleteI assume you are also using a modified 'hosts' file. eDexter works in conjunction with a 'hosts' file. You can change what page your web browser starts at by going into "Tools->Internet Options..." under IE. The address to start at is the first thing you will see. One possibility is that you have spyware installed. I recommend a combination of Ad-Aware and Spybot Search & Destroy to remove spyware. Next, check your computer with viruses (one of the best scanners is ClamWin). And then make sure everything is stable with VerifyMyPC. Search Google for these various products.
I forgot one other thing, I didn't write eDexter. This blog is a great resource to learn about the best tools to use. I am relatively unbiased when it comes to software - I enjoy finding the best product there is in any given category. I'm never content with the status-quo and therefore experiement with other applications.
ReplyDeleteHi Thomas,
ReplyDeleteThanks a lot for the quick reply. Yes, I just realised that spyware has been installed on my pc. I have downloaded Ad-Aware and Spybot Search & Destroy. They are very good.Now my pc is very clean.Thanks again for all your suggestions.
Sreenath
Screw IE, too much hacking required, and even you will admit there is a limit to the number of hacks until you must just redo everything. I use Firefox with AdBlock with FilterSet.G, RefControl, CustomizeGoogle, Tor, SwitchProxy, NoScript, IEView, PassiveCache, and SessionSaver. Might as well switch to Linux or something while you're at it.
ReplyDeleteI too vouch for Firefox - with a pop-up blocker built in, and two add-ons (adblock plus and adblock plus filterset.g updater) I don't see any ads anymore... They're not even downloaded, which means all pages load lightening quick.
ReplyDeleteI go a bit farther in my hatred of ads - I find ways to block the text ads as well. Stylish (Firefox add-on) is a CSS editor I use to hide the text ads I run across. I see them once and they're gone.
amandakerik at yahoo dot com
I prefer the DNS cache poisoning method because it works across multiple web browsers. Whatever browser I'm using at the moment is what I'll tend to surf with. I prefer IE6 because FF crashes _a lot_ and IE7's menu bar can't be moved above the address bar and I can't be bothered to work on browser source code. Opera is an interesting browser but I don't use it very often for some reason I can't put my finger on (probably all the color fade to yellow effects).
ReplyDeleteTo me a web browser is a web browser is a web browser. The only issues I care about are: Can I read the page? Does it look good? IE's rendering engine is, IMO, better than any other _visually_. (Don't even try to argue the point. I'm an artist and usability exper). And IE doesn't crash nearly as often as Firefox does (Actually, FF doesn't crash as often as it just freezes and refuses to close until it is killed by Task Manager). Haven't seen a web browser yet that doesn't crash.
Anyway, DNS cache poisoning works best for me and the way I work and eliminates most/all web advertisements under all web browsers. If you just use one web browser like Firefox, then good for you and your magical plugins. I wrote the article for people who are not you. Take your FF (2.x) propaganda elsewhere. I'm not the least bit interested even though I use Firefox. Don't ever tell me what I should or should not use as a web browser. I will determine that on my own. I use all web browsers to some extent (sometimes even Lynx) but mostly stick with an IE, Firefox, Opera combo.
Feel properly chewed out? You should. I'm the only one on this blog that is allowed to recommend products. Go recommend products on your own blog. If you want me to try out a product _YOU_ authored, then pick the most recent post and write there.
Hey,
ReplyDeleteI just wanted to thank Thomas for this blog. And the previous comments regarding Firefox.
.exe files scare me, but with the add ons to Firefox, advertisment free surfing!
So, don't freak out Thomas. You never know who's floating by and what they might find of value in others comments.
Cheers!
(Now I'm gonna go play with Linux, because Vista sucks.)
I use a MVPS hosts file and Opera browser with really super-simplified settings. Thanks for the article, but I really think you're being touchy about the guys above, they are only expressing opinions.
ReplyDeleteFinally, in the words of... er.. Conan the Barbarian, "... and if you do not listen then to hell with you!"
The MVP host file is pretty good too. I just use the other one because that's what I came across first.
ReplyDeleteAnd I prefer Conan the Librarian. And flying poodles. And red snappers. Mmm...verwy tasty!
a modified hosts file is wonderful.
ReplyDeletehowever if you use emusic, you have to comment out several addys in the common modified hosts files or else the emusic site will not load.
unfortunately, i have not found all of them yet (lack of time), so i temporarily switch back to unmodified hosts file when accessing emusic, grrr.
Okay, this is all over my head! lol I'm sure people who are more into computers than I am can understand what you're saying but this middle age person can't. I'll have to google again and see if I can find a website that has step by step instructions for a person like me who doesn't have a clue what you're talkig about even when it looks like you're trying to explain it. Haha. Thanks anyway.
ReplyDeleteI'm sorry to hear that this is over your head. I thought I had made the concept simple enough:
ReplyDelete1) Poison the DNS cache, which redirects web requests for ads to the local computer.
2) Since there is no web server on the local computer, ads appear as broken images.
3) To fix the broken image problem, use the specialized web server eDexter, which is designed for this very purpose.