Thursday, March 20, 2014

Why I run Adblock Plus and Ghostery...

A few topics came up on my radar recently that questioned whether or not AdBlock Plus is a security risk because several websites are now asking users to disable it for their website and claimed AdBlock Plus is a security risk. This got me thinking about why I really run both AdBlock Plus and Ghostery. I trust both plugins because they do their job VERY well, are generally trusted products by millions of people, and are open source software. However, the reason I run these tools is not the usual "ads are annoying" or "privacy is important" reasons that I see bandied about. I run them because NOT running these tools introduces security vulnerabilities and serious performance degradation into the web browser stack. Here are a few reasons as to why you should be running *at least* AdBlock Plus:

1) Ad server operators are notorious for running any ad, including ads that deploy malware. It is not uncommon for a hacker to use a stolen credit card to flight malware ads on an ad server platform. They send over their malicious creative and it runs without being analyzed. In some instances, the ad runs before payment even clears! If the flighted ad is placed on what is known as a "remnant ad provider", it can take 6 to 8 hours after discovery of the malware to get it taken offline. Meanwhile, the ad is being served up to all sorts of users around the world. This actually happens and it happens because there is no accountability in the ad server world and the people responsible are reactive instead of being proactive. AdBlock Plus (and, to some extent Ghostery) should be considered to be part of a comprehensive security solution beyond what your anti-virus software and hardware firewall solutions offer. This reason alone should be sufficient to immediately install AdBlock Plus (or equivalent) because, if the ad server can't serve anything in the first place, it can't deliver malware to your computer or other devices. These tools reduce the potential attack surface of the web browser.

2) Excessive web requests. Remnant ad servers are especially notorious for this. To request a single remnant ad position, the browser will generally contact an average of 15 different servers across the Internet. Each server request also requires talking to a local DNS server to get an IP address of the destination. If the local DNS server doesn't know the IP address of the target server (fairly common), it has to go and find out. DNS requests are fairly expensive. Throw 3 to 4 ads on a page and suddenly page load times skyrocket to at least 20 seconds per page. I've personally seen page load times in excess of 60 seconds on modern hardware. AdBlock Plus drops page load times to under 6 seconds in many cases by simply blocking the excessive web requests. Ad server operators don't know when to say "no" to money and constantly make exceptions. Therefore, they don't set rules on request depth and, even if they did, they would never stick to such rules because the drive for money outweighs common sense. I also use Ghostery more for the reason of excessive web requests than the "privacy" reasons that other people use Ghostery for - it shaves off another 1 to 3 seconds per page load with very few issues.

3) Those flighting ads also almost always do not know nor have the desire to know even very basic HTML. They will happily flight ads that output broken content onto the page, which then proceeds to destroy the layout of the page. Mismatched 'div's or other bad HTML code results in half of a page simply not loading or loading properly. It then takes up to several hours to diagnose the problem ad and then the ad finally gets taken down. Meanwhile, users suffer with an unusable website. A more stable website viewing experience is just one more reason to run AdBlock Plus.

4) Most ads are not compliant with the Americans with Disabilities Act. Ads that flash, rapidly change colors, have wild patterns (e.g. optical illusions), or otherwise move on a screen can trigger seizures even in those who have never had a seizure before. These triggers are scientifically proven. Therefore, AdBlock Plus is also a lifesaving medical device and brings website operators into some semblance of compliance with ADA regulations. The only ads that are remotely ADA compliant are those that are static images with muted color combinations. But since you don't know nor can control what ads will be served to you, the only solution is to install AdBlock Plus.

5) Animated ads, especially Flash ads, also dramatically hurt browser performance. Moving DOM elements around on a page causes DOM thrashing (for lack of a better term) and redraw operations at the OS level - combined, they take a lot of CPU power to pull it off and frequently lag. Fortunately, some browser vendors are blocking Adobe Flash by default now, but authors of ad creative are just switching to a "Javascript plus images" method, which doesn't help much. The only solution to this problem is to block all ads until the industry wakes up and realizes that animated ads aren't just annoying, they hurt the performance of the user's web browser.

6) Ad server operators don't demand that all ad creative fit in with their website design. It doesn't seem to matter which ad, they all look ugly and destroy what would otherwise be an elegant website design. This stems from no review process prior to flighting any ad. A good review process will reject both ads and advertisers that refuse to meet a set of well-defined requirements that result in ads that look good in relation to the rest of the website. This lack of concern over the ad creative that users will see demonstrates that there is also a lack of concern over the website's users. If a website operator can't be bothered to properly care for their users by only flighting ads that have been through an extensive review process, then AdBlock Plus is a great way to send the message that the users want to be cared about to the website operator.

7) Third-party server dependencies hurt browser performance. If just one third-party server goes offline in an unusual way, pages that depend on the third-party will never finish loading. A lot of sites depend on the "DOM ready" event to fire to execute important changes to the page. If the browser is waiting on some third-party server to return content before continuing and that server hangs for 30+ seconds, I'll generally just leave and go elsewhere. I've seen both ad servers and analytics servers hang for extended periods of time. AdBlock Plus and Ghostery dramatically reduces the number of third-party dependencies, which speeds up page load times while simultaneously helping improve site uptime.

Until all of these issues are addressed by the entire ad and SaaS industries, AdBlock Plus and Ghostery stay installed and active on my hardware.

Saturday, March 08, 2014

Writing software without copyright still needs a license

Let's say for a moment that you are writing some software that you want to release into the public domain. That is, you don't want to claim that you own a copyright on the software. This is very rare to see in the first place, but it does happen. Interestingly, Copyright Law doesn't do anything but protect others from copying and modifying your work. Neither public domain software nor copyright protected software protects the author from lawsuits from damages arising from use of the software. In other words, you still need a license to protect yourself from liability lawsuits.

Unfortunately, it seems like there aren't any OSI approved licenses for software authors that are prepackaged and ready for use with public domain software. The OSI actually doesn't have such a license because it believes it can't correctly define what Public Domain means to the author within the license itself. I disagree with that assessment.

Having researched numerous licenses over the years, I'm very comfortable with various licenses. The MIT license is, in my non-legal opinion (because I'm not a lawyer), the most liberal open source license that's as close the public domain without it actually being public domain. It basically says, "Hey, you can do whatever you want with this software just don't sue me if it causes harm. However, please note that I own the code and you have to include this license and my claim to copyright somewhere in your software." That last little bit is a sticking point if you don't WANT to claim copyright because you want it to be in the public domain.

Toward this end, here is my best attempt to satisfy the concerns of OSI regarding public domain with a modified MIT license:

Modified MIT License for Public Domain software

Public Domain or legal equivalent
Original authorship by [authors] (the "Authors") in [year]

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

The first line of the actual agreement ("Public Domain or legal equivalent") is intended to be treated as a title for the agreement, having legal impact wherever it applies (e.g. United States Copyright Law clearly defines what Public Domain means). If a locale has no such definition, the first line will most likely be overlooked and the second line ("Original authorship by [authors] (the "authors") in [year]") says who is the actual owner of the copyright (i.e. a legal fallback mechanism). "Original authorship" is a clever avoidance of the legally defined word "copyright" in many locales.

The first paragraph therefore will only technically apply to those locales where copyright law has no official definition of Public Domain. However, some or all of the terms may apply regardless. In essence, it clarifies the intentions of the author in regards to their hold on copyright. Should copyright still apply, attribution is included with the license to indicate who the claimants actually are. Obviously someone with intention to place their software into the Public Domain has no intention to ever claim ownership of their copyright in the first place, which effectively neuters any legal effect that the first paragraph might have. The definition of Public Domain is therefore clearly defined by the first paragraph. Whenever and wherever there may be doubt, define what you mean.

The second paragraph is the "covering your legal rear" paragraph. Of the two paragraphs, this is the most important one because it protects the authors from many types of lawsuits. However, wherever copyright law may still apply (and even those locales where it doesn't), the combination with the first paragraph adds extra protection by clearly specifying that the author allows the software to be used for any purpose, thus removing all legal liability (wherever legally allowed). I removed the words that referenced "copyright holders" since, by definition of public domain, there are no copyright holders. However, the use of '(the "Authors")' in the legal fallback mechanism used earlier means that the word 'AUTHORS' is tied to those who hold the copyright (if any). Even if copyright doesn't apply, the word 'AUTHORS' in that specific location protects the authors from legal liability for the software. This really wraps up the entire package, puts a fancy bow on it, and tells lawyers to go away.

It is my opinion that this license represents the closest to public domain that we as software developers can get legally worldwide while staying really far away from liability lawsuits.

---
Permission is hereby granted, free of charge, to any person obtaining a copy of this license (the "License", to deal in the License without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the License, and to permit persons to whom the License is furnished to do so.

THE LICENSE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE LICENSE OR THE USE OR OTHER DEALINGS IN THE LICENSE.
---

Hey, gotta cover my legal rear when writing licenses too. Interestingly, the license itself is a kind of circular reference.

Wednesday, February 26, 2014

Googlebot caught red-handed in Gmail's cookie jar...

Googlebot watches Gmail. I recently bought a new domain name and kept an active search window open for very specific keywords to make sure it didn't go live until I was ready for it to. The people who received the link to the domain are trustworthy to not blab about it (e.g. tweet, post, etc) until I'm ready.

That said, two days ago I sent a link to precisely one person with a Gmail account. This morning, Google search results returned positive for search discovery. Googlebot is definitively watching Gmail for references to new domains and the number of references passed around within Gmail might actually affect PageRank.

This shouldn't really surprise anyone, but it should be something to be aware of if you want to keep a product launch a secret. In my case, it doesn't really matter.

Friday, February 21, 2014

"Dumb Starbucks" - dealing with Copyright, Trademark, Patent Infringement

Over the years, I've had a vested interest in Copyright, Trademark, and Patent Law. As a software developer, having that knowledge at your disposal can be an invaluable tool to avoiding legal issues. We are in the business of writing complex software products, any of which may infringe at any time. Before I begin though, I am not a lawyer and what follows, while good common sense, is not legal advice. If you need legal advice, find a good attorney. I recommend reading the actual laws surrounding Copyrights, Trademarks, and Patents by visiting the various official .gov websites for yourself, which link directly to and have copies of the relevant laws available. If you are a software developer in a country other than the United States, you should make yourself aware of your own country's intellectual property laws. I'm hoping that what I have to say here is just reiterating what you already know as a software developer. What follows is opinion mixed with generally good advice as I follow an ill-conceived idea that popped up on my radar.

Recently I learned about "Dumb Starbucks" from a friend and, immediately, the thought "probable Trademark Law violation" popped into my head and I raised my eyebrows in the "Oh really" fashion. This person proceeded to tell me that the person behind "Dumb Starbucks" was a comedian and that the comedian said "everything was legal". Because comedians are also apparently lawyers, judges, and juries. Oh wait, they aren't. Lawyers can provide legal advice, but only a court of law (with a jury of peers - depending on the case) can decide whether or not something is truly legal. However, it is best to steer really far away from the courtroom as possible. This comedian, who I personally don't think is all that funny (to each their own), apparently wants to visit a courtroom really badly.

I went and watched a few videos on YouTube related to "Dumb Starbucks", including the one where the comedian addresses the legal situation from their perspective. I also went and re-read everything I know on the relevant legal topics. Based on my understand of the laws and precedents set, there's a strong possibility that this infringes on multiple fronts. I wouldn't blame Starbucks Corp if they sued the comedian into oblivion. Here's why:

Let's pretend you are Starbucks for a moment. You have a set of intellectual property (IP) that you have spent time, money, effort, and energy to create. I then proceed to come along and make a business, no matter how well-intentioned, and call my clone of your intellectual property "dumb". The dictionary has the following to say about the word "dumb":

dumb

1. lacking intelligence or good judgment; stupid; dull-witted.
2. lacking the power of speech (offensive when applied to humans): a dumb animal.
3. temporarily unable to speak: dumb with astonishment.
4. refraining from any or much speech; silent.
5. made, done, etc., without speech.

Some or all of these things have now been attributed to your brand, your intellectual property, your stores, your employees, and possibly your customers. There is actually significant precedent in several court cases where the "parody" claim did not fly. A quick Google search turns up (hey, correct use of a Trademark, BTW! Trademarks are correctly used as an adjective before a common noun):

Overview of Trademark Law

Harvard Law school - one would hope that they can understand law, digest it, and then communicate it so that the average person doing cursory research can understand the basics. All four referenced "parody" cases mean that the holder of the mark found sufficient evidence to go to court over the claimed parody. Remember the rule of thumb: You don't want to be anywhere near a courtroom. This means that there was a failure on those with the parody defense to do advanced due diligence. While the above is a summary, I see somewhat of an additional pattern in the cases that Harvard Law school surprisingly seems to have missed. The two cases that were deemed infringing seem to be much more denigrating to the brand than the two that were not deemed infringing. Respectively, the two cases associating the brand with poop and illicit drugs were deemed infringing while the cases regarding semi-nudity and the briefly mentioned name of a character in a movie that's a comedy were not infringing. Hopefully you can see the difference. All of them ended up in court though - a place you don't want to be or unnecessarily risk being in.

The last case struck me as odd because it is widely known that the Muppets franchise is frequently equated to cheeky parody that has its humorous moments, so I went and looked up more information about the case. It looks like Hormel Foods Corp. was concerned about the potential reduced sales of their merchandise due to merchandise that The Jim Henson Company was planning to sell and not the name of the character in the movie itself. Articles on the topic seem to be rather misleading in this regard. The actual reason of potentially lost sales makes more sense than the reason of a character's name in a comedy. However, instead of filing suit and making themselves a public nuisance and embarrassing themselves later as an example on Harvard Law school's website, they should have either formed a business relationship so that they got a (significant) cut of the profits of the merchandise (hey, practically free money from a big brand name!) or sent a cease-and-desist and then recorded losses in the sales of their own merchandise so that they would have demonstrably hard numbers for filing suit later after a reasonable amount of time had passed. The way they filed suit just strikes me as being draconian in their approach to running a business. Don't get me wrong, they are totally entitled to running their business that way, but it will cause them to end up as an example on Harvard Law school's website - as long as they are cool with that, sure go ahead. But businesses should try to attempt to have good public relations.

At any rate, let's get back to "Dumb Starbucks". First off, associating the word "dumb" with the mark is highly denigrating to the brand. Name-calling is never funny - if you called me "dumb" to my face, the conversation might degrade to something less than ideal. Putting that label on the brand in my mind moves it into the same territory as the aforementioned "poop and illicit drugs" cases. In addition, the parody attempt fails to actually say something culturally specific about the brand which the average person can pick out and say, "Oh yeah, that makes sense. Ha ha, that's funny." That seems to be the rough definition of the word "parody" in the context utilized and this attempt fails to meet that definition. Parody is best done as a loosely associated mention with lots of variations on the theme, not outright cloning of a mark and then slapping some minor modifications to it (in this instance, the word "dumb" all over the place). In addition, their claim of parody ironically has this against them:

Photo of a relevant document

The photo is loaded with poorly worded gems and is really all Starbucks Corp needs to take the "comedian" to court and sue him into oblivion over misuse and abuse of their intellectual property. The combination of "marketing purposes" with "not affiliated in any way with Starbucks Corporation" says this is a commercial venture of some sort meant to be in direct competition with the Starbucks brand. That alone puts them in hot water (har har). Second, I'm not aware of any such thing as "Parody Law". However, there are three sets of relevant laws to be concerned with in the United States with regards to intellectual property: Copyright, Trademark, and Patent Law. Within Copyright and Trademark Law, there exist clauses that roughly exclude "parody" via "fair use" but generally leave it up to the courts to decide what those words/phrases actually mean. It is up to a business or individual to bring suit to test the extent of those phrases. Fortunately, they are fairly well tested and defined by easily discoverable precedent. It would seem to me that the lawyers didn't do a good job of due diligence here. Again, you don't ever want to be anywhere near a courtroom.

Third, while they address on that piece of paper that their use of the word "dumb" is not out of "enmity", there is no indication of such anywhere else. This is especially true on the large sign with the modified logo(s) that the average person can see from the street as they drive by. Someone just driving by could confuse it with vandalism or an official marketing ploy by Starbucks Corp. and go inside thinking it was a real Starbucks franchise. I would anyway. If it were truly a parody, the sign would somehow attempt to indicate as such in the same or a larger font size to clearly distinguish that business from the original and official brand.

Finally, the document indicates eventual profits. This will provably cut into the Starbucks Corp. bottom line. In fact, it already has done precisely that by giving away coffee and thus drawing Starbucks customers away to this new, unaffiliated business.

If Starbucks Corp. also holds patents on specific processes for specific coffee preparation techniques, they also have reason for simultaneously filing for patent infringement. Due to the fact that most companies hold patents like nukes in a Cold War (i.e. they don't generally fire them), claiming patent infringement would just be a minor part of a larger suit that Starbucks Corp. might bring to bear against the comedian. It is generally accepted that patents are a legal minefield. Again, avoiding going to court is the best policy but also don't waste tons of time and money figuring out if something will infringe because there are a bunch of junk patents out there too that should never have been issued. My personal policy is to do due diligence but not worry too much - partly because of the whole "treble damages for conducting patent searches" aspect of Patent Law that needs to be removed from Patent Law because it effectively prevents proper searching of the USPTO database (i.e. damned if you don't, triple damned if you do).

There also may be grounds for Copyright infringement, but such would be less likely to succeed than Trademark infringement. It is difficult to explain why I lean in this direction. Copyright protects against outright copying and some derivation of protected works but doesn't necessarily cover trademarks. The only thing that might be Copyright infringing here is the (ab)use of the official Starbucks logo - specifically, the image of the logo can be considered artwork, which is protected by Copyright Law. Copyright generally demands that derivative works be sufficiently different from original works to be considered not infringing. Where that line is varies, but I personally estimate 25% to 75% of the copyrighted content has to be different, which this doesn't really meet the minimum qualification. Exceptions in copyright are made for parody under the "fair use" clauses but, as I stated earlier, I don't see any actual parody taking place and "fair use" in terms of the rest of Copyright has traditionally been isolated to education-ish things. Parody comments on society and its norms - this merely slapped the word "dumb" on an existing logo and started a real business based on it. A more correct use of "fair use" is Wikipedia, which uses a lot of company logos protected under Copyright but claims their use falls under fair use as they are distinctly dedicated to disseminating impartial information about the company whose logo they are using in an encyclopaedic format. Wikipedia's use feels a little awkward but is more in line with both Copyright and Trademark Law and the general understanding of what fair use means and then I combine that with editors who care about correct usage and it leans more toward being legal than infringement, but it does walk the line. This is because I'm sure that if a company demanded removal of their logo from the Wikipedia article about their company, Wikipedia would probably comply and not fight it, but demanding as much would be rather silly since the logo only helps to reinforce the brand in conjunction with impartial facts about the company - for most companies, this just helps their image and market position. While this is my view, it is entirely likely a court of law will see an entirely different take and decide that the artwork modifications are sufficient enough to warrant a denial of a copyright infringement claim. This could go either way and only a court of law can decide. However, risking copyright infringement here is not a smart decision because it has a pretty good chance to land the business in court - again, a place that you nor I want to be and should generally avoid.

Also, according to what I've read, the business was shut down, not by a lawsuit, but by the California Health Department for not being properly licensed. Ouch. While semi-irrelevant, it could be used as part of the arguments of a lack of care for Starbucks customers and the brand itself. Perhaps it is a "parody" by saying that Starbucks franchises don't meet health codes and because of that, Starbucks Corp is therefore "dumb". Starbucks Corp could then counter with something to the effect of, "Furthermore, we hold all of our franchises to the highest standards of quality and require them to meet or exceed all local, state, and Federal regulations, whether they are health regulations or otherwise. Dumb Starbucks did not bother to apply for the appropriate business licenses to operate a coffee shop in the State of California, which further tarnishes the reputation of our organization through the implication that our franchises do not attempt to meet health codes or other laws. We have had to issue statements to counter the damage in public perception of our brand regarding our strict compliance with health regulations." (I'm pretty good at that, aren't I? A lot of programmers I've run into have randomly become lawyers when they grow tired of programming. Apparently, it's a natural progression of sorts. I can't ever see that happening - too much exciting and innovative software is just waiting to be written.)

The comedian should fire their attorneys for not making a recommendation against treading into this very dangerous legal territory. Good legal counsel will give good, sound legal advice, addressing the risks and weighing them against the benefits. The risks here are multiple lawsuits, no financial gain, and significant financial loss. The benefits are either no financial gain (most likely) or barely breaking even since setting up a new business is an expensive proposition. This was a financial lose-lose situation from the get-go, should have been identified as such, and promptly and summarily declared a bad idea and rejected.

Now to bring it home: Once again, as a software developer, you never want to be near a courtroom. It is a waste of your time - time better spent making new and innovative things instead of copying someone else and getting taken to court over it. When we develop software, we have to be careful but still willing to take risks after doing due diligence. If it looks like a logo might infringe, change it - no big deal. Verify the licenses of the third-party libraries being used. Run regular Google searches on terminology. When in doubt, don't. These are all pretty common sense things that are easy to deal with up front, avoid legal hassles later, and you can still work on a whole lot of great software development projects that are innovative and also interesting, fun, and/or exciting.

Also, if you have the attitude of "I'm just going to keep innovating" instead of "I'm going to copy these things that other people have made and make a profit", you'll be way ahead of most programmers and businesses. Lazy programmers and businesses copy, great programmers and businesses innovate. If you are constantly innovating, if you get sued over one product, it won't be the end of the world or your business since you'll have a dozen other products that don't infringe to fall back on. You'll simply just take the infringing software offline, apologize and explain the situation in a positive light to former customers, and move on to the next project. The tendency of potential infringement is fear, but with an attitude of innovation you'll never be affected by it. It is important to just be aware of the various aspects of intellectual property and having a healthy respect for existing brands, doing due diligence to avoid landing in court before developing the product, being hospitable if someone has a legitimate claim, but all the while hoping there aren't issues down the road because, beyond due diligence, hope is literally the best you can do in a world with over 6 billion people all with their own ideas.

Thursday, November 28, 2013

An open source C++ snippet library...

For the past month and a half, I've been delving deep into C++ land, wrestling with the intricacies and annoyances of writing cross-platform code. My biggest gripe with the major libraries out there is that they all have a 'string' class and then proceed to use their 'string' class everywhere, which basically means that if you want to use a specific library, you end up using all of it. I also wanted to push the performance envelope to the limit, which means putting storage on the stack instead of the heap.

At any rate, check it out:

Cross-platform C++ Snippet Library

My favorite class is Sync::TLS, which implements a temporary memory allocator that outperforms system malloc()/free() by a factor of up to 19 times! When I saw the stats, my jaw hit the floor and I drooled a little. It was awesome. I was expecting for Sync::TLS to get crushed soundly.

I'm also rather happy with the little UTF-8 library that's included. A minimalistic Unicode implementation that makes sense, given what I know about Unicode...which is a LOT. It has some really clever conversion tools to get to and from UTF-8 into whatever system calls use (i.e. wchar_t, LPWSTR, etc.) without getting all OS-dependent. The UTF-8 stuff is the only "string" class in the library and I've intentionally severely limited its scope.

The reason I started the library was to have cross-platform, cross-process, named mutexes, semaphores, event objects, and reader-writer lock objects. The 'named' part is critical. Linux is pretty weak as far as named synchronization object support goes, but Windows could use some work as well.

Tuesday, September 17, 2013

Outlook 2007 and later supports 'blink'...

...WHY?!

For Outlook 2007, Microsoft removed the embedded Internet Explorer engine for displaying HTML e-mail and dropped in some half-baked solution. From Word. Apparently, the same engineer (an idiot) who thought that margins and padding in 'div's were bad (because Outlook is now pretty much incapable of that) also decided that support for blinking text is good.

The ancient, decrepit 'blink' tag isn't implemented, so how does one accomplish this atrocity? With 'text-decoration: blink'.

That's right, the same engineers who could have spent their time figuring out how to space things out on a screen with margins and padding, instead wasted those precious moments writing CSS support for blinking text. Sigh.

Okay, so HTML e-mail is bad to begin with, but good software developers don't implement stupid stuff until more important things are implemented. Please prioritize what is important. Got it? Cool.

Saturday, August 03, 2013

What are C and C++ pointers? A simple explanation.

The question constantly comes up. It is a sticking point and everyone has a different answer for new C and C++ developers. There was even a stop-motion animation created once upon a time to explain them (Pointer Fun with Binky). They are just one of the reasons that a lot of developers are cutting their teeth, so to speak, on other programming languages (but there are other reasons as well).

C/C++ pointers.

I've given a lot of thought over the years on the best method to explain C/C++ pointers in a way that can be more universally understood. Even if the programming language you are using doesn't have pointers, understanding pointers helps gain an understanding of how the underlying hardware operates behind the scenes to execute code. In other words, they are still relevant to you. A lot of what I'm going to say involves gross oversimplifications of several concepts, but the goal is to understand how pointers work as succinctly as possible.

First, we need a grid:

That grid represents RAM. A generous 64 bytes of RAM. Each byte is called an "address" - kind of like a street address but using only numbers. 64 bytes is more than plenty to demonstrate why pointers exist and are necessary. Let's say our computer program gets the first 32 bytes of RAM to store data. We'll refer to this as the "heap". The other 32 bytes represents the "stack". The heap stores data while the stack stores variables. Let me draw a dividing line to make it easier:

For the purposes of this demonstration, let's use a really simple C program (the same concepts apply to C++'s new/delete):
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
  char *MyStr, *MyStr2;

  MyStr = malloc(9);
  MyStr[0] = 'P';
  MyStr[1] = 'o';
  MyStr[2] = 'i';
  MyStr[3] = 'n';
  MyStr[4] = 't';
  (*(MyStr + 5)) = 'e';
  MyStr[6] = 'e';
  MyStr[7] = '\n';
  MyStr[8] = '\0';
  printf(MyStr);
  MyStr2 = MyStr + 4;
  printf(MyStr2);
  free(MyStr);

  return 0;
}
You may want to: Run the program

Let's assume this is a 32-bit computer. So 32 bits / 8 bits per byte = 4 bytes per pointer. So, in this example, 4 bytes are requires to store a pointer. In the case of the above program, for the sake of this example, it will put the variables (MyStr and MyStr2) on the aforementioned stack. A stack is generally small and used for temporary, rapid access storage purposes (~1 megabyte per thread). The heap is much larger (e.g. could be gigabytes).

Anyway, the stack is manipulated using "hardware registers" that can hold the size of the bit level of a computer (e.g. 32-bits - anyone noticing a theme yet?). Hold on though, what is a "hardware register"?

CPUs are built to perform operations very fast. Once you enter the world of hardware, you start thinking in terms of transistors, NAND gates, and the whole mess involved there to just make technology work. I'm not going to go into that level of detail. What you need to know is that because things get complicated at the hardware level, a CPU needs to operate in such a way that offers the most amount of functionality with the least amount of silicon in the fastest possible fashion. To do this, CPU engineers expose "registers" to end users (i.e. assembler programmers). There are only a handful of registers in a CPU and all computer programs ultimately rely on them. Assuming a 32-bit Intel-compatible CPU, let's see a few of the common hardware registers:

eax, ebx, ecx, edx - generic multipurpose integer storage, 32-bits
esi, edi - index registers, 32-bits, for performing fast memory moves
ebp - stack pointer top for the current function, 32-bits
esp - stack pointer bottom for the current function, 32-bits

As I said, just a handful are available to computer programs, which makes it pretty impressive that software works at all. On 64-bit Intel processors, there are 64-bit registers such as rax, rbx, rbp, rsp, etc. Basically, the 'e' becomes an 'r', but the concepts of how a program works stays the same and the number of available 64-bit registers is still fairly limited. 32-bit is slightly simpler to work with and understand.

So our program starts up, the 'ebp' and 'esp' registers are initialized to the value of 32 by the OS, because that is the location of the start of our stack. The first line of code that matters, char *MyStr, *MyStr2;, increases the value of 'esp' by 8 (2 pointers * 4 bytes per pointer = 8 bytes):

ebp = 32
esp = 40

We now have something that looks roughly like:

MyStr is really the result of the equation "ebp + 0" (32 + 0 = 32). MyStr is human-readable but it eventually has to end up as executable machine code in order for it to be executed by the computer. This is what a compiler does. It translates MyStr references into 'ebp + 0' and maintains that knowledge during the compilation process.

Moving along. In this example, the OS is protecting the first 32 bytes of RAM. Attempting to access that memory without getting permission first will cause the program to crash. Let's represent this important restriction in red:

The next line of code is a call to malloc(). It takes an integer as input, makes a request to the OS to allocate that much space for the program, and then, assuming all goes well, returns a value representing the address of the start of the allocated memory. I'm going to simply ignore how a function call like malloc() is made. It involves pushing and popping registers (ebp, eax, etc.) onto and off the stack but is also not really relevant to understanding pointers. Continuing along, let's say the OS allocates the requested 9 bytes starting at address 20 as a result of the malloc() call:

At this point, let's assume that the value 20, the address returned by malloc(), is stored in the register 'eax'. The next thing that happens is to store that value into memory:

mov [ebp + 0], eax

Which results in:

Remember that 'ebp + 0' is really the variable MyStr. As can be seen, the value 20 has been stored. Note that the image contains a decimal representation rather than the traditional hexadecimal representation. In hex, it would appear as "14 00 00 00" (1 * 16 + 4 = 20). I figure that just spelling it out is easier to digest. The number is probably backwards from what you might expect. I did mention earlier that this is an Intel CPU. Intel uses the "little endian" format to store values in RAM (mostly due to historical reasons). There are other popular CPUs that use "big endian" format, which would store the number, in hex, as "00 00 00 14". The takeaway is that computer technology is supposed to be fast, not necessarily human-readable.

The next few lines "dereference the pointer". The assembly code might look like:

mov eax, [ebp + 0]
mov [eax + 0], 'P'
mov [eax + 1], 'o'
mov [eax + 2], 'i'
mov [eax + 3], 'n'
mov [eax + 4], 't'
mov [eax + 5], 'e'
mov [eax + 6], 'e'
mov [eax + 7], '\n'
mov [eax + 8], '\0'

The register 'eax' gets the value of 20 from the 4 bytes of memory stored at [ebp + 0]. Next, 'P' gets stored at address 20, 'o' gets stored at address 21, and so on.

Which results in:

When someone is asking "How do pointers work?" they really are asking, "How does the computer and my program know where my data really is?" The value stored in MyStr in this example is an address to another location in RAM. The program knows where MyStr is stored because of the value stored in the 'ebp' hardware register. The lack of understanding how the hardware works is why the question is so frequently asked.

I broke out the first 'e' being assigned (i.e. (*(MyStr + 5)) = 'e';) as being different from the rest of the code to demonstrate exactly how so-called 'arrays' really work since most new C/C++ programmers learn about arrays early on. Note that the assembler output is identical (i.e. mov [eax + 5], 'e'). All an array access does is add one value to another to get a new address (20 + 5 = 25) and then dereferences it. MyStr[5] is a shortcut to the long-form of (*(MyStr + 5)). Dereferencing is a fancy way of saying "access the value at this location in memory" and the operation either retrieves it from memory into a register or stores a value into memory either from a register or a hardcoded value. In this case, the code is storing the zero-terminated string "Pointee\n" by storing each character individually.

The next line calls printf() (dangerously - don't do this) and outputs the zero-terminated string that was stored earlier. For this example, what is passed to printf() is the value 20, not the actual string. printf() then dereferences, starting at 20, and outputs characters until it runs into ASCII character 0 (hence the name zero-terminated string).

Take some time to digest everything so far. It is a lot to take in. Compare the source code to the assembly code to what happens on the grid. Everything you need to know about pointers is there. What follows is just more of the same.

The next line of code assigns MyStr2 the value stored in MyStr + 4:

mov eax, [ebp + 0]
add eax, 4
mov [ebp + 4], eax

Which looks like:

The next line makes a printf() call using the value of 24 instead of 20.

The next line of code calls free(). The value 20 is passed to free(), which, in turn, releases the memory to the OS so that it can be used for other purposes.

If you look carefully, you can still see the value that was stored and the address 20 is still stored at 'ebp + 0' and 24 is still stored at 'ebp + 4'. And, depending on the compiler library that manages memory, that memory may still be readable, but you should never attempt to access such memory because random crash bugs will crop up. Memory management behind the scenes gets weird and therefore some crash bugs become very difficult to track down.

And that, my friends, is how pointers work.

Disclaimer (for nerds): I realized, after I had generated all the images and most of this post, that I had messed up how stacks really work on Intel hardware. Stacks grow down, not up, which means everything should have been 'ebp - 0' and 'ebp - 4', but I contest any naysayers that being 100% accurate would have resulted in more confusion. The whole little endian vs. big endian thing was actually a tossup, but, if someone is going to be working with pointers, they'll eventually care about that sort of thing, whereas knowing how a stack grows is a lot less important. And I was too lazy to regenerate all of those images.