Saturday, June 20, 2015

How to call select() - the CORRECT way!

There is a TON of broken code out on the Internet with lots of programmers who enter the world of TCP/IP socket development and think they have figured out how to write socket code. They then disseminate their broken code to others who, in turn, disseminate broken code to other people.

One of the most egregious problems plaguing the world of software development today is the use, or abuse, of select(). Today, you are going to contribute to fixing this problem once and for all by reading what I have to say and then ingraining it into your brainz.

There are two types of file descriptors/sockets/what-have-you:

Blocking and non-blocking. Sometimes referred to as synchronous and asynchronous.

If you are using select() on synchronous sockets in your code, you are doing it wrong!

select() is ONLY for asynchronous sockets. Think of it this way: A synchronous socket is you telling the OS that you know the exact order of operations on that socket (e.g. POP3) and are willing to wait until hell freezes over for that read/write operation to complete.

Read that over again and you should come to the same conclusion: Calling select() on a synchronous socket is WRONG. Although, if you've been doing it wrong for decades, this fact becomes a lot harder to accept.

Where does this misunderstanding come from? A lot of people misunderstand select() because the book/teacher/website they learned *NIX Socket Programming from got it wrong because they learned the wrong approach from someone else. select() on a synchronous socket introduces bugs that are hard to trace and happen randomly. Also, most socket programmers start out using synchronous sockets with simple client-side applications and then later want a way to handle multiple sockets at one time. select() is referenced all over the manpages/MSDN and, when the programmer reads about it, it sounds like it will work, so they try it and it seems to work. That's the real problem: select() seems to work, which is why programmers use it improperly.

select()'s purpose in life is to offer a way to not have a busy loop in an asynchronous environment since no read/write operation will ever block. It is entirely possible, if you pass in a synchronous descriptor to select(), that select() will indicate that the socket is readable but when you go to read data, the synchronous socket will block. You might say that can't possibly happen but guess does happen! This is why select() being only for asynchronous sockets makes much more sense. Once you learn this, the code for asynchronous sockets becomes surprisingly cleaner and is only marginally more complex than synchronous socket code. If you ever thought your synchronous socket code using select() was kind of hacky/messy, then you now know why. This is a harsh lesson to learn for many people.

Therefore, to process more than one descriptor per thread at one time, use asynchronous descriptors and select().

The best way to fix this entire problem would be for select() to determine if a descriptor is synchronous and then reject it outright. Such a solution would break every major application and then, lo-and-behold, everyone would fix their code! The world would then have less broken code and we'd all be happier.

Friday, June 19, 2015

Dear WebSocket, 1980 called and wants its text mode back among other things

This is a mostly tongue-in-cheek response to RFC 6455, which defines the WebSocket protocol, which I recently built a client for and can be found in the Ultimate Web Scraper Toolkit. Certain things annoyed me.

Dear WebSocket,

FTP called (RFC 765, circa 1980) and wants its text mode back. Please return it to the nearest Internet Engineering Task Force (IETF) member as soon as possible. You may have shined it up a bit with UTF-8, which was basically designed on a napkin. Of course, Unicode has never had any implementation problems with it whatsoever. Ever.

Your masking key is for clients only and not even being optional for servers defies the core Internet tenet of being liberal with what you accept, strict with what you send. Technically, both are peers and therefore both are clients of each other since you are, after all, bi-directional and the client could easily function as a server after the connection is established. Because this has absolutely never ever been done before.

Your comments about the Origin header only being sent by web browsers enables differentiation from an automated script and a web browser. Or does it? No, it does not. For I can simply send to the target whatever Origin header I want and therefore look like any web browser of my choosing. Your false defenses are no match for my Kung Fu.

Your reserved bits per frame are silly and probably full of arbitrary limitations. From the comfort of my keyboard and screen, I fart in your general direction.

Your required closing packets are unnecessary and most likely a security risk with anyone assuming that such frames will ever be sent. TCP already has built-in shutdown mechanisms. Ping/pong frames are more than sufficient.

Since it is important to always remember the childrens, you get an F--. Enjoy.


The portion of the Internet that actually knows how to design rational communication protocols.