I got a great question from a reader of my sockets book recently. He asks:
[..] as a newbie both to Ruby and to network programming, [..] "What sort of things will you be able to build after reading this that you couldn't build before?" or, more simply, "What can you do armed with this knowledge?"
This is a great question. Some of us, myself included, just want to know how things work. Knowing how our tools work has gotten us out of situations that would otherwise have us banging our faces against our keyboards.
But some people haven't felt that pain yet. Even for those of us who have, why learn about socket programming? There's an infinite amount of things that to be learned for the modern software developer, so why bother?
Here are a few things from my own experience where I've benefited from understanding sockets:
1. Client libraries.
My introduction to socket programming involved writing a client library for a DNS registrar API. When I asked for the documentation, they told me the endpoint to connect to and provided me with a PDF outlining their custom API protocol. With this as my guide, I stumbled through the project. Since this was my introduction to socket programming, there was a lot of trial-and-error just getting connected to the endpoint. At the time, I didn't understand a lot of the methods I was using, but thankfully that client library is no longer in use :)
I've written a client library or two in the time since and had a much more fruitful, and enjoyable, experience given the knowledge I had built about socket programming.
There are lots of opportunities to write client libraries. When a new storage engine hits the scene, it often requires a socket-based client library. Think of redis, memcached, mysql, Tokyo Cabinet, etc.
If you want to be able to connect to some piece of infrastructure, and no client library exists yet, that's a perfect opportunity to write one.
2. Tuning web servers.
Given the abundance of great web servers in the Ruby community, most of us will never need to write our own (but that can be fun!). Unless you have a devops team that shields you from your infrastructure, you're probably responsible for deploying applications.
If you're responsible for deploying apps, you're also responsible for choosing the right pieces of infrastructure and ensuring they're tuned to handle the load.
Let's say you're deploying an app and decide that Unicorn is the right choice as a Ruby web server. Like other web servers, it provides sensible defaults so you typically don't need to worry about tuning. But when things go awry, or you start to handle more traffic, tuning becomes necessary. For example, Unicorn allows you to configure the size of the listen queue. You may want to decrease this so that failover can happen promptly when it's needed. You can also turn on and off certain socket options like TCP_NODELAY, TCP_NOPUSH, etc. Understanding how nginx interacts with Unicorn, where queueing happens, the effect that a client disconnecting has on nginx -> Unicorn can be crucial under heavy traffic (speaking from experience here).
All of these things can be understood once you've got a handle on the basics of socket programming.
3. Backdoor for data processing
One of my co-workers recently suggested a great idea. We have a Rails app that handles a lot of data input to a MySQL database. We wanted to be able to generate reports based on this data. In this case, the reports were probably going to be generated by scripts and had no real need to be 'web-ified'.
Rather than using HTTP, which adds some overhead, he suggested we provide a socket connection to the app that will return 'raw' data. It would still pull it out of MySQL, but instead of wrapping everything in an HTTP request, it can simply be streamed across a socket connection to whoever needs it.
This is a great example of socket programming coming in handy just because you know it. It's no secret that we're a little HTTP obsessed in the Ruby community. With a basic knowledge of sockets, you don't always have to turn to HTTP when you want to share data. Sometimes a socket can be simpler, and probably easier in the case of streaming, than using HTTP.
These are three examples of times that I've put my knowledge of sockets to good use. The last example that I can think of is conversations. There are lots of conversations that happen around our work. Whether it's choice of tools, ideas for optimizations, discussion of architectures, or whatever. The more knowledge and understanding you have about the layers underneath the stuff we work on, the more you can participate and lead these kinds of discussions.
Until next time,
As a Ruby developer interested in putting some of these suggestions into action, Working With TCP Sockets is written for you.
It will walk you through the lifecycle of client sockets and the basics of using protocols for communication, a given when working with most any network service.
In terms of tuning web servers, you'll see what some of the common socket options are and what they do. On top of that, you'll see a basic several basic server implementations using different kinds of network concurrency. Overall, this is really going to build up your understanding of your web server stack.
Check out a free sample chapter or the full version.