[Interview] Daniel Huckstep on Ruby's stdlib and golang
Published on July 02, 2013 by Jesse Storimer
I recorded a conversation with Daniel Huckstep about two topics that always arouse discussion: Ruby's stdlib and the Go programming language. Daniel is a Ruby developer, Go convert, and the author of Go: The Standard Library (ebook).
Here are the highlights with links straight to the transcript:
- Daniel's "Ruby Batteries Included" talk from Mountain West Rubyconf #
- Core library vs. Standard library #
- Our stdlib favourites #
- Gemifying the stdlib #
- The Go programming language has a fantastic stdlib #
- How to discover Ruby's stdlib #
- What's different about Go's stdlib? #
- What makes Go compile quickly? #
- Getting started with Go #
- Go code formatting vs. Ruby code formatting #
- Why you should give Go a try #
- Daniel's book about the Go stdlib #
Listen to the audio
MP3 Download (~58 minutes, ~56MB) : Here's the link. Listen in your browser or Save As to download it.
Jesse: Hello, everyone, and welcome. I am Jesse Storimer. I’m here today with Daniel Huckstep. How are you doing?
Daniel: I am excellent. How are you?
Jesse: Good. I remember the first time we met online, or had some interaction online. You were one of the wonderful people who bought my first book in the first couple of days that it existed and I remember seeing that coming in. I had read some of your blog posts about Mongrel2, so I reached out and got in touch that way. That was a couple years ago.
Daniel: That was the networking one, right?
Jesse: That was the UNIX Processes book.
Daniel: Oh, UNIX Processes, right. Networking was the previous one or the one after that?
Jesse: After that, exactly, yes. One of the reasons I want to talk to you today is not about Mongrel2, which I haven’t heard about in a long time. But about this really cool talk you did at Mountain West RubyConf called “Ruby Batteries Included: Ruby Can Standard Library, and So Can You!”
Daniel’s “Ruby Batteries Included” talk from Mountain West Rubyconf
Daniel: Yeah, that was fun. I had submitted that randomly. Just was like, “Oh, you know, they’re not going to pick this, whatever. I’m a nobody. They’re not going to pick my talk.” And then they picked it, and they announced it on Twitter. And I was like, “Oh, shoot, I’ve got to write a talk now.”
Jesse: Yeah, that feeling.
Daniel: I was like, “Oh, cool, uh… all right…I guess so.”
Jesse: Oh yes… oh no.
Daniel: I hear of a lot of people that have given multiple talks before, and they wait until the last week or whatever to write their talk or they’re writing it on the plane or something like that. I took at least two months to write mine and I gave it twice at the local meetup to flesh it out and get the flow going and figure out what topics I needed to drop and what topics I needed to emphasize. I think it showed, I think I did all right. People said they enjoyed it and learned some stuff from it. Apparently Ryan Davis learned things; you don’t teach Ryan Davis things.
Jesse: Interesting. Are you generally a perfectionist when it comes to these things?
Daniel: Like talks and things like that?
Jesse: Yeah, talks and, I guess, being in public and putting stuff out in public.
Some of them I plan a little bit more in advance and have nice slides for. Other ones, it was just kind of like, “Well, we need someone to talk tonight.” I’m like, “All right, well I can blabber on about this for 20 minutes,” and I make some notes in a text file and I go with that. But when I do that I make it clear like, “Hey, I just these 20 minutes ago, so let’s not be too critical.”
Jesse: Okay, cool. I wanted to get to some of the contents of your Mountain West talk. It was kind of like a safari tour of the standard library, right?
Daniel: Yeah. I had a lot more. I just kind of scrolled through the rdocs and just tried to prune it down to the useful and the interesting stuff. Specifically for the final structure of the talk, I had it kind of grouped into the three things, like basic, performance, and beyond. It was like Bed, Bath and Beyond.
Core library vs. Standard library
Jesse: I remember that slide. I think before you went into the standard library, you talked about the difference between the core library and the standard library. And this is something that I still sometimes get it wrong, as to saying, “Oh yeah, that’s definitely in core,” and then it turns out that it’s in standard library. So, for people listening, what’s the effective difference between the core library and the standard library?
Daniel: So, the core is basically stuff that Ruby needs. If
Net::HTTP wasn’t around, Ruby would still run; you could still do stuff. But if
String wasn’t around, well, you’re going to have a bad time. Things like
Enumerable; those are in the core because they’re required for the language itself to function. The standard library is just cool stuff that’s already packaged with the distribution of the language. It’s sort of like Ruby gems that you get for free. You still have to require them, but they’re there.
Jesse: So, is it fair to say that anything that you have to require is not in the core library?
Daniel: Yeah. Typically, if you’re requiring it, like if it’s in the standard library, then you usually have to require it. Rails might tweak things for you, like, it might auto-require some stuff. It might seem like it’s in the core, but if you’re using Rails and you do something and it’s like, “Oh, that just worked,” well, maybe it auto-required it. Sometimes it doesn’t. I know I have to require Set in Rails, if I want to use Set. But I wouldn’t be surprised if it randomly auto-required some things for you.
Jesse: Or, some gem you depend on, pulls something in for you?
Jesse: So, do you have a couple of favourites from the standard library? Or maybe a couple of things that people probably often forget about? What was the stuff that Ryan Davis didn’t know was in the standard library?
Our stdlib favourites
Daniel: I can’t remember, honestly. One of my favorites isSetbecause everybody kind of forgets about it. You have
Array and you can do
uniq on it to, like, get rid of unique values. But that’s kind of cheating because if you don’t want unique values in it maybe you should use a data type that enforces that and then your data type says something about this blob of data. If you use a set, well, there are no unique values in it. But if there’s an array, well, who’s to say that you can’t put unique values in it.
Jesse: Yeah, use what you need, right?
Daniel: To do certain things like set operations, it’s a lot faster. You have a
Set and you say “
.include?(thing)” it’s going to happen essentially in constant time because it’s basically just a hash underneath. Whereas if you have an array, it’s got to walk the whole array and if it’s not in the array, well, you have to go through the entire array.
And, if you’ve got a bigger array, it’ll be really slow. I think the basic idea of the
Array versus a
Hash is the whole: Ruby being slow at requiring things when you get a lot of things that you have required. And someone wrote a patch for it, and that was a basic thing. It kind of just stuffed everything into an array and, even in C, that’s slow when you have an array of thousands of thousands of things that you have to go through and compare all of these strings. So, a guy wrote a patch to basically do it as a hash table, or a set.
Jesse: I think that patch ended up getting rejected and there was some other optimization made, but require is faster now.
Jesse: There’s Set and there’s one that is also defined in the same file, which is SortedSet, which is a subclass of Set but things are automatically sorted for you. I’ve only used it a couple of times, but when I’ve used it I’m like, “Oh, this is super easy, I’m glad it’s here.”
Daniel: Yeah. It does what you need it to do because you’re representing your data properly, I guess.
I think Ryan Davis forgot about SimpleDelegator. You make a wrapper object that you can define new methods on. You make a class, inherit from
SimpleDelegator, then you can instantiate that class, pass in some other thing like a string, and then you can call, like, .length on it and it will delegate down to the string. But then you can define other methods within your other class that are now on it. So, you’ve kind of done a DCI without blowing out your method cache.
Jesse: Cool. I’ve definitely seen that one from people talking about DCI within the past couple of months. It seems useful.
Benchmark is one that I’ve used quite a bit and I think a lot of people know about it. I think it’s fairly well known.
Daniel: Oh yeah. Everybody knows about it. I don’t know if everybody knows how to use it properly.
Daniel: Benchmarking code is hard, right? There are so many things that conflate and screw up your numbers. It tries to do a decent job of giving you a nice little API to do the warm-ups so your cache is fresh and you can report things so that you have a nice little table of, “Yes, this is this code and this is this code.”
But, who benchmarks things anyways?
Jesse: Well, we have all of these higher level tools, like, the New Relics and things of that nature. We’re typically not doing too much micro-benchmarking, probably.
One that I’ve never seen before is GServer. Is it just like a really simple TCP server?
Daniel: Yeah. I think the G is short for General or Generic, maybe. You inherit from
GServer and you implement
serve, which takes an IO object and it’s basically just a TCP connection and you can read from it and you can write from it. Boom. It’s threaded, so, if you’re on JRuby you get actual concurrency. Remember, use your mutexes as you pointed out, what was that yesterday? Or a couple of days ago?
Jesse: Probably, yes.
Daniel: You’ve had a few good posts on the Ruby GIL and how it doesn’t actually work.
Jesse: Yes. You still need to care about thread safety, even if you’ve got a GIL. It does not save you. It does not protect you.
Daniel: But, yeah,
GServer. You just implement
serve and what I used it for in the talk was that you read some little command and then return some data. It’s like no code. There’s nothing there. Just read the command, check what it is, and return the thing- the absolute minimum. And then you just start your server and away it goes.
And you can have multiple of those things running because there’s a method to start the server, and then that returns and the thing is running and then if you want to block on it you call server.join. So, you can have multiple servers listening on multiple ports and they’re all independently threaded and what not. You could do some pretty crazy TCP stuff with this and JRuby, I bet.
Jesse: Interesting. I never knew about that one.
Daniel: I think PStore blew a lot of people’s minds.
Jesse: That one I have seen, but I don’t know of any uses besides, you know, examples or somebody saying “Hey, have you seen this?” I’ve never really seen anybody using it.
Daniel: Yeah. I still have to write my Google Reader replacement using only the standard library.
Jesse: Yeah, well, you’ve got RSS in there, you’ve got storage here. You’ve got a threaded server.
Daniel: Well, you can use WEBrick, you got ERB, it’s all there-it’s all you need.
Jesse: That’s pretty cool. I think you said at the start that there’s, what, 500 files in the standard library? 552 classes.
Daniel: Yeah, I just did a really terrible find, grep, sort, hack, and, you know, it printed out 552 and so I was like, “Yeah, that’s a good number.” There’s at least that many in there. There’s a lot of code. I mean, like, in the very end where I’m just listing off other fun things- there’s an RSS package. Okay, there’s an XML package, fine, but there’s an RSS package. Beyond just basic XML, there’s full RSS parsing and generating. It’s like, cool, but why is that in the standard library? That seems a little out there, but, alright, whatever, it’s there.
Gemifying the standard library
Jesse: One thing I’ve seen in all of these announcements on Ruby 2.0 is that they’re working on gemifying the standard library. And, as far as I understand, the intended goal of that is, when you install Ruby, you’ll have a bunch of gems pre-installed, which will be the standard library. And you’re welcome to, obviously, install more gems to replace those. Have you heard anything about that?
Daniel: Only whispers in the wind. I think what they’re kind of going for is the ability to update and improve the standard library without releasing a new version of Ruby, which I guess kind of makes sense. We can actually work on the
Net::HTTP API or just make a new one that will replace it or something and you can do that outside of Ruby.
Jesse: Did you get any impression from doing the research of how things get included in the standard library? Because you brought up the point about RSS, where, it’s kind of surprising that there’s this RSS library in the standard library but we don’t have, for instance, like Nokogiri in there or some other really useful or really popular gems.
Daniel: I didn’t research terribly far into that range of things. I know a lot of these have been in the standard library for a while. They’re there in 1.8.7 and probably a lot of them are there in 1.8.6 and even earlier than that. So, I’m not exactly sure when the RSS library came in. Some of these things have probably been there for a while and nobody really cared because either nobody was using them or it worked well enough and then it just kind of hung out and stayed there.
So, I think that’s part of the problem is that nobody-like, how many people used the RSS library? Probably not too many. So, if it’s there, a) people forget about, and then it doesn’t get worked on, so it doesn’t get improved, which means people don’t want to use it which means it doesn’t get worked on; it’s kind of a cycle. If it’s a bad API or a bad thing then you don’t want to use it and then nobody wants to work on it.
Jesse: And then, if you were to remove it you would technically be breaking an API that Ruby had, right?
Daniel: Yeah, and that’s hard, right? It’s kind of like “Oh, we don’t want to do break this API.” Well, nobody’s touched the code in eight years, maybe we should rip it out, I don’t know.
Jesse: Yeah. That reminds me of this blog post I just looked up from Mike Perham a couple of years ago: "Ruby’s standard library is a ghetto” He’s just calling out the fact that, like you’re saying, a lot of the stuff is eight or ten or more years old and is not exemplary Ruby code and maybe is not a Ruby-ish API but, for a lot of these reasons we’re talking about, it just doesn’t get removed.
Daniel: Yeah. They’ve got to do it at some point.
Jesse: You think so?
Daniel: There’s been more talk about it, like this blog post and other people have said, “We should rip this out and rip that out.” I think somebody’s just got to do it. Maybe if we get Aaron Patterson a little tipsy at a conference, he’ll just start to do stuff; he’s got commit bit.
Jesse: I follow along with the Rubinius developments and I know that they’re planning to go ahead with the gemifying, whether MRI makes the move or not so they’re planning to make a move in that direction. Just to get the ball rolling, so to speak. I think when this blog post was published, a lot of people were jumping behind it saying, “Yeah, hell, yeah, we should do something about this,” and I don’t think anything has really changed.
Daniel: There was a little bit of talk, I think, before Ruby 2.0. Because, if you’re going to rip anything out, let’s do it now. But, I don’t think anything got ripped out. We got refinements and a better GC and some other stuff, but we still have an RSS library.
Jesse: Yeah. And we still have REXML in the standard library as opposed to what everybody’s using.
Jesse: Anyways, we could probably talk about that as much as we want but we haven’t done anything about it, so, let’s move on. I was curious, like, what inspired you to do the talk and get looking into the standard library?
The Go programming language has a fantastic std lib
Daniel: Initially, it was the Go programming language because it has a fantastic standard library. It’s a different library because it’s a different language, so, of course the API is tailored for Go and how you’re supposed to write Go programs. But, there’s so much there, and it’s at, I think, the correct abstraction layer. It plays well with the language. There’s XML libraries, fine, there’s no RSS library but it’s trivial to write your own RSS parsing library given the language itself and how it deals with data structures and then the XML library that comes with Go. So, you can just define your structure and say, “Hey, de-serialize this XML to my structure,” and boom, it’s done. You didn’t have to think terribly hard about it.
So, it’s got a fantastic standard library. But then you see people on the golang-nuts mailing list, they’re like “Oh my God, I wrote this cool thing that does-” something that is in the standard library that you can do in like two lines. I applaud you for working on stuff and learning along the way and releasing code for everybody can use, but, we’ve already got this.
Jesse: That’s an interesting point.
Daniel: And then other places will be like, “Yes, we have this in the standard library, but it doesn’t do these things, and so we release an argument parsing library that does these things.” And you’re like, cool, I guess. Do you need those features? I don’t know; maybe in your app, not in mine. It’s about choice and it’s super easy to install stuff. But, going back to the whole reasoning, I guess, is that I’d see people doing stuff that the standard library already did.
So, I was like, “Obviously, I need to educate people about the wonders of the standard library.” That sort of led me into wanting to write a book about it, which is taking forever because I’m terrible at scheduling time to write. Then I was kind of like, “Well, when I’m done with this one, in like 2016 or whenever the hell that is, I’ll do a Ruby one.” Because that would be the logical next step.
Jesse: The one you’re working on now is about the Go standard library, right?
Daniel: Yeah. The first one I chose Go because I wanted to do more Go, so I thought, “Hey, I’ll go through the Go standard library.” Maybe, in retrospect, I should have done Ruby, I don’t know. But, then I thought, “Well, I’ll do this talk and that will kind of lead me in to see what the reception would be to something like that.” It was like, “Well, I think I could do a book on that and people would buy it and learn stuff from it.”
Discovering Ruby’s std lib
Jesse: I want to talk more about your book, but first I want to talk about how you discover stuff in the standard library. As someone who is thinking, “Oh, I need an option parsing library for Ruby,” like, there’s an easy way for us to search GitHub and find existing gems and search RubyGems.org and find existing gems so that we don’t end up; we can find other people’s libraries. But, there’s no really canonical way to search the standard library. If it’s got 500 classes, you can just grep through it if you have the source there, but I’m not sure how else you would find out what’s in the standard library.
Daniel: If you go to , it does have a search box. How good it is, I don’t know. But, more importantly at the top, you have Core and Std-lib links. So, if you open up the Std-lib link you get a nice little 1998 frames page. But, on the left, you have your table of contents. It’s a big standard library, but it’s not so big that you can’t scroll through the list and look at something and think, “Hmm… that kind of sounds like what I need to do.” So, you just kind of scroll through here and we’re using option parsing… where is it? Oh,
getopt. That sounds UNIXy,
getopt. And, what was the other one? There’s, like,
Jesse: I’m getting distracted by these names I haven’t seen before. Like, “Oh, I wonder what DBM is and I wonder what this is.”
Daniel: Yeah, it’s exciting.
Jesse: Yeah. Definitely some stuff to check out there. That’s a good tip, to just kind of head there. If you were thinking of writing something just head there and see if there’s a name there that sounds like what you’re working on.
Daniel: Yeah. You know the joke, it’s like, “Oh, I need to do this thing and I’ll use a regular expression.” Now you have two problems. “Oh, like, I need to do this thing and I’ll use the standard library.” Now you have no problems. It’s like the opposite of that. You can just check the core language. See if the language can do it with its syntax or built in types. Maybe there’s just a method on
String that can do what you want.
And, if that fails, check the standard library. It’s not that big and the more you go back to it, the more you’ll remember and the more you’ll say, “Oh yeah, there’s this thing that can do that and I can plug that into that and, okay, that will work.” And then you thought about it for five minutes and your problem was solved, versus adding another dependency to your application by installing something off rubygems.org that increases your startup time, you know.
Jesse: Or writing something that you have to maintain, and all of these kinds of things. It’s all cost.
Daniel: Yeah. There’s a time and a place for everything. There’s nothing saying that you can’t use the thing in the standard library for a bit, find out that it doesn’t fit your use case and then go down the path of, “Well, we need this other gem, or we need to write this thing by hand.” If the standard library can get you going, there’s not really too many reasons why you wouldn’t want to at least start there.
What’s different about Go’s std lib?
Jesse: So, let’s hop back over to Go for a minute. So you said the standard library is really superb, and I’ve definitely heard that, too. People saying, “If you want to learn how to write idiomatic Go code, have a look at the standard library.” Which is something people don’t usually say about Ruby. So, why do you think that is? What is Go doing differently about their standard library?
Daniel: A lot of the authors of Go itself, they’ve been working on the language for a while. I think it’s already, what, two years old now?
Jesse: At least.
Daniel: I thought it was announced in, like, 2009. That might be wrong, but…
Jesse: Google race.
Daniel: Even before then, they were writing code …
Jesse: Yeah, it appeared in ‘09.
Daniel: Before they even announced it, you know, they were working on stuff, working on the standard library, and they were fleshing out the language with the standard library. I think. So, they’re sitting there writing the standard library in Go, and they’re seeing that, “Oh, this doesn’t work very well so let’s do it this way,” so then they’d add a feature to the language or something like that. And they’ve also thought very hard about the features that they add and the syntax of the language and what not, so, they’ve had a long time to tweak things and bring the standard library up to date.
Jesse: So, it might be fair to say that while Ruby’s growth was kind of accidental; it started out as Matz’ hobby project and he didn’t really expect it to take the world by storm. Whereas Go, from the beginning, its intent was to be a production ready language and to have a great standard library and all these kinds of things.
Daniel: Yeah. It was made with some very clear goals in mind. It was basically designed while waiting for their C++ projects to compile. That’s the thing that Rob Pike always says. He starts his compilation on this big C++ app and he turns around and then they have discussions for twenty minutes about how they can make this all better and then, well, they did it.
What makes Go compile quickly?
Jesse: Yeah. Pretty fantastic. Now, maybe you don’t know this, it’s about the core of the language, but, is the reason that it compiles so quickly because of the way you define dependencies and the way you import and export stuff in your packages?
Daniel: Yeah. So, they have a really simple mechanism for defining the visibility of stuff. There’s no private/public/protected. It’s capital letter or lowercase letter that starts your identifier. That could be a type or it could be just a variable or a constant or a function, or functions on types. So, if you export a type, but then there’s a method on it that starts with a lowercase, that’s not visible outside the package.
Jesse: Okay. So if I make an RSS package, and I have a function called Parse and I make it capital-P, Parse, and then other code which imports my library, that’s essentially a public method to them.
Daniel: Yeah. So, then I would import RSS and then I could call RSS.capital-P Parse, then, that calls your thing. But, if it was lowercase-P Parse, then, as far the compiler says that method doesn’t exist when you try to compile your code. So then, to go along with that, they know what things are exported and they put that at the top of the object file so that when the linker goes along in the linking phase, because Go is compiled, remember, for all of you Ruby folk out there, when the linker goes along and links things, it doesn’t have to look very far into the files and everything is explicit.
It’s not like in C or C++ where you include something which includes this which includes this, and goes down into this gigantic chain of things and you’re parsing files multiple times. The Go compiler can be smarter about that and it only really has to read a small chunk of the file at the very beginning of the file to figure stuff out. So it’s not doing a bunch of extra work that it doesn’t need to. This whole fast compilation was one of the requirements from the beginning. They didn’t want to sit there and wait for minutes to compile things. The entire Go compilation for the language and standard library is less than a minute. It’s really fast.
Jesse: Yeah. I’ve seen Rob Pike do it on stage during his talk. He just kind of starts it up and then before he finishes a sentence, the compiler has bootstrapped and it’s compiled itself and all of the libraries and everything.
Daniel: You can install everything from scratch and run all of the tests and it’s a couple minutes. It compiles quickly and it runs quickly-it’s kind of a win/win. And with the way they’ve designed it, you’re very productive and you can get things done quickly just banging your fingers on the keyboard. Stuff happens quickly. Even though it might be a little bit more verbose, there’s a little bit more error handling, but, once you get going, you go pretty quick. So, you’re churning out code quick, and you’re compiling it quickly and then it runs quickly, so everybody wins.
Jesse: So, is your background in lower-level stuff like C and C++ or did you get your start in Ruby and high level languages?
And then I went to University at the University of Alberta, here in Edmonton and I did the engineering program. The class that I did the best in, in my first year, which is kind of a general first year that all engineers take, my best class was the one programming class. We used C++, but really only for IO for like standard out, with the double-shovel, shovel operator, everything else was just C.
I was like, “All right, that’s cool. I like computers. Let’s do computer stuff.” So, I went down the software engineering co-op program stream and then I started to do more specific things, like, “You’re doing C and you’re doing assembly programming and you’re learning about computer architecture and how CPUs deal with caches and memory…”
Then you graduate and you’re like, “Well I’m never going to use that.” Until I knew the answer to a problem I saw on Stack Overflow just from reading the question. It came up on Twitter and it was like, “Why is this thing so much faster when you sort the array beforehand,” and I’m like, “It’s probably branch prediction.” And I go and look at the thing, and I read the question, read the first accepted response, it’s like, “It’s all branch prediction.” I’m like, “University, paid for itself.”
Jesse: In stack overflow karma.
Daniel: We did C, that was probably the first official programming class I did in University was C. We did a class where we used Ruby. That’s kind of where I got my Ruby start. We did GUIs with-I can’t remember what we used-I think we used the TK Library.
Daniel: Yeah. In this one class we had a multiplayer Connect Four game, which I wrote the AI for. I also took the AI class for giggles. My friends ended up going to the functional programming ones so they learned LISP and Prolog. It’s like, “I’ll learn Prolog later. I’m going to do AI.” And, then, of course, a lot of Java.
Learning Go as a Rubyist
Jesse: The reason I asked is that I was curious about…I assume you picked up Go in the past couple of years, after doing a bunch of Ruby. I’m just curious about what the transition was like. How hard was it to pick up the language and start working with it?
Daniel: I didn’t find it that difficult but then again I had a C-type language background. Yeah, you’ve got your curly braces and what not. They tried to make the syntax pretty easy. It has a few Pascal-isms. There’s a difference between defining a variable and then initializing it. So you can say, like,
var i Int, or you could say, like,
Jesse: And the colon-equals is only used on initialization, right?
Daniel: Yeah, it’s only used on initialization. It says “Here’s my variable, I’ve defined this variable and it is equal to zero.”
Jesse: Okay, kind of like, you infer the type for me kind of thing?
Daniel: Yeah. It does the type inference on things like that.
Getting started with Go
Jesse: Interesting thing to trip up on. Okay, so, for other Ruby guys who are interested in Go, is there a good place to get started?
Daniel: If you go to , there’s, on the documents tab on the top, there’s Installing Go, Getting Started, How to Write Go Code and Effective Go, which are excellent. There’s also just the Go references, where you have the package references, which is all of your standard library docs, the command documentation, which is the Go tool itself, that you do everything with on the command line.
And then there is the language specification which, if you know compilers and stuff, it has all of the grammar productions in there. It’s actually a very readable spec of the language. It’s very, very complete. They did an excellent job of putting everything together in here. So you can just go here and you can read about floating point literals, imaginary literals and, like, how they’re defined.
Jesse: [jokes] It’s a good way to start with the language, to learn about the spec first, right? At least if you’re into that kind of thing.
Daniel: It’s really straight-forward. In Ruby, there’s like four different ways to do things. In Go, there’s, like, one and it’s very well thought out. If statements and for-loops; there’s only one kind of loop: the for-loop. That’s it. There are a couple of different syntaxes with how you can use that, but there’s the for-loop. And you have to use braces.
Go code formatting vs. Ruby code formatting
Whereas, like, in C, you can have a while loop where you just have i =, or i++ = j++ and you don’t need the braces. Well, in Go, you need the braces. In fact, the opening brace has to be on the same line. Whereas, you see in C programs they put the brace on the second line-that’s a compiler error; that’s not valid Go. Whether you like it or not, stop whining about it. Your code’s not going to compile if you don’t do that.
Jesse: That goes hand-in-hand with the Go format tool, right?
Daniel: Yeah. So, then, the natural progression of that is, “This is how your code should look, so we’ve written a tool that will make it look that way.” It just gets rid of a whole class of arguments. Just run
go fmt and it indents your code properly, it lines up equals signs and it just makes your code look the way it should look. So that, you look at this project you look at that project and everybody’s looking at the same code.
Jesse: Whereas in the Ruby world, when you switch teams and switch companies, they show you their style guide as one of the first things, saying, “We don’t indent here, we put spaces here, we like our braces to have this many spaces.” That whole discussion is just off the table, now, because there’s an official language decision saying, “This is how Go code should be styled,” right?
Daniel: Yeah. There are a few tweaks you can make. When I do my
go fmt, I use spaces instead of tabs, because their default is tabs, and I indent it four spaces and I think their default is eight. But those are just, like, arguments to gofmt, right? And if you have a project, well, it’s trivial to have some little script that just kind of does it.
Jesse: I see. So you’ve got a bit of choice there.
Daniel: Yeah. Sublime Text has a plug-in called GoSublime where you can set up the gofmt arguments, and when you save it just formats your code. Along with the other things that sublime does, like removes trailing white space, so, like, there’s no excuse for you to have trailing white space, there’s no excuse for you to not have properly formatted Go code. You don’t have to think about it. You can just write whatever and it can look terrible and then you can run gofmt on it just fixes it and it’s beautiful.
Jesse: I think that they’ve caught on to something there that’s definitely appealing.
Daniel: What makes that possible is that the standard library has all of the libraries to parse Go itself. So, there’s a lexer and a parser and you can pull out comments and that’s how all of the godoc stuff is generated. It just reads all of the code and pulls out all of the comments and formats it nicely in HTML for you and that’s all in the standard library.
Jesse: Interesting. I was just looking at the website, here. For people that are interested in trying Go and haven’t yet, I have done theGo tour, which is an in-browser introduction to the language. It’s really simple. It takes you through step-by-step and you can try everything in the browser and it definitely helped me at least understand what the language is about and get a little head start.
Daniel: Yeah. There’s the tour of Go, there’s more than a few videos, talks at conferences, screencasts, and whatever else that people have done. A lot by Andrew Gerrand, who is the developer advocate, I believe that’s his official title. Rob Pike does a few; he’s one of the main guys on the language. Rob Pike’s done more than his share of stuff.
Jesse: Anything with his name on it I tend to like. He’s been involved in UNIX and Plan 9. Have you ever seen videos of his ACME text editor?
Daniel: I have one that just kind of sits perpetually in my YouTube Watch Later. Russ Cox does a good one. It’s like a 25 minute thing of just Russ Cox using ACME and I just look at that once in a while, like, “[Sigh] I wish, I wish.”
Jesse: I’ve done the exact same thing. I feel like there’s some potential to learn that and achieve productivity.
Daniel: You need that three-button mouse thing.
Jesse: Yeah, I can’t really get over how it’s so mouse-driven. It’s definitely an interesting video.
Daniel: But the stuff that he does with the mouse, though, it’s out there. He’s getting a lot done, somehow.
Jesse: I think they call that mouse chording, where you’re actually doing, like, “Click this one, then that one,” in quick succession or doing the opposite in quick succession. That’s a different operation than button one then two in quick succession. You have all these different kinds of combinations.
Daniel: Then, of course, holding shift+control+option+command, whatever, all those keys, are your modifiers.
Jesse: It’s almost like playing a first-person shooter where you have one hand on the keyboard and one hand on the mouse.
Daniel: Sometimes, that’s what programming feels like.
Jesse: Yep, how many frags today?
Why you should give Go a try
Daniel: Yeah. So, if you haven’t checked out Go, you should at least go look at it. They thought they were writing it for C++ programmers who wanted to get more done with less code and shorter compile times, but then they found that the C++ programmers didn’t want to come to Go because they were giving up stuff. Like, “Oh, you can’t be super-specific about the memory layout there, and you can’t be super picky about when things are allocated,” and they didn’t want to give up that control.
But then you get the Ruby and Python folks that, well, you can work just as fast; you end up with better code. The error handling is a lot more explicit, versus just writing code and waiting to see what exceptions get thrown, but then you also get, “Well, okay it compiles quickly,” as opposed to in Ruby you don’t compile at all but this you compile quickly so it’s effectively like it’s not happening. And then your stuff runs fast, runs a lot faster than your Ruby code. You get actual concurrency; well, you know, you can use JRuby or something like that, but …
Jesse: Do you find the language as expressive to use as using Ruby?
Daniel: There was a good blog post the other day about “Unremarkable Go Code”. It was kind of like, you find a solution to a problem and you read the code and there’s nothing surprising or fancy about it. It does what it says it does.
Jesse: It’s not, like, a clever code.
Daniel: Yeah. It’s not clever. It’s just correct, it makes sense, and you move on. It doesn’t break. Whereas, you do some crazy stuff in Perl land where you have random variables that spring out of nowhere and what not and you’re like, “Oh cool, I did this stuff in one line,” and it’s a line of nonsense that you don’t understand the next day.
Jesse: You get a lot of expressive power with Ruby but it’s easy to abuse.
Daniel: Yeah. They have things in Go to iterate over things. You have your one type of loop, but you get the range operator so you can very easily iterate over an array or a slice of things without having to do the whole
i = 0; i++ nonsense. Okay, maybe it’s not quite
foo.each and then pass a block, but it’s pretty close. There’s different expressiveness that fit the language, I guess.
Jesse: The one thing I have heard from people really new to Go, like myself, is that the syntax for, for instance, initializing a slice, has more ceremony than you have in Ruby and it’s a little hard to parse the first couple of times. A couple of curly braces and square braces in there that you’re just kind of like, “What is this one for and what is that one for?” It’s not quite simple.
Daniel: Yeah. They have, for certain things; you can make an object literal notation for your structs. You can build those pretty easy, to just like build an array, or as Go calls them a slice. Yeah, you’re doing like a make, and then you’ll give it the type, which will be like square bracket of the type, square bracket string and then zero, or something like that. You use that in a few places and then you’re like, “Oh, you know what, I know the exact size this has to be.”
So you can just pre-allocate the whole thing and you don’t have to rely on Go expanding the thing in the background. You can just say, “Make this thing an empty slice but pre-allocate it to 500 slots so that when you are adding things to it, it never has to re-allocate anything.” It’s one of those things where once you get going with it; once you do more coding in it you just kind of skip right over it. It’s just something your fingers do.
Daniel’s book about the Go std lib
Jesse: So, you’re writing a book about the Go standard library. What’s it called?
Daniel: Go: The Standard Library.
Jesse: I did hear you mention that on the Golang website, they have a standard library reference. So how is your book different from the reference docs?
Daniel: One thing I’d see a lot of in reference docs and various other books, and I can’t name any names, but you’d see a lot of, “Yeah, here’s this cool thing, and they’d show you, like, one combination of the five arguments that this function takes and not in any context. Here’s the function call and it’s like… “Cool? What about this other thing that is relevant to my situation that nobody ever talks about?” So, I’d see a lot of that.
So, my emphasis with this book, and with any future books I decide to write, will be on full code. Like, “Here is a file. You can run this file. It’s a complete program and it demonstrates what I’m trying to explain to you.” You can look at the whole thing in full context as opposed to, “Yeah, here’s a function call.” Which is a lot of reference documentation, like, it describes the function. It’s sometimes hard to see how that works in the big scheme of things.
Jesse: Are you, like, using the reference documentation as a starting point and then adding examples to it? Or are just starting from scratch and, like, working your way through the code and working up your own explanations and examples?
Daniel: I’m just going through alphabetically, as far as the order of things. And I’m learning a lot of stuff as I go, too, it’s kind of like, “Oh, I haven’t used this package before.” I’ll look through it and look at all the things and try to figure out how I can group them. Some things don’t need explaining. Like, if you show them this thing, but then there are these five other functions that will work basically the same way, then maybe you don’t need to cover all of them in explicit detail.
A lot of crypto stuff and, like, the hash package-pretty much all of the things in the hash package are, like, make a hash, write stuff to it, and then call the sum method to get out the hash. They all work that way because they all implement the one interface. So, you don’t need to explain this versus that. It’s just, “Okay, if you want this hash, you call A
dler32.new, if you want the FNV you call
FNV.new, past that it’s all the same.
For things like the Go package, where there’s an AST package and a lexer and a parser and a scanner, okay, well, now we’re getting into, “Well, here, we’re going to scan Go code.” And then in the example for the lexer, I point out, “Oh, yeah, there are all of these semicolons that show up but they’re not in your file. That’s just Go’s semicolon insertion at work.” Other packages were similar but a little bit more interesting. Like the debug package has Elf, Mach-O and P windows portable executable packages, so they’re all sort of similar, but they’re just different here and there so you kind of have to cover them all.
Jesse: So, your book is not finished. Have you covered the whole standard library yet?
Daniel: Oh God, no.
Jesse: About how far are you?
Daniel: I think HTML is my next one.
Jesse: That’s almost halfway through the alphabet, isn’t it? That’s why you were saying 2016 because you are finding it still to be a lot of standard library to cover?
Daniel: Yeah. There’s a lot of content and it takes a while to write. You’ve written a couple of books, you know that writing a book isn’t an easy thing to do. Part of the hardness of it is just that it takes a while. When I see the Leanpub emails come in, “Oh, somebody’s purchased a book,” that spruces me up and gets me excited.
But then I remember I’ve got to do this thing and I’ve got to do this thing. Oh yeah, then I’ve got to visit the photographer tonight because I’m getting married in a year. Tomorrow, a year from tomorrow is when I’ll be getting married. So, we’re thinking about this stuff and that stuff, and we’ve got to do this thing at work, and then by the time you get home…
Jesse: It gets put on the backburner.
Daniel: You just get sidetracked doing this other thing. I wrote a load balancer in Go over the weekend. Why? I don’t know I wanted to learn about load balancers when we were debugging stuff at work. The whole weekend I’m like, “Oh, I should work on my book,” that didn’t work out too well.
Jesse: I know that I found, doing writing, just sitting down and banging it out is sometimes the hardest part. Once you get going, your progress is good or bad, but the hardest is just making time to sit down and do it.
Daniel: Yeah. And you’ve got a couple of kids and you’ve got other stuff that’s just, frankly, more important. And if all of that more important stuff takes up your time and you’re dead tired at the end of the day, well… Do you work on the book for an hour and just be more tired the next day, or do you go to bed and try again tomorrow and see what happens tomorrow?
Jesse: I know it’s definitely a first world problem, but… the only other solution I found was that I traded sleep so I could get progress done on the book and was more tired the next day and just dealt with it. Either that or I would, one night I would stay up half the night writing, and then the next three or four nights I wouldn’t do anything. But at least I got some progress done and then I’d sleep the next couple of nights. It’s certainly a challenge to try and fit it into an already full schedule.
Daniel: At one point I tried, like, I said, “Oh, I’ll get up an hour earlier.” And I’m not a morning person so that didn’t work out. I did it, like, once and then the next day I was like, “Ah, snooze button; I’ll work on it tomorrow.”
Jesse: I think we all dream of getting up early and getting our day’s work done before the sun rises.
Daniel: You envy people like Michael Lopp from Rands In Repose. He’s a morning person. He gets up and he gets writing and he’s got, like, a blog post out by 11:00. It’s like, “Really, come on, you’re making us look bad.”
Jesse: So, a logical follow-up question is, with regards to this Go thing-the larger idea is to not just do Go. You mentioned a couple of times that maybe once this is done, you could do the same for Ruby or the same for Python or some other language too, right? But you’re thinking Ruby could use something like this, too, so people are aware of what’s in the standard library and how they might use it.
Daniel: Yeah. People whine about Net::HTTP and it’s like, “Oh, maybe if we had some good, solid examples, then maybe it wouldn’t be that bad.” Because you look at some of the APIs and there’s a variety of different ways that you can do things, but, I don’t know-
Jesse: Net::HTTP is one that’s so unintuitive that, good examples are certainly a good thing to have, but the fact that you need examples each time you use it is kind of a point against it. Whereas something like Set, for example, has a really obvious API and it’s really easy to use. But the HTTP one is just a challenge every time.
Daniel: Looking at some of the stuff in there, like, its API isn’t…the philosophy behind the API isn’t intuitive. Like, okay, you make a request object and then you tell the thing to do that request, as opposed to, like, RESTclient where it’s just like
RestClient.get. One thing I always fight with RESTclient, and any of those libraries, is that I want to get with a hash of params and have them be URL params, and they never do it. It’s always like get, the path or the URL and then extra headers. So, if you want URL params, you’ve got to build them yourself.
Jesse: Like another string?
Daniel: Yeah. And, if you’ve got active support, then you just to_query it.
Jesse: Well, we’ve been going for about an hour here so I want to kind of steer us towards the end. I want to give you chance to tell people where they can find you. I know you have a blog and you’ve got your book and stuff, so where can people find all of these URLs to find out more about you?
Daniel: I blog at http://VerboseLogging.com. I tweet @darkhelmetlive. A lot of other places I’m DarkHelmetLive. GitHub is the exception where I’m just darkhelmet. There’s more than a ton of code up there that you can poke around and play with and file issues for and fork and swear at me for and whatever else. The standard library books. I say books but it’s half a book right now, is http://thestandardlibrary.com. And it’s the prettiest bootstrap site you’ve ever done seen.
Daniel: My other app that I’m super proud about that we haven’t actually mentioned because it’s not super relevant to the talk is http://Tinderizer.com. And that’s written in Go, that will send articles on the web to your Kindle.
Daniel: There’s the blog, there’s Twitter, there’s GitHub, there’s the book, Tinderizer…
Jesse: Is there anything else you want to leave off with. Like a little tidbit of advice or something cool people should check out or the one thing that people should do or… final words?
Daniel: I think everyone should go and, at least, check out Go. Try to write a real application in it because that’s where it really shines. It’s not like an academic language that you just kind of discuss in mailing lists.
Along those lines, actually use the language before coming to complain about it in the mailing list. Because there’s a lot of, “Oh, Go needs this feature,” and everybody’s like, “Well, have you actually written anything in it?” “No. I’m going to if Go has this feature.” So, check out Go, and use it before complaining about it. And, I guess, that would apply to almost anything. Use the library and the standard library before you decide that it’s not going to work.
Jesse: Good advice in programming, for sure.
Daniel: Yeah. Read the code. Check it out. Play with it and then make an informed decision as opposed to just saying, “Ah, screw it. It’s in the standard library it must be bad.”
Jesse: All right. Well, thanks for chatting with me, Daniel.
Daniel: Thanks for having me.
Jesse: Take care.