Straylight Run

I first heard the term technical debt when I was learning about Scrum.  It immediately struck a chord because it made so much sense!  Technical debt described those times I coded the quick and dirty way (incurring debt) and not the way I wanted.  Technical debt described all those times I wanted to refactor an ugly system (and pay down debt), but couldn’t due to deadlines and the fact that it’s so hard to demonstrate the value of better code when the business output is the same.

As Steve McConnell says about this attitude towards debt:

I’ve found that business staff generally seems to have a higher tolerance for technical debt than technical staff does. Business executives tend to want to understand the tradeoffs involved, whereas some technical staff seem to believe that the only correct amount of technical debt is zero.

Like financial debt, a little technical debt is okay.  After all, if the viability of the business depends on releasing a product, saving the business is more important than feeling good about your architecture.  But you have to service the debt at some point, and fight the common business notion: if the software works, then it’s good enough.

Because the tax man will come to collect.  And you will know when he does when you attempt to change someone else’s (or perhaps your own) old code and you see that you have to change 40 files to make a tiny feature because the whole system is a big ball of mud and you grumble something about doing it right the first time.

I thought about all this when I read Jay Pipes’ advice to MySQL.  Years between releases… Bug fixes that cause bugs…  It sounds like the tax man has come to collect at MySQL.  He argues for taking a year break to pay off the technical debt in MySQL.  Ouch.  That’s quite a bill.

Now, I consider Jay to be one of the smartest persons I’ve ever met, but I have to disagree on this one.  The thought of stopping new work to radically alter a huge working system and ultimately release it a year later in a Big Bang terrifies me.  If you were really, really, good, you could be successful at this and maintain quality and measure output and do tons of integration testing, regression testing, etc.  But one thing that I believe is oft overlooked is that developers like to release code.  And the more time that goes by without any released code, the more it feels like days are just wasting away.  That’s been my experience anyway.  But I digress.

Much like our credit markets these days, after accumulating so much debt, you find that you cannot move.  You feel stuck and trapped and spend all your energy trying to stop moving backwards, instead of moving forwards.

If you find yourself wanting to echo or print something to the screen, go ahead and do it, but be sure to add a debug-level logging call for the info too. Chances are, you or someone else will want to see the same info sometime in the future.

Silicon Valley is inexplicable, a phenomenon unto itself.  You can start an internet company anywhere, but your experience starting one in the Valley will be remarkably different from starting one elsewhere.  The Valley’s perfect blend of support (funding, legal, media coverage, and social) plus an inexhaustible talent pool makes it hard to replicate.  But what is it about the Valley that makes it so special?

Paul Graham wrote an article a while ago about how to be Silicon Valley.  He claimed that the two main requirements to duplicate the Valley are nerds and rich people.  So, Pittsburgh has lots of nerds, but no rich people.  Hence you don’t hear about the next Google coming from the Steel City.  The same applies to Miami, which has lots of rich people (and professional athletes), but no nerds.  In considering things like personality, creativity, and youthfulness of a city, I like the following passage:

…Most good startup ideas seem a little crazy; if they were obviously good ideas, someone would have done them already.

That’s the connection between technology and liberalism. Without exception the high-tech cities in the US are also the most liberal. But it’s not because liberals are smarter that this is so. It’s because liberal cities tolerate odd ideas, and smart people by definition have odd ideas.

Conversely, a town that gets praised for being “solid” or representing “traditional values” may be a fine place to live, but it’s never going to succeed as a startup hub.

Being “a fine place to live” describes Cincinnati to a tee.  If our company is successful, it will have been in the face of the extraordinary limitations of where it started.  I love this town, but this is no place to start an growth internet startup.  Advertising or manufacturing, maybe.  One of the reasons I joined was the idea of being a part of maybe the first real consumer internet success in this city.  But I digress.

This article in the NY Times, “Seattle Taps Its Inner Silicon Valley,” talked about how Seattle is becoming the next Silicon Valley.  The article rightly found Seattle following in Silicon Valley’s footsteps, with its influx of venture capital and transformation from timber and aerospace to internet, based around U-Dub and of course Microsoft and Amazon:

“A start-up ecosystem needs social networks, support businesses and a business culture that views failure as a badge of honor, not shame. All of that is in place in Seattle.”

That article generated a small firestorm of opinion.  Glenn Kelman, Seattle resident and CEO of Redfin.com, took issue with the comparison in his blog “How Green Was My Valley.”  Kelman’s pointed out why Seattle is indubitably not Silicon Valley, and never will be, thankfully.

“My first roommate spent four years building a company in San Francisco without ever buying furniture. When his startup went bust, he packed for the trip home to Toronto the same day. Seattle is different. People live in Seattle because they love Seattle.”

Michael Arrington of Techcrunch, and defender of all things Silicon Valley, reacted with “An Outsider’s Flawed View Of Silicon Valley,” a post defending the region from the Kelman’s blog.   Arrington eloquently, if not arrogantly, proclaimed Silicon Valley as the supreme center of all things internet entrepreneurship, and anyone claiming that it’s better to start a tech company anywhere else is delusional.  I wouldn’t disagree, I guess (emphasis his):

“But the best of the best come to Silicon Valley to see if they’re as good as the legends that came before them. It’s a competitive advantage to be here. And if you aren’t willing to take advantage of every possible advantage to make your crazy startup idea work, perhaps you shouldn’t be an entrepreneur… Making lifestyle choices is fine, but don’t delude yourself into thinking those choices are anything but a tradeoff. If staring at lakes and skiing after work are important to you, don’t pretend to be surprised when your startup doesn’t cut it.”

Doing what I do, where I do, certainly comes with a cost.  But the benefit is that I don’t abruptly move my family to California (oh to be 22 again).  Whether the cost or benefit was the bigger factor remains to be seen.

In any system, the biggest bottlenecks will usually be related to I/O. What this means practically is two things:

1. Memory is faster than disk.
2. Disk is faster than network.

But moving across the boundaries of memory, disk, and network is usually cumbersome.  For example, storing things on disks is programmatically easy, but slow.  Storing things in memory, in a persistent way, can be hard.  This is more true for a shared-nothing architecture like PHP rather than Java, so you may have to deal with some shared memory libraries and SysV IPC-style calls.

Enter tmpfs, the linux shared-memory file system.  You can mount it just like ext3, create files, and otherwise treat it like a normal disk, but it’s in memory!  Awesome!

On RHEL, Fedora, CentOS – not sure about others – there is a tmpfs drive mounted under /dev/shm by default.  One other note: since it is memory, its contents will be lost upon reboot.  I usually re-create any directories I need in the /etc/rc.d/rc.local script.  Note, however, that this is the last file to run on boot, so if you have a service or daemon that assumes a folder in /dev/shm, you will need to create it in the service’s startup script (usually in /etc/init.d).