Straylight Run

If you find yourself wanting to echo or print something to the screen, go ahead and do it, but be sure to add a debug-level logging call for the info too. Chances are, you or someone else will want to see the same info sometime in the future.

Silicon Valley is inexplicable, a phenomenon unto itself.  You can start an internet company anywhere, but your experience starting one in the Valley will be remarkably different from starting one elsewhere.  The Valley’s perfect blend of support (funding, legal, media coverage, and social) plus an inexhaustible talent pool makes it hard to replicate.  But what is it about the Valley that makes it so special?

Paul Graham wrote an article a while ago about how to be Silicon Valley.  He claimed that the two main requirements to duplicate the Valley are nerds and rich people.  So, Pittsburgh has lots of nerds, but no rich people.  Hence you don’t hear about the next Google coming from the Steel City.  The same applies to Miami, which has lots of rich people (and professional athletes), but no nerds.  In considering things like personality, creativity, and youthfulness of a city, I like the following passage:

…Most good startup ideas seem a little crazy; if they were obviously good ideas, someone would have done them already.

That’s the connection between technology and liberalism. Without exception the high-tech cities in the US are also the most liberal. But it’s not because liberals are smarter that this is so. It’s because liberal cities tolerate odd ideas, and smart people by definition have odd ideas.

Conversely, a town that gets praised for being “solid” or representing “traditional values” may be a fine place to live, but it’s never going to succeed as a startup hub.

Being “a fine place to live” describes Cincinnati to a tee.  If our company is successful, it will have been in the face of the extraordinary limitations of where it started.  I love this town, but this is no place to start an growth internet startup.  Advertising or manufacturing, maybe.  One of the reasons I joined was the idea of being a part of maybe the first real consumer internet success in this city.  But I digress.

This article in the NY Times, “Seattle Taps Its Inner Silicon Valley,” talked about how Seattle is becoming the next Silicon Valley.  The article rightly found Seattle following in Silicon Valley’s footsteps, with its influx of venture capital and transformation from timber and aerospace to internet, based around U-Dub and of course Microsoft and Amazon:

“A start-up ecosystem needs social networks, support businesses and a business culture that views failure as a badge of honor, not shame. All of that is in place in Seattle.”

That article generated a small firestorm of opinion.  Glenn Kelman, Seattle resident and CEO of Redfin.com, took issue with the comparison in his blog “How Green Was My Valley.”  Kelman’s pointed out why Seattle is indubitably not Silicon Valley, and never will be, thankfully.

“My first roommate spent four years building a company in San Francisco without ever buying furniture. When his startup went bust, he packed for the trip home to Toronto the same day. Seattle is different. People live in Seattle because they love Seattle.”

Michael Arrington of Techcrunch, and defender of all things Silicon Valley, reacted with “An Outsider’s Flawed View Of Silicon Valley,” a post defending the region from the Kelman’s blog.   Arrington eloquently, if not arrogantly, proclaimed Silicon Valley as the supreme center of all things internet entrepreneurship, and anyone claiming that it’s better to start a tech company anywhere else is delusional.  I wouldn’t disagree, I guess (emphasis his):

“But the best of the best come to Silicon Valley to see if they’re as good as the legends that came before them. It’s a competitive advantage to be here. And if you aren’t willing to take advantage of every possible advantage to make your crazy startup idea work, perhaps you shouldn’t be an entrepreneur… Making lifestyle choices is fine, but don’t delude yourself into thinking those choices are anything but a tradeoff. If staring at lakes and skiing after work are important to you, don’t pretend to be surprised when your startup doesn’t cut it.”

Doing what I do, where I do, certainly comes with a cost.  But the benefit is that I don’t abruptly move my family to California (oh to be 22 again).  Whether the cost or benefit was the bigger factor remains to be seen.

In any system, the biggest bottlenecks will usually be related to I/O. What this means practically is two things:

1. Memory is faster than disk.
2. Disk is faster than network.

But moving across the boundaries of memory, disk, and network is usually cumbersome.  For example, storing things on disks is programmatically easy, but slow.  Storing things in memory, in a persistent way, can be hard.  This is more true for a shared-nothing architecture like PHP rather than Java, so you may have to deal with some shared memory libraries and SysV IPC-style calls.

Enter tmpfs, the linux shared-memory file system.  You can mount it just like ext3, create files, and otherwise treat it like a normal disk, but it’s in memory!  Awesome!

On RHEL, Fedora, CentOS – not sure about others – there is a tmpfs drive mounted under /dev/shm by default.  One other note: since it is memory, its contents will be lost upon reboot.  I usually re-create any directories I need in the /etc/rc.d/rc.local script.  Note, however, that this is the last file to run on boot, so if you have a service or daemon that assumes a folder in /dev/shm, you will need to create it in the service’s startup script (usually in /etc/init.d).

When looking for something in an array of values, it is very tempting to use in_array(). After all, that’s what the name says. However, searching through an array, even with best-case search algorithms, will never be faster than a single index lookup, which is where isset() comes in. With isset(), you can use one operation to see if a value exists, provided those values exist as keys. I don’t know if it’s truly random access, but it’s pretty darn close.

So, instead of something like this:

$exclude = array(1, 4, 6, 8);

for ($i = 0, $size = count($data); $i < $size; $i++)
{
   if (in_array($data[$i]['id'], $exclude)
   {
      // do something
   }
}

do something like:

$exclude[1] = true;
$exclude[4] = true;
$exclude[6] = true;
$exclude[8] = true;

for ($i, $size = count($data); $i < $size; $i++)
{
   if (isset($exclude[$data[$i]['id']]))
   {
      // do something
   }
}

So does this make a difference? Let's write a little benchmark script.

#!/usr/bin/php

We fill two arrays with 1000 random integers. One is the haystack - what we will search through. The other is the list of needles - we want to search for each one. For each needle, we look for it in the haystack. Then, we repeat this 1000 times.
Executing this, the script takes around 37 seconds:
% time ./bench.php 

real    0m37.400s
user    0m37.282s
sys     0m0.068s

Now, let's change the last for() loop to this:
for ($i = 0; $i < 1000; $i++)
{
    $tmp = array_flip($haystack);
    foreach ($needles as $needle)
    {
        if (isset($tmp[$needle]));
    }
}
The new output:
% time ./bench.php 

real    0m0.778s
user    0m0.764s
sys     0m0.008s

Execution time drops from around 37 seconds to 0.7 seconds.

A while ago, we were struggling with the question of whether or not to use a framework with some new code.  Specifically, did we want to use Zend Framework or not?  (The reasons for settling on ZF vs. others is the topic of a different post.)  We had been using our own little framework for almost a year, and it had served us relatively well.

Reasons to use ZF:

• Many have worked on it and will continue to work on it, and it enjoys community support. After all, it was created by the PHP people.
• Why re-invent the wheel?
• Accelerates development and time to release.

Reasons not to use ZF:

• It will impact performance. After all, it is plainly more code. The question is how much?
• There will be a learning curve to learn the “framework” way, when we already know our own code.

My #1 concern was with performance.  So I ran some tests using Apache Bench and Zend Framework 1.0.  I won’t concern you with the details of the test because 1.0 is now a bit outdated, and the performance of Zend Framework is not really the point of this post.  But I will say that ZF was much slower than what we were currently using.

Then I got to thinking about all the websites out there.  Zend is a very nice framework – perhaps my #1 choice for frameworks at the time (though that changes frequently). And getting “Hello World” to work was easy and enjoyable.

But different websites have different audiences. If I am making a local storefront website, or even a chain website, the traffic patterns are going to be very different from a website aiming to be a national destination. If I were cranking out storefront websites every month, I think ZF is the way to go. It’s quick, and I’m sure I would be doing similar things over and over again. ZF is very good for that. Same goes for a shrink-wrap application (by shrinkwrap, I mean something downloadable and installable). If I wanted to make the next great blogging platform, and thousands of people would come to my website just to download it and put on their own hosts, again, ZF is very good for that. (In fact, I would make the application in ZF, and the website to download it from in ZF!)

But for my company, it’s different. If we are to become that national destination that we want to be, it will need every ounce of performance squeezed out. It will have a very custom environment, and require very custom features. That is why Zend Framework, and other frameworks, fail.