Friday, May 23, 2008

Questioning Tog: Does the Mouse Really Rule?

Did you hear this one? According to Tog, the keyboard feels faster than the mouse, but it's actually slower. (This came via Tim Bray and John Gruber).

Tog says Apple spent $50 million on user interface related R&D by 1989, and learned, among (hopefully numerous) other things, that:

  • Test subjects consistently report that keyboarding is faster than mousing.
  • The stopwatch consistently proves mousing is faster than keyboarding.
Tog's explanation: using the mouse is easy and boring, so it feels slower than recalling keyboard shortcuts, even though "the stopwatch will tell you" that it's actually faster.

No matter how many times I've read this very general statement, it never sounded right to me. Nobody seemed to ever mention any details. When I finally took the plunge and read Tog's actual analysis (parts 1, 2 and 3), I found out that the details still weren't there.

I would really like to know the answers to the following questions, for example:
  1. How many different functions did the subjects have to perform in these tests?
  2. What exactly did accomplishing a task by "mousing" and by "keyboarding" involve?
  3. How many key combinations did the user have to remember for a test? How difficult were these key combinations to enter? How easy were they to remember?
  4. How many mouse targets needed to be acquired, how big were they, and how far apart were they positioned?
  5. Did the users have any previous experience with acquiring similar mouse targets to those featured in the test?
  6. Did the users have any previous experience with using similar (or identical) keyboard combinations to those in the test?
  7. Was the user manipulating an object (like a text box or an image) between these "tasks"? If so, how big was the visual representation of this object? Did the user have to return to the object after acquiring the target needed for the task? If so, was it also timed?
I think Tog's "mousing is faster than keyboarding" statement is akin to saying that "walking is faster than driving," and neglecting to mention whether you were running your tests on a speedway or in a staircase.

Tog's test revisited

Tog does provide us, though, some insight into the methodology of at least one of his tests in the third part of his discussion:
The test I did I did several years ago, frankly, I entered into for the express purpose of letting cursor keys win, just to prove they could in some cases be faster than the mouse. Using Microsoft Word on a Macintosh, I typed in a paragraph of text, then replaced every instance of an "e" with a vertical bar (|). The test subject's task was to replace every | with an "e." Just to make it even harder, the test subjects, when using the mouse, were forbidden to just drop the cursor to the right of the | and then use the delete key to get rid of it. Instead, they had to actually drag the mouse pointer across the one-pixel width of the character t o select it, then press the "e" key to replace it.

The average time for the cursor keys was 99.43 seconds, for the mouse, 50.22 seconds. I also asked the test subjects which method was faster, and to a person they reported that the cursor keys were much, much faster. This was a classic example of the difference between subjective time (the passage of time the user experiences) and objective time (the passage of time the clock experiences). Put simply, the more mentally engaging the task, the shorter the time appears.
While I'm not a user interface expert, this passage leaves me scratching my head. It does seem to prove an interesting point on perception vs. reality, but it definitely fails to prove that the mouse is faster in general.

All it proves is that acquiring targets (vertical bar characters) scattered over a large area of the screen is more efficient with the mouse than with the keyboard. As we'll see in a minute, moving to smaller distances (especially single steps) would be much easier with the keyboard.

The same way, if you need to pick up 20 parcels which are miles apart, you'll probably take a car or a bike, but not if the same parcels are placed on the first 20 steps of a staircase.

As the vertical bars can be pretty far from each other, moving your pointer between them constitutes a significant part of the test, so it should favor the mouse, the faster vehicle – so I thought. But when I ran the test* on myself as a (somewhat questionable) test subject, using TextEdit and a paragraph from an Ask Tog article, I got the following results:
  • Mouse: 143 s
  • Keyboard: 133.5 s
I simply kept pressing the right arrow key until a vertical bar came up, at which point I selected it using Shift-Right arrow, and replaced it by pressing the "e" key. Surprisingly, and contrary to Tog's data, it was actually the keyboard that turned out to be marginally faster in this test.

By the way, having performed this test, I really fail to see how much it has to do with what Tog calls the "high-level cognitive function" of deciding "upon which special-function key to press." Maybe I'm really smart, but I didn't find that holding down an arrow key would tax my cognitive capabilities that much.

I also noticed how unnatural it felt to be using the mouse for the task. I was struggling to exert the right amount of force needed for precision alignment. It felt unbelievably tedious and even physically painful – my right wrist actually hurt afterwards. (Could all this suffering be another reason why using the mouse for the task seems longer than it actually is?)

*These test results are meant to serve little more than entertainment purposes. One (biased) subject, who happens to be the same person who devised the test, is less than statistically convincing. Yet I've made efforts to make these tests as meaningful as possible. I performed every task several times, and averaged the results. When I felt that fatigue, proficiency achieved by practice, or other factors interfered with my results, I kept re-doing the tests, switching their order, and sometimes taking long breaks from them, until I found that my results stabilized around a value, and that any difference between the values I got for the two or three tasks I was comparing was representative rather than accidental. I also honestly tried to achieve a reasonably good score in each test, without going into extremes. (I basically pretended that I had to perform actual work, in an office setting, on a looming deadline. I'm an average typist, and a power mouse user, having spent a decade working in Photoshop, Illustrator, InDesign, and Final Cut Pro, all particularly mouse-heavy applications.)

The mouse fights back

It occurred to me later that my results may have been affected by the font size I happened to be using. It was 13 points (Verdana) on a 17" screen at a 1440 x 900 resolution; Tog might have used a larger font and/or a smaller screen resolution almost two decades earlier.

Sure enough, setting the font size to 22 points turned the results around, improving the speed of the mouse by a large margin, and letting it score a narrow victory over the keyboard (though nothing like the double speed that Tog reported). Keyboard times also improved a bit:
  • Mouse, large font: 113 s
  • Keyboard, large font: 122.5 s
So Tog seems to have neglected a very important factor in his keyboard vs. mouse test: font size. Larger mouse targets are obviously easier to acquire, so the point size of the text you're editing can decide which method is better or more efficient.

That got me curious. What happens if we increase the font size to a ridiculous 200 points? I found out that the difference between the mouse and the keyboard almost disappeared in this dreadfully impossible task. In addition, the test results varied greatly, and repeating the test again and again helped improve times by leaps and bounds as I learned new tricks on the way. The keyboard started to win again, though mouse performance improved by a small margin every time I repeated the test. I averaged the results when I felt that the curve of the mouse improvements was flattening out:
  • Mouse, huge font: 155.5 s
  • Keyboard, huge font: 133 s

Beating the test

But there's more. Trying to speed up my "keyboarding," I tried the Alt-Right Arrow key combination, which jumps one whole word ahead, making navigation faster – and stumbled upon a "cheat": TextEdit considers the vertical bar a word boundary, thus whenever one shows up, Alt-Right arrow will stop the pointer right before it. Using this cheat (which involved more cognitive brain functionality, as well as some keyboard acrobatics), I needed more concentration than before, so the task felt much more difficult, and I also had to stop and correct some errors (something I didn't have to do using the other two methods). However, it cut my keyboarding time by a fourth, beating the hell out of the mouse at both normal and large font sizes:
  • Keyboard "cheat," normal font: 95.5 s
  • Keyboard "cheat," large font: 94.5 s
Noticeably, increasing the font size did nothing here. My theory is that finding each vertical line character was somewhat automatized by the cheat, so better visibility wasn't much help.

One side effect of the cheat was that I ended up inadvertently typing a lot of uppercase "E"s. I think the reasons are that here (1) I had to use two modifier keys (Alt and Shift), (2) I had to switch between them pretty fast, and (3) almost every second navigational keystroke (i.e. Alt-Arrow) was followed by a selection (Shift-Arrow), then immediately by a replacement ("e") keystroke, requiring me to switch between three different key commands very often, making me mess up one of them a lot.

Using a huge font size slowed down this task as well, but it was still the fastest way at that size:
  • Keyboard "cheat," huge font: 123 s
The table and graph below summarize how the font size change affects the three input methods tested. The straight lines connecting the dots should by no way be interpreted as implying continuity, they are provided mostly for better visibility (and as hints at very vague trends).

I could come up with several pages' worth of analysis, comparing and contrasting how font size affects each test case, but suffice it to say that without specifying everything down to the smallest details, it's just about impossible to decide whether the keyboard or the mouse is faster even in a very simple test like this. Tog seems to have oversimplified things a lot.

The keyboard's turn

In order to investigate things a bit further, I also came up with a test of my own, one where I felt the keyboard would have the edge, as it required smaller, more precise navigational movements.

In my test, the subject needed to replace every second letter of every word in a (much shorter) paragraph with the letter "p." I was the test subject again, and these were my results:
  • Mouse, normal font: 137 s
  • Keyboard, normal font: 90 s
  • Mouse, large font: 118 s
  • Keyboard, large font: 76.5 s
Boy, did I manage to contrive a test that favors the keyboard! As expected, it performed better as a means for selecting every second character of a word. Rhythmically pushing the right arrow twice would always do the trick, whereas with the mouse, I had to move past small characters of varying width*, and had to strain myself to position the pointer accurately. It seemed unnatural.

Replacing the letters was identical in both cases, so it didn't influence the results. (Or did it? We'll get back to that very soon.)

Somewhat surprisingly, a larger font size helped both tasks equally well. I expected mousing to benefit more, as the targets became larger, thus easier to acquire. Investigating the reason for this anomaly is beyond the scope of my blog post, but it, again, proves my main point: that things are complicated.

If you think by now that I'm some sort of a keyboard evangelist and mouse hater, rigging tests so that they will always let the keyboard win, then well, you've got that mouse pointer hovering over the wrong guy. No, I'm the guy who keeps rigging tests in small ways that change the results in big ways, trying to show that simplistic statements like Tog's are simply wrong. And ultimately, I just want the help the case of redundancy and freedom of user choice in user interface design.

So then I changed a little aspect of my test: instead of the letter "p," the subject now had to enter the "¶" character in place of every second letter of every word. This turned the test on its head, making the previously victorious keyboard lose by a Tog-uesque 50% margin.

What gives? Well, it's simple. While the subject (i.e. yours truly) could easily rest his fingers on the "p" key while navigating with the keyboard in the previous version, that was no longer possible with the hard-to-reach Alt-"7" combination needed for the "¶" character. Thus, every time I had to switch between entering that symbol and navigating/selecting, I had to lift my fingers off the keyboard and reposition them, losing a lot of time. (With practice, my times improved greatly. From an initial dysmal 184 seconds, which I excluded from my calculations as an anomaly, I reached a much more respectable average of 149 seconds by the time I finally decided to give the whole frustrating mess a rest. As always, the number I provide is an average. However, an interesting trend to note is that this is an initially huge margin that gets significantly smaller with practice.)
  • Mouse, large font: 126 s
  • Keyboard, large font: 149 s

So, guess what? You can't say that "the keyboard is faster." You can't even say, "The keyboard is faster when replacing every second letter of every word in a text." No, you even have to say what character you're replacing it with. It gets as specific as that.

*Switching to a monospaced font didn't help much, though. I ran the test with Monaco, and didn't notice any statistically significant improvements.


After testing some very narrow areas of the huge field of "user interaction by keyboard vs. mouse," all I can say is that both input methods have their strengths and weaknesses, and their optimum fields of use should be analyzed and investigated much more carefully than Tog appears to have done.

Personally, I think it's best to let the user decide what method(s) to use, and provide for both keyboard and mouse-based functionalities, whenever it makes any little sense. But even providing several methods isn't going to be enough: you also have to get them right. Designing key commands and mouse targets is pretty easy to mess up, having the user end up with useless input methods even in their preferred contexts.

Also, even though some specific input methods may have intimidating learning curves, in some cases, they may be worth the effort (e.g. in the case of the iPhone keypad). Obviously, users may be a bit tough to sell on such user interface choices, so they must be explained in a careful and convincing way.

Finally, I also wonder whether it's always speed that matters. It may easily be the case that the faster way of performing a task is more exhausting, less natural, or simply less fun. Speed may or may not be important; the developer simply can't imagine every possible way his or her product will be used. Maybe a task you imagined to be pretty rare as a developer will end up being the one that your user needs to perform a thousand times in the course of two hours – so there had better be a fast way of doing it. And while you're at it, why not also create an easy way, and a fun way? Redundancy can be a great thing in the world of user interfaces.

That is, unless you can find a perfect way, one which is easy, fun, fast and natural in every situation – I'm hard-pressed to find too many examples, though.


Tuesday, March 25, 2008

The Magic Text Field: searching, URLs and bookmarks

Cabel Maxfield Sasser notes how advertisements in Japan display search boxes with recommended search terms, as opposed to URLs. It certainly feels more user-friendly (and natural) to enter real words into one box (the search box) than to type something as geeky as a URL in another box (the URL field), and the author even speculates that future versions of Safari may transition to displaying a more prominent Google field and a less prominent URL field.

This has got me thinking. These are my closely related, highly anecdotal observations on URL entry vs. searching:

  • Some people (especially Firefox users) enter everything, even URLs, in the Google box. (By default, a new Firefox window has focus on the Google box.) These users will have to click again at the Google-displayed URL, but they don't seem to mind.
  • I sometimes catch myself mistakenly using the URL field instead of the Google field, hinting that search may be taking over from URL entry in my use as well.
  • Firefox actually accommodates (perhaps encourages) this previous behavior: entering expressions in the URL box yields Google search results or top hits. Other browsers simply display the predictable error messages caused by such attempts at connecting to malformed URLs.
But then I also have some more loosely related observations as well, mostly concerning bookmarks and history items:
  • I hardly ever use the bookmarks menu for opening a page, relying on the autocomplete feature of the URL field instead. That is, if I add something to my bookmarks, I will navigate to it later by starting to type some words from its URL (if meaningful), rather than choosing it from the menu.
  • But then I hardly ever add any bookmarks either. If I want to bookmark a page for later retrieval, I don't have to do anything: it will be automatically added to the History list, and thus it will be available for autocomplete, too.
  • In Leopard, Safari's history items and cached pages are indexed by Spotlight, even enabling users to find visited pages by searching for words in their contents.
The most perfect solution for me would probably involve one huge box with hooks for a lot of AI magic working behind the scenes, listing Google hits, URL matches, Wikipedia and Dictionary and other results, cached page search terms, all categorized, ranked in an ultra-smart way with the most likely results first, all displayed in real time, with easy keyboard-based navigation possibilities.

I think web pages are verbal, and the most natural way of relating to them is by typing text.


Wednesday, February 13, 2008

MacBook Air: harbinger of the tablet Mac?

As I was re-reading, for some reason, my old post on the November tablet Mac rumors, and got to the part where I speculate on the features of the purported device (such as ports and disk drives), something suddenly occured to me.

As I speculated back then, a tablet Mac would very likely need to do away with some traditional Mac features, such as an optical drive. However, I didn't think it would lack Ethernet or FireWire: I thought those would be too extreme omissions.

Guess what: Apple has just shipped a Mac without any of these things. It remains to be seen how exactly people are going to respond to such a radical elimination of items whose presence in a Mac have been taken for granted for almost a decade, but looks like Apple is on to something there.

The Air is more of a breakthrough in what it lacks than it is in what new features it adds (basically, a MultiTouch trackpad), and I'm sure Apple is eagerly anticipating feedback.

If it turns out that there exists a significant enough class of users who don't mind the radical departure this Mac represents, Apple can be more confident in launching yet another product category: a tablet Mac, taking the Air's ultraportability concept yet one step further.

MultiTouch cannot be forever confined to cellphones and trackpads.


Thursday, January 31, 2008

MacBook Air: iMac or Cube?

The introduction of the new Mac laptop is a bold move by Apple, as the subnotebook radically eliminates some components that may be considered essential.

Every single Mac that Apple has released since 1998 has had Ethernet connectivity, as well as an optical drive. Since 2001, every Macintosh has also shipped with a FireWire port.

The MacBook Air does away with all three.

Sure, there are workarounds, but all are cumbersome. Ethernet is available as a $29 dongle that Apple sells separately, though it would occupy the single USB port of the Air. External optical drives also exist (Apple sells one exclusively for the MacBook Air), but carrying such additional devices around somewhat defeats the purpose of having a super thin, super light notebook. The external drive would also need to fight over the single USB port with competing devices, unless you buy yet another companion product, a USB hub.

You can also hack into the optical drives of neighboring Macs or even PCs with a piece of software that is reputed to "just work," as one expects from Apple. However, it's kind of creepy to be constantly asking favors from fellow computer users, even installing software on their machines, whenever you want to use an optical drive. And those computers had better be equipped with WiFi, too.

And as for FireWire, you're pretty much out of luck there.

So, what the hell were they thinking? How could such a device ever sell?

Well, Apple did make a similarly radical move back in 1998, when it introduced the original iMac. Steve Jobs was back with a vengeance, and he chose a pretty dramatic way to show everyone he means business: he released a sexy-looking, simple entry-level Mac that lacked a floppy drive, and eschewed traditional ports such as SCSI or ADB in favor of USB. So what happened? The iMac sold as hot cakes, and peripheral makers started to build USB keyboards, mice, scanners, etc. The floppy disk was already on its way out, but the iMac's snub might have been the last nail in its coffin. So, the iMac pretty much changed the world around it.

Will the MacBook Air do the same? Will USB flash drives kill optical disks? Will WiFi drive Ethernet into extinction? Is Apple knifing its own FireWire baby?

It certainly looks like Apple would very much like all of this to happen. Just as modems started to disappear from Macs, Ethernet may be next, surviving only in professional machines. Optical drives may still stick around for a while, though, but Apple doesn't think they will be missed from the MacBook Air. While you'll still need to leech the drive of a neighboring computer for software installs, the Mac maker would prefer if you turned to its products and services instead of using an optical drive: get music and video off iTunes, use iPods instead of burning CDs, and buy Time Capsule for backups. Clever.

There certainly is method in this madness. Anyone in the market for a $1,800 notebook must have some cash to burn on these products and services, so each Air sold (especially in countries where the iTunes Store is available in its full glory) should generate some guaranteed extra revenue for Apple. Besides, these relatively wealthy people probably already have a Mac at home anyway, helping them overcome most of their objections to the Air.

If sales of the Air reach a critical mass, the new Mac could help reform the computing landscape, just like the iMac did a decade ago. If sales end up failing to go off the charts, but remain respectable, then, well, Apple can still boast a successful niche product, and I'm sure that a hundred bucks or two may go off the price eventually, if needed.

But what if Apple has made a major miscalculation, like the one in the case of the Power Mac Cube? Wasn't it also a relatively underpowered pro-level Mac that the market deemed too expensive? Pundits crucified the Cube for putting style over substance, and weak sales of the radical-looking new Mac spelled serious trouble for the still-vulnerable Apple.

I don't think there's any reason to anticipate a similar fate for the MacBook Air. There may be some superficial similarities to the Cube, but the differences are more significant:

  • As far as tech specs go, the Cube was clearly a weaker product than the Power Mac, yet it cost more. The MacBook Air is also a weaker product than the MacBook Pro, but it costs less as well. (However, it may also be compared to the MacBook, and it wouldn't fare so well in that comparison: the Air is the less capable and more expensive of the two notebooks.)
  • Miniaturization is not such a strong selling point for a desktop Mac as it is for a notebook. The beauty of the Cube was mostly skin deep, whereas the thinness of the Air is also very practical.
  • Apple's revenues in fiscal 2001 were $5.65 billion, whereas in 2007, they were $24 billion. Apple can better afford to risk less-than-optimal initial sales of a new experimental niche product now than it could at the time of the Cube.
I definitely don't think one should worry too much about the fate of the Air. It may not necessarily become a second iMac, but it really shouldn't be a second Cube either.


Thursday, January 24, 2008

Yet more whining on Apple secrecy

I'm sure Jens Alfke is a great guy and a great engineer, and I understand if he leaves Apple due to creative differences. Yet some of his comments seem to warrant a big "Duh!"

I think Apple’s policy on blogging is one of the least enlightened of major tech companies; Microsoft in particular is surprisingly open.
Well, Apple has been all about secrecy for the past decade or so, while Microsoft, perhaps the world's greatest vendor of vaporware, has always embraced blabbering as one of its main communication tools. Isn't it as simple as this?