Purpose Behind this Article

My previous article on OS X (released immediately after Public Beta but still relevant today) was written because it was obvious many mac users would not give it a fair shake. Much as I hate to say it, there are a lot of unimaginative, inflexible people in the mac community who would despise anything in OS X which didn't resemble what they were currently using. They remind me of DOS WordPerfect users who were throughly stuck in a groove where they expected function keys to do something very specific and would literally panic if a single character was unrecognized.

I hoped to inject a less religious perspective on the whole graphical interface revolution and hopefully get people to see themselves as a bunch of traditionalists who may me missing a productivity gold mine as a result. I pointed out nearly every UI advancement of the mac was invented by a 3rd party thus all Apple has to do is fix a few problems with the foundation of OS X which I listed. I also made a technical and strategic case for treating OS X like a completely new platform.

This is one of at least two follow-up articles I'll write on Apple's progress in fixing (or at least addressing) these key problems. I also want to clarify the specific issues raised by these problems, technical and otherwise. I won't touch issues I feel Apple has addressed already (privately or otherwise).

Apple's Preliminary Aqua HI Guidelines

While bemoaning the problems of UI conformity I placed all blame on Apple's current Mac HI Guidelines for being out of date and widely ignored due to it's incompleteness. As if my magic Apple did revise their human interface guidelines to be more thorough including list/matrix cell and text selection.

These guidelines are the key to gaining productivity with a graphical user interface. Without a general method to accomplish tasks common to various applications there is virtually no point to having graphical means of doing mundane tasks like setting preferences (to pick a random example). I use *NIX every day and avoid X11 apps like the plague, the command line is far less chaotic.

As I pointed out in my previous article, NeXT and Apple followed a different strategy to make applications behave similarly. NeXT used an abstract API while Apple used guidelines. There is virtually nothing programmatic keeping mac applications in check. Only the mac community and their reluctance to buy anything less than excellent (they did buy macs after all) held the common interface together. In fact I consider the mac community to have been far more successful than the NeXT community in demanding adoption of innovative UI conventions. In the NeXT community less emphasis was placed on making ideal interfaces. In their minds it was already taken care of thus NeXT programmers had far more leniency to create uniquely weird interfaces.

Now, more than ever, Apple's revised HI guidelines have paramount importance. Apple is supporting no less than three APIs: Carbon, Cocoa, and Swing. They are in danger of losing one of the few marketable htmlects of their product which they currently have a virtual monopoly: a unified interface. Seemingly on purpose the preliminary Aqua HI Guidelines do not address current differences in the implementations of these three APIs. I consider this a tactical and technical mistake which they will rue if not corrected. We all need to know what the eventual guidelines are if we are to get there in the least amount of time. Even if Cocoa or Swing do not follow the guidelines today at least we (programmers) will know what they will be and can plan accordingly. I'll list three omissions from the preliminary Aqua HI Guidelines to serve as an example:

• Command-clicking for selecting individual non-contiguous items was covered, however not command-dragging. In a list/array of items many applications allow one to select groups of items by command-dragging across them instead of having to command-click each item. This is very useful and I don't see why it shouldn't be standard.
• The guidelines only implicitly says that white spaces at the end of lines should move the cursor. This, in my opinion, ought to be made clear especially since Cocoa doesn't yet follow this behavior (white spaces near the edge do not move the cursor, and it's impossible to know if there are several spaces instead of one as of public beta).
• The guidelines do not say what text is to be selected when the selection extends to the end of one line, like when somebody drag-selects to the end of one line. In MacOS the implicit behavior (as well as Carbon's TextEdit behavior) was the carriage return is not selected while in OPENSTEP (and Cocoa in OS X PB) the CR is selected, similar to Windows.

As we'll soon discover this is a mere taste of what ought to be made clear in this very document.

OS X's Schizophrenia Revisited

In my previous article my gravest concern was OS X's schizophrenic nature, or its danger of making it's users schizophrenic. I use this term pretty much literally:

Main Entry: schizo·phre·nia
Pronunciation: "skit-s&-'frE-nE-&
Function: noun
Etymology: New Latin
Date: 1912
1 : a psychotic disorder characterized by loss of contact with the environment, by noticeable deterioration in the level of functioning in everyday life, and by disintegration of personality expressed as disorder of feeling, thought (as in hallucinations and delusions), and conduct -- called also dementia praecox
2 : contradictory or antagonistic qualities or attitudes (both parties... have exhibited schizophrenia over the desired outcome -- Elizabeth Drew)

Like a form of multiple personality disorder, OS X has severe identity problems. It supports three APIs plus classic which have barely been localized to each other. OS X has three sets of color pickers, three ways of selecting text, four pairs of open/save dialog boxes each with a different way to navigate, three different menu layouts, four different appearances, four different title bar drawing routines, three different menu bar drawing routines, three different sets of menu item states, the list goes on and on. I don't care much about appearance for my personal use (strategically it's more important), however when I have to adjust to the different behaviors of these applications my productivity nose-dives.

Many of these are obvious problems. For example the Swing open dialog box is basically a copy of Windows. Classic's open dialog box is the old navigation services and Carbon's is totally broken. Apple had made an attempt to make them similar looking, however I would rather have them behave the same or ideally THE same. Same with color pickers, no reason why Swing must have it's own. Cocoa's color picker is nice and probably ought to be used by all applications (with the exception of Classic). Copy+paste works for some data types like plain text, but styled text doesn't get converted when copying from Swing. Drag+drop doesn't work at all in Swing as far as I can see. Swing menus are designed for Windows and X11. Allowing programmers using Swing to comply with Apple's HI guidelines may require a change in the API to define more UI elements and standard items/categories so they can be placed correctly. Feel free to complain to programmers using Swing or this may not be fully fixed ever. These are all obvious and I've wasted enough time nit-picking.

A more subtle and problematic difference deals specifically with Cocoa. I mentioned this problem before but, based on the feedback I got, I don't think I was clear enough.

When selecting text we (mac users) tend to select left-down not right-up. I'm not sure how OPENSTEP users typically select text, but the only way to get the same behavior with Cocoa applications is to select right-up. The specific behavior difference deals with the carriage returns at the end of lines. When selecting from right to left without moving down to the next line, Classic, Carbon, and Swing will not select the CR at the end of that line. There are various reasons why this behavior is ideal. I'll list each separately with the consequence of the specific situation:

1) Nearly every time I select text I do not want the CR. I'm often pasting into text fields or a command line. Even when editing a document like this one I don't want to select the CR. In the rare case that I do I add it after pasting.

For this reason selecting without the CR ought to be easier even if deleting the CR is equally easy (which it isn't).

2) Adding the CR is always possible, removing unwanted CRs isn't (a CR will sometimes 'enter' text).

For this reason selecting without the CR ought to be the default.

3) CRs are paragraph delimiters. The general HI guideline is when selecting a 'word' then removing it and double word delimiters are to be avoided, the first one is deleted not the last one (whether in Eudora, BBEdit, or TextEdit.app). Selecting the first paragraph delimiter would be consistent, Cocoa clearly isn't consistent with itself by selecting the last.

For this reason selecting the first CR should be easy: by dragging down from the previous line on the right margin will select the first CR with the line thus making document editing easy. This drag action in Cocoa selects both CRs and is a pain.

4) Every other human interface uses a right-down drag motion like hierarchical menus (not to mention typing words is right-down), the only exception is Cocoa's text selection. Again, where is the consistency_

For this reason the desired selection ought to be possible using a right-down drag.

5) Just to avoid a schizophrenic interface where one uses a right-down drag motion in all apps except Cocoa where the motion is left-up.

For this reason the selection behavior ought to be uniform for all applications.

For the above reasons and more, I'm inclined to believe the mac selection behavior is far better than Cocoa's. Apple has already altered Cocoa's text handling behavior to comply with previously implicit (made explicit in Apple's HI revision) mac conventions. For example the down arrow moves the cursor to the end of a line in a text field (being logically consistent with dragging down) and pgup/pgdown/home/end keys scroll instead of move the cursor. Clearly Apple isn't blind to those differences however have failed to change the selection behavior. Either they aren't aware of this, or there is a NeXT clique in Apple resistant to change.

This isn't to say Carbon is without fault. Carbon would do well to adopt virtually everything else Cocoa's text handling does like triple-click, double-click-drag, shift-option-arrow modifiers, etc. There is evidence Apple is already doing this by updating TextEdit (Carbon function). Hopefully WASTE will follow these changes. Carbon's font rendering should be scrapped too.

Missing APIs and Opportunities

I would like to point out Apple hasn't released APIs and source which would allow 3rd parties to extend the user experience, and more importantly increase productivity. Apple has nothing to gain by keeping such things private. Their competitors can't use it outside OS X, only OS X developers and the wider mac community would gain.

As I pointed out in the previous article I love the Dock, but purely for strategic reasons. It's easily grhtmled and could allow Apple to break into new markets. However if Apple continues to micro-manage it's evolution most people will just get fed up with it's limitations. Apple should open source the Dock. I don't care if they write a license where they have control over any changes to the main tree, they have everything to gain and nothing to lose by allowing programmers to submit changes. As it is now I keep a really small (16 pixels high) Dock merely to keep me informed which apps are launching, for everything else I use Drop Drawers. This includes app switching, app launching (I have hundreds of apps categorized into several drawers), clipping storage, note storage, url bookmarks and sorting, etc. The full potential of the Dock has barely been scratched and it's a shame.

The OS X Finder uses Preview modules to show the contents of specific files. In fact if you preview a QuickTime movie link it will automatically download the movie via http/rtp/rtsp and display the streaming movie. Apple should release this API so 3rd parties can write their own Preview modules. Let's say StairWays Software writes a Preview module for a partially downloaded file. You can have controls in a Preview as evidenced by the QuickTime movie Preview thus you can have a button labeled "Resume" and after clicking it get a progress bar of the download. This is just one example, I'm sure many 3rd parties would love to write a Preview module instead of merely using 128x128 preview icons.

Apple still hasn't released the Preferences Panel API which would allow 3rd parties to write Preference Panels. This is no different than control panels in System 6 and before. I don't think I have to explain why it would be a good idea to release this API.

Apple should release the source to their Carbon shim and PEF launcher. There are other Classic libraries which ought to be available to PEF and Mach-O applications in OS X, for example Quesa (www.quesa.org).

Menu Management

All I have mentioned to this point are obvious problems with current technology, nothing revolutionary. I could mention countless other bugs, however I would rather end this article with at least one idea which could dramatically improve the way we use current applications. Something for Apple to consider for OS X 2.0.

There are several reasons why the evolution of MacOS was driven almost completely by 3rd party hacks. Primarily it's because Apple is a large corporation without a creative bone in it's body. They didn't invent the multi-finder, they didn't invent desktop publishing, they didn't invent floating windows or ubiquitous drag+drop text or hierarchical apple menu or 99% of the widgets we use daily. In fact they did invent a lot of stupid stuff which would have never succeeded as hacks like desktop mounted volumes. Anyway the second reason why hacks were so instrumental in evolving MacOS is because such hacks were possible to begin with.

MacOS doesn't have memory protection in the typical meaning of the term. This is to say each application doesn't have it's own 4 Gig memory space which no other process can access. Obviously it's a good thing that OS X has memory protection in that one application cannot directly cause another to crash, however it also has other consequences. In MacOS the structures created for system 'managers' existed in one global structure. This structure could be read and altered by anything else, including hacks. However in OS X each application maintains it's own structures (Apple had to overcome this discrepancy when designing CarbonLib by creating accessor functions). Now a 3rd party hack which requires access to these structures becomes near impossible.

I wanted to write such a hack for OS X but can't, so I'm forced to lobby for Apple to implement it instead. Every time I describe this functionality in the past I'm misunderstood so I'll try to get it right this time.

Mac HI convention properly states that all commands ought to at least be mirrored in the menu bar hierarchy. This means that virtually every command of every application is available to you via a menu item. What if you had absolute control over how you display and access all these commands_ By that I mean implement any method your mind could imagine to organize, display, script, and engage all commands currently at your disposal. Out of the zillions of menu items available to you, how many do you use in a typical day_ How drastically would your productivity increase if you were able to access those commands without using the menu bar hierarchy the programmer has selected for you_

If you would like to do any of the following:

• Set the hotkey of any menu item
• Drag menus off the bar
• Replace the bar with something else like NeXT-style menus
• Replace menus with something else entirely, like toolbars or whatever
• Reorganize the menu hierarchy
• Create custom menus with menu items from other menus
• Create global custom menus with menu items from other applications
• Have total scripting control
• Have more responsive menus
• Queue menu commands like you can currently queue mouse clicks
• Anything else imaginable

...then you're in favour of my proposal.

OS X currently has every application manage it's own menu. Each application calls a shared library (Carbon, AppKit, Swing, whatever) which registers menus and items in a local data structure. This is then accessed by other library calls which draw the menu and interpret mousing calls. The API doesn't specify how menus are managed, it only allows the application to create menus and items then interpret menu selection events. Two things ought to be immediately clear about this present situation:

1) When an application stalls when switching apps you either see no menu, or the menu of the inactive application which isn't clickable. Either way when you see a menu it's being managed in that application's process.

2) The hack I'm proposing is impossible.

There is a very good reason why the menu is managed each application's process. As you're using the menu the application is immediately aware of any selection you make because it's in the same process. If I were to write a hack whereby the menu was a separate process it would have unacceptable latency problems because for one process to notify another requires an IPC call (inter-process communication). However there is a way to have the menu structure in one location AND to avoid an IPC call, manage the menu between the application and kernel*. The obvious place to do this is the Window Server. Mousing and other events are sent from the kernel to the Window Server then to the applications. This is the current process:

Application registers the menus:
application->menu registering code

User clicks menu:
mouse->kernel->Window Server->application->event interpreting code->menu drawing code->display server

User selects item:
mouse->kernel->Window Server->application->event interpreting code->application responds

However if the Window Server managed menus the process would be:

Application registers the menus:
application->menu registering code->Window Server

User clicks menu:
mouse->kernel->event interpreting code->Window Server->menu drawing code->display server

User selects item:
mouse->kernel->Window Server->event interpreting code->application

Thus we have no delay and the Window Server has complete and utter control over how the menu behaves (interprets events) and is displayed. It should also be more responsive because the Window Server doesn't have to wait for the application to respond before the user can access his menus. This brings up a minor decision: how to handle cases where the Application isn't immediately responsive. This is important when the application's state can change after selections have been made. This is no different than problems you have when clicking an application with the cursor when it's not responsive. There are various ways to handle this situation:

• Menus are gray until application is responsive.
• Menus are red until application is responsive, yet selectable. Commands can be queued just like mouse clicks are currently queued. Queued commands which cannot be completed due to changing menu state are flushed and result in a beep or menu bar flash or something.
• Menus become invisible or lock up like they do currently.

Given the previous two choices why settle for the current behavior which is the third_ To keep everything consistent if the first option is chosen the user shouldn't be able to queue mouse clicks as we can now. Perhaps gray the cursor instead of that weird spinning color disc.

As you can see there are many reasons to move the menu management code into the Window Server even before we introduce the massive productivity potential of allowing 3rd parties to make custom global menus and whatnot. The work involved for Apple to implement this is not massive, nearly all the code to register, draw, and interpret events is already written, the difference is where it's executed. The bulk of the work is patching specific calls like changing menu state which now involves telling the Window Server to change the menu state and flush the command queue for that item. However I consider the huge potential this project to be well worth it. Apple would have a truly unique interface users can globally customize to their maximum potential and retain a unified interface!

If anybody is interested in petitioning Apple to include this in their next major revision please contact me.

(*some people have suggested this would be possible simply by patching the menu libraries, however this is incorrect. The only way to avoid an IPC call with a global menu is to execute that code between the kernel and application)

Summary

This was mainly an article to clarify some of OS X's current problems and some of the progress Apple has made in these fronts. Hopefully we can put more wood behind the arrow in making specific suggestions for Apple instead of the "It's not MacOS 9, waaaa" fluff Apple has had to deal with the past year. Looking forward to the future, aren't you_ |-)