Day 11 Reading Summary

Today's readings started looking at how usability fits into the open source community, and considers several case studies where usability is being applied to popular OSS project (such as GNOME). The readings are:

• Getting to know you: open source development meets usability
• Professional usability in open source projects: GNOME, OpenOffice.org, NetBeans
• Why free software usability tends to suck
• Why Free Software Has Poor Usability and How to Improve It
• The Luxury of Ignorance: An Open Source Horror Story
• The Luxury of Ignorance: Part Deux
• Daring Fireball : Ronco Spray-On Usability

The paper, Getting to Know You: Open Source Development Meets Usability, introduces a new track of study in the SIGCHI 2002 conference, which is around the issues of usability in the open source world.

The paper begins by reiterating the point that most Open Source endeavors are a form of Computer Supported Collaborative Work (CSCW), as contributors are scattered across a wide geographic area, and use computer tools to aid in communication and co-ordination. In fact the open source community is likely the largest community using CSCW on a daily basis and to an extreme degree. The study of usability in such a large CSCW community is of particular interest to the HCI community for several reasons. Firstly, it tells how the development of products occurs in non-hierarchical organization which doesn't have geographical boundaries. Secondly, usability practitioners can study a community which contributes their time in many cases free of charge for the development of products, and what impact that has on issues such as the quality of products. Finally, the benefits and potential costs for making product development follow a participation chaotic and open process to product development, can be better studied.

On the other hand, the Open Source community has much to learn from the HCI community. For one thing, as the value of open source software begins to be realized by corporations and the adoption of open source software starts, usability becomes an increasingly important topic. While the open source community tends to be enthusiasts that can handle the eccentricities of OSS projects, corporations are less patient when usability issues arise. In addition, due to the chaotic nature of OSS projects due to the lack of hierarchy and control structures makes coming to a consensus on a wide variety of issues difficult. By applying HCI methods, a set of goals and priorities can be formed from these tools that provides a basis for sound design in the future. HCI methods therefore can become the standard by which the meritocracy of the community use in evaluating designs. There are challenges in educating the open source community to appreciate these concepts. Likely the most important thing that the OSS community needs to learn more about is that there are people out there that can help them with usability .

The article argues that there are three sore spots in usability in the open source world, namely:

1. Lack of systems that are usable by ordinary people
2. Lack of application of consistent usability principles across contributors
3. Lack of attention to accessibility issues in OSS

The paper, Professional Usability in Open Source Projects studies the contributions that Oracle/Sun (formerly Sun Microsystems) made to three projects : OpenOffice.org, GNOME and Netbeans. In addition, it also discusses the usability challenges that these three projects face.

The OpenOffice.org project is an open source spinoff of the Sun Microsoft's Star Office. The OpenOffice.org project has access to the StarOffice User Experience team which the author claims includes professionals from all fields of HCI. Formal user requirements for future releases of StarOffice/OpenOffice come from several sources including:

• Sun's Marketing Team for Business Requirements
• The OpenOffice.org Bug reporting system provides a venue for the Open Source community's requirements

The design and specifications for future releases go through a formal iterative process which engages the community through several mechanisms (such as newsgroups, community events, etc.) The author's don't seem to express any weaknesses as far as usability is concerned in the OpenOffice.org project, which seems troubling as any usability project can be improved

It makes the controversial argument that providing usability know-how to OSS projects leads to usable and used products which in turn is beneficial for developers, corporations, and users. However, there have been several studies which have shown that many developers in OSS projects see usability as a disincentive for writing applications as they have to field questions from beginners, and accommodate a group of users which don't represent the user base that the developer created the project to, "scratch an itch," for.

The usability of GNOME is another area where Oracle/Sun has contributed its resources. One of the major contributions and advancements in usability the paper argues was the adoption of Human Interface Guidelines (HIGs) and the establishment of the GNOME Usability Project (GUP). As a result, the paper argues, usability many usability bugs in GNOME have been caught and addressed. Accessibility technologies have also been added to GNOME which in turn makes GNOME approachable for those with various disabilities. Finally, documentation has been improved over time and translated to several different languages so that developers and users a like have reference material for using GNOME.

The challenges that still face GNOME, the author argues are:

• An approachable communication medium for non-technical contributors
• The GUP doesn't provide consultancy for applying GNOME HIGs to their applications which can cause an inconsistent user experience
• The vision of GNOME's target audience is not well documented.
• Lack of application of a rigorous usability process
• Many OSS developers don't contribute to the usability aspect as they feel Oracle/Sun can pick up the slack

Finally the Netbeans project is an Open Source Java IDE. The author claims that Sun performs regular usability studies of the program, with usability issues being given equal priority to functionality bugs. In addition, to usability studies they also created usability guidelines for the development process of the project. Beyond the functional usability aspects of the project, the graphics assets of the project are also evaluated for usability as well. Finally, accessibility and localization of Netbeans to several languages also means that the software is more usable for those with disabilities or who communicate in different languages.

The biggest challenges of Netbeans the author argues are:

• The lack of a unified communication system (currently scattered between mailing lists, and bug reports)
• Are we targeting and engaging our audience through appropriate mechanisms (i.e. mailing lists)?
• A lack of ownership for functionality, as both developers and designers contribute

The paper also makes the argument that Open Source projects should lend themselves more easily to usability techniques. Indeed the cathedral method of development encourages quick and often iteration over careful monolithic, "cathedral," design. Iterative prototyping techniques, for instance latch on the cathedral development philosophy. Nevertheless, making usability techniques approachable to contributors whose incentive is to contribute some of their spare time to causes that they find fun and interesting will be an important factor that needs to be considered when determine how to make usability more approachable and applied in the Open Source community.

Why Free Software Usability Tends to Suck is a laundry list of usability issues that the author notes in open source projects, and include the following:

• Few non-paid dedicated interface designers, who tend to be less experienced
• Every contributor to the project tries to do interface design, which leads to inconsistent vision and detail of design
• Dedicated interface designers are given less weight in OSS projects as many don't know how to contribute patches and may lack the technical skill the meritocracy seeks
• OSS projects copycat Microsoft,Apple and other proprietary software companies that may have poor design and which leads to a lack of usability innovation
• The software is designed for power users as it is made by power users
• Minor details (such as fine tuning error messages) are ignored by developers as they are seen as unsatisfying to work on
• Many features and capabilities are options hidden away in preference dialog boxes due to disagreements that occur between contributors, as well as for credit
• Releasing early and often means that the community experiences buggy, slow and unusable systems, which causes users to push back the proposed changes

The article, Why Free Software has Poor Usability and how to Improve it, is a follow up article by the author of the prior article, several years after the article was written. The author argues that while some minor incremental improvements in usability have occurred over the years, usability still needs improvement.

Suggested improvements to Open Source usability the author argues are:

1. The Open Source community needs incentives for making software usable, much like proprietary software companies have the incentive of increased revenue for usable products.
2. Have developers learn the principles and concepts of usability.
3. Create roles in FOSS projects specifically to handle usability issues.
4. Make design suggestions have equal footing with code patches
5. Implement concrete usability measurements that can be monitored over time
6. Incorporate frequent user testing
7. Get the design right before getting the code right ; don't make design be an after thought
8. Interface design should be conducted by dedicated professionals who can create a consistent user experience
9. Innovate in design - don't copy others who may have usability issues of their own
10. Encourage simplicity in design as opposed to a design suitable for power users
11. Focus on the small details of interface design, they really make all the difference
12. Use strong project management to remove the option overload found in OSS projects
13. Find alternative mechanisms for giving contributors credit than adding items in options dialog box
14. Use high bandwidth communication mediums for discussing design (such as VoIP, virtual whiteboards,etc.), as design is a rich medium.
15. Make user's expectations resemble final product expectations when releasing early and often
16. Test modules/components against a variety of different use cases, so that it can integrate better with front-ends
17. Increase co-ordination and communication across project and team boundaries

The article, An Open-Source Horror Story, discusses Eric Raymond's difficulties in setting up CUPS (Common Unix Printer System), to connect to a networked printer in his house. He attempted to install the printer in what seemed to him to be an intuitive way (Selecting System Settings -> Printing). He then proceeded to add a printer queue as it would seem to him that he wanted to add a printer. However, as he later discovered the way the developers implemented the system, you are supposed to join an existing printer queue rather than add one. In any event, he mentions that when he proceeds to the next screen of Add New Printer Queue it presents him with a variety of different options, that seemed confusing to him, as well as to a beginner user, he feels. When he attempts to access help, the context sensitive help only discusses adding a new local printer, and not the actual problem he was trying to solve (add a new networked printer). The story goes on to discusses what he feels are less intuitive things, such as fields filled in by default but which have no meaning to the, "typical user."

Eric Raymond's central argument in this article is that while open source usability is improving, it still seems focused on the style of what is archetypically considered usable software (i.e. GUI interfaces, online help,etc.) He argues is that what the Open Source community needs to think about is the substance usability. Part of this he argues, is putting themselves in the shoes of novice users. Novice users, he argues by definition won't understand the technical jargon that is found in many OSS projects, and frustrated will stop using OSS with the perception that all OSS is hard to use. Other things that he felt OSS lacks include discoverability, as many features and capabilities are hidden away or are made to not be the default option. In addition, local documentation is lacking, and one tends to need to Google the solution, which in turn may contain pages with half-solutions and half-truths in them. He goes on to make the controversial remark that if you want to make software for novice users, the users shouldn't (and likely won't) consult the user manual to complete the tasks they want done.

The Luxury of Ignorance Part Deux, is another article by Eric Raymond where he reviews the community feedback that he received several days after he published An Open Source Horror story. An interesting aspect of the article is that the author claims to have received many e-mails from readers more or less saying, "Gee. And all this time I thought it was just me...". This implies that there many users have pent up frustration with many OSS tools, but never act on their frustration, namely making more public that the problem is not with them, the user, but rather with the system and the way in which it was designed.

The e-mails that Eric Raymond received included many users (from beginner Linux users to veteran UNIX user of 20 years), which each discuss a frustration that they have in using an aspect of OSS. Some have hinted that they were thinking of switching back to proprietary solutions are they are easier to use. Many of the writers discussed the time spent (and arguably wasted) trying to figure out how to setup something in Linux. Others argue that Linux users should not be embarrassed or, "flamed," for having difficulties using Linux, and that these complaints ought to be taken seriously. Finally he argues that it doesn't matter who is at fault for these usability issues, but what matters is that these usability issues be fixed.

Finally, the article Ronco Spray-On Usability is a critical review of Eric Raymond's Open Source Horror Story, as well as to further satirize Eric Raymond.

The review of Eric Raymond's Open Source Horror Story, begins by arguing that Eric Raymond's insistence of having developer's put themselves of the shoes of Aunt Tillie (a novice user) is dangerous. For one thing, he argues, how many novice users want to setup a shared networked printer? More often than not, he argues, they will want to connect to a locally attached printer. The central theme behind this is that the novice user may very well have different typical use cases and scenarios than a power user, and hence one should study how people actually use the system. This is in stark contrast to asking, "How do I make this feature easier for a novice?" What was interesting he noted was that non-novice users (i.e. power users) were having difficulty completing many tasks in OSS, as they have mentioned in the e-mails sent to Eric Raymond, even though they thought that they were the only ones experiencing those types of problems.

He makes another important argument which is that user-experience, and user interface design aren't something that you just spray on a technically competent solution. He mentions several times that in Eric Raymond's Open Source Horror Story, he mentioned that UI design, was not, "rocket science," or difficult. He implies that the hard part (getting the low-level technical development correct) has already been done and all we (the community) need to worry about is strapping on an easy to use user interface. However, as everyone in the HCI community knows, this isn't the right way to get user experience right. In fact techniques such as Contextual Design, are processes where the low-level and high level design of a solution are built around the needs of users.

Another particular reason why Open Source usability is not done well is the fact that in most open source development projects, people are not paid or compensated for their work. However, it is a well known fact that good user interface design is the result of long, hard work by talented developers and designers. Having many developers and designers work part-time on a project creates a lack of focus and consistency in user experience. In fact many open source supporters are switching over to closed source solutions due to these frustrating technical problems for which the closed source community got right!

Finally, he ends by discussing the engineering adage, that you can build something that is, "fast, good,cheap : pick two." While Linux is both fast (release early, release often) and cheap (low cost to free), it is not good. The fact that Linux is cheap may appeal to Wal-mart shoppers and people who are genuine cheap-skates, most people are willing to pay for the luxury of ignorance (i.e. good systems). This is precisely the market that Windows and Mac OS X serve (fast and good) and why Linux is struggling to overtake these two proprietary solutions.