Книга: Agile Software Development

Shaping the Crystal Family

This chapter describes how I resolved the dilemmas involved in methodology design: the difficulty of communication, the need for people to be people within the methodology, and the need for multiple methodologies. I chose to construct a family of methodologies, along with principles for tuning them. This is not a kit of methodology parts for you to assemble on your own, but a set of samples that you adjust to your circumstances.

"Crystal" is the family name for the methodologies. As with geological crystals, each has a different color and hardness, corresponding to the project size and criticality: Clear, Yellow, Orange, Orange / WebStream, Red, Magenta, Blue and so on. Each one Is people-and-communication centric Gets adjusted to fit its particular setting Works from the project tolerance level and the bottleneck activities to an answer that matches the project ecosystem. This chapter describes three members of the Crystal family that have been put to use on live projects: Crystal Clear, Crystal Orange, and Crystal Orange / WebStream. For each one, I Describe the characteristics where the methodology is appropriate Describe the methodology itself Reflect on the construction of the methodology The reason for including this chapter is to show one way of working through the problems and principles surrounding methodology design, and to give you something to copy and alter when you start on your own.

Shaping the Crystal Family

Two plausible responses to the problem of needing multiple methodologies are to Create a kit of methodology parts that the project team assembles for their project, and to Create a copy-and-alter family of specific methodologies that get tailored on each project.

Rational Corporation used the kit approach in the first generation of their methodology product, the Rational Unified Process (Krutchen 1999). RUP is a framework for constructing methodologies for individual projects. It is centered around processes, work products and tools, with a collection of "best practices" to guide the practioner along the way

The assembling of a correct methodology for a RUP project hinges around creating a "development case" for the project, and then assembling the parts from the RUP kit that fit the development case (Larman 2001).

The standard mistake managers make is not to do that assembling and tuning. They drop the library of work products on the development team and say, "Do that." The developers do one of two things: They recognize that producing all of those work products will damage the project, so they ignore the manager's instructions; or They do as they are told and produce all of those work products (and damage the project accordingly).

RUP is not incompatible with the principles developed in this book, but it doesn't naturally lead people to focus on the two key success factors, communication and community.

My hope is that after reading this book, managers who buy RUP will allocate time to get it tuned it to their projects. I also hope that the people who do the tuning will cut down the required work products produced to the smallest possible set, and augment RUP with attention to communications, community, concurrent development and so on.

Crystal Family

An alternative way of approaching methodology-per-project is to collect a set of concrete, sample methodologies that have been used on projects, and let the people on the project use the tailoring techniques described in the last chapter to adjust them on the fly.

This is the approach Jim Highsmith and I are following. We are collecting examples of successfully used, communication and community based agile methodologies that people can use as starting points.

By seeing one that is already written, the new project team can see how the communication and community issues were addressed in a real situation. By having a set of examples to choose from, the team can find the one that most closely matches their situation.

I nickname the ones I design, "Crystal."

The word Crystal serves two purposes.

First, it is just a pleasant name. In the book, Crystal Clear, I create a protagonist called Crystal to personify the methodology, and argue for its design.

Second, it provides a metaphor that supports the first two degrees shown in the project grid in Figure 4-21.

Moving right in the grid means coordinating more people, which means a heavior methodology is needed. In the crystal metaphor, moving right corresponds to choosing a darker color (clear quartz, topaz, ruby, sapphire).

Movement up in the grid corresponds to more potential damage from the system., and the use of more rigor and ceremony. In the crystal metaphor, moving up means increasing "hardness" (in the mineral hardness scale, diamonds, the hardest stone, receive the harness number 10).

Thus, in the crystal metaphor, two people programming the overtime food menu are working on a project that calls for a soft, clear quartz crystal methodology. The two people programming the movement of boron rods in a nuclear reactor are working on a project that calls for a diamond-category methodology.

Of the two dimensions, I find the color dimension the more useful as the project index. The hardness dimension can be more easily picked up in the methodology tuning workshops.

I therefore index the Crystal methodologies by color: Clear, Yellow, Orange, Red, Magenta, blue, violet, and so on (Figure 6-1).

Figure 6-2. The Crystal methodologies are named by color. In Figure 6-1, I omit life-critical systems from the shaded areas. This is because I have not worked on or interviewed life-critical-system projects, and so don't have enough information to say exactly how an agile life-critical project looks.

The gray E6 box, outside Crystal Clear, indicates that Crystal Clear does not explicitly address "essential moneys" projects, but that the team may be able to stretch Crystal Clear to such a situation.

The other restriction on Crystal methodologies is that they only address colocated teams. As discussed earlier, none of the distributed and off-shore development projects I have seen would count as methodologically successful. They only recommendation I have for such projects is to put the team together at one location.

Crystal does not aim to be upward or downward compatible. In using computer hardware, there are large financial consequences to changing hardware, which cause compatibility to be a key issue. I don't see similar consequences in moving up and down the methodology scale. People working on a four-person project that grows to become a 20-person project shouldn't ask, "How do I preserve our former working conventions?" They should ask, "What is a good way for 20 people to work together?"

Core Crystal Elements

The core Crystal philosophy is:

Software development is usefully viewed as a cooperative game of invention and communication, with the primary goal of delivering useful, working software, and the secondary goal of setting up for the next game. Two consequences of that philosophy are that different projects need to be run differently, and the amount of modeling and communication that people need to do is just the amount they need to jointly move the game forward.

Members of the Crystal family share

Values and principles

On-the-fly tuning

Two values are that Crystal methodologies are intrinsically

People and communication centric High tolerance

The former means that tools, work products and processes are there only to support the human component. The latter recognizes varying human cultures. Within the tolerance of the Crystal family, though, the team can choose to work in a high-ceremony or high-discipline manner (adopting parts of PSP or XP, for example).

Seven principles were discussed in Chapter 4, "Methodologies." The principles are roughly summarized as follows:

The team can reduce intermediate work products as it produces running code more frequently, and as they use richer communication channels between people. Because every project is different and evolves over time, the set of conventions the team adopts, must also be shaped and evolve. The shifting bottlenecks in the system determine the use of overlapped work and stick information holders.

The two rules common to the Crystal family are: The project must use incremental development, with increments of four months or less (with strong preference to one- to three-month increments). The team must hold pre- and post-increment reflection workshops (with strong preference to also hold mid-increment reflection workshops).

The two base techniques in Crystal are: The methodology tuning technique: using project interviews and a team workshop to convert a Crystal Clear is a methodology for D6-category projects. You should be able to stretch Crystal Clear to an E8 or D10 category project with some attention to communication and testing, respectively. I expect it to be difficult to extend beyond that, since Crystal Clear contains no communication structure for more people than can conveniently work in the same room, and lacks the system validation elements needed for life-critical systems.

Brief Description of Crystal Clear

There is only one team, seated in one or adjoining offices.

The roles needing separate people are: Sponsor, Senior Designer-Programmer, Designer-Programmer, User (part-time at least)

One of those people may pick up the assignment of being Project Coordinator. Some one will be the base methodology to a starter methodology for the project. The technique used to hold the reflection workshop.

You are welcome to replace those two techniques with others, if you have another way of acoomplishing their goals.

Attending to the above issues creates something that has a particular feeling. As someone wrote, it is possible to make another methodology look like a Crystal project, without making it feel like a Crystal project. A visitor to a successful Crystal project will notice communications and community in action, pragmatism in reaching the two goals of the game.

The following three sections describe the three Crystal methodologies I have so far constructed and seen used.

The structural differences between them are obvious. See if you can spot the commonalities.

Clear

Business Expert, and either one or many people will share the role of Requirements Gatherer.

The Senior Designer-Programmer is the key person on the team. The other Designer-Programmers may be at some mixture of novice and medium levels as the Senior Designer-Programmer and the problem can support. Hiring people who know their jobs of course helps.

The policy standards are that Software is delivered incrementally and regularly, every 2-3 months. Progress is tracked by milestones consisting of software deliveries or major decisions, as opposed to written documents. There is some amount of automated regression testing of application function. There is direct user involvement. There are two user viewings per release. Downstream activities start as soon as upstream is "stable enough to review".

Product and methodology tuning workshops are held at the start and middle of each increment.

The policy standards are mandatory, but equivalent substitution is permitted, as would be the case if Scrum work scheduling (Schwaber 2001), XP, or Adaptive Software Development (Highsmith 1999) policies were used.

The work products that are produced include: Release sequence

Schedule of user viewings and deliveries Annotated use cases or feature descriptions Design sketches & notes as needed Screen drafts A common object model Running code Migration code Test cases User manual

The following are left as "local matters," to be set and maintained by the team: Templates for the work products Standards for coding and user interface Standards and details of regression testing

Crystal Clear does require project documentation to be created. Just what that documentation consists of is not spelled out by Crystal. That is left as a matter of local judgement. The combined team must decide how to present their design notes to future team members.

The most important tools the team can own, besides a compiler are:

A versioning and configuration management system A printing whiteboard.

You should be able to cost-justify several printiing whiteboards on any project, based on just the time people save typing design documents and meeting

Crystal

Crystal Orange is a methodology for D40 category projects: up to 40 people, sitting in one building, working on a system that might cause loss of discretionary moneys, summaries, and lost communications cost when people do not copy down whiteboard contents.

The techniques used by the individual roles are left entirely to the discretion of the individuals.

Substitution of elements from from similar methodologies is permitted. For example, the team could decide to Scrum or DSDM's timeboxing and dynamic prioritization policies, Scrum's daily stand-up meetings, pair programming from XP, and so on.

Reflection on Crystal Clear

Crystal Clear is the most tolerant, low-ceremony small-team methodology that I can find that still works.

It contains those elements claimed by my interviewees to be the cause of their success: Focus on close seating and close communication Frequent delivery Information from real users Code versioning tools.

The printing whiteboard has proven more valuable than any of its higher-tech replacements, with the possible exception of the newest generation whiteboard capture software (see Pixid URL). People usually start a discussion thinking it won't be worthwhile recording. They discover after their discussion that it would be good to have a record of.

Crystal Clear provides a place to fall back to if you try and give up on XP. Any part of XP can be substituted for Clear, since XP meets all Crystal Clear standards except for documentation. If you move from XP to Crystal Clear, you will have to add documentation. I don't think you can get any sloppier than Crystal and still plan on having better-than-even odds of completing successfully.

Orange

Crystal Clear calls out more team structures and more team coordination than is needed on a 20-person project. It is lacking in the subteaming structures that are needed on an 80-person project, and it is missing design- and code verification activities as would be used on life-critical systems.

Crystal Orange receives given 18 pages of description in Surviving Object-Oriented Projects (Cockburn 1998). It is characterized there as "for a medium-sized production project in an industrial setting. The characteristic of such are project are:

* 10 to 40 people total.

* 1 to 2 years duration.

* Time-to-market is important.

* There is a need to communicate with present and future staff, and a need to keep time and costs down.

* It is not a life-critical system.

It is a common sort of project, requiring trade-offs between complete, extensive deliverables and rapid change in requirements and design. I have kept the number of deliverables low, to reduce the cost of maintaining them, yet included enough to keep the teams communicating. I tailored job assignments and teams to allow the fluidity usually needed on this kind of project. Many other sorts of projects also need provisions for fluidity and can take advantage of this methodology. "

Recalling that lighter is better as long as it lasts, a team on an E50 type of project might extend Crystal Orange with some additional verification testing elements, rather than shift to a Red methodology targeted at 80 people.

The roles on the project include Sponsor Business expert Usage expert Technical facilitator Business Analyst/Designer Project Manager Architect Design Mentor Lead Designer-Programmer Other Designer-Programmers UI Designer Reuse Point Writer Tester

They are arranged in several teams:

System planning

Project monitoring

Architecture

Technology

Functions

Infrastructure

External test.

The larger functional team is split into cross-functional groups using the Holistic Diversity strategy (Cockburn 1998). Each such group cantains a Business Analyst-Designer, a UI Designer, and one to three Designer-Programmers. Each group also contains a database designer and reprentatives of other technologies if several technologies were in use on the project. Each group may have a Tester.

The structure of the teams has to be adjusted to account for possible shortages of certain specialists. The point of having a cross-functional group is to reduce deliverables and to enhance local communication. The people are evauated as a single group, so that each sees a purpose to contributing wherever he is needed, not just in his job description.

The work products include Requirements document Release sequence Schedule Status reports UI design document Common object model Inter-team specs User manual Source code Test cases Migration code

Each work product is developed until it is understandable by colleagues, to a level of precision and stability that permits peer review.

The policy standards are identical to those of Crystal Clear, except that the incremental delivery period may be extended to three or four months.

As with Crystal Clear, the policy standards are mandatory, but equivalent substitution is permitted, as would be the case if Scrum XP or Adaptive Software Development policies were used.

Work product templates, coding style, user interface standards and details of regression testing are left as local standards, to be set and maintained by the team. The techniques used by the individual roles are left entirely to the discretion of the individuals.

Reflection on Crystal Orange

Crystal Orange is not a structure to impose on a group of only 10 people. It is much too heavy. However, for 40 people working in three or four technologies, it is very light. It is held together by close communication within the Holistic Diversity

Crystal Orange / Web is a methodology we created for eBucks.com, a company delivering their code to the web in a continual stream. It differs from Crystal Orange in that this methodology does not deal with a single project, but with a continuous stream of initiatives requiring programming, each initiative's results being merged with the growing code base being used by the public.

This methodology is still in its trial run. I include it here because An increasing number of companies are finding

themselves in this sort of situation. It represents the most recent application of the ideas in

this book. It has a it a different shape than Crystal Orange.

The eBucks.com situation was interesting for a second reason (the first being the continuous and criss-crossing web of demands from different customer groups). The company had already established a web presence. They were no longer driven by time-to-market pressures, but were moving into one governed by the cost of defects. Customer calls, arriving in exponentially increasing volumes, could easily negate their profit margins. Thus, they functional groups and the frequent viewings by the users.

This methodology has been used successfully. That experience is described in the project Winifred report, in Surviving Object Oriented Projects,.

While I would use the basics of the methodology again gladly, technology has shifted so that the specialists who show up on the project are different today than then, and their work products and needs for interaction are different. Also, the bottlenecks on the next project will probably be different than they were on project Winifred.

On a new project, I would use Crystal Orange as a base methodology and shape it using the methodology tuning technique described earlier.

Agile were shifting from productivity to defect freedom as their top priority.

There were about 50 people to coordinate in this situation, executives, business people, managers, analysts, programmers, testers. I classified this as an E50 situation.

The group was relatively new, so some process definition was needed to make clear who made which decisions, and who handed what information to whom. Otherwise, people generally knew who they had to talk to in order to get their job done.

I performed interviews, as called for in the methodology tuning technique. I interviewed people in each job role, from marketing through testing and system operations.The interviews revealed that: Convection currents of information were quite good.

Everyone was on one floor. The had movable glass and whiteboard partitions as walls, so they could see and signal to each other, while still keeping some privacy. Ongoing distractions were keeping people from having the quiet time they needed to make progress on their assignments (in all job roles).

Each person was working on multiple initiatives, with frequent interruptions.

Crystal Orange / Web

Attitude, amicability and morale were still quite good, but sinking because of the frequent interruptions and lack of progress. Also, the programmers sat on one side of the building, while the business specialists sat on the other. This meant that the chit-chat in each group drove negative commentary about the other group.

The company was less than a year old, meaning that old habits had not yet set in, so people were open to inventing new work habits and conventions. In keeping with the idea that a methodology is the set of conventions the group agrees to, we wrote the methodology as a set of conventions, in five categories. Here they are:

1. Regular Heartbeat, with Learning

The purpose of this category is establish a core procedure for getting feedback on "how we work around here" and taking the time to reflect and improve on that. Every convention except the post-cycle reflection workshop can be altered as an outcome of the reflection workshops.

2-week development cycles. Overall production runs in fixed-length development cycles of two weeks. After each delivery, each team may opt for their next delivery to be either two or four weeks, depending on what they can deliver of use to the public. Each team must deliver something useful to the public every four weeks.

Post-cycle reflection workshop, suggestions visibly posted. At the end of every cycle, the company meets to discuss what worked well, what didn't work so well, and what ideas to try out on the next cycle. The outcome of the meeting is a posted list of things to keep.

2. Basic Process

The purpose of this category is to organize who creates which pieces of work and who makes which decisions, in order to avoid duplication or gaps in the effort, and to look far enough ahead to spot potential troubles early. The process aims for delivery of business initiatives live to the web.

A business owner writes a business use case and a system use case brief (Cockburn 2001). The business use case illustrates the proposed new system features in operation, paying particularly close attention to the manual business processes that get invoked when things go wrong.

The brief is used by the technology group to estimate the work involved in creating the features. The business executives review the business use case, technology estimate and value of the initiative before agreeing to further work. The user interface designers work with marketing and the detail business analysts to incorporate the features into the overall site flow, and then produce screen designs and the XML for the screens.

The detail business analysts produce detailed use cases and data descriptions, which go to the user interface designers, the server programmers and the servlet programmers. The servlet programmers work from the XML for the user interface, the use cases, the data descriptions and the server interfaces, producing the servlets. The server and servlet programmers produce regression tests for their code, peer-review the test cases. When the test cases are deemed good and the code passes the tests, the code is passed to the integration testers, who perster the developers to fix whatever remaining errors they find before the major deployment.

The integration testers post the changes going out in the new release to the internal group and also to the call center.

For live code, the call center returns bug reports to a special SWAT team whose sole purpose is to fix problems in production. The SWAT team is selected from the development group on a rotating basis every two cycles.

3. Maximum progress, minimum distractions

The purpose of this category is to ensure that people are working on what is of greatest value to the company, and have time to focus and make progress on that work.

The top corporate key initiatives are prioritized & visibly posted for each two-week production cycle. They are allocated to individual people so that each person knows their top two or three personal priority items for the cycle.

Work is broken into what can be completed and tested in the two week cycles, further broken down into things that can get accomplished in 1-3 work days. Each person working on more than one initiative is guaranteed at least two consecutive days to work on any one initiative without being pulled onto another assignment.

The developers post on the whiteboards outside their office the current status of the work they plan to complete this week. Every morning, the developers meet with the business owner of their current work initiative, in a short meeting to determine the current state of the work, the top work priorities and to discuss any questions. The business owner is not permitted to ask for status again the rest of the day. The period 10:00 - 12:00 each day is declared "focus time," in which no meetings take place, and everyone in the company is encouraged to turn off their phone.

4. Maximally defect-free

The purpose of this category is to construct a culture of "kill bugs here!"

Every server and servlet class will have a set of automated regression unit tests, written by programmer for his/her own code, using JUnit and HttpUnit, or equivalent. Programmers only release code to integration test when the tests have passed the scrutiny of a peer developer. the integration tester therefore get the code, the test cases, and a note from another programmer saying she/he will vouch for the quality of the tests.

The server contains a loopback mechanism so that the integration testers can maintain their own, controlled test database (which other people can use).

There will be a small, pidgin language that can be used by business people to construct sample business transactions and name an expected response. This little language allows integration testers, business owners, and the servlet writers to construct a test scenario and add it to the test database.

Screen-click statistics from the call center are posted in a visible place so that everyone can see where the public is having difficulties, whether navigation difficulties or programming defects

5. A community, aligned in conversation

The purpose of this catgory is to indicate the long-term target toward which the company is aiming.

Eventually, the programmers, user interface designers, testers, business owners, marketers, and so on, should sit in cross-functional teams to maximize the effect of conversation around delivering initiatives across specialty boundaries, and to minimize the effect of rumoring about others specialties. This will have to be balanced with staffing levels and growing space needs.

Reflection on Crystal Orange / Web

Two things strike me about about this methodology.

The first is the reduced role of process and work products in expressing the methodology. They are present, but occupy only a fragment of the space usually devoted to them.

The second is the general absence of concurrent development, which is one of my favorite development speed-up techniques. Concurrent development is missing because of the bottlenecks in the system.

The programmers had an enormous work backlog, no spare capacity, and were being constantly interrupted. The people were quite inexperienced in both developing software and in the business domain. These two points together meant that the programmers were not able to do overlapped development and hold the requirements in an oral culture. They needed stickiness in the information, which meant having specs written down for them.

With time, this should change, and as it does, I hope they will reduce the paperwork and increase the conversation. In the meantime, they need the paper.

Six Months Later

I present this methodology as it was constructed as the starter methodology. We would expect to see some drift over time, both as people thought up new ways of working, and as they drifted away from the high-discipline practices.

Michael Jordaan, CEO of eBucks.com, made these comments about the group's work habits six months later: "Obviously, when you left some disciplines survived while others did not stand the test. The survivors include the forthighly heartbeat with carefully planned cutoff times, which allows for developers and business owners to plan, testers to test rigorously and customers to be informed upfront of scheduled upgrades.

We discussed a three week heartbeat, but this was considered too long. More complex issues than can be solved in two weeks are run at twice the heartbeat (four weeks), but we still encourage incremental rollouts. The post-heartbeat meeting is strictly enforced and it has become one of the few times that I get to speak to the entire team. I have made quite a point of paying tribute to those involved in succesful upgrades. Hopefully this public recognition is motivating.

And What Should I Do Tomorrow?

Whether you use Crystal or not, increase morale and communication on your project so that the people trade information a little bit better. This applies for any base methodology.

Mistakes are discussed and suggestions for improvements have been made at every meeting, supporting the learning culture we are creating. The quality of code going live has improved greatly as the testing team has a veto power, to prevent bad code from going live (and this can be embarrasing). The SWAT team, dedicated to eliminating live bugs have also made great strides in responding to customer and call centre queries.

Focus time is still adhered to (and we still ring a bell every morning at ten). If I go a single day without these two hours I now start panicking, so useful has it proven to be.

Some things that did not survive was the habit of posting current priorities/ work progress on a board. Maybe interruptions are less of an issue now, as people work from home or maybe relationships between business owners and developers have stabilised. Maybe people are just lazy.

Most developers have a maximum of three tasks at any given time, except for the two key people working on the back end, who may easily have a list of 15 each. Moreover they are still interrupted by live issues, which interfere with their completion of tasks and lead to much frustration by other developers and business. The issue here is lack of skilled resources. It is the age-old problem that training employees to assist, while undoubtedly the right medium term solution, takes longer than simply doing it yourself."

In those comments, what I notice with some satisfaction is that the team still uses the core elements of the process: heartbeat with learning, and have found ways to modify even that heartbeat to fit their needs.

I notice the discussion of talent and skills as being critical to the project, and I notice the drifting away from what probably was embellishment in the methodology.

Get your experienced developers at Level 2 in methodology design. If your project doesn't have anyone at that level, do two things: Study your base methodology.

Start holding reflection workshops so that someone Compare what your team is doing with the three gets up to Level 2 soon. methodology samples given in this chapter. Choose a few ideas to apply on your own project.

Оглавление книги