The Central Problem: Information Decreases with Time While Authority Increases
One specific lesson we've learned is that UX, or any upfront documentation, can, break agile by becoming a defacto waterfall process.
Put another way, UX is the primary reason agile is important, yet formal UX practices often induce a waterfall model.
One reason for this is simple psychology. With time, any specification becomes a defacto authority and replaces conversations that are long lost once the software is actually being built.
Challenging Ancient Texts of Wisdom
How do you challenge something that was the result of hard won agreements three months ago? And if you reopen a conversation, what is the impact on the rest of the carefully architected UX cathedral you have built? As is stands, a basic truth of design and a primary tenet of Lean UX, is that you must do this. Almost all early design is a tenuous hypothesis that hasn't been validated and more importantly, is based on priorities that haven't been tested. Even if the design is right, the priorities are almost certainly wrong and the economics of what should be built and when, aren't properly understood.
So those discussions must be reopened, but without a Lean UX process the team ends up largely rehashing old ground and the result is fatigue and disengagement, or, more likely, the discussions aren't reopened and the endeavour falls into a waterfall process.
Reality is Bigger than we give it credit for
Design only captures part of the picture, as pictures and written words always woefully under-represent reality. That's actually not a bad thing– It means that reality is cooler than we give it credit for. However, that also means that when we kick in agile to 'fill in' and 'iterate' the design we are in fact missing a lot of the context that drove the design. Yes we have more context from testing and user feedback, but that's often a microcosm of the entire picture. We are left with design entropy and information loss from our original UX work because our design's context is lost with time.
The Front of the Process - Capturing the Design
Information captured in meetings and conversations is far deeper and nuanced than can be captured effectively on paper. With the multitude of factors that influence design, we are essentially putting a lake into a bucket.
The Back-end of the Process - Implementing the Design
Now that we move to implementation, we are getting new information through validation from testing with users, but this information is skewed to the users you are testing with, and may not reflect bigger market goals. In addition, the information is based on a subset of the solution that may not capture the value proposition and it is interpreted by a team that is removed from the deeper market perspective developed earlier in the UX design cycle.
The design from the front-end of the process is suffering information entropy and losing meaning. Inputs from the validation process are compared to distant, corrupt memories of past conversations, creating the implementation of a design that isbased on faulty inputs from both the past and the present.
There are 3 key components to solving this challenge.
Shorten the Time Between UX Work and Validation
The conversations MUST happen close to the work regardless of how it's captured. If your UX design is being done months before construction and is a prescriptive form with only the outputs captured, you have, in fact, defaulted to a waterfall process where you have underspecified behaviours. You'd probably be better off doing a structured design process from the 70’s.
Capture Design as Theory, not as Prescription
… Not theory as in hand-waving overly abstract conjecture, but theory as in a design architecture that is resilient to validation and is effective as input to emergent design during the plan/build/validate process of iterative design.
This is often done as a style guide that provides a playbook for future incremental design as well as design implementation.
Adopt a UX Lifecycle Model
A UX lifecycle model will allow you to treat your upfront UX work as a hypothesis for certain purposes, and invest properly in capturing the detailed information closer to implementation as in point 1.
Stage 1: Client Research
Who is building the product? This could be research into your own organization. Basically, knowing what the motivation and vision of the product owner or organization is, becomes a critical step in understanding what to build or where. This is part of the conceptual ‘why’ step that is critical to the later success of a project.
Stage 2: Market Research
Once you have clarity on the person or organization building a product, you need to clarify what market you are building it for. Generally, this is strongly shaped by the client’s experience and outlook.
Stage 3: User Research
This is where the deep conversations have to happen close to implementation. This is the ‘who’ whom you are building for, and the nuances here make or break a UX project. Ideally, these detailed conversations happen right before building the software.
Stage 4: Working Software
This is the ultimate design reference, which along with updated design documentation, provides the main reference for what is being built. The final features and updated design documentation should be enough to justify the feature, and guide future refinement without having to go back to stage 1, 2 or 3. But a key aspect of this is an implicit trust that the feature, its acceptance and the updated design documentation reflect a deeply shared reality born of the conversations in stage 3 and the validation in stage 4.
If you are interested in a deeper dive into the UX lifecycle model presented here, you may be interested in my presentation from FITC Toronto - Lessons Learned from One Dozen Projects