Regular, well-run retrospectives are critical to the ongoing improvement of an Agile team. In theory, scaling this process into an enterprise-wide activity sounds like a great idea. The reality of large scale retrospectives is quite different, however. The steps you would normally take to get a team to collaborate in an intimate setting just don’t scale. 

For large organizations to leverage the power of the retrospective, a different approach is required. This takes some planning, but the results are worth it. Done properly, large scale retrospectives can:

  • Identify the right areas for improvement — and help overcome obstacles to change
  • Build alignment across the enterprise
  • Establish the means to measure future progress

When is it time for a Large Scale Retrospective?

Telltale signs it might be time for a Large Scale Retrospective are: 

  1. When your team retrospectives start to fail
    Nearly all team retrospectives uncover problems to address outside the scope of the team. Initially, this has little impact on the team’s progress because there is so much room to improve inside their scope. However, over time the impact of outside impediments will loom larger.

    The result? People get bored and the time retrospecting no longer feels productive. This is a clear indication that a Large Scale Retrospective is needed.

    At this point, leadership has the responsibility to remove enterprise impediments to agility. Large Scale Retrospectives can then help leaders prioritize solutions while generating organizational alignment.                                                                                                                                                                      
  2. When you are scaling change
    When organizations are involved in scaling change, Large Scale Retrospectives can provide leaders with the chance to communicate urgency and reinforce the vision for the future. A retrospective can also reveal obstacles and align teams through collaborative prioritization. Ongoing Large Scale Retrospectives (every 6 months or so) can reinforce success and anchor the new way of working. 

How to Organize Large Scale Retrospectives

The process for running Large Scale Retrospectives boils down to this: engage every team in their own online retrospective session using the same framework for every team so that you can compare results across teams. 

In our view, the only tool that makes this really easy at scale (10 – 100+ teams) is SAFe® Collaborate (formerly called Conteneo’s Weave platform.) SAFe Collaborate makes it simple to view the results of a single team AND the combined results of all the teams that participate. That said, it is possible to get good results with other online collaboration platforms if you do not have access to SAFe Collaborate. Just plan on spending a bit more time at the end organizing the results into a single data set. (If you have more than 10 teams, contact us. We can partner with you and use SAFe Collaborate to scale your event to 100s of teams.)

Step 1 – Pick a Framework and Write a Central Question

The first step is to select and tailor a retrospective framework that each team will use during their session. We like to use a variation of an original Innovation Game®, Speedboat, called “The Voyage.” The metaphorical framework includes an island representing the vision, rocks representing risks, and a Sailboat representing the team and organization. 

(A guide for “The Voyage” framework is provided at the end of this article.)

Next, create a Central Question that each team will consider during their retrospective session. We recommend increasing the period of time teams consider from weeks to about the last six months. Here is an example: 

“We are embarking on a large effort to scale Agile across the company so that we can better meet our customer’s needs more quickly. We know the effort will be challenging at times so we would like to get your input early. As you conduct this retrospective with your team, spend time describing your Vision for the ideal future state. What does success look like, how will we measure it? What risks can you see that will make this change difficult? What difficulties and problems do you encounter today that will hold you and your team back from success, or slow you down? What strengths do you have that can be leveraged? As you answer these questions, think about how you and your team collaborate with other parts of the company and other teams.”

Step 2: Schedule and Facilitate the Team Retrospective Sessions

Each team retrospective should last 60 to 90 minutes. Assign a facilitator to each team and consider scheduling special, separate sessions for leaders and managers so that team members feel comfortable being as honest as possible without managers present. 

The Facilitation Team will need to meet ahead of the scheduled sessions to confirm the Central Question, agree upon both the instructions they will give participants and the intended actions that will come after the results are analyzed. Facilitators who understand how the results will be analyzed and the overall goals will ask better questions during the retrospective sessions. Better questions usually means asking for examples, and responses to those types of questions improves the information gathered and helps immensely during the analysis phase.

Step 3: Analyze the Results

Every item added to The Voyage framework during a team retrospective should be tagged adding two additional dimensions for analysis. This tagging can actually begin during the retrospective session or the facilitator can begin this process immediately after the retrospective ends. Every item is categorized as People, Process or Tools and then by Scope of Control (the smallest segment of the company that can either address a problem or risk, amplify a strength or realize a part of the Vision.) 

We often use this Scope of Control framework to help the analysis team make the best determination about where responsibility or agency lies.  

 (A framework guide for “Scope of Control” is provided at the end of this article.)

Step 4: Identify Themes

Once items are tagged, we then group the items into themes. While every Large Scale Retrospective is unique, we are not surprised to find themes such as Stakeholder Engagement, Culture & Mindset, Strategic Alignment, and Communication & Collaboration as major themes.

In this graph generated by an Excel spreadsheet downloaded from SAFe Collaborate and further manipulated by adding tags, it is interesting to note that many of the issues around Communication & Collaboration are within a Team’s scope of control. However, the combination of these issues from Location, Division and Department are certainly worthy of closer inspection. Issues with tools are primarily found at the Division and Enterprise level while Planning & Delivery impact teams, Divisions and the Enterprise. 

The recommendations stemming from this Large Scale Retrospective included three areas of focus and approximately twelve initiatives for consideration. The expectation was not that every recommendation would immediately be acted on as the cost was too great. However, the company did see some obvious wins and communicated the results of the Large Scale Retrospective by including the decision to immediately fix some obvious issues. The participants understood immediately why the action was approved.

[Note: The steps above are also summarized in the recent webinar “Large Scale Retros” we led in partnership with Bay Area Agile Leadership.]

Building Alignment Across all Teams

A second major benefit of the Large Scale Retrospective is to allow teams to align across the organization. Keep in mind the fastest way to alignment is often not through announcements and edicts drafted in small meetings behind closed doors. People waste months after these announcements trying to course correct, initiate “do-overs” or even actively oppose mandates. Resistance to mandates should not be a surprise as the foundation of a high-performing Agile team includes such concepts as “self-organizing” and “autonomous.”

Instead, to effectively build alignment, often a better option is to take the recommendations stemming from the Large Scale Retrospective and use a Participatory Budgeting engine, like the one available in SAFe Collaborate. (For more about how Participatory Budgeting works, see our blog The Power of Collaborative Cost Cutting.) The same teams that engaged previously are invited to prioritize the initiatives based on a limited budget – either the actual budget or by using relative sizing and points. The struggle to fund the most important initiatives through collaboration and negotiation generates the best decisions and creates support for whatever the final outcome may be. For example, Anne may have felt that Initiative E was more important than Initiative C, but if C is the next approved action, Anne still has a very strong appreciation for why C was chosen. The need to spend months explaining or socializing the value of Initiative C to Anne and hundreds like her is saved.

 (A framework guide for “Participatory Budgeting” is provided at the end of this article.)

Measuring Progress Over Time

The third major benefit of the Large Scale Retrospective is to allow teams to measure progress over time. We have seen patterns in this regard.

  1. Evolution of Impediments – Over time, as the organization focuses on amplifying strengths and removing impediments, the results of periodic Large Scale Retrospectives will show progress. For example, we often see Tools & Technology as a category with many challenges that lie outside a team’s scope of control. As companies upgrade or standardize tooling for example, the feedback picture shifts and the results show real progress and reinforce the wins across the team’s that participate. 
  • Inferred Alignment – We commonly note that in the first Large Scale Retrospective results, we see a lack of alignment around future goals. The leadership may have a thoughtful strategic vision that they believe has been communicated far and wide, but the results often show weak alignment around the vision. In one project, unique descriptions of the ideal future state were plentiful and often at odds with other future success characteristics. Our conclusion was that people and teams were not aligned on the future state and were pursuing goals at odds with each other. Achieving any success was hard and this lowered morale. Within six months, the variation in describing a goal state was greatly reduced and our conclusion was that people and teams were much better aligned on where the organization was headed, how success is defined and their role in achieving the desired goal state. 

Taking the Next Step

If you would like to talk with us further about how we can help with your next Large Scale Retrospective, please contact us.  You can also watch a recorded meetup with Bay Area Agile Leadership where we had the opportunity to talk about the value of the Large Scale Retrospective.

Frameworks for Large Scale Retrospectives

Framework name: The Voyage
Participants: Ideally, all levels of organization including team members.
Framework goal: Measure alignment around the Vision for the future, uncover impediments, identify risks to success and strengths that can help the organization succeed.
Framework guidelines:
  1. Start by asking participants to describe the goal using stickies on the island, what does success look like.
  2. Balance time between all four areas of the framework.
  3. Ask for examples and stories to clarify input, encourage discussion. Agreement is not the goal, clarity and understanding is. 
Boundaries provided:
  1. Ideal number of participants of ~8 per facilitated forum (additional support staff to facilitate or capture information not included)
  2. Schedule separate forums for managers and leaders.
  3. Include everyone impacted by the change.
  4. Maximum time for each session of 90 minutes.
Resources used: In-person: Poster print of The Voyage framework, 4 colors of sticky notes per group and black sharpiesOnline: SAFe Collaborate (formerly Conteneo’s Weave)
Actionable outcome(s):
  1. List of items describing success.
  2. LIst of items describing risks.
  3. List of items describing strengths.
  4. List of items describing weaknesses.
  5. Insight into the degree to which participants are aligned with each other and with the Vision for the change.

 

Framework name: Scope of Control
Participants:
  • Teams determining where agency lies for ideas generated during a retrospective
  • Large-scale Retrospective Analysis Team as a tool for determining where agency lies for ideas generated during a retrospective
Framework goal: Determine the smallest organizational unit that has the ability to address an obstacle or risk, amplify a strength or validate a goal.
Framework guidelines:
  1. Tailor the region names to represent the terminology of the company.
  2. Organize items into the best location.
Boundaries provided:
  1. Ideal number of participants of ~8 per forum (additional support staff to facilitate or capture information not included)
  2. Maximum time for each session of 60 minutes.
Resources used: In-person: Poster print of Cope of Control framework, Items printed on note cards, one per card, sharpies and tape

Online: SAFe Collaborate (formerly Conteneo’s Weave), Miro, or another online collaboration platform

Actionable outcome(s):
  1. Items organized by Scope of Control

 

Framework name: Participatory Budgeting
Participants: Ideally, all levels of organization including team members.
Framework goal: Prioritization of remaining investments.
Framework guidelines:
  1. Use the actual budget remaining to distribute to participants.
  2. Use estimated remaining investment costs for evaluation (Ignore sunk costs).
  3. Stress that the outcome of the session will contribute to overall investment decisions and that the results of the session are not final.
Boundaries provided:
  1. Maximum number of participants of 8 (additional support staff to facilitate or capture information not included)
  2. No more than 20 investments per group of 8 participants. If more than 20 investments remaining, group by representative or consolidated descriptions (contact us [link] for more complex collaboration structures)
  3. For large organizations, form diverse groups of participants from different business units or divisions and analyze results. Results from dozens of sessions may be consolidated.
  4. Maximum time for each session of one hour.
Resources used: Investments remaining from Strategic Alignment/Cost matrix

Total actual budget

In-person: Play money; Cards for investment name, estimated remaining costs, and potential value; name cards; grid for investments by row and columns by name.

Online: Weave

Actionable outcome(s):
  1. List of funded investments.
  2. LIst of unfunded investments.
  3. Qualitative insights to decision making by groups.