Walmart lets AI do its Agile Planning
Find out how Walmart uses an AI agent for a nuanced use case.
Generative AI has the potential to solve some of the biggest problems.
But those huge transformations will only come from an accumulation of multiple smaller solutions.
We recently surveyed business leaders (survey soon to be published), and found that a mere 14.6% of organizations use Gen AI for 5 or more functions.
So many companies are leaving too much on the table.
From simple use cases like text summarization to complex ones like medical imaging, GenAI can augment a wide range of your business functions.
And there are so many ways to identify these opportunities. You can look for them yourself, empower teams to try out GenAI solutions, follow leading GenAI avenues (cough* The AI Edge *cough) to discover new use cases, and so on.
In today’s edition, we discussed one such nuanced GenAI use case from Walmart.
There’s so much that they’ve done right here.
The team did a great job of identifying a valid use case
They gave sufficient weightage to planning
They kept the execution relatively simple
The team is still looking for opportunities to improve the AI solution
So here it goes:
The problem
Walmart wanted to solve the challenge that developers face during Agile sprint planning.
The developers often struggled to write concise task descriptions and testable acceptance criteria.
The sprint planning process can be laborious, and Walmart decided to streamline this process by using an AI solution.
By automating Agile planning, they intended to make it faster and more efficient for developers to create sprint plans, leading to better communication, estimation, and overall progress in software development projects.
The solution roadmap
They interviewed individuals involved in agile planning to get a better understanding of the overall workflow.
Source: Walmart Global Tech Blog
Once they mapped the workflow, they broke down the process into smaller tasks that an AI could execute.
The tasks would look like:
Fetching goal details
Dividing features into tasks
Creating description and acceptance criteria
Estimating the level of effort
Finalizing plan with the team
They also identified the relevant sources of information for the AI to get these tasks done.
They again referred to their previous experiences to determine the right information sources. Given that they had been doing this for so long, it was straightforward.
The implementation
They called the solution Agile Hemingway and here’s the step-by-step of how they built it.
1. Create a design flow
Agile Hemingway Version 0 comprises three main components that work together to automate the Agile planning process. These components are:
- The Sprint Planner
- The Story Groomer
- The Story Sizer
Source: Walmart Global Tech Blog
Each of these components is a call to a Language Model (LLM) like GPT
2. Set up the tech stack
Walmart used the following tools for implementing the solution:
- GPT Instance: Used as the base LLM for their model.
- LangChain Framework: A library that enables easy implementation of Retrieval-Augmented Generation (RAG) pipelines.
- PgVector: An extension for the Postgres Database that allows storing and managing text embeddings.
3. Curate data sources
The AI model needs data from different sources to carry out the planning process. These sources include:
- The Goal: The PI (Programme Increment) Goal in the Agile framework.
- The Product Requirement Document (PRD): A detailed description of product requirements.
- Corporate Guidelines: Company-specific guidelines on Agile processes and story creation.
- Previous Sprint Stories: Stories created in previous sprints for reference and learning.
These data sources were curated, and their text embeddings were stored in the PgVector store.
4. Develop LLM models for handling various tasks
a. The Sprint Planner: This is a prompt that plans the timeline and divides a given goal into a series of tasks. It uses the guidelines data as well as inspiration from the standard processes. The Sprint Planner fetches goal details and breaks the goal into tasks using data from the data sources mentioned earlier.
b. The Story Groomer: This LLM call creates a story headline, description, and acceptance criteria for each task generated by the Sprint Planner. It uses task details from the Sprint Planner, supportive information from the vector database, and guideline information. The Story Groomer generates descriptions and acceptance criteria based on these inputs.
c. The Story Sizer: This component estimates the Level of Effort (LoE) based on the acceptance criteria generated by the Story Groomer and information available from similar stories in the past. The Story Sizer uses two kinds of information:
- Story details (description and acceptance criteria) created by the Story Groomer
- Semantically similar story details from past sprints
The result and improvements
The AI solution works well for Walmart.
They can do it in minutes, which usually takes days.
But they are not done yet and plan to improve upon it continually.
The plans include creating repositories of good and bad examples, building validation frameworks, bringing human feedback into the loop, and including automated testing.