The Delphi Method (also called Delphi Technique) is a structured expert judgment process used to achieve consensus on a topic through anonymous, iterative rounds of questionnaires and controlled feedback. Developed by the RAND Corporation in the 1950s for forecasting and decision-making, it's now widely used in project management for estimation, risk assessment, and expert consultation.
Core Characteristics
Key Features:
- Anonymous participation - Experts don't know who else is participating or what others initially said
- Iterative rounds - Multiple cycles of questionnaires (typically 2-4 rounds)
- Controlled feedback - Facilitator summarizes results and shares with group between rounds
- Statistical aggregation - Results compiled and analyzed quantitatively
- Consensus building - Process continues until reasonable agreement emerges
The Name: Named after the Oracle of Delphi in ancient Greece, where people sought wisdom and prophecy.
The Process
Standard Delphi Process Flow
Round 1: Initial Input
- Facilitator identifies problem/question
- Panel of experts selected (typically 10-30 people)
- Questionnaire sent to all experts independently
- Experts provide estimates, opinions, or forecasts anonymously
- Facilitator collects and analyzes responses
Round 2: Controlled Feedback
- Facilitator compiles Round 1 results (statistical summary, range, median)
- Summary shared with all experts (without identifying who said what)
- Experts review group results and rationale for outliers
- Experts revise their estimates based on others' input
- Experts can explain reasoning for their position
- Facilitator collects Round 2 responses
Round 3: Convergence (if needed)
- Facilitator shares updated statistical summary
- Highlights areas of agreement and disagreement
- Provides rationale from experts holding divergent views
- Experts make final revisions
- Facilitator collects Round 3 responses
Round 4+: Additional Refinement (optional)
- Continue if consensus hasn't emerged
- Most Delphi studies reach stability by Round 3
- Diminishing returns after Round 4
Final Analysis:
- Facilitator compiles final results
- Documents level of consensus achieved
- Reports median, range, and distribution of responses
- Captures key insights and rationale
Project Management Applications
1. Estimation
Duration Estimation:
- Ask experts to estimate task durations
- Achieve consensus on realistic timeframes
- Reduces individual bias and optimism
- Produces more reliable estimates than single expert
Cost Estimation:
- Estimate project or component costs
- Build budget from expert consensus
- Identify assumptions affecting estimates
- Understand range of possible costs
Resource Requirements:
- Estimate team size needed
- Determine skill requirements
- Forecast resource capacity needs
2. Risk Identification and Assessment
Risk Identification:
- Experts independently identify potential risks
- Consolidate comprehensive risk list
- Capture risks individual experts might miss
- Build complete risk register
Risk Analysis:
- Experts estimate risk probability and impact
- Achieve consensus on risk ratings
- Prioritize risks based on expert judgment
- Reduce subjective bias in risk assessment
Example Risk Delphi:
- Round 1: Rate risk probability (1-5 scale)
- Round 2: Review group median, adjust ratings
- Round 3: Finalize with rationale for outliers
- Result: Consensus-based risk priority ranking
3. Requirements Prioritization
Approach:
- Experts rank or score requirements
- Identify highest-priority features
- Build consensus on what's essential
- Reduce political influence on prioritization
4. Vendor Selection
Process:
- Experts evaluate vendor proposals
- Rate vendors on multiple criteria
- Achieve consensus on best selection
- Reduce bias from individual preferences
5. Forecasting and Planning
Strategic Planning:
- Forecast future trends affecting project
- Predict technology evolution
- Anticipate market changes
- Plan for long-term uncertainties
Scenario Planning:
- Experts propose possible future scenarios
- Assess likelihood of each scenario
- Plan responses to different futures
6. Problem-Solving
Complex Decisions:
- When multiple expert perspectives needed
- Technical decisions with high uncertainty
- Novel problems without historical data
- Controversial issues requiring objectivity
Advantages
1. Eliminates Group Dynamics Issues:
- No dominant personalities controlling discussion
- Introverts contribute equally
- Political hierarchies don't suppress junior experts
- Reduces groupthink and conformity pressure
2. Reduces Bias:
- Anonymous participation reduces anchoring
- Experts can change minds without losing face
- Peer pressure minimized
- Focus on merits, not personalities
3. Leverages Distributed Expertise:
- Geographic barriers don't matter
- Time zone differences accommodated
- Experts participate on their schedule
- Access to broader expert pool
4. Structured and Documented:
- Clear process and methodology
- Traceable reasoning and rationale
- Documented evidence for decisions
- Defensible results
5. Convergence Toward Consensus:
- Typically achieves reasonable agreement
- Outliers have opportunity to influence group
- Statistical aggregation reduces individual errors
- Group wisdom emerges
6. Cost-Effective:
- No need to gather experts in one location
- No travel costs or time
- Asynchronous participation
- Less time than multiple meetings
Limitations and Challenges
1. Time-Consuming:
- Multiple rounds take weeks or months
- Each round requires time for response and analysis
- Slower than single meeting
- May not work for urgent decisions
2. Requires Expert Commitment:
- Participants must complete multiple rounds
- Attrition between rounds
- Declining response quality in later rounds
- Needs motivated, engaged experts
3. Facilitator Skill Critical:
- Proper summarization essential
- Bias can enter through framing
- Must present feedback neutrally
- Quality depends on facilitator competence
4. May Not Reach Full Consensus:
- Some issues remain divisive
- Forcing consensus can be artificial
- May converge on compromise, not optimal solution
- Outliers may represent important insights
5. Limited Interaction:
- Experts can't debate directly
- Misunderstandings harder to resolve
- Creative synergy reduced
- Complex reasoning difficult to convey
6. Quality Depends on Expert Selection:
- Must identify true experts
- Need diverse perspectives
- Too homogeneous = narrow thinking
- Wrong experts = poor results
7. Anonymity Can Be Double-Edged:
- Reduces accountability
- May encourage careless responses
- Experts might not invest sufficient thought
- Hard to probe reasoning deeply
Best Practices
Expert Selection:
- Choose recognized subject matter experts
- Include diverse perspectives and backgrounds
- Avoid conflicts of interest
- Aim for 10-30 participants (optimal range)
- Ensure expertise relevant to specific question
Question Design:
- Clear, unambiguous questions
- Specific and focused
- Quantifiable when possible
- Avoid leading or biased framing
- Pilot test with small group
Facilitation:
- Maintain strict anonymity
- Provide neutral, complete feedback
- Share statistical summaries (median, quartiles, range)
- Include rationale for outlier positions
- Don't manipulate toward predetermined answer
- Set clear deadlines and expectations
Feedback Between Rounds:
- Share distribution of responses (box plots, histograms)
- Report median and interquartile range
- Provide anonymous rationale for extreme positions
- Highlight areas of agreement and disagreement
- Don't identify individuals or create peer pressure
Stopping Criteria:
- Predetermined consensus threshold (e.g., 70% within range)
- Stability across rounds (responses no longer changing)
- Maximum number of rounds reached (typically 3-4)
- Diminishing returns evident
- Time constraints require conclusion
Managing Outliers:
- Don't force conformity
- Request explanation of reasoning
- Share rationale with group
- Outliers may have unique valid insight
- Include in final report with context
Documentation:
- Record all responses anonymously
- Document process and methodology
- Explain how consensus was determined
- Report level of agreement achieved
- Capture key insights and rationale
- Note remaining disagreements
Variations
Modified Delphi:
- Starts with face-to-face meeting to frame issue
- Then proceeds with anonymous rounds
- May conclude with group discussion
- Balances structure with interaction
Real-Time Delphi:
- Uses software for immediate feedback
- Participants see updated statistics as others respond
- Faster than traditional rounds
- Loses some iteration benefits
Policy Delphi:
- Designed to explore policy options, not reach consensus
- Focuses on identifying diverse perspectives
- Useful when disagreement is expected and valuable
- Goal is understanding, not agreement
e-Delphi:
- Conducted entirely online
- Uses web-based survey tools
- Automated analysis and feedback
- Faster turnaround between rounds
Delphi Conference:
- Combines Delphi with synchronous elements
- Uses technology for structured group sessions
- Maintains some anonymity features
- Faster than pure Delphi
Example: Project Duration Estimate
Scenario: Estimating duration for software development project
Round 1:
- Question: "How many weeks will it take to complete the XYZ software module?"
- 20 experts respond independently
- Results: Range 8-24 weeks, Median 14 weeks, Mean 15.2 weeks
Round 2:
- Facilitator shares: "Group median is 14 weeks, range 8-24"
- Includes rationale from extremes:
- 8 weeks: "Assumes we reuse 60% from existing module"
- 24 weeks: "Includes integration testing often underestimated"
- Experts reconsider and resubmit
- Results: Range 10-18 weeks, Median 13 weeks, Mean 13.8 weeks
Round 3:
- Facilitator shares: "Median now 13 weeks, range 10-18"
- Notes: "Integration testing concern widely shared, reuse assumption questioned"
- Final estimates collected
- Results: Range 12-16 weeks, Median 14 weeks, Mean 14.1 weeks
Outcome:
- Consensus estimate: 14 weeks
- Contingency planning for 12-16 week range
- Key assumptions documented
- Risk of integration testing delays noted
When to Use Delphi vs. Other Methods
Use Delphi When:
- Expert judgment needed but can't meet in person
- Risk of dominant personalities skewing results
- Controversial or political issues
- Time permits multiple rounds (weeks available)
- Need documented, defensible consensus
- High uncertainty with no historical data
- Geographic distribution of experts
Use Alternative Methods When:
- Urgent decision needed (use expert interviews or single meeting)
- Group synergy and debate important (use facilitated workshop)
- Factual data available (use analogous or parametric estimation)
- Simple question with clear answer (use quick consultation)
- Team building important (use collaborative planning)
Alternatives:
| Method | When to Use Instead |
|---|---|
| Brainstorming | Need creative ideas quickly, group synergy valuable |
| Nominal Group Technique | Need consensus in single session, face-to-face possible |
| Expert Interviews | Time-critical, need depth from few experts |
| Facilitated Workshop | Complex problem requiring real-time discussion |
| Wideband Delphi | Estimation with team discussion between rounds |
| Planning Poker | Agile estimation with team in same location |
Measuring Consensus
Quantitative Measures:
Interquartile Range (IQR):
- Difference between 75th and 25th percentile
- Smaller IQR = greater consensus
- Target: IQR < 20% of median
Coefficient of Variation:
- Standard deviation / mean
- Lower percentage = more agreement
- Target: < 0.5 (50%)
Stability:
- Percentage of respondents changing estimates < 10%
- Indicates convergence achieved
Consensus Threshold:
- Percentage within predetermined range
- Example: 70% of responses within ±15% of median
Qualitative Indicators:
- Rationale for positions aligning
- Fewer outliers in later rounds
- Decreasing standard deviation
- Narrowing range
Integration with Other PM Techniques
Three-Point Estimation:
- Use Delphi to generate optimistic, most likely, pessimistic estimates
- Combine with PERT calculations
- Produces statistically robust estimates
Risk Management:
- Delphi for qualitative risk assessment
- Follow with quantitative methods (Monte Carlo)
- Build comprehensive risk profile
Requirements Prioritization:
- Use Delphi to score requirements
- Combine with MoSCoW or Kano model
- Develop consensus-based priority list
Vendor Evaluation:
- Delphi for multi-criteria scoring
- Objective selection based on expert consensus
- Reduces procurement disputes
Key Concepts
Delphi Method - Structured expert judgment technique achieving consensus through anonymous, iterative questionnaires with controlled feedback.
Facilitator - Person managing Delphi process, distributing questionnaires, analyzing results, and providing feedback without influencing content.
Expert Panel - Group of subject matter experts participating in Delphi study, typically 10-30 individuals.
Anonymity - Experts don't know others' identities or initial responses, reducing bias and groupthink.
Iterative Rounds - Multiple cycles of questionnaires (typically 2-4) allowing experts to reconsider based on group feedback.
Controlled Feedback - Facilitator's summary of group responses shared between rounds to inform experts without revealing identities.
Consensus - Reasonable agreement among experts, measured quantitatively or by stability of responses.
Convergence - Process of expert estimates moving closer together across rounds.
Outlier - Response significantly different from group median or outside typical range.
Stability - Point where expert responses stop changing significantly between rounds.
Median - Middle value when responses arranged in order; preferred measure for Delphi consensus.
Interquartile Range (IQR) - Difference between 75th and 25th percentile; measures consensus level.
Groupthink - Phenomenon where group pressure leads to conformity and poor decisions; Delphi designed to prevent this.
Wideband Delphi - Variation including group discussion between estimation rounds; common in agile for planning poker.
Modified Delphi - Variation starting with face-to-face meeting before anonymous rounds.
Real-Time Delphi - Software-enabled variation providing immediate statistical feedback as experts respond.
Policy Delphi - Variation exploring diverse perspectives rather than forcing consensus.