This post was modified from one I developed several years ago for the Mercure AACE 2013 program I took, called W5_PM_PERT-Challenges. I am still surprised that with the number of project managers I meet who recommend this technique, which effectively puts the Flaw of Averages on steroids.
This post is not specifically about sustainable change delivery. It does provide useful considerations, tools and techniques for consideration though for project controls in sustainable projects.
Context
The PERT (Program Evaluation and Review Technique) has been around since the late 1950s, and is frequently mentioned in Project Management estimating, scheduling and costing references.1 There appears to be several “camps” in regards to scheduling with PERT and presenting this information to senior management. This post will provide a high-level analysis of PERT and conduct an assessment of a simple MS Project schedule using both PERT and Monte Carlo to evaluate the differences in the response.
Development of the feasible alternative
This post will propose that for the purpose of identifying scheduling and costing information to be used for senior management reporting, one of these two alternatives is correct:
- PERT is a generally sound estimating approach
- PERT is a limited estimating approach
Development of the outcome
This post intends to provide a factual analysis of PERT to assist in making decisions on future usage.
Selection of criteria
The following selection criteria / attributes for reference materials were identified:
- Are there any statistical / mathematical constraints or concerns
- Are there situations where PERT is not appropriate
Analysis and comparison of the alternatives
Methodology
For the purposes of this analysis, a simple previous project was used with activities for Peter Milsom as Project Manager, plus an Analyst and a Developer all at $100 and hour for simplicity. We conducted this analysis using MS Project 2003 (yes this is from an older post), the PERT Entry Screen for MS Project 2003, the PERT Entry Screen for MS Excel, and MS Project 2003 using a Monte Carlo add on (using Risk+ and Barbecana Full Monte). In all instances for PERT and Monte Carlo the same optimistic, expected and pessimistic numbers were used. Also for both the Beta distribution was used. The final plan of record results are provided at the end.
0. Sample MS Project schedule baseline for analysis:
1. PERT Entry Screen for MS Project 2003 (using the MS Project 2003 PA_PERT_Entry Table):
1a. PERT Entry Screen for MS Excel 2007 (only the critical path activities were included up to 3 Sigma):
2. MS Project Monte Carlo (using Risk+ and Barbecana Full Monte) Entry Screen (the curve is a beta curve distribution) with no conditional branching:
Though the same data, activities, costing and scheduling were the same, the manner in which the activities and schedule are analyzed are completely different (i.e. Monte Carlo ran 10,000 simulations). The following are the results:
1. Resulting outcome of PERT Analysis (running the MS Project 2003 Calculate PERT Function):
2. Resulting Outcome of Monte Carlo Assessment:
2a. The following provides a range of the costs and percentage probability
2b. The following highlights those activities with the highest cost sensitivity or uncertainty variance. Focusing on reducing the uncertainty in these activities would reduce the risk and cost variance
2c. Cumulative S-curves of Project Cost
2d. The following provides a range of the project early finish and percentage probability
2e. The following highlights those activities with the highest time duration sensitivity or uncertainty variance. Focusing on reducing the uncertainty in these activities would reduce the time and cost variance
2f. Cumulative S-curves of Project Early Finish
Selection of Alternative
Once again the alternatives were:
- PERT is a generally sound estimating approach
- PERT is a limited estimating approach
The analysis has referenced several respected sources that have identified a number of statistical and mathematical constraints or concerns with the PERT approach. Also, running the PERT calculations “properly” for the author was challenging at best. Though commonly referenced, and for simple schedules perceived as easily executed (though to do properly is not straightforward), there are many situations where PERT is not appropriate.
“Scheduling and managing the activities involved in completing a complex, large scale project can be overwhelming. Many managers use project management software that is built on some basic models, but they are not familiar with the underlying models.”
“The results are appealing because they appear scientific… However, the results are misleadingly precise.”7
Based on a very simplistic assessment of PERT and Monte Carlo, the following results came out:
Based on the literature, and the “quick and dirty” test, for the purpose of identifying scheduling and costing information to be used for providing senior management information to make informed decisions, PERT is only recommended in limited situations.
Recommendations
This was particularly challenging finding straightforward guidance on employing PERT and Sigma for scheduling without concerns over validity or constraints. The recommendation is to use a Monte Carlo based approach. Please refer to the sister post “Sustainable Estimating” for recommendations on how to properly provide sustainable project estimating.
References:
1 Malcolm, Roseboom, and Fazar, (1959). Application of a technique for Research and Development Program Evaluation. Operations Research. 7 (5), pp.646 – 669
2 Galway, Lionel, (2004). Quantitative Risk Analysis for Project Management: A Critical Review. 1st ed. USA: Rand Working Paper, WR-112-RC. http://www.rand.org/content/dam/rand/pubs/working_papers/2004/RAND_WR112.pdf
3 Reginier, Eva, (2005). Activity Completion Times in PERT and Scheduling Network Simulation. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 12, pp 1 – 9. http://edocs.nps.edu/npspubs/institutional/newsletters/DRMI/2005/Apr05.pdf
4 Reginier, Eva, (2005). Hidden Assumptions in Project Management Tools. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 11, pp 1 – 4. http://www.nps.edu/Academics/Centers/DRMI/docs/1jan05-newsletter.pdf
5 Kerzner, Harold (2013-01-28). Project Management: A Systems Approach to Planning, Scheduling, and Controlling (Kindle Locations 15789-15792). Wiley. Kindle Edition.
6 Alleman, Glen B. (2009). Why PERT Has Problems. [ONLINE] Available at: http://herdingcats.typepad.com/my_weblog/2009/08/why-pert-has-problems.html. [Last Accessed 15 February 2013].
7 Reginier, Eva, (2005). Hidden Assumptions in Project Management Tools. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 11, pp 1 – 4.
8 Hubbard, Douglas W. (2009). The Failure of Risk Management: Why It’s Broken and How to Fix It. Wiley. http://www.amazon.com/Failure-Risk-Management-Why-Broken/dp/0470387955/ref=sr_1_1?s=books&ie=UTF8&qid=1442807076&sr=1-1&keywords=the+failure+of+risk+management
Hubbard, Douglas W. (2014). How to Measure Anything: Finding the Value of Intangibles in Business, Third Edition. Wiley. http://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/ref=sr_1_1?ie=UTF8&qid=1442806937&sr=8-1&keywords=How+to+Measure+Anything%3A+Finding+the+Value+of+Intangibles+in+Business
Savage, Sam L. (2012). The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty. Wiley. http://www.amazon.com/Flaw-Averages-Underestimate-Risk-Uncertainty/dp/1118073754/ref=sr_1_1?ie=UTF8&qid=1442807005&sr=8-1&keywords=The+Flaw+of+Averages
Clef two-factor authentication
Hi Peter,
Not withstanding the fact that technically, you failed the course for not fulfilling what you agreed to when you signed the team governance agreement (by taking and passing one or more of the AACE certifications), I was thrilled to see you still using the 7 step process. Proof that you not only learned how to use this valuable technique but retained and are using it.
Having said that, you missed explaining two important constraints or limitations of PERT in your paper. The first is that PERT, which is a top down approach, is only appropriate for use in either (AACE) Class 5 or Class 4 estimates. And isn’t the margin of error for Class 5 estimate +100% to -20% and a Class 4 estimate +60% to -15%? So why didn’t you explain that in your paper?
If you want to see what a recent successful graduate of our program has written on this same topic, check out this article just published in the PMWorld Journal- http://pmworldjournal.net/article/comparison-8-common-cost-forecasting-methods/ See what Steve has done (you will surely recognize the case study!!) and see what his conclusions were.
Keep up the good work and I truly do hope that you finish up what you agreed to when you signed the team governance agreement. It will surely help you with your work for GPM.
BR,
Dr. PDG, Jakarta, Indonesia