I’m on a journey to become a better software developer by reducing the number of defects in my code. The Personal Software Process (PSP) is one of the few proven ways to achieve ultra-low defect rates. I did a deep dive on it over the last few months. And in this post I’m going to tell you everything you need to know about PSP.
(As an Amazon Associate I earn from qualifying purchases.)
The Promise of the Personal Software Process (PSP)
Watts Humphrey, the creator of PSP/TSP, makes extraordinary claims for defects rates and productivity achievable by individuals and teams following his processes. He wrote a couple of books on the topic but the one I’m writing about today is PSP: A Self-Improvement Process for Software Engineers (paid link) (2005).
This chart shows a strong correlation between the formality with which software is developed and the number of defects in the delivered software. As you can see, PSP/TSP delivered a shocking 0.06 defects/KLOC, which is about 100 times fewer defects than your average organization hovering around CMM level 2 (6.24 defects/KLOC). Impressive, right?
Humphrey further claims:
Forty percent of the TSP teams have reported delivering defect-free products. And these teams had an average productivity improvement of 78% (Davis and Mullaney 2003). Quality isn’t free–it actually pays.
Bottom line: defects make software development slower than it needs to be. If you adopt PSP and get your team to adopt TSP, you can dramatically reduce defects and deliver software faster.
It seems counter-intuitive but Steve McConnell explains why reducing defect rates actually allows you to deliver software more quickly in this article: Software Quality at Top Speed
Of course, you can only get those gains if you follow a process like PSP. I’ll explain more on that later but I want to take a small detour to tell you a story.
Imagine your ideal job
Imagine you work as a software developer for a truly enlightened company–I’m talking about something far beyond Google or Facebook or Amazon or whomever you think is the best at software development right now.
You are assigned a productivity coach
Now, your employer cares so much about developer productivity that every software developer in your company is assigned a coach. Here’s how it works. Your coach sits behind you and watches you work every day (without being disruptive). He keeps detailed records of:
- time spent on each task
- programming tasks are broken down into categories like requirements, requirements review, design, design review, coding, personal code review, testing, peer code review, etc.
- non-programming tasks are also broken down into meaningful categories like meetings, training, administrative, etc.
- defects you inject and correct are logged and categorized
- anything else you think might be useful
Your coach runs experiments on your behalf
After you go home, your coach runs your work products (requirements, designs, code, documentation, etc.) through several tools to extract even more data such as lines of code added and modified in the case of code. He combines and analyzes additional data from your bug tracker and other data sources into useful reports for your review when the results become statistically significant.
Your coach’s job is to help you discover how to be the very best programmer you can be. So, you can propose experiments and your coach will set everything up, collect the data, and present you with the results. Maybe you want to know:
- if test-first development works better for you than test-last development?
- are design reviews worth the effort?
- does pair programming work for you?
- do you actually get more done by working more than 40 hours a week?
- are your personal code reviews effective?
Those would be the kinds of experiments your coach would be happy to run. Or your coach might suggest you try an experiment that has been run by several of your colleagues with favorable outcomes. The important point is that your coach is doing all the data collection and analysis for you while you focus on programming.
Doesn’t that sound great? Imagine how productive you could become.
Unfortunately this is the real world so you have to be your own coach
This is the essence of the Personal Software Process (PSP). So, in addition to developing software, you also have to learn statistics, propose experiments, collect data, analyze it, draw meaningful conclusions, and adjust your behavior based on what your experiments reveal.
What the Personal Software Process gets right
Humphrey gets to the root of a number of the problems in software development:
- projects are often doomed by arbitrary and often impossible deadlines
- the longer a defect is in the project, the more it will cost to fix it
- defects accumulate in large projects and lead to expensive debugging and unplanned rework in system testing
- debugging and defect repair times increase exponentially as projects grow in size
- a heavy reliance on testing is inefficient, time-consuming, and unpredictable
Here’s an overview of his solution:
The objective, therefore, must be to remove defects from the requirements, the designs, and the code as soon as possible. By reviewing and inspecting every work product as soon as you produce it, you can minimize the number of defects at every stage. This also minimizes the volume of rework and the rework costs. It also improves development productivity and predictability as well as accelerating development schedules.
Humphrey convinced me that he’s right about all aspects of the problem. He has data and it’s persuasive. However, the prescription–what you need to do to fix the problem–is unappealing, as I’ll explain in the next section.
Why almost nobody practices the Personal Software Process
PSP is too hard to follow for 99.9% of software developers. The discipline required to be your own productivity coach is beyond what most people can muster, especially in the absence of organizational support and sponsorship. Software development is already demanding work and now you need to add this whole other layer of thinking, logging, data analysis, and behavior changes if you’re going to practice the Personal Software Process.
Most software developers just aren’t going to do that unless you hold a gun to their heads. And if you do hold a gun to someone’s head PSP won’t work at all. They’ll just sabotage it or quit.
Here’s a quote from Humphrey’s book just to give you a taste for how detailed and process-driven PSP is:
The PSP’s recommended personal quality-management strategy is to use your plans and historical data to guide your work. That is, start by striving to meet the PQI guidelines. Focus on producing a thorough and complete design and then document that design with the four PSP design templates that are covered in Chapters 11 and 12. Then, as you review the design, spend enough time to find the likely defects. If the design work took four hours, plan to spend at least two, and preferably three, hours doing the review. To do this productively, plan the review steps based in the guidelines in the PSP Design Review Script described in Chapter 9….
It goes on from there…for about 150 pages. I can see how you could create ‘virtually defect-free’ software using this kind of process but it’s hard for me to imagine a circumstance where you could get a room full of developers to voluntarily adopt such an involved process and apply it successfully.
But that’s not the only barrier to adoption.
Additional barriers to adoption
The Personal Software Process is meant to be taught as a classroom course
The course is prohibitively expensive. So, the book is the only realistic option available to most people.
You’ll find that reading the book, applying the process, figuring out how to fill out all the forms properly, and then analyzing your data all without the help of classmates or an instructor is hard.
The book assumes your current process is some version of waterfall
TDD doesn’t not play nice with the version of PSP you’ll learn in the book. Neither does the process where you write a few lines, compile, run, repeat until you’re done. It breaks the stats and the measures you need to compute to get the feedback you require to track your progress. It’s not impossible to come up with your own measures to get around these problems but it’s definitely another hurdle in your way.
Languages without a compile phase (python, PHP, etc.) also mess up the stats in a similar way.
The chapters on estimating are of little value to agile/scrum practitioners
There’s a whole estimating process in PSP that involves collecting data, breaking down tasks, and using historical data to make very detailed estimates about the size of a task that just don’t matter that much in the age of agile/scrum. In fact, I didn’t see any evidence that the Personal Software Process estimating methods are any better than just getting an experienced group of developers together to play planning poker.
It’s very difficult to collect good data on a real-world project
If you’re going to count defects and lines of code, you need good definitions for those things. And at first glance, PSP seems to have a reasonably coherent answer. But when I actually tried to collect that data I was flooded by edge cases for which I had no good answers.
For example, a missing requirement caught in maintenance could take one hour to fix or it could take months to fix but you are supposed to record them both as requirement defects and record the fix time. Then you use the arithmetic mean for the fix time in your stats. So that one major missing requirement could completely distort your stats, even if it is unlikely to be repeated.
Garbage in, garbage out. Very concerning.
By the way, there’s some free software you can use to help you with the data collection and analysis for PSP. It’s hasn’t been very useful in my experience but it’s probably better than the alternative, which is paper forms.
Your organization needs to be completely on board
Learning the Personal Software Process is a huge investment. I think I read somewhere that it will take the average developer about 200 hours of devoted study to get proficient with the PSP processes and practices. Not many organizations are going to be up for that.
PSP/TSP works because the people using it follow a detailed process. But most software development occurs at CMM Level 1 because the whole business is at CMM Level 1. I think it’s pretty unlikely that a chaotic business is going to see the value in sponsoring a super-process-driven software development methodology on the basis that it will make the software developers more productive and increase the quality of their software.
You could argue that the first ‘P’ in PSP stands for ‘personal’ and that you don’t need any organizational buy-in to do PSP on your own. And that may technically be true but that means you’ll have to learn it on your own, practice it on your own, and resist all the organizational pressure to abandon it whenever management decides your project is taking too long.
Is the Personal Software Process (PSP) a waste of time then?
No, I don’t think so. Humphrey nails the problems in software development. And PSP is full of great ideas. The reason most people won’t be able to adopt the Personal Software Process–or won’t even try–is the same reason people people drop their new year’s resolutions to lose weight or exercise more by February: our brains resist radical change. There’s a bunch of research behind this human quirk and you can read The Spirit of Kaizen (paid link) or The Power of Habit (paid link) if you want to learn more.
Humphrey comes at this problem like an engineer trying to make a robot work more efficiently and that’s PSP’s fatal flaw. Software developers are people, not robots.
So, here are some ideas for how you can get the benefits from the Personal Software Process (PSP) that are more compatible with human psychology.
1. Follow the recommendations without doing any of the tracking
Your goal in following PSP is to remove as many defects as possible as soon as possible. You definitely don’t want any errors in your work when you give it to another person for peer review and/or system testing. Humphrey recommends written requirements, personal requirements review, written designs, personal design reviews, careful coding in small batches, personal code reviews, the development and use of checklists, etc.
You can do all that stuff without doing the tracking. He even recommends ratios of effort for different tasks that you could adopt. Will that get you to 0.06 defects/KLOC? No. But you might get 80% of the way there for 20% of the effort.
2. Adopt PSP a little bit at a time
To get around the part of your brain that resists radical change, you could adopt PSP over many months. Maybe you just adopt the recommendations from one chapter every month or two. Instead of investing 200 hours up front, you could start with 5-10 hours and see how that goes.
Or you could just track enough data to prove to yourself that method A is better than method B. And once you’re satisfied, you could drop the tracking altogether. For example, if you wanted to know if personal design reviews are helpful for you, you don’t need to do all the PSP tracking all the time. You could just setup an experiment where you could choose five tasks as controls and five tasks for design reviews and run that for a week or two, stop tracking, and then decide which design review method to adopt based on what you learn.
3. Read the PSP book and then follow a process with a better chance of succeeding in the long run
Rapid Development (paid link) by Steve McConnell is all about getting your project under control and delivering working software faster. McConnell, unlike Humphrey, doesn’t ignore human psychology. In fact, he embraces it. I believe most teams would follow McConnell’s advice long before they’d consider adopting the Personal Software Process (PSP).
Most of the advice in Rapid Development is aimed at the team or organization instead of the individual, which I think it the correct focus. PSP is aimed at the individual but I can’t see how you’re going to get good results with it unless nearly everyone working on your project uses it. For example, if everyone on your team is producing crappy software as quickly as they can, your efforts to produce defect-free software won’t have much effect on the quality or delivery date of the finished software.
What I’m going to do
I develop e-commerce software on a team of two. My colleague and I have adopted a number of processes to ensure only high quality software makes it into production. And we’ve been successful at that; we’ve only had 4 critical (but easily fixed) defects and a handful of minor defects make it into production in the last year. Our problem is that we have quite a bit of rework because too many defects are making to the peer code review stage.
I showed my colleague PSP and he wasn’t excited to adopt it, especially all the tracking. But he was willing to add design reviews to our process. So we’ll start there and improve our processes in little steps at our retrospectives–just like we’ve been doing.
Rod Chapman recorded a nice talk on PSP and I like his idea of “moving left”. If you want go faster and save more money you should move your QA to the left–or closer to the beginning–of your development process. That sounds about right to me.
Additional resources
Here are some resources to help you learn more about PSP:
- Nice overview of PSP by Rod Chapman (video)
- Watts Humphrey speaking about TSP/PSP (video)
- A PSP case study supporting PSP (pdf)
- Research paper that casts doubt on the benefits of PSP (pdf)
- PSP: A Self-Improvement Process for Software Engineers (paid link) (book)
- Link to the programming exercises for the book (website)
- Process Dashboard – a PSP support tool (website)
Takeaways
It’s tough to recommend the Personal Software Process (PSP) unless you are building safety-critical software or you’ve got excellent organizational support and sponsorship for it. Most developers are just going to find the Personal Software Process overwhelming, frustrating, and not very compatible with the demands of their jobs.
On the other hand, PSP is full of good ideas. I know most of you won’t adopt it. But that doesn’t preclude you from using some of the ideas from PSP to improve the quality of your software. I outlined three alternate paths you could take to get some of the benefits of PSP without doing the full Personal Software Process (PSP). I hope one of those options will appeal to you.
Have you ever tried PSP? Would you ever try PSP? I’d love to hear your thoughts in the comments.
How did your PSP experiment turn out?
I did a couple of the exercises and then got stuck with a problem with the PSP logging software I was using. Then I got busy and haven’t gotten back to it.
I found it difficult to relate to the process. I work in an interpreted language (which PSP want designed for) and I like to do TDD and PSP doesn’t like that either. It would be easier to adopt if someone adapted PSP for more modern development practices.
I’m not going to do the full PSP at my job. I believe it’s overkill but I’m open to being proven wrong. My plan is to add the elements of PSP to my process that I think will benefit me the most. I’ll add a little bit at a time, measure my results, and keep what works.
Very nice perspective It’s so cool to see people having so much discipline to try things like you mentioned. So few of us try 🙁 Small steps are essential in such cases!
In my oppinion:
• I took the PSP course and got certified long ago, I remember some of the issues you addess:
-There was this question about “what about the scripting languages like PHP?, in those cases there are no compiling stage!”; the instructor aproach was that the compile phase is just an end mark of the end of the coding phase (you don’t acctually have to take statistics of the compilation process). It is just a milestone.
-The scenarios where we as programers are writing a few lines of code, then compile, then change code, and then compile again, and so on, those scenarios commonly are for prototyping or testing a technology that you are getting to know or testing; and maybe you don’t need to get those scenarios through PSP, since they might not be scenarios for producing an end product.
• “Moving left” sounds really good; I’m not really convinced of Test Driven Development, I guess that the “design” phase always occurrs, no matter how brief it is or if it only takes place in your mind and not in a diagram or any other formal format, there is always a design! So thinking of a development being driven by (only) test (test cases, etc.) sounds odd to me; in my oppinion there could be a design phase where of course one can design test to be applied to the design and eventually to the code and the program, a design review, coding, coding review, and test. But not a process, where there is not a design that is the root of how to solve a problem and the source of what you’re about to code.
• I totally agree on being able to take small pieces of PSP little by little in order to adopt a reallistic process, one that meet all the expectatives in your team (PM, SM, QA, etc.) and that at the same time helps you to improve quality, prediction and efforts tracking (after all, those are expected issues that we as software developers are suppose to manage!); acctualy that’s my recomendation to every place where I work, sadly lots of software developers falls into the category that you descrive: our brains resist radical change (or even small changes). It is hard to convience managers and developers to adopt an aproach like this, they don try it, and most of the times they acctually doesn’t effort enought to understand it.
• There are a lot of missunderstandings around PSP, as they are to many software metodologies/frameworks. I’ve heard lots of odd simplifications (made without ground) and lots of absurd polarization between “trendy” metodologies and “old” metodologies. The bottom line for me is: real chalenge is to find when and where to apply any of the frameworks and metodologies; and not to discard one of those just because confort or lack of knowledge or “trend”.
Thanks for all your comments. I’m curious what, if anything, from PSP you use in your daily work? Do you follow any of the practices? If not, why not?
Also do you achieve low defect rates in delivered code with the methodology you are using right now (whatever it might be)?
Recently I moved to a software developer rol again, and I’m getting used again to take doing thins like the ones I did when I’ve learned the PSP.
So, talking about my past experience, at first it consisted on gain awareness about the phases of my proccess, things like: realizing that I’ve had to first plan my phases and figure out every task that I would need in my process; I’ve started to take the time to design the solution properly (not just doing diagrams or algorithms in my head anymore), also I’ve started to commit my self to start writing code only if I have had no doubt about the software requirements or architecture or technical issues I was about to confront (again, if it was a requirement where I needed to do a prior research -to understand a component, object, framework, etc, that was another effort, those hours were not part of my current development effort, those task had to be worked prior to this PSP execution).
I understood that doing it so helped me to dig into the problem to solve early in the process, and ultimately to getting knowledge/conciousness of what kind of effort would be required (kind of Sun Tzu’s mindset: solving the problem before going to the field, in this case before the coding phase).
So, I’ve started to write down:
-Tasks
-Estimating effort (using relative sizes)
-Gathering statistics of time per phase (I used PSP Dashboard)
-Recording my defects. Actually, understanding which kind of defects I used to inject in my software, helpedme to gain awareness and to avoid them, it’s funny how sometimes we tend to repeat behaviours and once we get over it actually we get better.
More or less that was what I remember doing back then; everithing was at a personal level, and it helped me to be more confident about estimates, to decompose problems early in my process (and thus, to ask for support or point out risks to the PM on time), commiting my self to my estimates. All that effort took away that feeling of uncertainity to the process of software developing.
Later I was assigned to lead an Architect team, and we took some of this habits!. One of our services as area was to implement research and proofs in order to adopt technological components for the buisness (doing research about commponents/technologies, doing benchmark kind of exercises, compare technologies, executing PoC, etc.); we used PSPDasboard, and since we had our process so detailed (and I really think this was a key point) we’ve started to measure it, and we’ve gain experience and it gave us the confident (and supporting elements) to defend for our time estimates.
Based on those experiences I could say that taking lets say 40% of the PSP framework gave me so much in exchange: maturity, confident to make forecasting, gain awareness at the design and an improve on the quality of my product, elements to defend my estimates, etc.
Based on this PSP adoption level, would those projects/efforts become a TSP certified project? No way, but this level of adoption of PSP helped us to improve.
Sorry for the long story, I hope I have answered properly your questions. And by the way I’ve enjoyed your article!. It seems that there is no much interest of new generations with PSP around this days on the web.