Beware Minefield Documentation
Has this ever happened to you?
You find a really cool software tool and they have documentation - yay!
The docs look good and so you happily start working your way through your project until... Aaaahhhhh!!! You just hit a landmine. The documentation is broken.
You followed the steps exactly, but then something went wrong and there's no indication of what to do next. You are a beginner at this tool, so you have no idea what to do. So, you spend the next 6 hours trying to debug the problem.
You thought it would only take 5 minutes and you feel much sadness since you just wasted most of your day.
Finally you resolve the issue. What a relief! So you continue on the project until... BOOM!!! It happens again.
By this time you realize that this documentation has never been user-tested. Rather than continue through this minefield, you decide to look around for another tool that won't be so painful to work with.
Unfortunately, this is the default
Nearly all documentation I've come across has painful landmines in it. Just yesterday, there was this discussion about AngularJS's docs. If you look at the comments, you can see just how many users this affects and how deeply painful and time consuming poor docs are.
Plea to software vendors (including open-source)
If you want to gain traction and new users, then please please please do actual user testing of your docs. There are a lot of vendors that think they are doing this, but they're not. Having your docs in the wild and being open-source is not the same as actual user testing.
User testing is this:
- choose a project a first time user would do
- find a user that has never used your tool
- have the user attempt the project
- observe the user and see where they hit landmines
- remove the landmines from your docs
- repeat
Fundamental priority error
Most software vendors (again, this includes open-source backers) make one of the most common mistakes in the business world: Focusing on their own awesomeness instead of the awesomeness of their users.
What does that look like? Well, shitty docs full of landmines is the first indicator. Secondarily, you will see vendors focusing on winning awards, marketing how awesome they are, sponsoring conferences, having booths at all the major conferences, and yet can't spend a fraction of their time or budget for user testing.
No, the "community" doing your user testing for you is not sufficient.
Only your advanced community members would be willing to do the testing for you - but guess what? They are also the worst possible people to do it. Why? Because at this point they are experts and know the path through the minefield at a subconcious level. That means that they will not hit the same landmines your new users would.
Only brand new users will be useful for this. However, unless you organize formal user testing, you will never get a clear picture of the experience from new users. They will give up very quickly and search for a different tool. They will generally not bother contacting you to point out landmines in your docs.
How to actually do it
There are plenty of ways to do this. If you can somehow arrange for new users to come on-site, that's awesome, but that's very difficult to do for most companies.
The simpler approach is to:
- hire remote developers (via oDesk or similar site)
- assign them a sample project and ask them to only use the docs
- observe their attempt to create the project (via screenshare etc)
- note where they get confused, stumble, etc
- update the docs
- repeat (this should be a regular event, not a one-time thing)
Competitive advantage
I can pretty much guarantee that none of your competitors will actually do this. It can be tedious work and why do it when you can just apply for more awards and spend heavily to promote yourself, right? That's what your competitors will think. In the meantime, you can do user testing and give your new users an amazing experience.
Original: 30 Oct 2013