You raise an interesting matter, Josh. If I’m reading the article well, the interviewees compare crowd-sourcing to self-assessment by the proprietor. Actually, there’s a third method: screenings by an external professional agency. All these methods have their strengths and flaws.

Self-assessment by the owner and crowd-sourcing both have a more personal aspect, as they both involve a directly interested party – either the supplier or the customer/visitor, while an external auditor will (hopefully) try to avoid anything that’s too biased. It’s like when a film director will talk about their latest movie, compared to a viewer and a film critic.

Crowd-sourcing gives people something to commit to, it’s personal and it’s probably cheaper to organise, compared to professional screenings (if you don’t count the costs to keep people motivated). For this and other reasons, crowd-sourcing will be interesting when you want to cover a large amount of audits in a short time, like on an action day.

On the other hand, a professional screener will be more accountable for their work, and they can be expected to make a more thorough analysis considering a variety of accessibility requirements. The screenings are also easier to harmonise and to coordinate if you’re working with only a limited number of screeners.

If there’s a budget available, I think that a combination of methods would be best in the long run. Crowd-sourcing can give you an indication of where you’ll want more details. People will be happy to keep an eye out for any outdated information, if you provide a feedback tool. Likewise, it’s not unusual to ask a proprietor some basic information before an external agency gets involved. But I’d certainly prefer paid researchers if the screenings have any official consequences (certification, grants, etc).