As you read this, individuals, hacker spaces, activist communities, and organizations are rapidly creating a host of new technologies. We’ve written about issues of redundancy and the impact this can have on the use of tools and how new tools get better over time with attention and refinement. With this in mind, we’ve been thinking about how our work can best improve communication between technologists and the people on the ground using their tools.
By sharing information about what’s available (and pointing to the great organizations developing, reviewing, and training on new tech) we’ve sought to passively facilitate smart technology adoption. Now, we’re looking at projects and methodologies for building productive beta testing communities that are tailored to the needs of technologists developing for advocates.
Why Beta Testing Is Important
When a technologist or tech firm launches a new tool in the social advocacy field, there are a few components that are likely the same:
- It’s open source, free and easy to download
- The launch is planned using existing network contacts
- Feedback from end users comes from either those close contacts directly contacted, or users in the open source community
- Iterative development strategy ultimately relies upon feedback from this small number of end users
To improve data collection and develop more representative samples of end user feedback, tech developers need to take a cue from their mainstream counterparts (i.e. anything that’s in the Itunes app store right now) and put resources into creating dedicated beta testing networks. What’s the engine room’s methodology for building such networks? We’re working on it, but first let’s look at how it’s been done before.
How we’ve seen it done in the past
Close contacts — like other technologists, friends, and colleagues — coupled with pools of beta testers defined by basic demographic categories review apps for usability, report back to the technologist team who iterates before a launch. When the tool is launched, technologists and organizations sometimes do a blitzkrieg outreach operation, work to promote the use of the tool, and open up the code to be improved upon by other developers.
If (a big if) there are resources, development that is based on user feedback will continue with the leadership of the original development team; if not, the app evolves organically and in a way not always in line with intended priorities or what users want most.
How the engine room plans to do it
To be effective, beta testing will need to take into account the wildly different ways in which people use tech (and want to be using tech) at the country level, and constraints on users that are imposed by things like censorship and infrastructure. We’ll develop tailored testing networks made up of on-the-ground advocates and, as data comes in from people testing the tool, make sure it’s as actionable as possible. Tools will be developed based on the needs of their users instead of the needs of developers’ friends. Plus (!) this data will inform not jus iterative development but also security strategies, the development of user guides and outreach to get more users.
Why tech developers shouldn’t create testing networks themselves
Technologists developing tools should not collect data directly from end users for several reasons:
- Technologists have ingrained expectations for end user input and this may affect the way that they incorporate data in the iterative development of technologies;
- Technologists have ingrained expectations about the appropriate user groups to be targeted;
- Technologists’ time is best spent on refining the technologies that they are developing.