There’s an abundance of data collection tools out there which promise to save time, streamline data processes, and ensure data quality. But how do you know which one is best suited to your needs? Choosing the right tool for your project and organisation may seem like a daunting task. To make it easier, we’ve put together a blog with three main questions to ask yourself before you pick your new data product.
The focus in this blog is on mobile-based survey applications, since they allow you to collect a large variety of data and development projects usually do their baseline and data monitoring exercises based on questionnaires.
Above: A member of the Kopernik team measures water quality using Akvo Caddisfly. Indonesia, November 2018.
Most mobile-based data collection tools support capturing data as text, numbers, dates, GPS, photos, video and audio. But some projects might need to be able to capture other types of data, such as geographical shapes, which are commonly used in agricultural projects to map farm plots, in conservation projects to track deforestation, or in WASH projects to map water resources. You may want to scan barcodes or QR codes, or input biometric details of your respondents. You might want to enrich your data with water or soil quality measurements, air pollution data, drill logs, or other sensor-based data. When selecting your tool, check what other data types the tool provides or if integrations with other data sources are possible.
Furthermore, when analysing data you might want to combine your field survey data with other datasets to enrich your analysis. Does the product support importing data from other sources? If so, in what format?
Let’s start with the overall data scale. Do you need to collect data from 100 respondents or 100,000? Will you use the tool at an organisational or national level? The larger the scale, the better the product needs to be to support traffic without limiting the data collection efforts or slowing down the analysis.
How many questions will one survey hold? Not only does your tool of preference need to be able to support long surveys and work flawlessly in the field, but the more questions you ask, the more variables you will have in your dataset. Does the tool support large datasets, not only in terms of rows (submissions) but also columns (questions)? Does it have a functionality in place to make it easy for you to clean, analyse and work with such large datasets?
Do you foresee using the tool only for one project, meaning you’ll fill out only a handful of different questionnaires, or will you use it across different projects? If so, you’ll need to find a tool which supports not only more data, but multiple surveys and users. Some products have built-in data limitations to ensure the performance is steady. When deciding on your product, it is worthwhile asking about any scale limitations that are in place.
Are you searching for a tool just to conduct a one-time data collection and analysis exercise or are you planning to work with data on a regular basis? The more frequently you capture data, the greater volume of data the product will need to support. Moreover, you will need to be able to easily handle and analyse data coming in at a higher frequency to avoid bottlenecks with cleaning, reporting and analysis.
Frequent data collection often happens when you’re monitoring your projects. So with frequency, you also need to look at how you monitor. Do you gather data from a representative sample? Does the product support that? Or do you want to go back to the source, a specific household, water pump, farmer or school, to collect another round of data? Does the tool allow you to link data about your source to the following data rounds so that you can easily track the changes over time?
One thing is having a mobile-based survey tool that makes it simple to collect data, but you also need to be able to use the data, analyse it, learn from it and act upon the insights you’ve gained. How does the product support you in cleaning and analysing the data? Does the product also make it easy for you to create the reports you need? Does the product provide integrations to share your data externally or to push the data to other tools you use? Are you going to export your data to a dedicated analytics platform? Or do you need an easily accessible dashboard for multiple stakeholders to review your data in real time? These types of considerations help determine whether you need a simple export to Excel, more complex data visualisation features, or even full platform integrations with specialised reporting software.
Many mobile-based data collection tools support offline data collection. If you are working in areas with low connectivity, or none at all, it is important to check not only if you can capture data offline, but also how will you get the surveys onto the devices and the data from the devices for analysis. And what about survey creation, data cleaning and data analysis? If your team works in an area where Internet connection is not stable, find out whether the tool provider offers offline support for these tasks too.
How does the tool fit your organisation?
Above: Members of the SMARTseeds partnership reflect upon data captured using Akvo Flow. Indonesia, November 2018.
The first thing to look at is the team in your organisation that will be working with the product. Who will be creating the questionnaires, collecting data, analysing it, etc? How many people will be working with the product? Some products come with restrictions on the number of users.
The more people working with data, the more you need to think about user access. Will all the users have access to everything or do you want to organise your data to ensure people only have access to data they work with? Does the product support different levels of user access and does it have ways to keep things organised?
Last but not least, do not forget to look at language support. First of all, you need to make sure that the product comes in the languages your teams speak. Many of the current products are multilingual to accommodate international needs. If the product does not support your language of preference, is the tool provider willing to add the language? Secondly, language support for your surveys can be quite a deal breaker. Often, data is analysed centrally, in one language, but the questionnaires are asked in remote areas where the respondents speak their local language. Does the product provide options to translate the surveys?
As with any other product or new system, you need to learn how to use it. You might need to adapt your workflow and roll it out to other team members. How much effort is required to get started? Does the platform require custom development by an engineer in order to fit your needs or are all required features available in a self-service mode for you to start right away? Does the product supplier provide training on the product? Do they have documentation easily available and accessible? Is there a channel in place where you can contact them to ask questions or report issues?
With mobile-based data collection products, there are two main aspects to keep in mind. First is the actual data collection part. Who will collect the data in the field? Are the data collectors familiar with smartphones and other tech tools? Is the mobile app simple to use? Will they need to be trained? How easy is it to get a data collector started? How quickly can you get the survey to the data collector?
Secondly, you need to think about the team creating the surveys and analysing the incoming data. Will you create the surveys from scratch and is it possible to independently create and modify surveys within the product? What about data analysis? Do you have an in-house analyst who will use the product or do you expect the product to provide basic analytics for you?
You can choose from free products to higher end ones, but the main thing to focus on when it comes to price is what it is that you get. With free products, you might not get the support you need or you may be limited in scale. With more expensive ones, you might get features and services that you won’t use. Many organisations first look at the price when choosing a product, but we advise understanding your data and organisational needs before you look at costs. Introducing a new product will always require investment from your end, so it is better to invest in something you can fully utilise in the long term.
Secondly, the cost of the product also depends on the devices you need to run it. Will you need to set up a server internally to run the product? Does the mobile data collection app support any device or are their certain requirements that may limit your purchase choice? When purchasing devices, it is important to also think about overall security. What are the risks and costs of a device getting lost, damaged or stolen?
Above: A World Wildlife Fund (WWF) team member analyses data captured using Akvo Flow. Indonesia, 2018.
When you choose a data tool, you are also going for the organisation behind the product. Where are they based? Will they be on hand to help you get started? What about if you need help later on in your journey? Is the product regularly maintained and further developed? When it comes to choosing a product you aim to use for a longer period or on a large scale, you want to make sure the product will be there, working flawlessly, for as long as you need it.
Secondly, you should make sure that the tool provider handles your data following the legislation of the country you work in and according to the principles of responsible data management. How is the data secured? Where does it flow once it’s collected and where is it stored and hosted?
Thirdly, we recommend understanding who owns the data, who has rights over the data, and who decides if and how it can be shared. This is important if your data is confidential, holds sensitive information, or contains private data, such as respondents’ names. It is important to have answers to these questions, not only for the duration of your partnership with the tool provider, but also for once your contract ends. How will the data be handled, archived, or removed?
Picking the right data product is a very important task, and it’s well worth putting the time and thought into picking the right one for you so that you can simplify the way you work with data. Do you want more tips on ensuring reliable data collection? Download our eBook!