Progressive Best Practices

At Vault, our consultants specialize in research design and methodology and we apply our knowledge across all industries, rather than focusing on one sector. Our experience base is both deep and broad, and this history has afforded us the ability to develop and implement best practices in our field. With these best practices at the core of our work, we approach every client partnership with integrity and a progressive mindset. We invite you to learn more about our practices below. 

Sample Quality

We ensure a representative sample by utilizing:

  • Randomization of all client-provided or purchased lists
  • Telephone listings are randomized in replicates of 100, opened slowly for estimates of productivity, with 4 attempts to reach each respondent
  • Nationally balanced and representative online panel sources, passively recruited for market research purposes only, with stringent ongoing multi-step validation processes

Panel Representivity

We employ the following approach to carefully vet our panel partners for quality and data integrity:

  • Panel Composition and Quality
    • Passive, invitation only recruitment methods and acceptance verified with double opt-in procedure
    • Diverse recruitment pool — normalized and representative
    • Recruited solely for online market research purposes
    • Panelists are deeply profiled up front for a variety of demographic, occupational, personal interests and experiences
  • Data Integrity
    • Multi-step validation processes, with security checks from the panel registration
    • Subscribe to the CASRO Code of Standards and Ethics for Survey Research
    • Panel activity is tracked for quality purposes
    • Limited surveys annually per panelist
  • All panelist information is verified
  • Member privacy is carefully protected to encourage honest answers
  • Use an established incentive currency to promote high participation rates

Telephone Surveys

When conducting interviews by telephone, we apply the following practices to ensure strong representivity across geographies, socioeconomic groups and other demographic attributes:

  • Telephone listings are randomized in replicates of 100, opened slowly for estimates of productivity, with 4 attempts to reach each respondent
  • Quotas or booster samples are employed to reach specific populations by telephone
  • With cell phone use, appropriate geographic location is determined for accuracy
  • Supplemental execution methods such as mobile research center, door-to-door, community center CLT are employed to fill “hard-to-recruit” segments.

Survey Quality and Accuracy

We use the following methods to consistently develop and implement top-quality surveys: 

  • Thoughtfully Written Questionnaires
  • Incorporating industry accepted and tested scales to ensure reliability of the data
  • Inclusion of a ‘trap’ question (internet studies) to both stop speeders and weed out bots
  • Extensive Quality Control Checks
    • Ensuring accuracy of data collection and output.
    • Ensuring that the study remains within budget.
  • Pre-test of Programmed Logic
    • Pre-test CATI or CAWI program prior to field start
  • Soft Launch
  • Closely examine preliminary interviews for data accuracy, respondent comprehension, length of interview, initial response and completion rates – and make any necessary adjustments
  • Telephone IDI debrief with small sub-set of early completes
  • Daily Monitoring
  • Review data to eliminate to include but not limited to straight-liners and speeders
  • Review ongoing survey length, completion and drop-out rate, with the potential of increasing the incentive or adjusting a question’s format to increase/improve data output
  • Ascertain where, and to what extent, respondents terminate and take appropriate action, if necessary

Data Quality and Accuracy

We implement extensive quality control steps that essentially guarantee data accuracy:

  • Project manager lays out a tabulation and analysis plan to clearly identify how each question will be tabbed and what the bases are (what question to pull base from and/or what base should equal).
  • Denote when to use a balancing line, when to pull out “don’t know” responses, etc.
  • Provide client with recommendation for all aspects of data tabulation plan, e.g. cross-tab banner points, coding nets, summary reports
  • Quality Control / Base Size Verification
    • Banner by banner table
    • Consistent programming base
  • Thorough personal checking of final data output
  • Includes comparison of final tabulated data to pre-mapped data program

Reporting Accuracy/Efficiency

Tools such as our automatically populating presentation software enable us to deliver results efficiently and reliably, within hours of completing fieldwork:

  • With this tool, the final report shell is customized and perfected with the client during the preparation phase.
  • Ensures a quick update of tracking surveys – with raw data being the only material changing from period to period

Security

We take serious measures to ensure the security of data:

  • 24/7 server health monitoring and continuous backups every 15-minutes for disaster recovery
  • Online safe deposit box – as an alternative to emailing sensitive documents such as client lists
    • Forum to post large files such as telephone or in person interviews immediately after their completion so clients can list and react
  • Secure online programming/hosting
    • Ensures survey is taken only once
    • Disabled print screen and ability to copy flash files