Research NZ Logo
 


Some of our recent work

Given our Wellington-based location, almost all of our work is undertaken for public sector clients and most of this involves applying market research principles in a social policy setting.

For example, we assist them with:




Market description and segmentation

Ministry of Business, Innovation and Employment

Reducing the effort of conducting business trans-Tasman

This project has its genesis in the establishment of the Trans-Tasman Outcomes Implementation Group (TTOIG) in 2009, which was mandated to develop and carry forward a work programme to establish a single economic market between Australia and New Zealand.

The TTOIG’s mandate concluded at the end of 2014 and, while many outcomes/initiatives were completed, a number were outstanding. One such initiative was the alignment or mutual recognition of the Australian Business Number (ABN) and the New Zealand Business Number (NZBN) which, in a practical sense, would mean that the business number of trans-Tasman businesses would be recognised in both Australia and New Zealand, thereby reducing compliance, reporting and other costs, while increasing or enhancing the opportunities for greater business.

This qualitative research, which involved face-to-face business interviews with key personnel from 16 Australian and 16 New Zealand trans-Tasman businesses, was commissioned to help inform the development of the work programme to deliver the alignment or mutual recognition of the ABN and NZBN. The specific objective of the research was to understand the ‘pain points’ Australian and New Zealand businesses experience at different stages in the trans-Tasman business lifecycle and, in particular, their perceptions of the benefits that might accrue as a result of the alignment.

Ministry of Business, Innovation and Employment

NZBN Sense Check

The New Zealand Business Number (NZBN) is a unique identifier (13 digit number), available for every sole trader, business, corporate partnership or trust in New Zealand. The NZBN links to core publicly available business information about businesses, which is held securely online on the NZBN Register. It is expected that as more New Zealand businesses begin using their NZBN, it will become easier and faster for businesses to share and update critical business information and enable simpler and more reliable business decisions and transactions.

In 2017, MBIE commissioned research New Zealand to carry out a nationally representative survey of businesses in six industry sectors to identify their main and most important business relationships (by brand as well as category). More than 1,360 businesses were surveyed online and by telephone between April and June 2017. This information was being used by MBIE to identify the core suppliers that businesses used in different sectors, to help MBIE identify potential industry partners to promote awareness, uptake and usage of the NZBN among their suppliers and business customers.

Children’s Action Plan Directorate

Survey of Workforce Capability

In 2015, we were commissioned by the Children’s Action Plan Directorate to help profile the children’s workforce and establish a baseline of the workforce’s current core competencies. The core competencies are required so that all workers operate in the same way, professionally, when interacting with children and their families/whānau when keeping vulnerable children safe.

This involved contacting the workforce via the Directorate’s participating agencies (viz. the Ministries of Education, Health, Justice, and Social Development and the New Zealand Police), initially registering the workforce and then arranging the completion of an organisational and an individual (worker) online survey to collect the necessary information.

This information was subsequently analysed and triangulated so that comparisons could be made between what organisation’s reported in terms of policy and practices with regard to vulnerable children, and what individual workers reported experiencing in terms of their employers’ policies and practices.

The Royal New Zealand College of General Practitioners’ (RNZCGP)

Workforce Survey

The Royal New Zealand College of General Practitioners (RNZCGP) conducts an annual survey of its members. The survey provides the College with a robust evidence base to inform its decision making with regard the General Practitioner workforce in New Zealand.

Conducted online, Research New Zealand has been responsible for the survey since 2016.

Core subject topics covered include hours worked in general practice, after-hours general practice commitments, training and training provision, GP and practice nurse vacancies, retirement intentions, practice ownership, work-life balance and burnout.

In addition, modules on special topics are included each year. For example, in previous years, there have been modules on technology, compliance, closed books, and the extent to which patients defer appointments.

Another important feature of the survey is that, in addition to written published reports, the results are available via an e-reporting tool built and hosted by Research New Zealand.

Ministry for Primary Industries

Far North Employer Engagement

In 2016, Research New Zealand was commissioned by the Ministry for Primary Industries (MPI) to undertake research to provide MPI with an in-depth understanding of the issues facing the rural sector in employing local labour.

This research involved two initial qualitative stages, in the form of group discussions, with employers from the key sectors operating in the Far North, including forestry, horticulture, sheep and beef farming, and aquaculture. The first round of these discussions was used to provide an understanding of the motivations and inhibitors driving decisions regarding employing local labour. Building on these findings, the second round explored responses to potential short and longer-term solutions. Finally, the findings from the qualitative research stages were used to feed into a survey of Northland employers to gain responses from the broader rural-based employer community.

New Zealand Rugby Union

Surveys of Young Boys and Girls

In 2013 and 2015, Research New Zealand designed and conducted two surveys for the New Zealand Rugby Union; one with Year 9 to 13 secondary school boys and a similar survey with Year 9 to 13 secondary school girls (i.e. 13 to 17/18 years old).

The overarching objective of the surveys was to establish young New Zealanders’ interest in playing rugby and the obstacles to doing so. This was in response to falling recruitment numbers. For current players, the surveys also focused on establishing the playing experience and what could be done to optimise that experience.

The survey questionnaires were designed with the assistance of our clients at the New Zealand Union Rugby and when they were in draft form, cognitively tested before being finalised. An important requirement was that they could meaningfully be completed by both players and non-players.

One of the survey challenges was recruiting suitable samples of boys and girls for the surveys. After considering all possible options, ‘convenience’ samples were recruited with the assistance of the New Zealand Secondary Schools Sports Council, which has Regional Sports Directors in each region of the country. Regional Sports Directors have close relations with schools and were therefore able to recruit 26 schools from 12 regions for the boys’ survey and 33 schools from 11 regions for the girls’ survey.

Schools were incentivised to participate. Each school was asked to randomly select two form classes to complete the questionnaire for each survey. Completed earlier than the girls’ survey, the boys’ survey was conducted on paper, with Regional Sports Directors taking responsibility to distribute and collect the completed questionnaires from each school.

In contrast, the girls’ survey was conducted on-line, with survey links sent directly to a liaison person within each school. In the final analysis, a total sample of n=1,674 boys completed their survey and a total sample of n=743 girls completed their survey.

Ministry of Business, Innovation and Employment

A Survey of the New Home Buying Market

As part of its wider interest in housing affordability, the Building and Housing division of the Ministry of Business, Innovation and Employment commissioned Research New Zealand to conduct a survey of new home buyers, in order to enhance its understanding of the participants in the new home buying market (including who they are, and what influences their purchasing decisions) and its understanding of what (if any) impact a buyer’s desire to customise standard plans has on price and affordability (where the buyer has purchased from a home building company).

The Survey of the New Home Buying Market was conducted between 1 February and 6 March 2013, with a nation-wide sample of n=754 respondents. A mixed online-telephone interviewing methodology was used for this research.

Back to top



Product and service development

Careers New Zealand

Foundation Research to inform the “Skills Transition Project”

Careers New Zealand planned to develop and host an online skills self-assessment tool, targeting 25-34 year old New Zealanders who are described as having ‘low skills’. It called the project to develop this tool, the “Skills Transition Project”.

The purpose of the tool was to help young people relate their skills and experience acquired in the workplace or elsewhere to potential study or training opportunities. The ultimate aim was to encourage them to re-engage with formal training or study, thereby improving their chances of securing stable employment, in a relatively well-paid job.

Against this background, Research New Zealand was commissioned to undertake two key streams of work:

  • Firstly, to re-analyse statistical information sourced from Statistics New Zealand, in order to more precisely define and segment the target user group and identify its defining demographic and socio-economic characteristics.
  • Secondly, to provide a greater understanding of the values, attitudes and needs of the Target User group, and to provide useful insights for the “Skills Transition Project”. This stream involved the completion of six focus groups:
    • Two focus groups with 25-34 year old Māori with low qualifications.
    • Two focus groups with 25-34 year Pacific people with low qualifications; one Island-born and one New Zealand-born.
    • Two focus groups with 25-34 year olds of mixed ethnicity (one group with low qualifications and the other with university-level qualifications.

The results from this research were used by Careers New Zealand to inform the development of the online assessment tool and the information architecture that underpinned it.

ACC

Understanding clients’ expectations of ACC

The development of effective measures of the quality of ACC’s service delivery is an ACC Board imperative. To this end, CMT measures, which have been used in the past, fall short in terms of providing a clear understanding of what underpins the drivers of satisfaction for claimants, and as such, how to improve service delivery.

In a bid to understand more about the importance of meeting, or exceeding, claimants’ expectations of ACC’s services, Research New Zealand was commissioned to conduct a three stage research programme.

The primary objectives of this research were to provide a thorough understanding of:

  • Clients’ service needs and expectations in relation to each of ACC’s three service channels (viz. STCC, Branch and RIS).
  • The needs and expectations not currently being met, and to identify the features of ACC’s service delivery processes and procedures contributing to this situation.

This research employed a building blocks approach, involving three stages. The purpose of the first two stages of this research was to input into the development of the sampling framework and lines of questioning for the final stage. The three stages of the research were as follows:

  • Design and Build (interviews with key informants within ACC).
  • Interviews with frontline staff and Team Leaders.
  • Interviews with clients.

Tertiary Education Commission

Testing the Tertiary Education Commission’s Key Information Set webpage

As part of its Learner Information Programme, the Tertiary Education Commission (TEC) developed a Key Information Set (KIS), to assist learners to easily compare tertiary education organisations’ course requirements and return on investment. On the basis of less than optimal feedback from a piloted version of the Key Information Set at the end of 2015, this was revised in order to better meet the evaluation criteria and engage learners.

The overall objective of this research was to ensure the revised KIS webpage had learner appeal. More specifically, to ascertain learners’ responses to the webpage’s: content, look and feel and usability.

A small number of Year 12 and 13 learners participated in intensive individual face-to-face interviews in which they were shown examples of KIS from tertiary education organisations, followed by TEC’s KIS webpage. The webpage was tested in the TEC’s online pilot environment. Testing included observing learners exploring the webpage prior to discussing their experience of it.

Back to top



The development of public information and education (advertising) campaigns (concept and pre-testing)

Health Promotion Agency

Responding to Infants’ Hunger and Satiety Cues

In some countries, such as the United States, evidence shows that overweight prevalence in infancy has increased in recent decades. It is possible that a similar growth trajectory is occurring in New Zealand, particularly in light of the high overweight/obesity prevalence among two-to-four year olds.

Given the Ministry of Health’s strategic direction in obesity prevention that focuses on maternal and infant nutrition, HPA’s Nutrition and Physical Activity (NPA) team consulted with nutrition experts to inform the development of a work stream that involves working with key influencers on infant feeding practices.

Some evidence suggests that a feeding style which responds to infants’ hunger and satiety cues promotes the retention of infants’ ability to naturally self-regulate their food intake.

To assist and inform this work stream, the HPA commissioned Research New Zealand to carry out an exploratory project to learn more about mothers’ knowledge, attitudes, behaviours and experiences relating to their infants’ hunger and satiety cues.

The project involved conducting six focus groups with first-time Māori, Pacific, and low-income mothers of infants aged six to 23 months, to explore their perceptions of a feeding approach that recognises and responds to infants’ hunger and satiety cues.

Heritage NZ

Public Opinion of New Zealanders’ Attitudes Towards New Zealand’s Built Heritage

In 2010, the Historic Places Trust commissioned Research New Zealand to undertake a survey of the general public’s perceptions of New Zealand’s built heritage.

The objectives of the survey were to gain a greater understanding of: the ‘weight’ New Zealanders attached to the importance of New Zealand’s built heritage, in comparison to its natural heritage; the perceived benefits of protecting and preserving places and sites of importance to New Zealanders; and prioritising which built heritage places and sites were of greatest importance.

In 2014, Heritage New Zealand re-commissioned the survey with the aim of understanding to what degree New Zealanders’ perceptions about the importance of New Zealand’s built heritage had changed since the benchmark survey.

Both surveys were completed as CATI-based telephone surveys. For the 2010 baseline survey, a nationally representative sample of n=1,000 New Zealanders, 15 years of age or more, was surveyed and Māori were over-sampled so that their results could be examined with confidence. The 2014, 2015 and 2016 follow-up surveys were completed with n=500 New Zealanders.

A measure of the importance that New Zealanders place on built heritage is now a regular measure of Heritage New Zealand's annual survey. This survey is now completed with a nationally representative online panel of n=1,000 respondents.

Back to top



Monitoring customers' satisfaction with the services and products they provide

ACC

ACC Service Delivery Satisfaction Research Programme

Research New Zealand has been responsible for ACC’s Service Delivery Survey Satisfaction Programme for over 20 years. During this period, the survey programme has been reviewed and changed at least every three years in order to ensure it is fit-for-purpose.

The survey programme comprises a number of continuous surveys with key stakeholder groups and are completed primarily for service improvement reasons and for reporting reasons. The surveys provide a number of external KPIs.

One of the major surveys in the programme is the survey completed with ‘managed’ claimants. These are claimants receiving weekly compensation or other entitlements because of the severity of their injuries and the fact that they are no longer able to work or they have lost their independence. Understandably, many of these claimants are anxious and concerned with their situation, and this is often reflected in their manner when they are interviewed.

The interviewing team is highly experienced in engaging with these claimants and in situations where claimants become upset, we have an agreed protocol that they follow. Among other priority groups, Māori are of particular interest to ACC given the fact that they are less likely to present when they become injured as a result of an accident.

Finally, the ‘managed’ claimant survey also illustrates our capabilities with regard to cleaning and setting up data for analysis, undertaking analysis and providing informative reports, etc.

The survey is reported at a number of different levels:

  • Every quarter, as soon as the interviewing has been completed, we provide ACC with the quarter’s KPI results. This is in the form of a spreadsheet which shows the results against previous quarters and on the basis of a 12-month rolling average. This is provided within a week of the interviewing for the quarter being completed.
  • This is followed by the presentation of the quarter’s results, relative to the three preceding quarters, in the form of an infographic-style fact sheet. A feature of the survey is the inclusion of modules of questions from quarter to quarter. The fact sheets highlight the results for these modules, in addition to the time series for the core set of questions. Complementing the fact sheet is a short video report of up to 60 seconds which is posted on ACC’s intranet.
  • The results for the quarter are also uploaded to an easy-to-use e-report, which is similar to the tool we propose is built for Oranga Tamariki’s survey. This enables authorised ACC staff to view the survey results, as well as specify their own tabulations by sub-groups of claimants of interest. A verb mining tool also enables the qualitative response to open-ended questions to be examined.
  • Finally, an annual report is produced which includes the results of more detailed analysis. For example, recent reports have focused on the results of analysis aimed at identifying the key factors that influence satisfaction/dissatisfaction.

All the above analysis and reporting is completed by the Survey Team that is responsible for ACC’s survey programme. This reflects our company’s belief that the best people to analyse the survey’s data are those staff who understand the purpose of the survey and the client’s research objectives.

Another feature of the survey programme for ACC is the integration of the survey results with administrative data held on ACC’s system. This is undertaken for the purposes of ACC’s modelling.

Strictly following the procedure specified in our Code of Practice, the integration of the survey results with administrative data involves ACC providing us with a second copy of the original contact database following the completion of the interviewing. This has the relevant administrative data appended. Using a unique identifier, we then integrate this with the survey results for those claimants who were successfully interviewed, depersonalise the resulting dataset, and return this to ACC in the form of an anonymised file. The exchange of contact databases and datasets occurs through our secure SSL VPN secure claimant portal.

Department of Internal Affairs

SDO Customer Experience Monitor

The Department of Internal Affairs serves and connects people, communities and government to build a safe, prosperous and respected nation. They achieve this by providing New Zealanders and New Zealand organisations with valuable services through their Passports, Citizenship, Charities, Community Operations, and Births Deaths and Marriages services.

Given the critical services which it provides to New Zealanders and New Zealand organisations, the Department decided to seek feedback about its services from customers to ensure it delivers services people value, in ways that best meet their needs.

Of particular interest to the Department are its clients’ service experiences across the five different business groups noted above, through four different service channels: Department or Ministry of Justice branch offices, online, by post, and by telephone through the Department’s 0800 contact centre.

The SDO Customer Experience Survey is now on-going and completed each year, on a continuous basis, using a mixed telephone and online methodology, depending on the audience of interest.

The objective of the SDO Customer Experience Monitor is to ensure that the Department obtains statistically robust feedback from its customers about the quality of the service the Department has provided them, and whether this met their needs. Reporting on the survey’s results is completed on a quarterly basis, using a suite of online reporting tools that have been customised to meet the varying needs of different end-users within the Department.

Ministry of Health

Maternity Consumer Survey of Bereaved Women

The Ministry’s Maternity Satisfaction Survey is a tri-annual survey that is completed with women who had a live birth over a period of two months before the date of the survey. The survey over-samples Māori and Pacific women in order to be able to examine their results with confidence. The overall objective of the Maternity Consumer Survey is to measure the level of satisfaction amongst women who have experienced New Zealand’s maternity services. In addition, the survey:

  • provides the Ministry with a comprehensive analysis of women’s perceptions of maternity services;
  • enables the Ministry to assess the current framework for maternity services; and
  • provides information to inform future planning.

In 2014, the Ministry decided that a companion survey would also completed with a national sample of women who had lost a baby in the perinatal period between 20 weeks of pregnancy and 28 days following birth (i.e. bereaved women). This was the very first survey of its type ever to be completed in New Zealand.

Although the questioning in the Maternity Consumer Survey of Bereaved Women is aligned with the main survey of women who had live babies, particular care was taken to develop the survey given the personal and sensitive nature of the subject topic. In this regard, we worked closely with Sands NZ, which is parent-run, nationwide group, specifically providing support and information to families who have experienced the death of a baby:

In order to ensure complete privacy and that informed consent had been correctly obtained, the decision was made to adopt a recruiting approach that required bereaved mothers to ‘opt-in’ to the survey after receiving an explanatory letter from Sands NZ.

All interviews were completed by telephone, by a carefully selected group of female interviewers. The decision was made to complete all interviewing by telephone (as opposed to online or by paper) so that respondents would interact with an interviewer who had been trained to manage situations in which they had become anxious or upset. The training of the female interviewers who agreed to complete the interviewing was completed by Sands NZ.

For both the 2011 and 2014 surveys, interviews were completed with just over n=100 bereaved women. Importantly, no issues or complaints were recorded.

Environment Protection Agency

The Annual Client Satisfaction Survey

In 2012 the EPA commissioned Research New Zealand to conduct an annual online survey of clients and stakeholders who have had contact with the EPA in relation to: HSNO, the Emissions Trading Scheme, a Nationally Significant Proposal, Māori National Network activity and/or the EPA’s website in the 12 months prior.

Following on from the 2012 benchmark survey, the 2013-18 surveying period has involved the completion of a number of online surveys with various stakeholder groups, in relation to their contact with the EPA.

Importantly, the survey questionnaire is updated each year in relation to the EPA’s annual reporting requirements, against its Statement of Intent.

WorkSafe New Zealand

Service Excellence Survey

Since December 2013, the crown agency WorkSafe New Zealand has had a mandate to monitor, regulate and enforce workplace health and safety. WorkSafe is charged by the government with leading New Zealand to achieve safer outcomes in the workplace using engagement, education and enforcement.

WorkSafe’s Service Excellence Survey (SES) provides employers and employees who have had a recent interaction with WorkSafe the opportunity to give feedback on the experience. The SES provides a benchmark against which WorkSafe can track its progress in relation to embedding and promoting good workplace health and safety practices in New Zealand workplaces.

Research New Zealand has been the provider responsible for conducting the SES since its inception in 2011 as MBIE’s Service Excellence Survey. The main objectives of the SES are to: measure clients’ satisfaction with WorkSafe’s delivery of workplace health and safety services; inform changes to WorkSafe’s service mix and/or the delivery of specific services; and enable WorkSafe to meet commitments to report on client satisfaction and changes to workplace practices. WorkSafe also uses the survey to assess the extent to which clients consider its work as fair, consistent and value for money.

Back to top



Brand image and market positioning (trust and confidence)

Statistics New Zealand

Use and Trust in Official Statistics Survey

In 2010, Statistics New Zealand commissioned Research New Zealand to conduct a survey to measure the New Zealand public’s trust and confidence in the information produced by the Department and other government departments (i.e. the body of Official Statistics produced in New Zealand). In 2013, Statistics New Zealand re-commissioned Research New Zealand to conduct a follow-up of the survey.

The specific objectives of the initial and follow-up surveys were to measure the awareness and knowledge of statistics produced by New Zealand government departments; the use of such statistics; whether they were derived from the agency concerned or reported in articles or papers; what purpose they were accessed for; and whether they had met the information needs of the user. Beliefs about the confidentially of information provided to Statistics New Zealand in the course of the Census and its surveys was also measured.

The survey population was resident New Zealanders aged 18 years and over. The data was weighted to ensure that Māori and non-Māori were represented in the correct proportions in the population estimates.

The sample sizes for Māori and non-Māori were such that differences between the two populations could be measured reasonably accurately. The total number of persons interviewed was 1,845. The maximum sampling error based on this sample was is ± three percent, at the 95 percent confidence level.

Back to top



Monitoring stakeholder relations

New Zealand Transport Agency

Survey of Stakeholders’ Perceptions of the NZTA

The New Zealand Transport Agency recognises the importance of having positive working relationships with a range of stakeholders, in order for it to achieve its goals and objectives. These stakeholders include those in Central and Local Government, various supplier groups/agents, and industry and lobby groups.

The Transport Agency has developed a Stakeholder Relationship Strategy and commissioned Research New Zealand to conduct on-going survey research to monitor the current status and progress it is making to develop positive working relationships with these stakeholders.

The baseline survey was completed in 2012 and was updated annually until 2015.

Back to top



Monitoring the impact of policies and programmes

Social Policy Evaluation and Research Unit (Superu)

The Wellbeing of New Zealand Families or Whānau

Superu (now part of the Ministry of Social Development) publishes an annual report on the wellbeing of New Zealand families or whānau, based on wellbeing indicators to be sourced from existing sources (i.e. official statistics and administrative data). The first Status Report was published in 2013 and the report for 2014 has just been published.

To support the annual Status Reports, Superu plans to undertake primary research to collect information on the opinions and perceptions of New Zealand families or whānau. This research is intended to provide a new source of information about families or whānau, to complement the analysis of the wellbeing indicators published in the Status Reports.

In this regard, Superu commissioned Research New Zealand to complete a review of the research literature on the current practice of measuring the opinions and perceptions of families about their wellbeing. The purpose of this review was to identify the primary research options that are realistically available to Superu.

The main conclusion arising from the literature review was that there is currently no general consensus on the best approach to the measurement of family wellbeing. A number of recommendations were also made as a result of the review, including the need to undertake fundamental qualitative research with New Zealand families or whānau, in order to understand how they judged their wellbeing and the factors they took into account in this regard.

This qualitative research was subsequently undertaken (a.k.a. the "voices" study), involving two phases of research; an initial exploratory phase and a main phase in which the interviewing was completed.

In terms of family structure, there were five families of particular interest, based on the official New Zealand-based definition of family (and as recorded in the Census of Population and Dwellings):

  • Two parent families.
  • Single parent families.
  • Couples without children that are under 50 years of age.
  • Couples without children that are 50 years and over.
  • Multigenerational families.

The adult members of each participating family or whānau were interviewed together. A feature of the reporting for this qualitative study was the production of a number of vignettes for 10 of these families or whānau.

Ministry of Business, Innovation and Employment

Better Public Services for Business

For the last six years, Research New Zealand has been monitoring the effort of New Zealand businesses in their dealings with central and local government.

‘Effort’ is not only defined in terms of financial cost and time, but also incorporates such aspects of business as opportunity cost and emotional stress.

The qualitative research that Research New Zealand has conducted in relation to this monitor has focused on identifying the ‘pain points’ that result when businesses interact with government. The backbone of the research programme is a large continuous survey involving n=2,000 businesses every six months, which incorporates the ‘pain points’.

Businesses are initially randomly selected with the assistance of ACC. These businesses are sent a survey invitation letter (on multi-branded letterhead), inviting them to complete the survey, either online or by telephone.

The reporting of the survey results is in multiple forms, including via an e-reporting tool.

Ministry of Health

Green Prescription (Green Rx) Patient Survey

In 2011, Research New Zealand was commissioned by the Ministry of Health to take over the Green Prescription (Green Rx) survey that had previously been implemented and reported by SPARC.

The objectives of the Green Rx Scheme are to encourage New Zealanders to get more physically active, in order to improve their health and general wellbeing. The Green Rx Scheme is administered by 17 licensed providers throughout New Zealand, who until 2013, received funding directly from the Ministry to provide Green Rx recipients with advice and support to improve their levels of physical activity and their dietary habits.

In 2013, the funding for the Scheme was devolved to the various DHBs. However, the Ministry continues to take responsibility for monitoring the success of the Scheme. 

The Green Rx Patient Survey is undertaken each year to collect robust evidence of the Scheme’s success, as well as how the Green Rx providers (contract holders) are performing in relation to nine KPIs they are expected to meet.

The survey is conducted as a hybrid online, telephone and paper-based survey during April and early May each year. The population of interest for the survey is GRx patients who had contact with a GRx contract holder over the six months from July-December in the year prior.

A total of 10,000 participants are randomly sampled from the approximately n=18,000 Scheme participants put forward by contract holders, against an agreed stratified sampling scheme developed with the Ministry.

Each sampled patient is sent a letter on Ministry letterhead inviting them to participate, along with a paper copy of the survey, and a reply-paid envelope. In addition to introducing the survey and its purpose, the letter also includes instructions for completing the survey online, should respondents prefer to do so. Approximately three weeks later, non-respondents are sent a reminder letter about the survey.

The following week, all patients, who have not yet responded, receive a reminder call from Research New Zealand, and are given the option of complete the survey by telephone as a CATI-enabled telephone interview.

The survey data is weighted to be representative of the number of patients referred to providers/contract holders during the July to December period in the year prior. Survey results are reported in detail, at an aggregate level, for each provider/contract holder, with any statistically significant differences based on respondent demographic characteristics highlighted (e.g. age, gender, ethnicity, employment status and socio-economic status).

ACC

Attitudes to, and the use of, the Rehabilitation Progress Checklist

The Rehabilitation Progress Checklist (RPC) is a questionnaire used by ACC case owners to assist with the early identification of pain related disability factors. The RPC’s focus on pain is based on evidence that the earlier pain-related flags are identified and managed, the better the outcomes for clients. If the RPC results in a high rating, this directs the case owner to refer their client to put pain-related and/or psychosocial services in place to expedite their recovery.

A research programme, with three streams of work, was completed to determine the efficacy of the RPC in facilitating client rehabilitation and return to work or independence, and to understand how it was being used, including any barriers to its effective use (e.g. case owner attitudes, training). The overall objective of the research was to inform a possible national roll-out. The research programme included: an online survey of case owners; qualitative interviews with national office stakeholders, specialist support staff, and case owners, and; data analysis, comparing the service utilisation and service satisfaction of clients who had and had not had the RPC administered.

WorkSafe NZ and ACC

Safety Star Rating Pilot Evaluation

In 2016, Research New Zealand was commissioned by WorkSafe and ACC to complete an evaluation of the pilot programme for the Safety Star Rating Initiative (SSR). SSR is an injury prevention initiative that aims to lift the performance of workplace health and safety in New Zealand businesses. It examines how well a business is performing against 15 standards that are important for good health and safety performance and provides the business with guidance on how to improve.

This evaluation involved working with WorkSafe and ACC to develop the evaluation design and associated success indicators, of which there were eight criteria. In-depth qualitative interviews with 33 of the 38 businesses that participated in the pilot were subsequently conducted, investigating opinions of SSR’s relevance, its likely effectiveness to drive improvements in occupational and workplace safety practices, and business leaders’ experience of the implementation of SSR, as well as their opinions regarding how it could be improved to better meet businesses’ expectations and needs.

Ministry of Justice

Te Mana o Ngāpuhi Kōwhao Rau Trust Evaluation

Commencing operations in 2011, the Matariki Court was established in Kaikohe to address the disproportionately high imprisonment rates of Ngāpuhi within the Far North, and mandated Te Mana o Ngāpuhi Kōwhao Rau Trust (based in Moerewa) to support the Matariki Court’s objectives by taking a Ngāpuhi-specific kaupapa Māori approach to aid their people through the court process. In turn, the Ministry of Justice funded the Trust.

The evaluation was undertaken to inform the Ministry about the scope of work undertaken by the Trust which would, in turn, inform its decision-making with regard to its continued funding of the Trust. A qualitative approach was taken to complete the necessary fieldwork, with this being primarily conducted on a kanohi ki te kanohi basis early this year. This involved a range of stakeholders, including court participants and their whānau.

The evaluation was completed with the assistance of our Māori research partner.

Back to top



Social research

Education New Zealand

The value of international education to New Zealand

View the report here

The total estimated economic value of the international education sector in New Zealand was estimated at $4.5 billion in 2016, up from $4.3 billion in 2015. While this makes the sector a significant contributor to the New Zealand economy, Education New Zealand considers it important that New Zealanders also appreciate the broader benefits of international students coming to New Zealand to study. As a result, Education New Zealand commissioned this research to provide an evidence base for the broader benefits of international education to New Zealand.

The broader benefits include community-based and socio-cultural benefits, as well as educational benefits, resulting from the presence of international students in our schools and campuses and in our communities generally (and their families who come to visit), exposing us to different world views and ways of doing and thinking about things, as well as helping us to connect with other countries and regions of the world.

It also includes direct and indirect economic benefits, such as filling specialist domestic skills shortages, introducing innovation to New Zealand workplaces, and helping to build New Zealand’s global linkages, as well as tuition fees and other spending by students.

This research study involved two main streams of work which were completed between May and July 2017:

  1. A literature scan, which reviewed how other countries value contributions made by international students coming to study in their country.
  2. Case studies with New Zealanders who have had actual, first-hand experience of the benefits and contributions made by international students who have come to study in New Zealand.

To provide context, a brief overview of the statistical information about the international education sector in New Zealand was also prepared.

The case studies in particular showcase a number of diverse ways in which New Zealanders benefit from international education, and the important, long-term role it plays in building our connections with the wider world. They include a case study featuring:

  • Tim Cossar, the CEO of Te Puia in Rotorua. This case study helps us appreciate the benefits of international students to the New Zealand tourism industry, particularly in terms of ‘filling skilled labour shortages and hospitality roles’.
  • Caroline Bilkey, the New Zealand Ambassador to Brazil. This case study helps us appreciate the benefits to ‘soft diplomacy’ and international trade, as a result of the (often life-long connections) that are made between international students and their New Zealand counterparts.
  • James Bones, the Operations Manager of Building Services Australia, New Zealand and the Pacific for BECA. This case study helps us appreciate the benefits to business, innovation, and the workforce in general.
  • Andy Wood and Jenny Elder, the Principal and Deputy Principal of James Hargest School in Invercargill. This case study helps us appreciate the benefits of community-based, cultural and educational value.


Outward Bound

Providing an Evidence Base

Outward Bound is a not-for-profit organization, which aims to help create better people, better communities, and a better world, by helping young people and others reach their full potential through challenge in the outdoors.

Increasingly, Outward Bound’s sources of funding are becoming contestable. It receives no government funding and is completely dependent on the support of philanthropists and the revenue it generates from the courses it runs at Anakiwa in Queen Charlotte Sounds.

With this in mind, and spurred on by a project completed by the New Zealand Defence Force, Outward Bound approached Research New Zealand for assistance to develop a research methodology that it could continuously use to provide an evidence base of its long term impact.

Our involvement with this project began in 2014 with the completion of a literature review and the development of an intervention logic. In turn, this was followed by a series of intensive one-on-one interviews with alumni and the development of a draft survey questionnaire. Originally, the plan was to administer this questionnaire on two occasions; at the 6-month mark following the completion of the Outward Bound course and again at the 12-month mark.

After further consideration, the decision was made to survey alumni only once (i.e. at the 6-mark mark) and to obtain feedback from a ‘significant other’ (e.g. a parent, employer, etc.). A draft questionnaire for significant others was also developed based on the questionnaire for alumni. Both questionnaires were cognitively tested before being finalized.

The Outward Bound’s long term impact research programme has now been running for approximately three years. Each year, a total sample of approximately n=500 alumni are contacted to complete the alumni questionnaire online, with their contact details sourced from Outward Bound’s contact database. A smaller sample of about n=250 ‘significant others’ such as parents, siblings and employers is also contacted (after they have been nominated by the alumni). In both cases, the response rate is a little over 50%.

This is an ‘opt-in’ survey in the sense that informed consent is obtained at the point of contact, via the provision of information in the survey invitation email that we send all prospective respondents.

A feature of the research programme is an e-reporting tool that has been established to provide Outward Bound with the survey results on an ongoing basis. This tool allows authorized users within Outward Bound to monitor the survey results, including specifying their own tabulations by key sub-groups and examining their qualitative feedback.

Otago University Medical School

The Longitudinal Interprofessional (LIP) Study

In March 2015, Research New Zealand commenced a project for the Otago University Medical School which, for the next four years, will examine the career trajectories of recently-graduated students from the disciplines of dentistry, dietetics, medicine, nursing, occupational therapy, pharmacy and physiotherapy. The project involves two groups of students; one group which participated in inter-professional education, with the other group being a control group.

Given the 4-year period of the study and the mobility of young people, the research design has taken into account the need to ensure students in both groups remain engaged and participate in the annual surveys (e.g. by establishing a study website and Facebook page to encourage two-way communication with (and between) students, and sending communications, reminders and survey links through each student’s preferred method of contact that is, by text or email, and offering incentives for participation in each individual survey and for those who complete all five surveys).

Following an extensive design phase in which the survey tool and all of the communication channels were developed, the survey was initially piloted with a cohort of nursing graduates. The first of the main annual surveys was completed in 2015, the Year Two survey was completed in 2016 and the Year Three survey was completed at the end of 2017.

Life Education Trust

Providing an Evidence Base

Life Education Trust is a not-for-profit educational organization focused on helping young people make healthy choices. Every year, the Trust’s educators teach over 250,000 children about their body, friendships, their identity, food and nutrition and helpful and harmful substances.

Despite its long history of working in New Zealand primary schools, the funds on which the Trust is dependent to operate have become increasingly contestable. As a result, it approached us in 2012 to help it develop an evidence base by which it could not only support bids for funding, but optimize its product offering and method of service delivery.

This project began with a review of relevant research literature and then a scoping exercise to develop the key features of the research programme. As a result of this scoping exercise, the decision was made to initially focus attention on developing a survey tool that principals and teachers could use to provide their feedback about the performance of the Trust’s educators in their school (i.e. feedback from the people who decide whether the Trust and its educators will be invited back into primary schools in the following year).

Extensive qualitative research was completed with a cross-section of principals and teachers in a range of primary schools in order to develop the survey questionnaires for this tool. When these were in draft form, they were cognitively tested.

Running simultaneously with the development of the survey questionnaires, we also focused on a developing suitable processes by which the principals and teachers in schools participating in the Trust’s educational programme could be surveyed. In the final analysis, an on-line system was developed as this was considered most suitable from both a practical and cost-efficiency point-of-view. Surveys for each of principals and teachers are initiated by the educators as they prepare for their arrival at a school or as they leave, after they had delivered their course.

This surveying of principals and teachers has been conducted for the last five years and produces about 2,000 individual cases of feedback per year. An important feature of this survey is the e-report that has been developed which automatically captures and collates the survey results on an ongoing basis, and enables the Trust to examine the results from a number of relevant operational perspectives.

This year we have commenced the development of a parallel feedback system for children who participate in the Trust’s educational courses. At this stage, we have focused on students in Years 5 to 8 (i.e. those aged 9 to 12 years) and are developing an online feedback form which asks questions about bullying, physical activity, mental health, mobile and internet use, family interaction, chores and pocket money, and nutrition.

The design of this feedback form has taken into account survey design methodologies used successfully in other jurisdictions to measure the health and wellbeing of children. The form is still in the design phase and we have just completed the first round of cognitive testing. This has involved administering the survey in a test-situation with 12 students from two different schools (Decile 1 and Decile 10). The researchers have used Belson’s ‘double-back’ method to test the feedback form.

The key features of the current form include:

  • Simple, short questions.
  • The use of simple language (e.g. ‘Do you feel sad or stressed about things?’).
  • A relatively small number of questions.
  • The strong use of visual elements.

When finalized, the feedback form will be completed by children, under supervision in class, using school laptops and tablets.

Health Promotion Agency

New Zealand Youth Tobacco Monitor

The New Zealand Youth Tobacco Monitor (NZYTM) is undertaken by HPA every two years and examines young people’s attitudes and behaviour towards smoking tobacco. Research New Zealand has been responsible for conducting the last three iterations of the monitor. This involves a high degree of organization:

  • HPA provides the survey parameters, including the specific sample of secondary schools that need to be recruited for the monitor. In 2018, Research New Zealand successfully recruited 68% of the n=186 sampled schools.
  • The school recruitment process is based on recognized school contact protocol. This initially involves writing to the school principal and inviting them to participate in the monitor. Invitation letters are followed up by calls to the school. The purpose of the letters and telephone calls is to obtain their informed consent.
  • Working closely with a nominated liaison person in each school that agrees to participate in the NZYTM, a particular methodology is followed in order to randomly select one Year 10 class for the monitor.
  • At the same time, approximately 20 interviewers across the country are recruited and personally trained by our Field Manager. Many of the staff that are recruited have worked on previous iterations of the NZYTM. The recruitment process includes completing Police checks.
  • On arrival, the interviewer assigned to each school checks that a minimum percentage of students for the selected Year 10 class is present. Having established this, they then distribute the questionnaire to the class and remain for the hour that it takes to complete the survey. If the minimum percentage of students is not present, the interviewer reschedules with the school liaison person.
  • When the class has completed the questionnaire, the interviewer completes an initial check of each questionnaire before couriering them back to Research New Zealand for processing. Questionnaires from a total sample of approximately n=3,500 students are expected to be processed this year.

The above process involves a team of three researchers and support staff from Research New Zealand, under the direction of one of our Research Directors. The project is based on a detailed Project Plan, developed in collaboration with our client at HPA. Regular progress reporting, against agreed milestones, is an important feature of this project. In addition to regular face-to-face meetings with the client, all details relating to the recruitment, consent, class visits, implementation of the survey and all communication logs are also recorded on a customized e-report that the Research New Zealand project team and our client at HPA have direct access to.

ACC

Evaluation of Mates & Dates Programme

In early 2014, ACC developed the Mates & Dates Programme, a school-based healthy relationships course, to address the factors that contribute to sexual violence by targeting and attempting to decrease underlying risk factors and increase resilience factors:

  • Risk factors include beliefs about gender equality and diversity; norms that excuse the use of violence, previous sexual violence, and rape-supportive attitudes; and endorse rape myths, impersonal or non-intimacy-based approaches to sexual interactions and relatedly, the use of alcohol and drugs.
  • Resilience factors include the tolerance of diversity; healthy and respectful relationships; and help-seeking behaviour.

The course was developed based on national and international research, evidence, and experience of sexual and dating violence prevention. It was developed as a multi-year course, to be taught by trained course facilitators across Years 9-13, in five one-hour sessions (one male and one female facilitator per class). The course content was tailored to suit each age group, with the more sensitive topics only included for senior students.

Research New Zealand cognitively tested, piloted and then implemented the feedback survey for the Mates & Dates Programme in 2015-2016:

  • The Mates & Dates programme itself was initially piloted with eight New Zealand secondary schools in 2014, with a view to having the programme nationally available in 2015.

In preparation for the 2015 national roll-out, ACC’s student feedback questionnaire was extensively redesigned to better reflect New Zealand conditions. Research New Zealand was responsible for this design work and as an interim step, the questionnaire was cognitively tested (August 2015) and the research methodology piloted with a small number of secondary schools that had delivered the Mates & Dates course to their students in Term 3, 2015. Three of the four schools that were invited to participate in the pilot participated, and an analysis of the feedback provided by the sample of approximately n=250 responding students was completed.

The pilot schools were recruited following recognized school contact protocol, with contact initially established with school principals to gain consent for their school to participate in the pilot. Principals then assigned a member of their staff to be our liaison person.

The survey process involved students who had completed the Mates & Dates Programme anonymously completing the feedback questionnaire, online, during a school period. Survey implementation instructions and a direct link to the survey was provided to each school via the liaison person, prior to the Programme commencing. The feedback questionnaire was completed by students after the Mates & Dates Programme had been delivered.

In return for their co-operation, schools gained access to their survey results on a portal and were able to compare their results with those of other schools, an aggregated, depersonalised basis.

The pilot identified a number of important issues in relation to schools’ ability to successfully administer the questionnaire to all students that had completed the Mates & Dates Programme, largely as a result of internal conflicts (e.g. the timing of external examinations). Younger students (e.g. Year 9) had higher completion rates than older students (e.g. Year 13).

Notwithstanding these issues, students that had provided their feedback rated the Programme positively, although students expressed variable opinions about the course facilitators. New Zealand European students were generally less likely to report that they learnt ‘a lot’ from the Programme in relation to healthy relationships and gender equality compared with Māori and Pacific students.

Following the pilot, the Mates & Dates feedback survey was implemented to all schools involved in the programme through to the end of 2016.

Unfortunately, further funding for the Mates & Dates Programme was not made available, and the Programme has therefore been discontinued.

Back to top