Annual Statistical Report: Experience from the Americas

By Luca Di Gennaro and Edouard Legoupil, Data, Identity Management and Analysis (DIMA) in the Bureau of the America

The DIMA Statistics team within the Regional Bureau of UNHCR in the Americas applies multiple strategies to improve the data quality of the 2021 Annual Statistical Report submission in line with the United Nations Statistics Quality Assurance Framework and UNHCR’s draft Statistical Quality Assurance Framework, which will be published later in 2022. In this post, we will present and explain these strategies.

Low Quality Data, Poor Decisions…

First, what is data quality? Data quality is a crucial and complex concept for data and statistics. If you want to make data-informed decisions and your data is inaccurate, there is a high chance that you will also make the wrong decision! High data quality is the first requirement for policy makers.

As defined in the assurance framework, data quality can be assessed through multiple dimensions including:

• Relevance: Do you really require this information?
• Accuracy and Reliability: Is the information true/precise in every detail?
• Comparability: How does this information relate to other trusted sources?
• Timeliness and punctuality: no one is interested in old information.
• Completeness: How comprehensive is the information?
• Statistical confidentiality and data protection. Confidentiality in statistics is like cleaning in a restaurant, essential!

Examples of data quality issues include inconsistent data, incorrect data, undefined data, disorganized data, duplicated data and incomplete data

Vital concepts on data quality are mentioned in the “Fundamental Principles of Official Statistics”, impartiality: ‘Official statistics provide an indispensable element in the information system of a democratic society… made available on an impartial basis by official statistical agencies to honor citizens’ entitlement to public information’. And professional standards and ethics: ‘To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics…’

Engaging the network of Statistics Focal Points

Data focal points involved in the Annual Statistical Report (ASR) in the 14 UNHCR operations in the Americas receive regular annual refreshers to ensure professional standards and ethics,  accounting for their respective diversity of skills and backgrounds. Data focal points are at the frontline of the data quality journey: they gather the initial reports from governments and validate them with other sources whenever possible. To keep the staff focused and engaged, the bureau organizes meetings both bilaterally and in small groups. The Bureau also interfaces with the team in charge of statistics in the Global Data Service in Headquarters to streamline communication.

Using a Data Quality Monitoring Dashboard

The Statistics team in the Americas Bureau has built a simple dashboard with four dimensions specific for Annual Statistical Report (ASR) submission depicting:

1. Submission stages
2. Disaggregation levels
3. Documentation availability
4. Institutional communication

First, the stage of the ASR submission process is monitored regularly: for example: letter sent to the data providers, data received from data providers, pending internal approval, validation etc. With this, we are clearly attempting to improve Timeliness and Punctuality.

Secondly, we aim at improving the levels of disaggregation: age, gender, location and urban/rural. With disaggregated data, we are trying to improve Completeness.

By revising Standard Operating Procedures (SOPs) and encouraging operations to include sufficient details within those documents, we are trying to improve Accuracy. SOPs should describe in details every ASR category with the expectation of an improvement on Coherence and Comparability. For instance, it will become easier to check coherence among operations. A good side effect of having updated and revised SOPs is to foster a potential handover in case of a new appointment (this year, we had 4-5 new data focal points in Americas). Understanding and documenting data challenges is the first step in the data quality improvement process. The initial assessment of the current state of the data sets is the initial step during the initial development of SOPs.

Last, by revising existing Memorandum of Understanding (MoU) and any official letters sent to the governments, we can systematically check that the request for disaggregated data is introduced in line with the recommendations of the Expert Group on Refugee and Internally Displaced Persons and Statelessness Statistics (EGRISS), i.e., in line with International Recommendations on Internally Displaced Persons Statistics (IRIS) and International Recommendations on Refugee Statistics (IRRS). In doing so, we are trying to improve the data Reliability. For an update on EGRISS in the Americas, please see here.

Those four dimensions (submission stage, disaggregation levels, documentation availability, institutional communication) are summarized in below one-page dashboard:

The dashboard is a tool designed by the DIMA statistics team primarily to communicate with data focal points in operations, but also with senior management both in the operations and the Regional Bureau. This is helpful for senior management, insofar as it allows to easily follow the progress in the preparation of the submissions until completion.