Assuring Data Quality at the UNESCO Institute for Statistics CCSA Conference on Data Quality for International Organisations Helsinki, Finland, 6 7 May 2010
UIS Quality Framework
0. Prerequisites of Quality 1990s UNESCO data quality is compromised Evaluation recommends: 1. Autonomous Institute, guaranteed by its own Board 1999: UNESCO Institute for Statistic 2. Relocation from UNESCO Headquarters 2001: move to Montreal Consequences: UIS can do its own resource mobilization UIS can do its own recruitment
0. Prerequisites (cont d) Data quality is the first priority in the UIS Medium Term Strategy 2008 2013 Internal restructuring: Centralized services for common processes Regional teams (all 6 UN languages) Resource mobilization very successful Domain specialists in main fields of competence: teachers, funding for education, TVET, higher education, data sources, quality of learning
1. Assurances of Integrity Developing staff competencies by investing in institutional training: technical competences management Recruitment through competitive process
2. Methodological Soundness International standards: International Standard Classification of Education (ISCED) revision in 2011 Statistical Data and Metadata Exchange (SDMX) Framework for Cultural Statistics (FCS) Development of international indicators for monitoring: EFA Global Monitoring Report Indicators on teachers ICTs in education indicators
3. Accuracy and Reliability 4. Serviceability Assistance to countries: Workshops (regional, national) Site visits (20+) DQAF diagnostic studies Implementation of Data Plans UIS presence in the field: Cluster and Regional Advisors
Objectives: Data Plans describe metadata and specific instructions on how to complete the UIS questionnaires Ensure the comparability and quality of education data Provide essential metadata about national data Provide a clear mapping between national data sources and international data requirements Pre requisite: High level of commitment from countries
Data Plans (cont d) Content: Description of national education structure Mapping to international standards and definitions (ISCED) Specific instructions for completing UIS questionnaires Recommendations for improvement from the international reporting perspective
Data Plans: Outcomes For countries Collaborative relationships between different reporting authorities Clear instructions for use by current and future ministry staff For everyone Better quality data Essential reference metadata Detailed documentation regarding data sources and definitions (trained teacher in country x) For UIS Improved communication Essential guidelines to insure data quality
Objectives: UIS Presence in the Field Assist UIS National Counterparts by becoming a resource person Supporting and facilitating the implementation of the UIS initiatives and programmes Providing technical expertise to the UNESCO offices; Responding to the requests from other partners in the field (UN family, development banks, donors etc.)
UIS presence in the field
5. Accessibility Renovation of the UIS website (2010): easier access to publication, data and metadata improved lay out of texts, tables and charts Restructuring UIS Data Centre to improve presentation (2011) ISCED mappings published on website
UIS Quality Monitoring Performance indicators to assess the timeliness and completeness of: Survey processes (collect and process) Education Database For reporting to stakeholders and donors
Completeness C2. Response rate Response rate Countries 2007 2008 Gain Loss TOTAL 80.3 82.2 17 13 Arab States 95.0 80.0 1 4 Central and Eastern 81.0 85.7 1 0 88.9 88.9 0 0 East Asia and the Pacific 58.8 64.7 4 2 Latin America and the Caribbean 87.8 90.2 3 2 North America and Western 82.8 86.2 2 1 South and West Asia 88.9 100.0 1 0 Sub-Saharan Africa 77.8 80.0 5 4
Completeness C4. Coverage of key indicators 25 20 Change between 2007 and 2008 TOTAL 2007 76.4 2008 77.1 Change 0.7 15 10 5 Arab States Central and Eastern 100.0 86.6 78.5 89.3-21.5 2.7 0-5 -10-15 -20-25 Arab States Central and Eastern East Asia and the Pacific LAC North America and Western South and West Asia Sub-Saharan Africa East Asia and the Pacific Latin America and the Caribbean North America and Western South and West Asia 67.1 58.6 71.6 87.3 89.5 92.0 74.0 66.4 85.6 82.7 24.9 15.4-5.2-1.7-6.8 Sub-Saharan Africa 75.8 66.7-9.1
Completeness C1. Update rate 2008 in relation to target 85% 2007 2008 Target 120.0 100.0 80.0 TOTAL Arab States 89.3 100.0 86.3 78.9 85 85 60.0 40.0 Central and Eastern 94.7 89.5 85 20.0 0.0 Arab States Central and Eastern East Asia and the Pacific Latin America and the Caribbean North America and Western South and West Asia Sub-Saharan Africa East Asia and the Pacific Latin America and the Caribbean North America and Western South and West Asia 88.9 85.3 95.1 89.7 100.0 100.0 76.5 82.9 89.7 88.9 85 85 85 85 85 Sub-Saharan Africa 77.3 93.2 85
T2. Timeliness of data submission 2007 2008 Change Absolute change between 2007 and 2008 TOTAL 13.8 12.7 1.1 4 3 2 1 Arab States Central and Eastern East Asia and the Pacific Latin America and the Caribbean North America and Western South and West Asia Sub-Saharan Africa Arab States Central and Eastern East Asia and the Pacific Latin America and the Caribbean 12.9 18.3 11.1 13.1 11.2 11.0 17.1 9.4 13.0 11.0 1.9 1.2 1.7 0.1 0.3 0 North America and Western 19.5 18.5 1.0 South and West Asia 14.0 10.3 3.7 Sub-Saharan Africa 11.9 10.5 1.5
T4. Timeliness of internal processing Change between 2007 and 2008 TOTAL 2007 3.8 2008 2.8 Change 1.0 3 2 1 0 Arab States Central and Eastern Eur ope East Asia and the Pacific Latin America and the Caribbean North Am erica and Western Eur ope South and West Asia Sub-Saharan Africa Arab States Central and Eastern East Asia and the Pacific Latin America and the Caribbean 4.9 1.6 5.6 4.1 5.4 3.9 1.5 2.9 2.9 3.6 0.9 0.1 2.7 1.1 1.8 North America and Western 1.6 1.4 0.2 South and West Asia 2.5 2.5 0.0 Sub-Saharan Africa 3.8 2.9 0.9
T6. Data freshness 2007 2008 Change Change between 2007 and 2008 TOTAL 28.1 24.7 3.4 9 7 5 3 Arab States Latin America and the Caribbean South and West Asia Arab States Central and Eastern 25.6 29.1 25.8 23.3 27.8 17.6 2.3 1.3 8.2 1-1 -3-5 Central and Eastern East Asia and the Pacific North America and Western Sub-Saharan Africa East Asia and the Pacific Latin America and the Caribbean North America and Western 27.9 24.7 32.0 26.9 22.3 32.4 1.0 2.4-0.4 South and West Asia 25.3 23.9 1.4 Sub-Saharan Africa 30.7 20.8 9.9
Thank you! For more information: Hendrik van der Pol h.van der pol@uis.unesco.org Anuja Singh a.singh@uis.unesco.org