Executive Summary: Handbook on ICT Indicators, with a Pilot Study on Time-Distances in e-Commerce Adoption
Constructing meaningful ICT indicators – a challenge for statistics
The rapid adoption of information and communication technologies (ICT) in business, and the objective of anticipating possible policy implications of this development, has triggered a demand for statistical information on the diffusion of ICT in firms and about related activities. ICT indicators can be grouped in many ways, but a basic distinction can be made between four broad areas of measurement:
- Statistics on the ICT sector: measurement of the contribution of the ICT sector to the overall economy. This report makes references to OECD work in this area, and features EITO European Information Technology Observatory as one of the most important and authoritative sources for ICT market statistics in Europe.
- Statistics on ICT investment: measurement of the aggregate investments by firms on ICT. Statistics are closely related to ICT sector statistics, but take a demand-side perspective, focusing on buyers' expenditures and behaviour.
- Statistics on ICT use: indicators focusing on the adoption and use of ICT in firms and households. Indicators are mostly derived from representative surveys. The e- Business Survey by e-Business W@tch and the annual Survey on ICT Use in Enterprises by Eurostat are relevant examples in Europe.
- ICT service indicators: measurement of the availability, price and quality of ICT services, particularly telecommunication services. These are supply-side indicators, provided, for instance, by ITU (International Telecommunications Union) and by the OECD.
This study focuses on the third category. It discusses issues related to the definition of indicators on ICT use and presents challenges that are related to the corresponding data collection. Major challenges in this context are definition issues (what exactly constitutes "ebusiness" and should be measured?), the adequate translation of concepts into survey questions, and issues of adequate aggregation methods (e.g., weighting of results). The report mainly addresses policy-makers who occasionally work with ICT statistics, rather than ICT experts themselves. It does not address statisticians, as they are fully familiar with the issues presented in this report.
What users of ICT survey results should know
In many cases, confusion about statistical data is the result of misunderstandings, either because of unclear meta-information about the data, or because of misguided expectations on the side of the user. Users of ICT indicators should ideally possess a basic understanding of key issues and processes of data collection. Four concepts are key to a proper reading of data stemming from surveys:
- The population of the survey: What types of companies or organizations were included in the survey?
- The base of a figure: If a value is 28%, what exactly does "100%" stand for?
- The weighting scheme: Do small and large enterprises count equally in percentages, or have data been weighted, for instance by employment or turnover?
- The statistical confidence interval: What is the statistical accuracy that can be expected from the data presented?
The role of research projects vis-à-vis official statistics
The main strength of indicator related research projects is their flexibility in developing and piloting new approaches, without being bound by the "legacy" of existing statistics. The down-side is that these efforts are mostly one-off studies, and that they usually do not have the resources to collect primary data on a substantial level. Ideally, therefore, the successful parts of these pilot exercises should migrate into the regular surveys carried out by official statistics. This report features selected research projects, mainly from the R&D Framework Programmes of the EU that have addressed the issue of information society and economy indicators.
Compound indicators and recommendations for users
Notwithstanding the risks and limitations of compound indicators, the report encourages policy-makers to actively promote the development of compound indicators and to make use of them. There are many good examples of the usefulness of compound indicators, particularly in policy areas which do not lend themselves to be measured by one or two specific indicators only. Moreover, compound indicators are a powerful instrument to trigger public debate about policy objectives, which can be a desirable goal in itself in terms of fostering democracy.
Innovative approaches in indicator development
The study features two methodological approaches for adding value to simple indicators that could be particularly useful for policy: S-time-distances, and correspondence analysis.
- The new generic time distance approach (with the statistical measure "S-timedistance") offers a new view of data that is easy to understand and communicate: it shows differences in adoption rates (e.g. of technology) as differences in time when a given rate was attained by laggards and by the benchmark adopter. This method can be applied to e-business adoption. For instance, the relative disparity in the adoption of online buying activity between the UK and Spain (adoption rates 46% and 19%, respectively, in 2003) can be translated into an S-time-distance of about 3 years.
- Correspondence analysis (CA) can be considered as an alternative approach to compound indicators. CA is a descriptive, multivariate method that can handle small and large data-sets to produce a graphical output of the results. CA can reveal underlying structures in a large data set by reducing its complexity, without losing essential information. It is methodologically related to principal components / factor analysis.
Policy recommendations
Policy is not only an important user of statistics on ICT diffusion and impacts, but is also in charge of making the necessary provisions that these statistics are available in high quality. The report makes the following recommendations for adequate use of statistics and for establishing mechanisms to further improve the quality of indicators:
| Policy objective |
Suggestions for policy |
Provision of adequate indicators on ICT adoption
|
- Strengthen links between activities of official statistics and research projects
- Strengthen links between closely related regular surveys carried out by official statistics
- Carry on efforts to develop meaningful compound indicators for the information society and economy
|
Good use of existing ICT indicators in policymaking processes |
- Use compound indicators as a vehicle to trigger public debate
- Going beyond GDP: apply the Balanced Scorecard technique for monitoring EU information society and economy development
|
Download full report
