Using online job vacancies to understand the UK labour market from the bottom-up

machine learning
text analysis

Turrell, Arthur, James Thurgood, David Copple, Jyldyz Djumalieva, and Bradley Speigner. “Using online job vacancies to understand the UK labour market from the bottom-up.” Bank of England Staff Working Papers 742 (2018).

Figure from paper

Bank of England

James Thurgood

Bank of England

David Copple

Bank of England

Jyl Djumaliev

Data Science Campus

Bradley Speigner

Bank of England


July 2018


What type of disaggregation should be used to analyse heterogeneous labour markets? How granular should that disaggregation be? Economic theory does not currently tell us; perhaps data can. Analyses typically split labour markets according to top-down classification schema such as sector or occupation. But these may be slow-moving or inaccurate relative to the structure of the labour market as perceived by firms and workers. Using a dataset of 15 million job adverts posted online between 2008 and 2016, we create an empirically driven, ‘bottom-up’ segmentation of the labour market which cuts across wage, sector, and occupation. Our segmentation is based upon applying machine learning techniques to the demand expressed in the text of job descriptions. This segmentation automatically identifies traditional job roles but also surfaces sub-markets not apparent in current classifications. We show that the segmentation has explanatory power for offered wages. The methodology developed could be deployed to create data-driven taxonomies in conditions of rapidly changing labour markets and demonstrates the potential of unsupervised machine learning in economics.


      title={Using online job vacancies to understand the UK labour market from the bottom-up},
      author={Arthur Turrell and James Thurgood and David Copple and Jyldyz Djumalieva and Bradley Speigner},
      institution={Bank of England Staff Working Paper Series},