Transferable skills in a disrupted job market: a data perspective
By Kasper Kok, Product Owner of Knowledge Resources and Juliette Conrath, Team Lead of Proserv – Southern Europe
The global outbreak of Covid-19 has severely shaken up the job market, both in terms of demand and supply. Millions of professionals in tourism, hospitality and other areas are currently unable to perform their jobs and the possibility of an economic downturn has made organizations cautious to spend and hire. The recent changes in consumer behavior induced by the crisis also have a flip side, however: jobs in delivery services and telecommunications are trending and there is obviously unprecedented demand for medical personnel and volunteers.
A disbalanced market disruption calls for mobility: in a time where entire industries are paralyzed by governmental restrictions, many people are left with no choice but to deviate from their current career path. But where does someone go who has been working in the tourism, entertainment, or hospitality industry all their life? Wouldn’t any employer outside these industries reject them at first glance of their resume, based on a lack of relevant experience?
Here’s some good news: modern staffing and recruitment experts are increasingly interested in tools that can help them identify transferable skills. Job titles can vary from industry to industry and even from organization to organization, so organizations are focused on identifying skill sets and looking at how to transplant skills gained from one experience into the next.
When looking at jobs in terms of the competences that are required to execute them successfully, the opportunities for mobility inside and across professional domains are vast, and opportunities go beyond the obvious. In this article, we’ll provide a data-based perspective on transferable skills across domains. The analyses we present build on three assets that Textkernel has developed over the last two decades: a substantial collection of over 1 billion vacancies, high-quality document parsers that can transform these documents into structured data, and taxonomies of professions and skills according to which vacancies can be classified.
Crawlers, parsers and taxonomies: a recipe for reliable analytics
A first step toward reliable analysis of skills in the job market is access to a large set of job market data. Textkernel web crawlers spider the web for vacancies from a large collection of job boards and staffing portals, totalling tens of thousands of websites. These vacancies are then transformed into structured data using Textkernel’s parsers, which decompose the documents into individual bits of information: job descriptions, candidate requirements, benefits, skills, etc. The final step is to normalize this information. That is, to relate the words found in the texts to the units that are of interest to HR and TA professionals, such as skills and professions and education levels. Normalization, in other words, is about recognizing synonyms that refer to the same concept to ensure that the analysis will be robust to linguistic variation.
Normalization happens based on Textkernel’s various taxonomies. Our Skills Taxonomy contains over 11,000 skills and recognizes over 135,000 synonyms in 6 languages.
In addition to the skill taxonomy, Textkernel maintains a profession taxonomy, which contains 4200 professions. The analyses presented below are based on the millions of jobs in the UK job market, which are normalized according to these taxonomies.
Case studies: the booking agent and the warehouse worker
To show how a large database of parsed and normalized documents can help identify potential professional transitions, we look at two case studies.
Jane is a booking agent, responsible for booking hotels and other travel arrangements. But the office hasn’t seen any orders coming for many weeks now and will be forced to let go of most of their employees.
Jake is responsible for receiving and processing incoming stock in a commercial warehouse, but the warehouse has seen a drastic reduction in business and he is not optimistic that his temporary contract will get extended.
The current crisis has made both Jane and Jake wonder about the next steps in their career. Different from a normal job-transition, the overall decline of travel and retail activities impedes job transition within the same role of even professional domain. But what are their chances in a different role or even another industry, and which of their competences can be transferred to another type of job?
Frequent versus job-specific skills
First, let’s explore the core competencies we can assume Jane and Jake to have. Based on our enriched vacancy database, we can answer a simple question: what skills are most often asked for in job postings for booking agents and warehouse workers?
Skills most frequently found in vacancy postings
|Booking agent||Warehouse assistant|
|1||Customer Service||1||Stock Control|
|3||Sales||3||Packaging and Processing Duties|
|4||Attention To Detail||4||Unloading|
|7||Team-working||7||Hardworking And Dedicated|
|8||Telephone Skills||8||Logistics Operations|
Upon inspection of these skills, you’re likely to notice that some of them are rather generic. Skills like communication and passionate are very frequently required in these professions, but they are not specific to these professions. This ubiquitous demand makes them important skills to develop, but they are unlikely to be among the key reasons for a hire. Rather than looking at the frequency of skills in requisitions for different jobs, we therefore need a measure of how strongly skills are associated with certain jobs. Instead of raw frequencies, we’ll therefore use a metric which we will call job-skill association strength. It has a high value for skills that are in high demand for a given job but not for many other jobs. Going back to our case studies, the following lists show which skills are most strongly associated with booking agents and warehouse assistants.
Skills most strongly associated with the target professions
|Booking agent||Warehouse assistant|
|2||Property Management Systems||2||Forklift Trucks|
|3||Upselling||3||Handling and Load Carrying|
|7||Hotel Reception Duties||7||Packaging and Processing Duties|
|8||Amadeus CRS||8||Warehouse Management Systems|
|9||Front Office||9||Manual Handling|
|10||Attentive Service||10||Logistics Operations|
The next best job: skill-based job transitions
Now that we have a way to characterize jobs in terms of which skills are most strongly associated with them, we can measure the distance between jobs based on overlapping skills. Here are the top 10 closest jobs to those held by Jane and Jake.
Jobs closest to the target professions, based on overlapping skills
|Booking agent||Warehouse assistant|
|1||Head of Reservations||1||Order Picker|
|2||Front Office Manager||2||Warehouse Administrator|
|3||Receptionist||3||Forklift Truck Driver|
|4||Hotel Receptionist||4||Delivery Driver|
|5||Travel Agent||5||Stock Clerk|
|6||Travel Consultant||6||Reach Truck Driver|
|7||Night Porter||7||Truck Driver’s Assistant|
|9||Account Manager Telesales||9||Warehouse Associate|
|10||Head of Reception||10||Packer|
Visualizing the overlap between professions
To get a better grip on the overlap between our protagonists’ profiles and the jobs listed here, we can plot the skill-job associations for two professions against each other. In the charts below, we take the union of the 10 skills that are most strongly associated with each profession and plot their association strengths on the two axes of the chart. The upper-right corner of these plots then represent the set of skills that are associated with both professions. The top-left and bottom-right show those that are associated more with one of the professions than with the other. To aid the interpretation, we plot ellipses around the relevant parts of the axes of the plots, simulating a Venn diagram.
Let’s first have a look at a possible job transition for Jake. From the above list of closest jobs, we can see that many of the possible job options are warehouse-related and therefore perhaps not the most promising options. There’s one job in the list for which there surely won’t be a lack of demand in a time where many people are homebound: delivery driver. Let’s have a look at which skills a warehouse assistant and delivery driver have in common.
This chart suggests that both professions are associated with various skills related to logistics, loading and unloading, and packaging. Jake won’t need his knowledge of Warehouse Management Systems anymore after the job switch, but he will need to learn a few things about defensive driving and tail lift trucks. All in all, these analyses suggest that there’s not much in Jake’s way of transitioning to a role as a delivery driver.
Next, let’s look at the options for Jane. Here too we see that a lot of the most similar professions are within the same domain, or at least have to do with travel in some way. Assuming that hiring in the travel industry will be low in the considerable future, let’s look at a travel-unrelated option: Account Manager Telesales. Here we see how the typical skills of a booking agent overlap with those found in requisitions for telesales roles.
According to this data, both booking agents and account managers in telesales are acquainted with call centers, sales techniques and call reception management. Some further training in b2b principles and sales concepts might be required for her to make a job switch, but there seems to be a solid common ground between the two positions.
But maybe Jane doesn’t like the sales part of her work all that much? Maybe she aspires to a more radical career switch, and hopes that she can contribute to healthcare? Using Textkernel’s profession taxonomy we could restrict the search for job transitions to the domain of healthcare. Based on skill-overlap, our data suggests that the closest medical job to Jane’s current profile is that of Patient Care Coordinator. Let’s look at what these two professions have in common:
Some upskilling will certainly be required for Jane, but judging from this analysis, there are several competences and activities that intersect the professions of booking agent and patient care coordinator. The ones that are more unique to the latter, shown in the top-left section of the plot, can be seen as areas where Jane will need to look for targeted training if she is to take this job transition seriously.
Skill analytics beyond the crisis
Analyses akin to the ones presented here can have implications to mobility questions of any kind, not just those that arise in times of a pandemic. Challenges regarding internal mobility, outplacement, targeted upskilling and strategic workforce planning all benefit from a clear understanding of which skills are shared between different types of jobs. For instance, one might employ skill-overlap analysis to answer questions like “what skills represent the difference between an account manager and a commercial director?” or “what training would help a Java developer become a data scientist?” The charts shown in the appendix can help to answer these questions.
Not having access to the tooling required for skill-oriented profiling, recruiters and employers are often fixated on role descriptions and assume that different roles within or across industries don’t have enough in common to consider out-of-the-box hires. A good understanding of transversal skills is pivotal to a more agile approach to mobility challenges. Using the powerful combination of crawlers, parsers and taxonomies, new opportunities for job transitions or upskilling can be identified that are fruitful in any state of the job market.
Appendix: additional analyses
The below serve as additional example to show areas where skills overlap could benefit areas such as succession planning and workforce mobility:
The impact of Coronavirus (COVID-19) on the US job market is unprecedented. Textkernel and CareerBuilder have teamed up to share labor market insight into how Coronavirus impacts job inventory, top jobs currently available, top hiring employers and more.
Below is a selection of insights, for the latest information be sure to follow #CareerBuilderCovidData on LinkedIn.
Overall job inventory is down, but these employers are leading the way in staffing up to meet demand in industries like Logistics, Insurance, Food Retail and Healthcare.
There are employment opportunities available for a range of skill sets. 5 of the top 20 jobs available now are in the healthcare industry, 5 relate to sales and customer service, 4 require driving and delivery of goods.
Job inventory is down across all industries. Information Technology, Retail Trade and Wholesale Trade are showing the least amount of disruption.
The demand for all healthcare-related workers is up – but, critical care positions are more than 2x in need as COVID cases exponentially rise in the US.
We’ve seen certain pockets of demand in the Transportation & Warehousing industry, but as a whole it’s down 45% from the 3-year average.
Mississippi, Kentucky and West Virginia are seeing the biggest dips in job postings by state.
Jobfeed data powers analytics dashboards across Europe and North America, including:
- Emsi (US & Canada) – Job posting analytics dashboard – Track job posting trends by day, week, and month and compare to 2019 averages. You can also filter by region, industry, company, job, and skill.
Textkernel takes great pride in our strong ties to the academic community, particularly since we started from humble beginnings in 2001 as a private, commercial R&D spin-off with a focus on research into Natural Language Processing and Machine Learning at the Universities of Tilburg, Antwerp and Amsterdam.
And so with great excitement, our Head of Ontology, Panos Alexopoulos, has announced the release of his book: “Semantic Modeling for Data – Avoiding Pitfalls and Breaking Dilemmas”. Published by O’Reilly, the book serves as a practical and pragmatic field guide for data practitioners that want to learn how semantic data modeling is applied in the real world.
The book is available for purchase global from all good online and physical book retailers. For a complete list of where to buy, visit Panos’ site.
We sat down with Panos to discuss a bit about how the concept of the book came about, and what readers can expect:
Avoiding Pitfalls and Breaking Dilemmas
“The book is about the broader topic of Semantic Data Modeling which is actually the task and problem of creating representations and structures of data in a way that the meaning of the data is explicit and commonly shared and understood by both systems and humans,” Panos explains.
“That’s a general challenge that information technology has, and especially now with AI technology in place, it’s important that meaning is understood in an explicit way by humans and machines. The book fills a gap in the literature and the market, especially when it comes to book about practitioners and professionals. There are several academic books describing how to build an ontology, what is the underlying theory behind data semantics etcetera, but the problem is usually this information is sparse, all around the place, either in papers or in presentations, so it’s never gathered together. What is lacking is the industry perspective, the perspective from the side of a practitioner – what it means to build, use and maintain these kinds of models in the real world, in organizations in the industry. My work here at Textkernel has been one of the key inspirations of the book, so many of the things I’ve seen here both positive and negative have contributed to me being a better modeler and professional, and I wanted to share these experiences with the rest of the community. That’s how the book was born.”
The role of early feedback
“It’s always important when you write a book to get early feedback, and what O’Reiley, my publisher, allows you to do is provide the book online, provide some raw and unedited content on he platform so that any users can see the book and are able to share their opinion, give their feedback, find mistakes, find things that may be wrong or may want more information. That’s extremely useful feedback because in the end it’s all about removing ambiguity. And because this book is not addressed to only one community, it’s actually a wider community and there are different sub-communities in the data world that don’t necessarily use the same terminology or necessarily have the same experiences it’s important that all these sub communities have a an opportunity to say something about the book.”
“Semantic Modeling for Data” is expected to be published in November, and is currently available as an early release version.
Download our latest eBook to understand why Internal Mobility is becoming an important new tool for your talent management strategy.
Learn more about how Internal Mobility can
- Reduce rising talent acquisition costs
- Improve employee satisfaction and engagement
- Build your internal skills economy that meet shifting talent demands
Download the full eBook to find out more:
In 2003, Textkernel started aggregating job vacancy information for matching and analytical purposes under the label ‘Jobfeed’ in The Netherlands. Today, Jobfeed is available for Austria, Belgium, Canada, Germany, France, Italy, the Netherlands, Spain, United Kingdom and US and Textkernel is market leader in this domain. Due to its strong technological base and domain knowledge, Textkernel created a unique source of job market data, allowing users to gain insight into the demand side of the labour market.
The unique aspects of Jobfeed:
- a very large number of sources (thousands of websites) that are spidered daily
- detailed enrichment on the job information that allows the use of many search criteria, regardless of the structure of the original vacancy text
- a high quality and reliable discovery and extraction process, resulting from years of experience
- accurate deduplication of job postings
- coding of professions and other criteria to customer-specific taxonomies
- customised reporting
- an unprecedented history of job data for analysis purposes and the capacity to make these jobs analysable with new insights
Jobfeed provides the ability to draw a near real-time picture of the labour market, and creates the opportunity to do trend analysis based on historic information from its large job database.
The Jobfeed process
Jobfeed searches the Internet daily for new jobs via an automated process. Found jobs are automatically extracted, categorised and recorded in the Jobfeed database. The following diagram shows a schematic representation of this process.
In more detail, the Jobfeed process consists of the following modules:
Jobfeed obtains new jobs from the Internet daily through spidering. In order to achieve broad and deep coverage, Jobfeed uses two spider methods: wild spidering and targeted spidering.
The wild spider is a system that works automatically and dynamically. It continuously indexes hundreds of thousands of relevant (company) websites and discovers new job postings.
Targeted spider scripts are created to retrieve jobs from specific – usually large – websites, like job boards, and websites of large employers. Despite their size and complexity, the script ensures that all jobs are found. The targeted spider scripts run multiple times per day.
Additionally, Jobfeed searches Twitter for links to jobs (currently only in The Netherlands).
Job sites that only copy or repost jobs from other sites (so-called aggregators), are excluded from Jobfeed, because Jobfeed already indexes the original jobs. Furthermore, aggregators often lose or misinterpret important information from the original job, resulting in bad quality.
Classification involves determining whether a retrieved web page contains a job or not. By means of advanced language technology and using textual features on the page, Textkernel’s algorithm determines whether the page should be processed. The classification is tuned to accept as many jobs as possible while discarding as many pages as possible that are not jobs.
Classification is only needed for pages coming from the wild spider, because the targeted spider scripts only fetch pages that are known to contain jobs.
In order to make the jobs searchable, they are automatically structured by means of Textkernel’s intelligent information extraction software. This software is trained on finding data in free text and is therefore independent of the structure of the text or format of the source.
The extraction process consists of two steps:
- Cleaning the web page, by removing all non-relevant content (such as menus and forms), leaving only the actual job text. In case of PDF input, this step does not apply.
- Extracting and validating more than 30 fields from the text, such as job title, location, education level and organisation.
Normalisation and enrichment
Normalisation means that extracted data is categorised according to a standard format. This makes it easier to search the data and perform analyses. Normalisation takes place on fields like professions, education levels and organisations.
For example, normalisation of professions is done by means of a taxonomy. This is a hierarchal structure that consists of reference professions with synonyms. The extracted position title is matched to one of the synonyms. The match does not need to be exact. The job will be linked to the most similar profession. When searching for jobs for a certain profession, all jobs matching any of the synonyms for that profession will be found.
Enrichment is done in case of organisations. The extracted contact information from the job is used to find the corresponding record in a national company database (such as Chamber of Commerce table in The Netherlands). Because the information in the job is usually sparse, a technique called “fuzzy matching” is used. Using this technique, Jobfeed can find the right organisation, regardless of differences in spelling of organisation name, address or in case of incomplete information. From the company database other information can be derived, such as the amount of employees, the company’s primary activity and its full contact information.
Jobs are often posted on multiple websites, or multiple times on the same website. Deduplication is done by comparing a new job with all jobs that have been found by Jobfeed in the past six weeks.
Two job postings that are duplicates of each other, are usually not identical. Deduplication therefore requires a sophisticated approach. As with the classification and extraction, the deduplication system uses a machine learning algorithm. To determine whether two jobs are duplicates of each other, the job description and important features of the job are compared, such as job title, city and advertiser.
Duplicates are not discarded, but also saved in Jobfeed. This way, Jobfeed is able to show not only how many unique jobs there are, but also how many job postings there have been.
Each job posting’s original source is regularly revisited to check whether it is still active. “Expired” means that the job is no longer directly available from the original URL, or that the job is no longer retrievable by a normal user from the homepage of the original website. The expiration date is stored in the Jobfeed database.
Automatic processes for spidering, extraction, classification and normalisation are the only cost-effective way to realise a maximum potential from online job data. However, these processes are not error-free. The quality of Jobfeed data is continuously monitored and improved. This is done using a combination of automatic alerting and manual quality checks.
For more information about Jobfeed, contact Textkernel via firstname.lastname@example.org.
Jobfeed is a product of Textkernel BV. Textkernel specialises in machine intelligence for people and jobs, providing recruiting tools to accelerate the process of matching demand and supply in the job market: multi-lingual resume parsing, job parsing and semantic searching, sourcing and matching software.
The company was founded in 2001 as a private commercial R&D spin-off of research in natural language processing and machine learning at the universities of Tilburg, Antwerp and Amsterdam. Textkernel now operates internationally as one of the market leaders in its segment.