Article: Data Mining & Scraping: Where do you stop when assessing the candidates?


Data Mining & Scraping: Where do you stop when assessing the candidates?

In an insightful session on Day Two of TechHR 2018, HR leaders and experts deliberate upon the use of data mining and scraping trends while assessing candidates.
Data Mining & Scraping: Where do you stop when assessing the candidates?

As the world hotly debates the concept of consent and digital identities, and countries attempt to regulate data sharing and privacy laws, a learned panel takes a look at the impact of data mining tools and activities on the assessment process of candidates. Practices, challenges, and the future of such practices are discussed, and the riders which come alongside using technology are clearly spelled out. 

How much is too much?

Social media profiles and online identities are routinely screened while determining the fitness of a candidate for a role. While this makes every trivial piece of information about an individual accessible and helps recruiters identify if they’d be a good fit in a particular role or organization, the challenge is how does one not see what they don’t want to see? How do recruiters maintain objectivity and fairness in the process? If someone is privy to personal details of a prospective employee’s life, certain biases and judgements are hard to keep at bay. This creates a very real impact on the outcome of interviews and applications and is naturally creating apprehensions. The realization that predictive analysis digs deep into an individual’s life and identity can seem extremely ‘cool’ or ‘weird’, depending on which side of the fence you are on. Social media has forced the concept of consent to evolve.

Consider the case of photographs and videos. Not until very long ago, people featured in a picture or video had to give a written consent to the owner, explicitly giving permission to be clicked and the right to use the said photo or video. Today, just about anyone can flip out their phone and take a picture of anyone they wish. Similarly, as opposed to a single CV, there are, on an average, 7.6 social media accounts for every individual. This reflects just how restrictive traditional models and frameworks are. In order to predict holistic and comprehensive models and outcomes, the behaviors, performances, and social connections of an individual need to be taken into account. But how does one go about it?

The Dilemma: Weird vs. Cool

Data and social information can be a monster or a slave, depending on how it’s used. As the 2016 US elections show, we are still discovering the several ways in which the information, that individuals shared online, is being used. What started off as a leisure activity over a decade away has become a carefully-crafted record of every single thing a person has said, done, or ‘liked’. Anonymity, a much-cherished feature of social media platforms, is slowly fading as platforms are working overtime to eliminate bots and suspicious profiles. This has created an urgency to clearly demarcate between what information to seek and what is off limits – something that is missing today.

Take the example of Facebook and LinkedIn; earlier, it was pretty clear that the former is for personal use and the latter for professional. Today, however, the water’s become muddy and both are used for a variety of purposes. In this scenario, how do you stop viewing or listening to something you don’t need to? How does one mitigate unintentional exposure to information? Organizations are struggling to balance the freedom they accord to their employees and respecting employee privacy, and need more time to be able to handle both seamlessly. In this context, it would be favorable if organizations are allowed to frame their own guidelines and protocols for data privacy and freedom, instead of towing the line set by legislation. 

The Future 

We are still in very early stages of understanding the impact of technology, especially, bots, AI, and Blockchain, on our work and lives. Our generation might not even be able to fully grasp the said impact.  This embryonic phase will be marked by several mistakes and missteps, which is actually a good thing; because the sooner we fail, the sooner we will learn. Going forward, the sheer variety of technological solutions on offer will act as a double-edged sword and understanding the context of their usage will determine success from failure. There are no handbooks or guidelines to follow, and organizations will have to try, and keep trying until they succeed. AI and ML will be increasingly used for organizational engagement, to predict output and analyze different roles and performances. Right now, there is so much data, but not enough ways to use it effectively; going forward this will no longer be the case.  We will, however, have to be mindful as to ensure that the systems and algorithms we develop do not inherit our personal and innate biases. 

HR leaders and professionals are expected to deliver results using technology and predict the future using AI-based algorithms. But right now, the capability to do so is severely limited. Even the best tools today can merely predict the chances of success, and not indicate the performance or the likelihood of an individual to stay in a company. As we step into an unknown and volatile future, retention of top talent will be an important piece of the puzzle, and the current tools and solutions on offer will have to step up. Furthermore, these solutions cannot be mass-produced, for they will be unique for every role, organization, and individual. For instance, all salesperson positions aren’t the same: the culture, the methodology, and the values of an organization play an important role in defining the job and role of a salesperson. 

At the end of the day, we need to distinguish between what we have a right to do and what is the right thing to do. As a thumb rule, HR professionals can go ahead with any data collection and mining process only if they feel it is absolutely essential, if it isn’t causing anyone harm, and if it is ethically sound. This is merely a conversation-starter, and it is essential that everyone works together to ensure that the human element in tools, systems, and processes is retained. Technology can, no doubt, act as a great enabler, but we all need to agree upon where to draw the line. We need to preserve our natural wisdom while using intelligent tools and give prudence to our sense of judgement and collaboration in order to make the best use of the technology available. 

(This article is based on the session Data Mining & Scraping: Where Do You Stop When Assessing the Candidates? on Day Two of TechHR 2018. The panelists of the session were: Ryan Ross, Managing Partner, Hogan Assessment Systems; Emmanuel David, Director, Tata Management Training Centre; and William Tincup, President, RecruitmentDaily. The session was chaired by Dr. Pradnya Parasher, Founder and CEO, ThreeFish Consulting India.)

Read full story

Topics: Technology, #TechHR2018

Did you find this story helpful?



How do you envision AI transforming your work?