Big Data Gets Personal

Big data is more than yet another technological innovation that alters what we work on and the way we work.  Some of that big data is about us.

For many of us in IT, big data is another in a long series of technological innovations that alter what we work on and the way we work.  Better tools, like integrated development environments, help us be more productive.  Wireless devices and broadband give us more mobility. Big has changed what we work on especially if you are involved with business intelligence and data warehousing. 

It’s also getting personal. Some of that big data is about us.

We can monitor events and conditions ranging from the temperature in a conference room to the number of emails we send to our collaborators.  How do you feel about having your email, tweets, IM messages cross referenced with the number of files you edit and compile?  This question came to mind this morning as I was reading Michael Schrage’s post How Companies Will Googlefy Your Career.  He opens with:

The best way to understand the trajectory of your high performance career tomorrow is to look at what's happening to college undergraduates today. The Amazonified, Googlefied and Big Data-soaked — enriched? — nature of educational advice and assessment will increasingly define how you and your colleagues get hired, fired and promoted.

Schrage’s post and a related article in the New York Times go into more detail about how big data analysis are used in higher education and how it might be applied by companies looking for algorithmic insights into their employees; for example, imagine you are subject to:

...a review of your calendar and meetings says that you're spending too much time with your direct reports; analysis of your emails for the past six months confirms that you need to take an online course in "gender sensitivity.

HR professionals will have to deal with problems that are unique to measuring people: we change our behavior based on how we are measured.  We can measure every conceivable characteristic of inanimate objects in the office or factory floor and not alter the way they function.  Once humans understand what is being measured and how we are rewarded we adjust our behavior. If the boss is looking at our digital traces some of us will make sure we leave the kinds of traces we want the boss to see. 

An entire industry has formed around teaching students to improve performance on standardized tests like SATs and GREs. Search engine optimization experts continually try to improve their clients standing in search rankings while the search engines designers revise their algorithms to avoid the effect of that kind of manipulation. 

How long will it be before books and blogs start offering advice on how to improve your standing in the HR algorithm mill?

There is no doubt that big data is making inroads to HR. It may be in the early stages but it is worth considering how it will change how we work. C-level executives may be getting sold on the idea that they can quantify and tune human performance but that will only work with willing participants.  The value of that data is undermined if too much of what we are measuring has been “optimized” to make some of us look better than we otherwise would.

Dan Sullivan is an author, systems architect, and consultant with over 20 years of IT experience with engagements in systems architecture, enterprise security, advanced analytics and business intelligence. He has worked in a broad range of industries, including financial services, manufacturing, pharmaceuticals, software development, government, retail, gas and oil production, power generation, life sciences, and education.  Dan has written 16 books and numerous articles and white papers about topics ranging from data warehousing, Cloud Computing and advanced analytics to security management, collaboration, and text mining.

See here for all of Dan's Tom's IT Pro articles.

(Shutterstock image credit: Cloud Data Folder)