Kaleido Insights Impact Analysis on Emotional Recognition

By Industry Analysts Jeremiah Owyang, Jaimy Szymanski, Jessica Groopman, and Rebecca Lieb

As emotional recognition technologies make their way into the corporate ecosystem, use cases and impacts abound. Emotional recognition has the potential to add important new levels of information and data to marketing, sales, customer service, HR, training, engineering, and product development.

Kaleido Insights’ methodology for analyzing emerging technology assesses the impacts on humans, on businesses, and on the ecosystem. As part of our ongoing coverage, we’ll be covering a series of topics using our methodology to help business leaders first understand, and then see beyond the bright and shiny to cut to what matters.

In each post, all Kaleido Insights analysts conduct a joint analysis session around one topic (e.g. technology, event, announcement, etc.). In this post, we analyze the business and organizational impacts of emotional recognition technology.

Topic: Emotional Recognition

Examples:  Microsoft Azure Emotion API; Affectiva; Emotient; EmoVu; Nviso; Kairos

Impact Analysis: Organizations & Business

A woman enters, but doesn’t say a word. Is she a likely prospect or is she uninterested?  Is she a job candidate for a job? Browsing in an automobile dealership or department store? In on-the-job training? Trying a new product?

Is she impressed? Angry? Happy? Distracted?

Emotional recognition technologies can help get to the core of her feelings and intent. Biometric analysis, pulse or heart rate monitors, semantic analysis, and technology that analyzes tone of voice are the primary ways in which organizations can learn what a customer or employee really feels. Emotional recognition technology has the capacity to make organizations, brands, products and communications more empathetic.

Business Opportunities

  • Product development Analysis of a user’s emotional state, e.g. level or happiness or frustration, in using new products or responding to product iterations, A/B testing.
  • Advertising and marketing Understanding the emotional response of end users to messaging, and using that data to optimize communications for different cohorts. Also the ability to identify more precise personas. Case example:  Super Bowl ad evaluated for emotional response by Nviso.
  • Customer Service: Better routing of calls, improve efficiency and  information delivery. Case Example:  Carnect uses SABIO’s voice platform to reduce time spent handling customer inquiries.
  • Training and Education The emotional state of a trainee can influence what questions/tasks will be presented, and in what sequence.
  • Investing: Knowledge around empathy or acceptance of new business ideas and/or models can influence allocation of funds.
  • Healthcare: Understanding patients’ levels or pain and suffering, or psychological data will inform care and treatment.
  • Smart Home: Virtual assistants can use emotional recognition technology to perform better and with greater contextual relevance.
  • Autonomous Cars equipped with sensors will sense stress, distraction and other emotions in drivers and respond accordingly.
  • Data Emotional recognition technology can add rich new layers of data around employees, end users and other members of the ecosystem.
  • Robotics: Data gleaned from emotional recognition will play a large role in teaching robots to interact with humans.

Business Models & Use Cases

Customers/End-Users Customers may allow access to their emotions only if they are paid for impressions. If not, they may wear dark glasses or disguise their voices to avoid emotion detection.

Employees Employee satisfaction, the suitability of job candidates, employee training and monitoring job performance will soon be informed by layers of emotional data.

Partners/Ecosystem Marketers should be ready to pay extra money to Facebook and Google. They will have the ability to sort emotional responses based on interest level, providing real-time feedback without the need to survey.

Organizational Structure and Leadership

An even more symbiotic relationship with IT and business functions will be necessary for implementation, as emotional recognition relies on even greater amounts––and types––of data. How will each BU make sense of the data that matters? End-user employees will also be required to help design the programs, for example call center agents who will escalate or de-escalate cases based on semantics and tone of voice. These end users must also be trained to use the new systems.

Customer Service Use Case: A major bank’s customer service call center analyzes the tempo, tone, and grammar of callers to ascertain how angry or dissatisfied they truly are.

Senior leadership will be critical to solicit the involvement of employees on the front lines to help develop emotional recognition programs. They must buy in to programs, and be part of the build.

Change Management

HR: will work to determine the ethics of capturing employees’ emotional information.

Hiring use case: 12 Grapes has candidates undergo a questionnaire video screening that evaluates, facial and emotional cues.

Enterprise monitoring policy: While using a corporate phone, the device may analyze you. The same holds true for video cameras in buildings and retail outlets. Employee behavior may be monitored for quality assurance.

Sales teams: Sales staff will be able to identify enthusiastic customers on the showroom or tradeshow floor. Eventually, emotion recognition technology could combine with AR and smart contact lenses at the point of sale. When hearts start racing in showrooms, staff will know it. Sales teams would soon be at a disadvantage in the market not using these technologies.

Hiring use case: The Bank of New Zealand used Emotionscan’s technology to analyze consumer feelings about money. Tailored messaging then brought the bank to second from fifth place in the market 

Data Lifecycle

Emotional recognition data will impact product development. Teams will understand how users react to product changes and innovation, as well as competitive products.  User data can be aggregated from the crowd rather than the cloud.

More symbiosis will be required between data warehousing and the departments implementing emotional data.

Process, Governance, Compliance

The General Data Protection Regulation (GDPR) will very soon have to take into account the role of biometrics and emotion recognition.  Moral, legal and ethical questions will also soon arise. For example, what is a brand’s obligation if it gains unintended insights, e.g. that a user is on the verge of a heart attack, stress, panic attack, is taking drugs, or lying?


Emotional recognition technologies will soon establish new sets of metrics and goals across the organization. A future CMO may soon review a report that says net retina sentiment is up for their brand, down for a competitor’s.  Corporate culture and HR will be informed by employee satisfaction (voice stress is down 20% this quarter).  Data will also reveal if training scenarios are successful.

Challenges & Risk Mitigation

  • Customers will soon learn that a squeaky wheel gets attention. At the same time, apathetic customers run a risk of being ignored.
  • Customer privacy is an issue, as is protection of personally identifiable emotional data, especially as we cross country borders
  • Similar cultural and environmental variations can impact the accuracy of “reading” emotions.

Emotional recognition is just one of myriad technologies creating new uses cases for business and consumer adoption and value. And, these are just a few of the many impacts on organizations today. Kaleido Insights’ analysts are tracking these and other technologies closely to help you find clarity amidst the chaos. Please download our latest research.

Leave a Reply

Your email address will not be published.