Rocket Science and Human Factors: 2 Key Insights That Helped Us Succeed in Our Use of Predictive Analytics to Drive Better Claim Outcomes

By Dr. Gary Anderberg PhD, SVP Claim Analytics, Gallagher Bassett Services, Inc. And Sandip Chatterjee, VP Product Development, Gallagher Bassett Services, Inc.

Dr. Gary Anderberg PhD, SVP Claim Analytics, Gallagher Bassett Services, Inc.

Predictive Analytics (PA) is not new to the insurance industry. Actuaries and underwriters have been using it for years to model the cost of risk and make premium decisions. The advent of big data, easy access to multiple data sources, cheap computing power, and ready access to multiple advanced modeling algorithms are bringing new predictive applications to other areas of the industry. Our topic is applying these new techniques to claims management.

"Our goal was to put a radically improved, more accurate analytic process behind the simple reserve advice our old system had been providing"

The following five step approach, starting with the question of “What is the business problem we are trying to solve and Why?” works well for any predictive analytics initiative that you may be thinking of pursuing in your organization.

We will touch on two keys to success in any predictive analytics initiative focused on the claims management process. They are:

1. Feature Engineering part of data pre-processing (Step 2- Data & Prep)
2. Designing interfaces to share model results that meet the needs of specific user groups (Step 3- Workflow Integration)

Feature engineering is the process of transforming and extracting (engineering) features from the data/variables to drive higher accuracy and precision in the model outputs. It may be one of the best kept secrets in the world of PA. The data transformation aspect of Feature Engineering shows up in other areas of data analysis. A common application example is the use of log transformation to adjust for skewness.

Sandip Chatterjee, VP Product Development, Gallagher Bassett Services, Inc.

The process of feature extraction, also commonly known as “variable creation”, involves creating new variables/features from existing data. An example from the world of Workers Comp (WC) claims management is the concept of “Lag” (Lag = Accident Report Date–Accident Occurrence Date). Anyone who has handled a WC claim knows the importance of understanding the drivers behind the lag and how it may impact the overall outcome of the claim.

Feature engineering cannot be done in a vacuum and requires collaboration between domain experts and the PA team. This collaboration and the partnership play an important role in providing credibility to the model output and driving adoption during a pilot or broader rollout.

Software to Wetware

In our quest for improved model accuracy and precision using Feature Engineering, we cannot forget that while computers do some things better than the operating system we call Homo sapiens, V 1.0; one of the tasks they don’t do all that well, left to themselves, is to communicate effectively with humans. Building the right user interfaces, the human factors engineering, is a daunting task. Some of the questions that need answering include:

1. What are the model outputs? Are they easy to understand/interpret?
2. How are the model outputs displayed to the user? Is it going to contradict anything that is already in place or add to alert fatigue?
3. How do we drive the desired change in behavior?

While communication, training, adoption and measurement are important across the board, we need to take into consideration the level of disruption a new user interface is going to cause. Usually it is a function of the users’ ability to understand/interpret the model results and take the right next steps to drive better business outcomes. In our case, we decided to reverse the old saying about old wine in new bottles to minimize operational disruptions when we rolled out our Decision Support Solution called “Waypoint.”

New Data in Old Bottles

In our company we have more than 3,000+ resolution managers (adjusters/claim examiners), thus any major changes in system output raises training challenges. Our goal was to put a radically improved, more accurate analytic process behind the simple reserve advice our old system had been providing for years. We also wanted to simplify a potentially problematic learning curve.

We decided to, in effect, hollow out the screens our people had used in the old reserving system, and populate the same note formats with much better reserve advice. Structural change was kept to a minimum. There is always a strong temptation to redesign old screens in accordance with the newest ideas in information display. The screens which now display the model outputs may look sub-optimal from a purely technical perspective but they still get the job done and simplify the transition for end users. 

Resolution Managers know just where to find the information they need at a glance. Transition issues were minimal with little resistance to use of the new state-of-the-art tool.

Meanwhile, on the Bridge of the Enterprise

But we do have a small group of users for the same system outputs with very different needs. Our reserve analytics unit is staffed by highly qualified specialists who do deep dives into reserving questions on behalf of our clients. They do need to understand the details. For them we built a completely new set of dashboards using Tableau:

Other dashboards use related metrics to track reserve accuracy by class of claim and myriad other variables to monitor the effectiveness of Waypoint in daily use. Each output is thus designed around the unique needs of a specific user subset.

A Few Conclusions

Developing a completely new predictive application to replace an old generation rules engine is an experience that most carriers and claims management companies are going through. When we set out to build Waypoint as a broad based decision support solution for an array of vital claims functions, starting with reserving, we made a handful of key design decisions. We identified the functions that we wished to enhance, such as materially improving reserving accuracy; we agreed with Claims Operations to combine highly targeted functional improvements with minimal process disruptions and new training requirements; and we combined an “all in” approach to scooping up all available claim information from all useable sources while relying on feature engineering to give us serious additional lift. We also settled on an incremental, one-release-at-a-time rollout accompanied by an intensive testing and QA process spread over two years. Waypoint is still young with many more releases to come, but the early introductions are working well and, most of all, functioning harmoniously with our resolution managers. New data in old bottles looks like a winning concept.