Protocol 5 Validation of Encounter Data Activity 3

Start

Protocol 5:Validation of Encounter Data

Activity 3

Protocol 5: Validation of Encounter Data

In the first lesson, we discussed the tasks associated with Activities 1 and 2.
If you recall, for Activity 1, the EQRO reviews state requirements for encounter data.
In Activity 2, the EQRO reviews the MCP's ability to collect encounter data by reviewing the MCP's most recent ISCA.

Protocol 5: Validation of Encounter Data

To complete Protocol 5, the EQRO must perform five activities for each MCP:

Activity 1: Review state requirements

Activity 2: Review the MCP's capability

Activity 3: Analyze electronic encounter data

Activity 4: Review medical records

Activity 5: Submit findings

Protocol 5: Validation of Encounter Data

Activity 3: Analyze Electronic Encounter Data

Activity 3 is the core function used to determine the validity of the encounter data.
When the EQRO has completed the steps within this activity, it should know whether the data are complete, of high quality, and can be used for analysis of quality, access, program integrity monitoring, among other critical state activities.

Protocol 5: Validation of Encounter Data

Activity 3: Analyze Electronic Encounter Data

If the EQRO cannot confirm the quality of the data after completing this activity, it should not proceed to Activity 4, the Medical Record Review.
Instead, the EQRO should work closely with the state or plans to determine underlying problems or acquire additional information to determine the quality and usefulness of the data submitted.
Difficulties completing this analysis may need to be summarized for the state to document serious data quality issues.

Protocol 5: Validation of Encounter Data

Activity 3: Analyze Electronic Encounter Data

The EQRO uses the information obtained from these analyses, the ISCA tool, follow-up interviews, and the results of state edits to assess the completeness and accuracy of the MCP’s encounter data.
The results of this activity will help develop a long-term strategy for assessing the quality of the encounter data.
The EQRO will be able to design targeted validation strategies to identify problem areas requiring resource intensive medical record review.

Protocol 5: Validation of Encounter Data

Activity 3: Analyze Electronic Encounter Data

For this activity, the EQRO should perform the following steps:

Step1

Step 2

Step3

Step4

Macro-analysis of data integrity

Micro-analysis of analytic reports

compare findings to benchmarks

develop a data quality test

Protocol 5: Validation of Encounter Data

Step 1: Develop a Data Quality Test Plan Based on Data Element Validity Requirements

The EQRO uses the information collected from Activities 1 and 2 to develop a data quality test plan. The plan should:

Account for front-end edits built into the MCP’s data system so that it focuses on issues that the MCP may have missed or allowed for other reasons

Specify the areas to be tested and the expected results

Protocol 5: Validation of Encounter Data

Step 1: Develop a Data Quality Test Plan Based on Data Element Validity Requirements

What is a "Front-End Edit?"

Encounter data are only useful if they are complete and accurate. To ensure quality, encounter data needs to be analyzed for quality.
Typically, this involves the use of "front-end edits" to identify errors. A front-end edit flags low quality data.
Edits that set thresholds too high or low may create high rejection rates and confusion about why the system is rejecting records.

Protocol 5: Validation of Encounter Data

Step 1: Develop a Data Quality Test Plan Based on Data Element Validity Requirements

What is a "Front-End Edit?"

These are common errors that may be detected in a front-end edit, and result in rejected encounter claims:

Duplicate claims: State systems usually have a front-end edit to flag encounters that appear to be duplicates

Incorrect data:Beneficiary information does not match, or is missing in state's enrollment files

Missing NPIs: The provider's National Provider Identifier (NPI) is not found in the state's "roster" list

Other coverage: The beneficiary has third-party insurance that must be billed first

Protocol 5: Validation of Encounter Data

Step 1: Develop a Data Quality Test Plan Based on Data Element Validity Requirements

Developing a Data Plan: Missing Encounter Data

Overall data quality issues

The EQRO should develop a data plan that addresses the extent of missing encounter data.

Problem: The EQRO needs to determine the type and amount of encounter data missing

Investigation:The EQRO should investigate front-end edits and determine how much data is missing

Comparison:The EQRO should compare front-end edits with typical data on encounters for similar populations

Protocol 5: Validation of Encounter Data

Step 1: Develop a Data Quality Test Plan Based on Data Element Validity Requirements

Developing a Data Plan: Missing Encounter Data

Overall data quality issues

The EQRO should address the following questions when developing a data plan:

Question: What types of encounters may be missing?

Question:What are the data quality issues?

Question:Were there any issues during data submission?

MCPs that pay for “bundled” services (e.g., prenatal care) or that capitate providers (e.g., for primary care) may not receive complete encounter information from these providers.
The EQRO should apply specific knowledge about the MCP’s contractual relationships with providers to identify specific areas to look for missing services.
The EQRO should obtain information from the MCP on the use of bundled payment and capitation to inform its plan

The EQRO should identify specific data quality problems such as inability to process or retain certain fields, or limited coding specificity on the encounter data record

The EQRO should identify problems the MCP has compiling its encounter data and submitting the data files to the state

Protocol 5: Validation of Encounter Data

Step 2: Encounter Data Macro-Analysis

Step 2: Encounter Data: Macro-Analysis

Steps 2 and 3 of Activity 3 are closely related.
When the EQRO reviews the data for accuracy and completeness, it conducts both macro- and micro-analyses.
Step 2 describes the macro-analysis, while Step 3 describes the micro-analysis.

Protocol 5: Validation of Encounter Data

Step 2: Encounter Data: Macro-Analysis

For the macro-analysis, the EQRO should:

Analyze and interpret data in specific fields

Check the data for volume and consistency

Step 2: Encounter Data: Macro-Analysis

Protocol 5: Validation of Encounter Data

Step 2: Encounter Data: Macro-Analysis

Without duplicating the state's edit checks, the EQRO should analyze data in submitted fields. These are some questions an EQRO should consider when interpreting data.

Question #1

Question #2

Question #3

Question #4

Question #5

Question #6

1. Is there information in the field, and is that information of the type requested?

The EQRO should check each field to determine if its values are of the type and size found in the state’s data dictionary, or in nationally recognized standards.
For example, if CPT-4 codes are requested, the field should have five digits. If the state’s Medicaid and CHIP beneficiary ID is requested, the field should contain the correct number of letters and characters.

2. Are the values valid and in the correct format?

To what extent are the values in the field valid? For instance, if ICD-10 diagnosis codes have been requested, are the values in the diagnosis field valid for that standard?
Do critical fields contain non-missing values in the correct format and specificity (e.g., maximum number of characters in a diagnosis) and that values are consistent across fields?

3. Are the data available?

Are all required data elements reported?
Do the data exist for all service types?
When viewed by date of service, are there gaps in the data?

4. Do the data meet basic consistency expectations?

Is beneficiary enrollment consistent over time?
Are the number of encounters consistent over time?
When broken out by population subgroups or service types, does consistency persist?

5. Are the state’s identifiers (IDs) accurately incorporated into the MCP’s information system?

The EQRO should compare the encounter data file to the state’s eligibility file and check for accuracy of the IDs and other eligibility information (e.g., age, sex, and eligibility category)
In addition, the EQRO should determine whether there are encounter data for the expected proportion of beneficiaries in comparison to utilization norms for similar populations

Is the information for each critical field within required ranges and is the volume of data consistent with the MCP’s enrollment? For example, can the following types of questions be answered with the data:

What is the rate of emergency department utilization per 1,000 member months?
What percentage of beneficiaries have at least one encounter during the year?

Protocol 5: Validation of Encounter Data

Step 3: Encounter Data: Micro-Analysis

Examples of analytic reports that can detect broader data quality issues are:

Reasonability tests

Analyses by DOS vs. adjudication dates

Checks by provider types

Reasonability tests: The EQRO should develop frequency distributions of the values and compare them to normative data from similar populations to determine whether the values make sense. For example,

If “place-of-service” is a required field, the values should be distributed across a range of values (e.g., IP hospital, OP hospital, ED, or office).
The number of enrollees, the number of encounters, and counts and totals for various eligibility categories or demographic subgroups, diagnoses, or types of service.
Frequency distributions on specific fields, as well as on the variables created explicitly for data validation purposes (e.g., beneficiary age from date of birth).
Distributions on subsets of data, especially where there are specific concerns about data validity. For example, if the EQRO finds a low rate of utilization for outpatient services, it could analyze the data by provider zip code to determine whether the files are missing specific zip codes, causing the system to reject records. By taking a deeper dive into the data, the EQRO could detect a different problem than originally expected.
Univariate statistics (e.g., means, medians, quartiles, and modes) as appropriate. The EQRO should check the output of these reports for reasonableness and to detect specific problems such as entire categories of data missing from the regular data submissions.

Analyses by dates of service versus adjudication dates:

Analyzing the data by dates of service and by adjudication dates can detect issues in consistency over time. Inconsistent processing can indicate other problems within the MCP’s IS system, which may impact data validity.
After establishing the length of time between service and adjudication dates, the EQRO should compare them with standards or benchmarks for data submission and processing.

Checks by provider types:

The EQRO should review the data by provider type to identify missing provider types and examine fluctuations in patient visits by provider type for specified time periods.
The EQRO should compare the distribution of encounters by provider type to normative information. The EQRO should also examine diagnosis or procedure codes by provider type to ensure that the relationship between provider specialty and the services rendered is consistent

Protocol 5: Validation of Encounter Data

Step 3: Encounter Data: Micro-Analysis

Examples of analytic reports that can detect broader data quality issues are:

Relational analyses by service type or episodes of care

Analyses by demographic group or subpopulation

Analytic questions

Relational analyses by service type or episodes of care:

The relationship between ancillary services (e.g., labs, x-rays, etc.) and visits
The relationship between outpatient visits and the number of prescriptions dispensed
The relationship between primary and specialty care visits
Outpatient services associated with inpatient admissions
Grouped services expected in particular types of visit or episodes of care
Other relationships between service types previously identified as problematic through the ISCA, front-end edits, or other EQRO validation activities.

Analyses broken out by demographic group or subpopulation :

If not addressed already in the MCP’s front-end edits, the EQRO should conduct analyses that take a beneficiary’s age and gender into account.
For example, the EQRO could verify that a gender-specific diagnosis (e.g., endometriosis) or procedure (e.g., caesarean delivery) is consistent with the beneficiary’s age and gender derived from the encounter header record or the beneficiary’s enrollment record.

Analytic questions:

The EQRO should use information gathered in previous steps to select a question or series of questions it might answer using the encounter data as another step in determining quality and usability.
For instance, the EQRO could take a particular measure of interest to the state and replicate it across all MCPs or within MCPs, such as the number of beneficiaries per primary care provider.

Protocol 5: Validation of Encounter Data

Step 4: Compare Findings to State-Identified Benchmarks

Benchmarks set standards for expected utilization volume, and may vary depending on the populations enrolled in MCPs.
In constructing reports and benchmarks, it is important to ask the users of the data what they need to know, with what degree of confidence, and whether it varies by plan, beneficiary group, or type of service.

Protocol 5: Validation of Encounter Data

Step 4: Compare Findings to State-Identified Benchmarks

Benchmarks may be obtained from these sources:

State encounter data

Historical FFS data

Plans' financial reports

HEDIS benchmarks

National data sources

Aggregating data from all state Medicaid and CHIP MCPs for certain services or utilization standards may help in setting benchmarks for plans as long as the populations enrolled have similar health status.

Data on trends over time in the use of certain types of service, whether through FFS or managed care, can be useful benchmarks.
Comparisons may be made month-to-month or quarter to-quarter using encounter data alone to judge data validity.
Errors or omissions can become immediately apparent through the front-end edits or via back-end reports that compare current data with historical trends.

National Medicaid data can be used to establish benchmarks for judging whether service use reported in encounter data meets expected levels.
Although comparisons to national sources are not useful for plan-specific benchmarking, they can provide a state with standards for service use and quality measures for the program overall.

In most states, plans are required to submit financial reports to the state department of insurance (or its equivalent) in order to monitor plan reserves and solvency.
Comparing the volume of data and service information in the encounter data to these financial reports can help reveal gaps in encounter data reporting.
This can only be done with precision if MCOs are required to report on encounter data the amounts they pay their providers.

Many managed care plans already report quality indicators into the HEDIS undergo audits as part of the NCQA certification process.
States can garner ideas for comparing plan encounter data with these and other sources, to avoid reinventing the wheel.

Protocol 5: Validation of Encounter Data

Step 4: Compare Findings to State-Identified Benchmarks

For example, this data table displays the average number of days from last date of service to the date of payment.
As you can see here, the average days from billing date to paid date for professional encounters was 80.3 days for this MCP. The EQRO compares the average to the baseline which is 106.5 days.
The EQRO has provided a reference to demonstrate that the MCP is performing above average in comparison to the benchmark.

Protocol 5: Validation of Encounter Data

Step 4: Compare Findings to State-Identified Benchmarks

Comparison data may require investigation. The EQRO may need to understand why there is a significant difference in the data.
For example, emergency department utilization may be lower in managed care than in FFS plans. If there is a large swing in utilization. or differences from the benchmark may indicate incomplete or erroneous data.
The differences may be attributed to an emergency, such as a natural disaster, that may account for the unusual changes in utilization.
The EQRO should determine if further analysis is needed, and discuss findings with the state to assess any underlying factors.

Protocol 5: Validation of Encounter Data

Knowledge Check

Protocol 5: Validation of Encounter Data

Knowledge Check

Protocol 5: Validation of Encounter Data

Knowledge Check

Protocol 5: Validation of Encounter Data

Knowledge Check

Protocol 5: Validation of Encounter Data

Congratulations!

You have completed Protocol 5: Activity 3

You are now ready to move onto the next lesson, Protocol 5, Activities 4 and 5. If you have any comments or questions about this lesson, please leave us a message in the chat boards!

EQRO Division Trainer: Michelle Hoover

mhoover@qsource.org

Protocol 5 Validation of Encounter Data Activity 3

More creations to inspire you

DAILY NEWS

RUNNING TECHNIQUES

ABCS OF THE OLYMPIC GAMES

THE MEANING BEHIND THE MOST FAMOUS SONGS IN HISTORY

MEET THE GAME OF THRONES CHARACTERS

SOCCER GAME SUMMARY

THE 7 WONDERS OF THE MODERN WORLD

Transcript