Protocol 4: Validation of Network Adequacy

Protocol 4:Validation of Network Adequacy

Activities 3 and 4

Start

Protocol 4

Activity 3: Review Information Systems Underlying Network Adequacy Monitoring

Activity 3: Review Information Systems Underlying Network Adequacy Monitoring

• For Activity 3, the EQRO will determine if the MCP's information systems are able to collect and report accurate data related to each network adequacy indicator.

Protocol 4: Activity 3

Activity 3: Review Information Systems Underlying Network Adequacy Monitoring

Activity 3: Review Information Systems Underlying Network Adequacy Monitoring

Step 2: Assess Processes for Collecting Network Adequacy Data not Addressed in the ISCA

Step 1: Review the MCP's ISCA

Step 3: Interview MCP or Other Personnel

Step 1: Review the MCP's ISCA

Protocol 4: Activity 3

Step 1: Review the MCP's ISCA

• The EQRO determines if the MCP has completed an ISCA review within the previous two years.
• If the MCP has completed an ISCA in the past two years, the EQRO should review the findings.
• If the MCP has not completed an ISCA in the past two years, the EQRO must conduct one.

Protocol 4: Activity 3

Step 1: Review the MCP's ISCA

Step 1: Review the MCP's ISCA

After review of the ISCA, the EQRO should have an understanding of the following:

IT system architecture

Data processing procedures

Completeness of data

File structure

Accuracy of data

Information flow

08

Step 1: Review the MCP's ISCA

Protocol 4: Activity 3

Step 1: Review the MCP's ISCA

After review of the ISCA, the EQRO should also have an understanding of the following:

Processes for maintaining and updating provider data

Processes to identify providers or organizations excluded

Processes to update provider data each month

Step 1: Review the MCP's ISCA

Protocol 4: Activity 3

Step 1: Review the MCP's ISCA

The ISCA should be reviewed to determine if there are any weaknesses in the MCP's information systems required for network adequacy validation, including:

If the information system meets standards for complete and accurate data capture

If the system meets standards for data processing, capture and reporting

Where and how information systems may be vulnerable

Step 2: Assess Processes for Collecting Network Adequacy Data Not Addressed in the ISCA

Protocol 4: Activity 3

Step 2: Assess Processes for Collecting Network Adequacy Data Not Addressed in the ISCA

• The EQRO should identify any data sources included in the network adequacy validation that were not reviewed in the ISCA.
• This may include primary data the EQRO will collect for the network adequacy validation and data from non-MCP entities (such as the state or third-party vendors), as well as any data sources from the MCP not covered in the ISCA.

Protocol 4

Step 2: Factors to Consider for Each Data Source

Step 2: Factors to Consider for Each Data Source

• What system is used to collect and store this data?
• How are data entered (manually or electronically)?
• Which staff are involved in collecting, storing, and analyzing this data, and what is their level of training?
• Are there adequate staffing resources to collect and analyze data?
• How frequently are the data collected and updated?
• What errors may occur in the process of collecting, storing, and analyzing the data?
• What quality assurance systems are in place to prevent and fix these errors?
• To what extent are data missing?

Step 3: Interview MCP or Other Personnel

Protocol 4: Activity 3

Step 3: Interview MCP or Other Personnel

• After assessing the data sources needed for the network adequacy validation, the EQRO should conduct follow-up interviews with MCP personnel to supplement its understanding of the MCP’s information systems and processes.
• The EQRO should follow up with personnel in other agencies or organizations with any questions about information systems and processes used in the network adequacy validation that are not maintained by the MCP.

Protocol 4: Activity 4

Activity 4: Validate Network Adequacy Assessment Data, Methods, and Results

Activity 4: Validate Network Adequacy Assessment Data, Methods, and Results

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

Step 2: Assess the Methods Used by the MCP to Assess Network Adequacy

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

Step 3: Validate Network Adequacy Results Submitted by the MCP

Step 4: Summarize Network Adequacy Validation Findings

Validate Network Adequacy Assessment Data, Methods, and Results

Protocol 4: Activity 4

Activity 4: Validate Network Adequacy Assessment Data, Methods, and Results

In Activity 4, the EQRO evaluates each MCP's ability to:

• Collect reliable and valid network adequacy monitoring data.
• Use sound methods to assess the adequacy of its managed care networks.
• Produce accurate results to support MCP and state network adequacy monitoring efforts.

Protocol 4: Activity 4

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

• The EQRO will determine if the data used by MCPs to monitor network adequacy are accurate, complete, and timely.
• Ensuring the data meets these standards will help achieve meaningful and accurate results.

Protocol 4: Activity 2

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

Step 1: Assess the Reliability and Validity of MCP Network Adequacy Data

To determine the reliability and validity of data the EQRO should:

Determine if the required variables identified in Activity 2 are included in each data source

Review the MCP's process for updating data

Assess changes in the MCP’s data systems that might affect the accuracy or completeness of data

Protocol 4: Activity 4

Step 1: Questions to Ask to Determine Data Collection Procedures

Step 1: Questions to Ask to Determine Data Collection Procedures

Question 5

Question 1

Question 6

Question 2

Question 3

Question 7

Question 4

Question 8

Protocol 4: Activity 4

Step 1: Other Considerations for Assessing Network Adequacy Data

Step 1: Other Considerations for Assessing Network Adequacy Data

If significant changes are found, the EQRO should consult with the MCP to determine if external factors (such as changes in populations covered or demographic changes in the overall state population) may be the cause.

The EQRO should compare the MCP’s data from previous years against the most recent data to assess reliability, as major changes over time could reflect data quality issues.

If an MCP uses performance measures or encounter data in its network adequacy monitoring activities, the EQRO can use the results of Protocol 2 and/or Protocol 5 for assessment of data reliability and validity.

Step 2: Assess the Methods Used by the MCP to Assess Network Adequacy

Protocol 4: Activity 4

Step 2: Assess the Methods Used by the MCP to Assess Network Adequacy

• In this step, the EQRO should review the methods used by the MCP to calculate each network adequqcy indicator.

Step 2: Assess the Methods Used by the MCP to Assess Network Adequacy

Protocol 4: Activity 4

Step 2: Questions to Consider to Assess the MCP's Network Adequacy

Question 5

Question 1

Question 6

Question 2

Question 3

Question 7

Question 4

Question 8

Step 2: Considerations for Time and Distance Analyses

Protocol 4: Activity 4

Step 2: Considerations for Time and Distance Analyses

How is distance measured? Is it "as the crow flies" or using road distances?

How is time measured? Is it measured during low traffic or high traffic time periods?

Is time measured using driving distance, or public transit times?

Step 3: Validate Network Adequacy Results Submitted by the MCP

Protocol 4: Activity 4

Step 3: Validate Network Adequacy Results Submitted by the MCP

• The EQRO validates the calculations and results generated by the MCP for each network adequacy indicator.
• The EQRO assesses if the results are valid, accurate, and reliable, and if the MCP’s interpretation of data was accurate.
• The EQRO may use the data sources provided by the MCP to reproduce the MCP’s calculations.
• Additionally, the EQRO may also conduct studies, such as provider directory validation studies and appointment availability studies, to validate the MCP’s results.

Step 4: Summarize Network Adequacy Validation Findings

Protocol 4: Activity 4

Step 4: Summarize Network Adequacy Validation Findings

• Next, the EQRO will review the indicators validated in Activity 4, Steps 1-3 against the list of state network adequacy standards and indicators established in Activity 1.

Protocol 4: Activity 4

Step 4: Summarize Network Adequacy Validation Findings

Step 4: Summarize Network Adequacy Validation Findings

The EQRO will indicate whether the MCP:

Notes any indicators that could not be validated due to missing data

Addresses all state network adequacy indicators in monitoring activities

Provides the validation rating for each indicator

Knowledge Check

Knowledge Check

Knowledge Check

Knowledge Check

Congratulations!

Protocol 4: Activites 1 and 2

Congratulations!

You have completed Protocol 4: Activities 3 and 4

• Now you are ready to proceed to

Protocol 4: Activities 5 and 6

• If you have any questions or comments about this lesson please reach out to your Program Manager, Course Instructor or leave a message in the Chat Boards.

📌 Course Instructor: Michelle Hoover

📌 email: mhoover@qsource.org