A. Goals of Physician Profiling
The primary goal of profiling is to improve the quality of
health care by providing physicians with meaningful information on
their clinical performances. Hence, the success of profiling should
be measured by evidence of improvement over time in the structures,
processes, and outcomes of care.
Other important goals are to ensure physician accountability to
the needs of health care consumers and accrediting and regulatory
entities.
Physician leadership is essential in developing and implementing
clinical profiling activities to ensure their clinical relevance
and to help inform patients and the community about aspects of
health care that are particularly important to physicians.
Profiling must address local, as well as regional and national,
priorities if local needs are to be satisfied and active physician
participation is to be assured.
B. Development of Physician Profiling
Development of effective physician profiling requires
close collaboration among physicians, their health care
organizations, payers, and regulatory agencies.
Expected benefits of profiling should be weighed against the
burden and costs for the program as a whole, and for each
performance measure. The value of profiling will be increased by
the use of standardized measures and methods, avoidance of
duplication of effort, and steps to ensure the accuracy and
usefulness of results.
Ongoing profiling activities should receive regular external
evaluations. These evaluations should focus on the choice of
performance measures, data collection and analysis strategies, the
accuracy of the results obtained, and the appropriateness of
interpretation of results.
Organizations that perform physician profiling (provider
organizations and vendors) should disclose fully their profiling
objectives, policies, and methods, and make these readily
accessible to both the physicians being profiled and the
public.
The burden and costs of physician profiling should be fairly
allocated among those who will potentially benefit including
physicians, patients, health plans, payers, employers, and
regulatory agencies.
C. Distribution and Use of Profiling
Reports
Physicians and physician groups being profiled should be the first
to receive all reports that measure their performance. They should
be given an opportunity to review and comment on reports prior to
external release. In particular, "outliers" on a measure should be
contacted to detect any unusual circumstances that explain the
result. Documented errors should be corrected, and substantive
comments or explanations should be appended.
External distribution of physician profile results should be
governed by the "need to know" as defined by the responsibilities
of the entity and the content of the report. Criteria for external
distribution, including rules governing confidentiality of content,
should be explicitly stated and agreed to by all involved parties.
For example, the public should receive reports that are relevant to
facilitating the choice of physician, health plan, or hospital.
Regulatory agencies should only receive information specified in
their credentialing standards.
Organizations that use physician profile reports should publicly
disclose the types of information they need and how this
information will be used to improve the quality of health care.
Reports intended for public release should meet higher standards
of accuracy, reliability, and statistical validity than those
intended for internal use only. Reports should not be released when
there are too few cases to support a meaningful analysis.
Appropriate risk adjustment of results is essential. Reports
intended for public release should be audited by an independent
entity prior to their release.
All reports, whether for internal or external use, should be
clear and unambiguous and accompanied by materials that facilitate
proper interpretation.
Profiling reports used for internal quality improvement should
remain confidential between the physician or physician group being
profiled and their immediate supervisors. Such reports should be
protected from disclosure by peer review regulations, whenever
possible.
Profiles keyed to sentinel events should be used only for
internal quality improvement unless statistically valid patterns of
performance can be documented.
Patient-specific data may, where necessary, be released to the
patient's physician for use in internal quality improvement
activities. Broader release of patient-specific data, however,
should require explicit permission of the patient.
Selection of Physician Performance
Measures
A. General Principles
Performance measures should be clinically relevant to the
individual physician or group practice being evaluated. Markers of
importance include high prevalence; significant impacts on
mortality, morbidity, or costs; and high degrees of practice
variation where variations have well-documented relationships to
health outcomes.
Physicians should be evaluated only with respect to patients and
clinical services for whom/which they are directly responsible.
Where responsibility for care is shared, the team, group practice,
or hospital service should be the unit of evaluation. When
attribution is uncertain, evaluation should be at the higher level
of aggregation.
Performance measures should, to the maximum extent possible, be
firmly grounded in scientific evidence. Where the science base is
inadequate, professional consensus may be substituted. In either
case, sources of support for the measure and their validity should
be fully documented and readily accessible.
The process for selecting the range of performance measures to
be included should take into account the perspectives of all
involved parties including physicians, patients, health plans,
provider organizations, employers, payers, and regulatory
agencies.
B. Characteristics of Individual Performance
Measures
Measures should be based on data available to the
clinician in the real-time clinical setting and should have clear
implications for actions to improve the quality of care.
Measures should be standardized and capable of systematic and
objective measurement. Relevant data sources must be available,
accurate, and reasonably complete.
To the maximal extent possible, measures should rely on data
that are routinely collected during usual patient care.
The burden of data collection for a measure should be
tolerable.
Measures should be updated at regular intervals to reflect
changes in medical knowledge or the norms of practice.
Measures of clinical outcomes should be risk-adjusted so that
results appropriately reflect patients' severity of illness at the
time of presentation or time of clinical action. Methods used for
risk-adjustment should be accurate at all levels of severity of the
illness.
Measures and associated analytic methods should be clearly
defined and fully disclosed to those who "need to know." Measures
based on non-disclosed "black box" algorithms or software are not
acceptable.
C. Types of Performance Measures
Clinical outcome measures should be clearly related to processes of
care that are under the control of the physician or group practice,
and can be modified to affect the outcome.
Process measures should be clearly linked by scientific evidence
to having direct effects on important patient outcomes. They
usually relate to diagnostic and treatment decisions but may also
examine access to care or compliance with care regimens.
Patient perceptions of and satisfaction with the quality of
services are important. Patients should have input into the
selection of these measures.
Patients are often the best witnesses to assess the outcomes
that they experience.
Resource use and cost measures should be supported by evidence
that patient care will not be adversely affected. Decisions on the
use of such measures should include individuals with no direct
financial stake in the care being evaluated.
The primary purpose of profiling on resource use and costs
should be used to raise awareness and inform quality improvement
activities. Results of these profiles should not be used for
punitive purposes except in cases of flagrant overuse or clear
waste.
"Economic credentialing" that measures a physician's costs of
care compared to a budget should be discouraged. Health plans and
insurers should be held accountable for adverse outcomes linked to
such credentialing.
Principles for
Implementation
A. Data Sources
Each data source needs to meet explicit standards of
accuracy and completeness if valid comparisons are to be made among
physicians or practices. The bar should be "raised high."
The data source should be appropriate to the performance measure
being examined.
The data source should be readily available in all practices or
health plans being compared.
B. Data Collection
Data collection protocols should be explicit, as objective as
possible, and limited to essential items of data.
Data collection from medical records or by survey should be
performed by persons skilled in the methodology. Ideally, these
individuals should be selected and reimbursed in a manner that will
optimize objectivity and minimize bias.
C. Data Analysis
The level of analysis (individual physician, group practice, or
health plan) should be appropriate to the ability of data to
support meaningful analyses and the intended use of the report.
Sample sizes of events or cases that are too small to support
analyses at the level of the individual physician may be useful for
internal quality improvement but should not be released to the
public.
Analyses should be planned and conducted by individuals who are
skilled in appropriate analytic techniques.
Analytic techniques should be appropriate to the objectives of
the analysis and the database.
Reports should emphasize important differences between the
entities being compared or time trends in performance, and include
clear statements about the statistical significance and clinical
importance of results.
Reports that are to be released to the public should be based on
adequate sample sizes and accurate data, and meet high standards of
statistical validity. Independent external audits should be
performed prior to release.
Reports that are for internal use in quality improvement
activities can be based on smaller sample sizes and may not require
formal statistical analysis.
Methods of analyses should be described in sufficient detail
that results can be easily understood and, if necessary,
reproduced.
D. Risk-Adjustment
Adequate risk-adjustment is essential to achieving valid
comparisons among physicians, practices, or health plans on
clinical outcomes and the appropriateness of decisions to perform
surgical or diagnostic procedures.
Simple adjustment for selected patient characteristics such as
age, gender, and risk factors for the disease will be sufficient
for certain process measures (e.g., mammographic screening for
breast cancer).
Risk-adjustment models should be carefully tested before they
are used and should have demonstrated good calibration between
predicted and actual outcomes at all levels of severity of illness.
Generic risk-adjustment models can be used if they have been
demonstrated to be valid for the particular condition and the
particular type of clinical setting.
The risk-adjustment methodology should be well-documented and
open to inspection, preferably published in the peer-reviewed
medical literature. "Black box" systems are not acceptable.
E. Frequency of Profiling Reports
The frequency of reports depends on the intended purpose.
If the goal is to achieve behavior change and quality improvement,
frequent reinforcement by quarterly reports may be required. Annual
reports are usually sufficient for comparisons among health plans
or to satisfy accrediting agencies.
The burden of data collection and other costs of profiling will
be limiting factors both for the selection of performance measures
and the frequency of reports.
-- Adopted November 1998
