Section 5
Chapter 2 		   Measuring the
  Effectiveness of
  Case Management
As a case manager or an administrator of a case management program in your organization you may be called on to participate in the evaluation of the case management model, its effects on the organization, or its effects on patient outcomes. You may have direct responsibility for evaluating the effectiveness of the model, the outcomes of the case management plans in use, the success of the case manager's role, or the case manager's performance. Some of the evaluation criteria to be discussed in this chapter may require an organizational effort in terms of data collection and/or analysis. Some you may be able to facilitate as part of your own daily job activities.
      
Whenever possible, it is important to have an evaluation process and a plan set up before implementation of the model. This is particularly true for those outcomes that affect the organization. Categories such as length of stay (LOS), patient and staff satisfaction, clinical care outcomes, and costs of care should have baseline benchmarks against which the hospital can judge success or failure. Additional measurement points will be identified prospectively during the course of the hospital stay or course of treatment or intervention and then at the point of discharge or completion of services, depending on the care setting. Other indices, particularly those related to clinical or patient outcomes, will evolve over time.

OUTCOMES AS INDICES OF QUALITY

 Broadly speaking, outcomes can be grouped by those that have an effect on the organization versus those that have an effect on patients. These indicators may be the best measures of quality because they provide an understanding of the functioning of the organization and its effects on the product of services and patient care. Outcomes are the result of actions or processes. In patient care they are defined as the goals of the healthcare process. A good outcome is one that has achieved the desired goal.
      
Case management, through the use of tools such as multidisciplinary action plans (MAPs) and clinical pathways, and through case management team conferencing, allows healthcare providers to prospectively identify the expected outcomes or goals of care. There are no organizational or clinical processes or tasks that are carried out that do not have an expected outcome attached to them. Therefore outcomes in their narrowest sense allow us to understand the effects on an individual patient and in their broadest sense provide us with an understanding of the functioning (i.e., efficiency and effectiveness) of an organization or the healthcare system at large. These linkages provide us with an understanding of the structure (the organization), the process (the delivery system), and the expected outcomes (goals of care).
      
Expected outcomes of care are of different types. Some are related to the organization's performance but are not directly related to the patient's health. Others are related to the patient's health but are not directly related to the organization's performance. Still others are solely related to the clinical processes, meaning they are purely clinical in nature.

OUTCOMES NOT DIRECTLY RELATED TO THE PATIENT'S HEALTH

 Not all outcomes are related to the patient's health. Some are related to the organization's performance, such as those presented in Box 2-1.

Patient Satisfaction
 Understanding patients' satisfaction with the care and services we provide helps us to improve those services over time and to continuously improve patients' level of satisfaction with the care they receive. Patient satisfaction data are either collected toward the end of the hospital stay or soon after discharge. All hospitals and many other healthcare organizations collect and monitor data on patient satisfaction. Unfortunately there may not always be a mechanism for feeding that information back for the improvement of system processes or expected clinical outcomes.
      
Patient satisfaction data collection instruments must be carefully analyzed to determine whether the instrument is valid and reliable. Some instruments, if not tested and determined to be valid and reliable, can be used only to monitor trends and not for statistical analysis. Some organizations mail questionnaires to patients after discharge. Some administer them just before discharge while the patient is still on the institution's premises. The advantages and disadvantages of both methods are summarized in Table 2-1.
      
Whether organizations mail their customer satisfaction surveys to patients after discharge or distribute them on-site with discharge papers or through some other means, there exist some pros and cons with either method. Mailing questionnaires means that patients can respond at home, where they may feel safer to share their true feelings. Unfortunately the majority of responses will reflect those patients who had either a very positive or a very negative experience. Those who had a positive experience will want to share. Those who had a negative one will be lodging a complaint. This skewing of the responses may mean that the sample of people responding does not reflect the experiences of the average patient. To reduce the effect of such bias and concern, some organizations mail the customer satisfaction surveys to a randomized sample of patients.
      
On-site distribution can be accomplished in various ways. Some organizations include the survey as part of the discharge process. Some may include it on the patient's meal tray sometime before discharge. These methods also have pros and cons. On-site responses may not reflect the patient's candid and true feelings because the patient may fear retribution from the hospital staff if responding negatively. On the positive side, the patient's immediate feelings can be captured and addressed accordingly.
      
On-site collection is the more labor-intensive process. It requires the distribution and collection of the surveys, whereas the return process is built in when the questionnaires are mailed in. One method of collecting on-site responses that may reduce fear of retribution is to manually distribute the questionnaire to the patient at the bedside 24 to 48 hours before discharge. Along with the questionnaire, provide the patient with a pencil and an envelope. Give the patient a short period to complete the questionnaire and ask him or her to seal it in the envelope when finished. Tell the patient that sealing the envelope ensures there will be no association made between patients and their responses, the sealed envelopes will be retrieved at a later time, and responses will be anonymous. Come back and pick up the envelope later. This technique is effective in getting a more representative sampling of patients, and the responses are then timelier. Unfortunately, it is a labor-intensive process and may require a dedicated employee or volunteer. Not all organizations may have such resources at their disposal. In addition, make an effort to randomize the sample of patients chosen to be included in studying customer satisfaction. Randomization may be applied to both procedures that are followed in the distribution of the questionnaires, on-site and mailed.
      
Whenever possible, use a questionnaire that is scientifically valid and reliable. One such instrument is the Hinshaw and Atwood Patient Satisfaction Instrument (Hinshaw and Atwood, 1982). Validity indicates that the instrument is indeed measuring what it is supposed to be measuring; reliability is a measure of the instrument's consistency, meaning that the results are consistent from one data collection time to the next. It is advisable for an organization to select an instrument that is widely used so that benchmark data are available for comparison. An example of such instruments is the Press Ganey Patient Satisfaction Survey that is available in different versions that are specific to the patient care setting, including ambulatory, acute, and emergency department, or specific to the patient population, such as pediatric and obstetrics. Using the Press Ganey instrument allows an organization to benchmark its results to national and regional databases.

Staff Satisfaction
 Measuring and monitoring staff satisfaction carry many of the same issues in interpretation of the data as those presented by the patient satisfaction data. It is difficult to prove a direct correlation between changes in staff satisfaction scores and the use of the case management model. There are so many variables at play in an organization at any one time that only relationships may be concluded from any data. Care must be taken in measuring staff satisfaction by using instruments that are valid and reliable whenever possible. If the results are statistically significant, a stronger and more powerful argument can be made for the correlation between the implementation of the case management model and the changes in staff satisfaction scores.
      
Staff to be tested should include all those directly affected, such as registered nurses, physicians, social workers, and physical and occupational therapists. Staff should be questioned before implementation of the model and then consistently thereafter in appropriate intervals that reflect change over time. Generally a minimum of every 6 months or once annually is sufficient. Unfortunately, during these longer periods, staff members may leave their position and new staff members may be added. Matching samples over a long period will be difficult to maintain because of staff attrition and turnover.

Turnaround Time for Tests, Treatments, and Procedures
 Turnaround time (TAT) can be used as a measure of the organization's process improvement and efficiency after the introduction of case management. Facilitating care and managing the patient through the healthcare system should improve the TAT for completion of tests and procedures. The TAT should be measured from the time the physician places the order until the order is completed and results are recorded in the medical record. Acceptable timeframes should be decided in advance. For example, the completion time for computed tomography (CT) scans may be 24 hours from the time the order is written until the results of the CT scan are placed in the medical record.
      
Monitoring of these periods can be done concurrently or retrospectively through the medical record. Concurrent data collection is always preferred because it is both more accurate and timely. If any problems or delays are identified, they can be addressed immediately. Retrospective data collection, on the other hand, may be more difficult because of lapses in documentation in the patient's medical record or simply the inability to obtain the necessary information. In addition, when problems are finally identified, it is rather late to try to resolve them.
      
Finally, relationships should be shown between the reduction in TAT and the LOS. As before, it may be difficult to prove sound relationships between LOS reductions and TAT because many other factors may have an effect on the LOS.

Cost of Care
 Clinical cost accounting methods are being used more and more as a means of understanding not only hospital charges but also the true costs of care. This information can be used to negotiate realistic and appropriate managed care contracts because the hospital understands exactly what costs are associated with the care of a specific population of patients. Cost accounting can also be used as a way of measuring the financial impact of case management on the organization. Although understanding that reduction in LOS of a particular patient population is clearly important, it is also important to determine the amount of resources consumed in the management of that population. Organizations often focus on reducing the LOS but neglect coupling this act with an effort to improve the practice patterns related to tests, procedures, and treatments (pharmaceuticals and others) and eliminating the unnecessary activities. Sometimes they even distribute the same number of tests and procedures across the days left after reducing the LOS. This act keeps the cost of care the same even though the LOS is reduced. Reducing the LOS but consuming the same amount of resources is not as valuable and should be avoided. This will not have the same long-term benefits of shortening the LOS but also reducing the amount of resources used in the care of that case type of patient (Cohen, 1991).
      
The two main goals of clinical cost accounting are first to identify the organization's standard use of materials for a particular diagnosis-related group (DRG) and then to define the standard cost of each clinical service. An understanding of these costs allows the organization to assess its costs relative to the normal reimbursements, such as Medicare, Medicaid, and other payers. This information also provides a frame of reference or benchmark against which the organization can compare itself with competitors. This can be particularly useful during managed care contract negotiations when the hospital wants to make the most competitive bid possible (Schriefer et al, 1996). Using standardized case management plans allows an organization to calculate the expected cost of care for a particular case type and facilitates the comparison of the actual cost to the expected cost. Such efforts enhance the development of an action plan for improvement.
      
Internally, clinical cost accounting helps the hospital measure its internal treatment patterns. This information can be linked to the medical staff to determine which physicians are rendering the most cost-effective care. Allowing the physicians to compare their cost per case with the expected cost or that of their colleagues may provide them with information they can use to improve their practice and in the revision of case management plans (see Section 4, Chapter 3).
      In the fictitious example in Table 2-2, Physician C has the greatest LOS and the highest cost per case compared with other physicians caring for the same type of patients. This report reflects the intensity of Physician C's use of resources such as medications, antibiotics, radiology, blood products, and other related supplies. This information might be used by Physician C and his colleagues to develop standard protocols that address the use of resources and how it can be reduced or controlled.
      
Clinical cost accounting can be used by physician department heads or chiefs of services as part of staff education programs. The cost information can be used to help them gain a better understanding of the costs of clinical services and their contribution to those costs. Where differing treatment patterns exist, they can review the patient outcomes and relative costs.
      
On a managerial level, clinical cost accounting can be used as a component of departmental performance reports, providing administrative staff with financial information related to the efficiency of their departments. Medical staff reports such as that shown in Table 2-2 can provide clinically related information to guide physicians in changing their clinical practice patterns and can provide information to the finance department in terms of the cost versus volume versus profit to the organization.

Length of Stay
 LOS is a broad umbrella term that can be interpreted in various ways to indicate the amount of time allotted to the care, treatment, or recuperation of a patient. In the inpatient setting (e.g., acute, subacute, or skilled nursing facilities/nursing homes) it can be measured by the number of bed days or the number of days the patient remains in the hospital. In the home care setting it is calculated by the number of visits to the home and the number of hours or minutes per visit or the total number of hours. In the emergency department the LOS may be measured in hours or parts of an hour (15 minutes). LOS statistics are most commonly used in the hospital setting. They are often used as an indicator of the success of case management in conjunction with or in the absence of a cost accounting system.
      
To determine success or failure of case management and case management plans and their effect on LOS, hospitals must have a clear understanding of what their LOS goals are and compare those with the current LOS statistics in the organization. Comparisons can be made between the hospital and a variety of benchmarks. The first should be the Medicare and non-Medicare DRG average LOS. Although DRGs are not the primary reimbursement system in every state, they are still used for analytical purposes. It is important to understand the history of the organization so that realistic LOS reduction goals can be set. The hospital should also benchmark against comparable hospitals. These hospitals may or may not be close geographically. National databases can be used for this purpose, such as the University Health System Consortium.

Interdepartmental and Interdisciplinary Communication
 Measuring changes in communication may be difficult to do. The best way to capture such changes is through anecdotal comments from the team members affected by implementation of the case management model. Try to capture these comments before implementation and then at designated intervals after implementation. It is suggested that this be done in the form of focus groups. Questions asked should focus on the team's ability to communicate in a timely fashion, the level of respect afforded them by other team members, and their sense of team spirit or esprit de corps. In addition, patient care–related questions should also be asked, such as staff members' perceptions of the effectiveness of case management and case management plans on patient care delivery and efficiency, cost, quality, and so on. Ask them to identify any barriers to communication, as well as to give their ideas about how to remove those barriers.

OUTCOMES RELATED TO PATIENT HEALTH

 Each time a clinical intervention is applied in healthcare, there is an associated expected outcome (Box 2-2). Case management provides the structure for identifying those outcomes prospectively. Clinical outcomes should be interdisciplinary and come as a result of the collective efforts of the entire clinical team.

Avoid Adverse Effects of Care
 The hospital environment can be a dangerous place, and one goal of care is always to get the patient in and out of the hospital without doing any harm. Many of the quality indicators traditionally used in healthcare have focused on errors or problems that are associated with the way care was provided. These have included falls, nosocomial infections, medication errors, returns to the operating room, readmissions, morbidity and mortality reports, and deaths. These indicators are focused on the negative, untoward effects of the care provided to the patient and less on the identification of areas for clinical improvement that appear as patterns or trends. Nevertheless, it is important to continue to track these untoward outcomes after implementation of the case management model or case management plan.

Improve the Patient's Physiological Status
 The next set of indicators is concerned with the patient's clinical response to treatment. A goal of care is for the patient to be discharged in a better clinical condition than he or she was in at the time of admission or the episode of care. One measure of this is the patient's physiological status, which refers to the functioning of the various parts of the patient's organs and other body parts. The physiological status can be measured by such things as vital signs, laboratory values, and physical assessment. It is anticipated that these measures improve between the time of admission and the time of discharge.
      
The patient's physical abilities are assessed through a thorough review of the major body systems. These include the cardiovascular, respiratory, gastrointestinal, genitourinary, neurological, musculoskeletal, and integumentary systems.

Improve Signs and Symptoms
 Signs and symptoms are the first stage of illness. In this stage three things generally occur:

(1) the physical experience of symptoms such as pain, shortness of breath, or fever;
(2) a cognitive awareness of the symptoms and a placing of meaning on them; and
(3) emotional response to this awareness in the form of fear or anxiety (Ignatavicius and Bayne, 1991). At this point the person may self-treat or seek a medical opinion. In either case it is anticipated that the signs and symptoms will be reduced or eliminated. If this occurs, the patient returns to the optimal level of wellness.
      
It is therefore anticipated that after the clinical interventions of the case management team, the patient's signs and symptoms will be improved.

Improve Functional Status and Well-Being
 Functional status and well-being address the patient's ability to perform in a variety of areas. These areas include physical health, quality of self-maintenance, quality of role activity, intellectual status, attitude toward the world and sense of well-being related to self, and emotional status. Functional ability refers to the patient's ability to perform activities of daily living (ADLs). ADLs include an assessment of the patient's ability to perform personal care, ability to communicate, and perception of needs (Ignatavicius and Bayne, 1991).
      
There are a variety of tools to measure functional status. Among these is the commonly used Functional Independence Measure (FIM), developed by Granger and Gresham (1984). The FIM helps to quantify what the patient actually can do, regardless of the clinical diagnosis. Assessment categories include self-care, sphincter control, mobility and locomotion, communication, and social cognition. The FIM helps care providers measure the level of dependence/independence of their patients in an effort to decide what kind of help they may need after discharge to the community.

CLINICAL OUTCOMES

 When developing clinical outcomes, the previously discussed categories should all be considered. Some will be more relevant to the clinical picture than others. One approach for monitoring the expected clinical outcomes is through the identification of intermediate and discharge outcomes. These should be specific to the clinical issue at hand. Intermediate outcomes are those expected goals that occur during the course of the hospital stay. They also are triggers for change or progression in the treatment process and indicate that the patient is progressing toward meeting the discharge outcomes/criteria. Achievement of the goal or outcome should be based on the patient's expected response to treatment. These expected outcomes can occur at any point in the hospital stay and usually trigger the move to the next phase of treatment.
      
Discharge outcomes are those expected outcomes that the patient must achieve to be discharged from the hospital. The intermediate and discharge outcomes should be the basis for the case management plans/ tools. The expected outcomes must be identified before the clinical course of events can be determined. Determination of the expected outcomes should be based on an assessment of both the appropriateness of the care (a determination of who should receive what care) and the effectiveness of care (how good the outcomes of the care are). This review of the evidence helps to relate the process of care to the expected outcomes. This review should be based on all the available evidence rather than solely on the consensus of the practitioners involved in its development (Crosson, 1995). Physicians, when presented with the factual and scientific evidence behind the case management tool or guideline, are more likely to favor it and use it to guide their practice.

Examples of Outcomes
 The following examples (Boxes 2-3, 2-4, and 2-5) of expected outcomes are presented to help understand the differences and the relationships between intermediate and discharge clinical outcomes. Each example does not include an exhaustive list of the outcomes related to the diagnosis or procedure under which they are listed. Members of the interdisciplinary team working on developing the particular case management plan are the ones who decide on the expected clinical outcomes. They are finalized after a thorough discussion of their implications on the care of the patient, LOS, cost, and quality and are always included as an integral part of the case management plan.

PATIENT CARE VARIANCES

 A variance occurs when what is supposed to happen does not take place. It is defined as a deviation from a standard or omission of an activity or a step from a predetermined plan, norm, rate, goal, or threshold (Strassner, 1996). Generally, variances are expectations that are not met. According to Webster's Third New International Dictionary (1986, p. 2533), variance is defined as "the fact, quality, or state of being variable or variant … a difference of what has been expected or predetermined and what actually occurs." In relation to patient care, variances are outcomes or healthcare providers' actions that do not meet the desired expectations. In relation to case management, variances are deviations from the recommended activities in any of the care elements delineated in the case management plan (Ignatavicius and Hausman, 1995; Pearson, Goulart-Fisher, and Lee, 1995; Tahan and Cesta, 1995; Cohen and Cesta, 1997). Variances often result from delays, interruptions, additions, or omissions of patient care activities and processes. They may sometimes be related to expediting patient care (e.g., performing a patient care activity before it is due is considered a variance).
      
In an era of increased competition in healthcare, case management plans have emerged as the most desirable tools for improving patient care quality through the elimination and/or prevention of variances, reduction in duplication and fragmentation of care elements, and the standardization of patient care activities. When followed appropriately, case management plans result in consistency in the practice patterns of physicians, nurses, and other healthcare professionals and thus reduce variations in patient care. With this comes the significance of patient care variance data collection and analysis, which are integral elements of case management. Variance data collection cannot occur until the expected outcomes of care, as they relate to the case management plan, have been identified. These expected outcomes become the benchmark against which variation in patient care can be determined.
      
Variance data collection is important because it provides the basis for improvement in patient care activities, processes, outcomes, and quality. The mechanism of variance data collection is usually decided on by the steering committee charged with implementing the case management model and the use of case management plans. Some institutions have delegated this responsibility to a case management department or a quality improvement committee/council. Regardless of who is responsible, the process should be made consistent across the various care settings that exist in the same institution.
      
There is no standardization in the method of classifying variances. Variances are classified into different categories in different institutions. Traditionally the most common broad categories used to classify variances are patient/family (Box 2-6), system (Box 2-7), and practitioner (Box 2-8). Patient/family variances are the result of the patient's behavior or activity or the behavior/activity of a family member (family is used here in its generic sense to denote a patient's spouse, caregiver, significant other, or family member). Variances may be refusal of treatments, or they may occur as a result of changes in the patient's condition or complications of a medical or surgical procedure (e.g., refusal to sign a consent for an operative procedure, infection, fluid and electrolyte imbalance, or family unavailable to accompany the patient home on discharge). Some institutions separate patient variances from the family-related ones. They may classify changes in the patient's condition as they result from the disease process under a separate category (e.g., physiological). The family-related variances may be classified into a separate category and labeled as community variances.
      
Practitioner variances occur as a result of behaviors of healthcare providers. Examples of practitioner variance are omission of a treatment, test, or procedure; giving the wrong medication; incomplete follow-up and documentation of the patient's response to treatments; or a visiting nurse not showing up for a prescheduled home visit. These variances represent the areas that healthcare providers may have the most control over, and if prevented they can influence positive patient care outcomes, lead to timely discharge, eliminate unnecessary work, and reduce cost.
      
System variances are those related to the way an institution operates, and they result in delays in patient care processes. They occur because of inefficient operations and systems, and they may be called operational variances. Mostly these variances require administrative attention or intervention for resolution. System variances can also be classified as internal (i.e., within the walls of the healthcare facility [institution-based]) or external (i.e., outside the walls of the healthcare facility [community-based]). Examples of system variances are lost laboratory requisition slips or specimen, failure of an infusion pump, no nursing home bed available, payment denial, or a managed care organization not approving certain patient care services.
      
Variances are also classified as positive or negative (Bueno and Hwang, 1993; Hampton, 1993; Ignatavicius and Hausman, 1995; Tahan and Cesta, 1995; Mateo and Newton, 1996). A positive variance is defined as a desired outcome that occurs before it is expected (i.e., before the timeframe that is indicated/projected in the case management plan). It is also a justified type of variance. Examples of positive variance are switching a recommended antibiotic to a different one because of a patient's allergy, changing the diet of a cardiac patient who is admitted for the management of heart failure from the salt- and fat-restricted diet recommended in the case management plan to include diabetic restrictions because the patient is also diabetic, and a patient's early discharge because all of the outcomes are met earlier than expected.
      
A negative variance occurs when a patient care activity is delayed and the patient does not meet the expected/desired outcomes (i.e., the recommended patient care activities in the case management plan are not achieved within the specified timeframes). For example, a patient was on anticoagulation therapy and required prothrombin time (PT) testing every 6 hours on the initial day of treatment. He was due for a PT test at 12:00 noon, but the test was not completed until 4:00 PM and the result could not be retrieved from the laboratory information (automated) system until 5:30 PM. The result of the PT test was found to be very high, and the patient required an immediate intervention, putting the anticoagulation therapy on hold temporarily. In this example, a delay in performing the prescheduled PT test was identified as a practitioner variance. This variance resulted in a delay in changing the anticoagulation therapy/plan of care, and the patient was required to stay an extra day in the hospital.
      
Another variance category is an add-on variance. It is defined as an unplanned or extra patient care activity or process. This type of variance occurs as an addition to what is indicated in the case management plan. An example of such variance is added laboratory tests. An add-on patient care activity is considered a variance because it may contribute to an increase in cost or a delay in discharge. Most often this type of variance results in duplication of services or performance of unnecessary patient care activities.
      
Because there is no standardization in classifying variances, it is difficult to share variances across care settings or institutions for the purpose of benchmarking. It is even more difficult to conduct a joint trending analysis of variance data from several healthcare institutions located in a particular community, an analysis that could sometimes be important for improving healthcare in a whole community rather than a particular hospital population. To avoid this, Hoffman (1993) recommends that the standardized critical elements of patient care included in case management plans be used as the classification system for variances (e.g., assessment and monitoring, treatments, medications, patient/family education, discharge planning). If this classification system is followed, then data become transferable within and across institutions, which is highly beneficial for improving the quality of patient care.

Variance Data Collection and Analysis
 Designing an effective method for documenting, collecting, and analyzing variances remains a great challenge for most healthcare institutions. Whether the process of variance data collection is automated or manual, most institutions have made the case manager the one responsible for collecting variances. Variance data could be collected anytime during the patient's hospitalization. However, the best time is at the time the variance happens or when it is identified. Timely identification and resolution of variances result in the delivery of cost-effective and high-quality care and increase patient/family satisfaction. A variety of sources for variance identification can be used, such as the progress notes of physicians, nurses, and other healthcare professionals; verbal communication with other members of the interdisciplinary care team during a case conference or one-on-one; or communication with other departments.
      
The extent to which variance data should be collected is extremely difficult to generalize and varies from institution to institution. How to collect data and what is needed should be prospectively determined. Some institutions collect data at random or as they relate to every single patient care activity without considering their impact on the LOS and quality of care. In such situations, data might become overwhelming and unmanageable. Because of the volume, collected data can be difficult to analyze, trend, or use to efficiently generate reports that can be used for quality improvement. It is recommended that the interdisciplinary team developing the case management plan spend some time defining the significant patient care activities that need to be evaluated for variance data collection. A decision as to what should be included should be individualized to the specific diagnosis or procedure of the case management plan. In addition, variance data collection should be limited to data that affect the predetermined outcomes of each case management plan. The steering committee overseeing the process of implementing case management systems and developing case management plans is the best group to guide the interdisciplinary teams in this process. Members of the interdisciplinary team may use a set of questions (Box 2-9) as a guiding tool for better decision making when faced with the dilemma of what variance data should be collected.
      
Variances must be identified and corrected as soon as possible for betterquality patient care outcomes and prevention of unnecessary delays in the patient's discharge or prolongation in the LOS. Regardless of who is made responsible for variance data collection, there should be the following:

If the primary nurses are given the responsibility of collecting patient care variances, then they, rather than the case manager, are responsible for responding to the identified variance. However, the ideal way of correcting variances is for the interdisciplinary team member who has identified the variance to immediately begin to try to correct or resolve it. This approach will then result in the timeliest results and should help to avoid delays in the LOS or deterioration of quality. Members of the interdisciplinary healthcare team should be made aware of their responsibilities toward variance data collection and resolution. Open lines of communication should be established to promote effective and timely communication of variances to the appropriate administrative personnel, particularly when direct care providers such as staff nurses are made responsible for identifying and resolving variances as they occur.

Strategies for Handling Variances
 When a patient care variance is identified, certain questions should be answered immediately to resolve the situation and improve outcomes. The answers to these questions will determine the urgency and seriousness of the situation and indicate the corrective action plan. The questions appear in Case Manager's Tip 2-1.
      
After careful collection of variance data, the data are analyzed. The variance analysis process (Figure 2-1) is a systematic and scientific interpretation through categorization/grouping, trending, and statistical analysis of the data collected. Variance data are usually compiled and analyzed over time to allow for better opportunities for quality and patient care process improvement. The ideal way of dealing with variance data is to link the process to the quality improvement efforts taking place in the institution. Evaluating variances and constructing and implementing an appropriate action plan for improvement ensure better patient care outcomes and prevent the situation from happening again.
      
Improvement efforts should be spent addressing the recurring variances rather than the isolated, random, or single events because better patient care outcomes are affected by the extent to which the recurring variances are eliminated or prevented. The isolated variances are known to happen as a result of special causes (i.e., not directly related to the systems, operations, or processes an institution has in place). However, recurring variances are the opposite of isolated variances and take place as a result of common causes. These variances require an evaluation and analysis of the systems and processes of patient care the institution follows, as well as the policies and procedures and in some cases the standards of care and practice. It is suggested that efforts to address the isolated variances be decided based on the individual situation, particularly if these events interfere in the LOS, patient/family satisfaction, and cost and quality of care.
      
It is not enough to identify and resolve a variance when it occurs. It is equally important to track variances. The purpose of tracking variances is to conduct a trending analysis of each variance category. The results of such analyses are the basis for revising and improving the case management plan, reducing the incidence of variances in the future, and studying the data for necessity of quality improvement efforts, particularly those that are system/operations-related.
      
Variance data collection and analysis allow healthcare executives to look beyond LOS and cost of care. They are ongoing processes that are helpful in continuously improving the systems of the institution, case management plans, patient care activities, processes and outcomes, and quality of care. It is through this process that standardization of the best/ideal approach to patient care can be achieved and maintained.
      
Establishing variance data collection tools and formalizing/standardizing the process are integral to the accuracy and reliability of the data collected (Case Manager's Tip 2-2). Before the initiation of variance data collection, it is imperative to determine what should be measured and how and when assessment and evaluation of care are conducted. It is not uncommon that a variance tracking tool be developed as a part of the case management plan (Figure 2-2). In this case, the tool is made specific and individualized based on the diagnosis or procedure of the case management plan. An institution may elect to use a generic tool (Figure 2-3) that is applicable to any case management plan regardless of the diagnosis or procedure. In both situations, the tool is not made a permanent part of the medical record. It is dealt with as defined in the institutional policy/procedure for variance data collection and analysis.
      
Providing feedback regarding the effectiveness of case management plans is important. Integral to any strategy for improving patient care is a provision for feedback to the clinicians and administrators who are involved in patient care processes and activities. A formal mechanism for communicating important and appropriate data must be in place in each institution. Reporting variance data should also be done on a regular basis with specific time intervals. Variance data reports should be distributed to members of the interdisciplinary team that was involved in the development of the case management plan and to the related administrators for initiating relevant quality improvement projects.
      
Patient care monitoring and timely variance identification and resolution are keys to the success of case management plans and are crucial for the delivery of cost-effective and quality care. However, establishing a standardized process for variance data analysis and management remains the best strategy for improving patient care outcomes. In addition, feedback among healthcare administrators and direct care providers is extremely important in refining the use and the effectiveness of case management plans.

OUTCOMES MANAGEMENT AND OUTCOMES CLASSIFICATION

 Healthcare providers and administrators are constantly engaged in the evaluation of the outcomes of care and the care delivery system (i.e., the case management model). The interest in outcomes management is integral to case management model evaluation. One must examine both outcomes management and case management to ensure more acceptable care quality and outcomes to the patient and the healthcare organization. Some organizations have made outcomes management as one dimension of the role of the case manager. Other organizations have integrated quality management into the case management department and under the auspices of the administrator/executive responsible for the management of this department. Combining the activities of outcomes management and case management results in having the case managers accountable and responsible for outcomes measurement, monitoring, management, analysis, interpretation, and reporting and for performance improvement.
      
Case managers are often found to be collecting data regarding variances (e.g., delays in care), adherence to case management plans, and the outcomes of care activities of both the clinical and administrative processes. This is important for case managers because of their responsibility to

"always track outcomes data for the patients in their caseload to determine the effectiveness, efficiency, and efficacy of the case management services provided. It is the ability of … case managers to use outcomes data intelligently to manage, plan, facilitate, expedite, advocate, coordinate, and evaluate the delivery of patient care that ultimately will result in increasing the degree of success of the case management system" (Flarey, 1998, p. 100).

     
To facilitate the role of case managers in outcomes management and case management model evaluation, it is important for the healthcare organization to establish an outcomes classification system. Such system must aim at evaluating the case management model and must be standardized across the organization and used by all case managers. It also must identify the outcome indicators to be measured and delineate the process of data collection, aggregation, analysis, interpretation, and reporting. Moreover, such system must adopt the use of predetermined outcome indicators reflective of the interest of both the organization/providers of care and the patient population/ consumers of care. The classification system must address the standards and requirements of accreditation and regulatory agencies and the expectations of the payers for healthcare, including managed care organizations.
      
One example of such classification system is shared in Figure 2-4. There is no one best system to be used. However, it is important for an organization to identify the system that works best for its providers and customers. For example, some organizations may consider the patient's functional ability as a separate classification and as independent of the clinical indicators; others may combine the two. The examples presented in Figure 2-4 do not make an exhaustive list of indicators. Others may still be added; however, it is important to define whether the indicator is a patient/family- or healthcare organization-related indicator. The examples shared here are not presented in a measurable format but rather as themes or broad classifications. A healthcare organization to be using this classification system must indicate the outcome indicators in a measurable way, for example cost per case type or an episode of care/illness or managed care reimbursement denial rate. The organization also must delineate the frequency of data collection, the sample, the formulas to be applied in the analysis, and the reporting format. Other ways of classifying outcome indicators are presented in the following sections.

Cost Versus Quality Outcomes

 Case managers and case management departments are evaluated in multiple ways. As has been discussed in this chapter, we may choose to look at outcomes in terms of those that directly affect the organization, or we may choose to look at them in terms of those that indirectly affect the organization. Another way to classify outcomes is by those that affect cost versus those that affect quality. We can further break this down to those used to evaluate the case manager versus those that are used to evaluate a department of case management (Case Manager's Tip 2-3).

ORGANIZATION-SPECIFIC OUTCOME MEASURES—EVALUATING THE DEPARTMENT OF CASE MANAGEMENT
 When evaluating a department within an organization, certain measures of cost can be selected and used as they relate to the department's goals, mission, and expected outcomes. Examples may include LOS, cost per day/episode of care, cost per case, and cost associated with reimbursement denials. As each is selected a frequency for reevaluating each outcome should also be determined. These intervals or frequencies should be appropriate to the selected outcome and should not be too frequent as to reduce the sensitivity of the data, nor so infrequent as to lose meaning over time.

Cost Measures
 Length of Stay
 LOS continues to be the most commonly used outcome measure in case management. If your organization or department measures LOS as an outcome indicator then the following factors should be considered:

Cost per Day/Case
 Before selecting cost outcomes, consider the following question: Can you access true cost per case using a cost accounting system? If not, other strategies can be used, such as measuring specific resource utilization (e.g., tests, medications, or radiological exams). If you have access to a cost-accounting system then the following are potential measures to be used: Third-Party Payer Denials
 Denial of payment by third-party payers is quantifiable and affects the bottom line of the organization; therefore it is commonly used to measure the impact of case management on the organization. The following data elements should be considered when looking at this outcome: Quality Measures
 Measures of quality are as important as those of cost but may be more difficult to quantify. Measures of quality may include patient and family satisfaction, staff satisfaction, clinical outcomes, and transitional planning outcomes. Consider the following examples, knowing that without cost savings attached to them they may carry less weight.

Patient/Family and Staff Satisfaction Clinical Quality Outcome Measures Transitional Planning Issues ORGANIZATION-SPECIFIC OUTCOME MEASURES-EVALUATING THE CASE MANAGER

 In evaluating the case manager, many of the outcome measures are consistent with those selected for measuring the case management department. The only difference is that the data will be aggregated in terms of the case manager; that is, based on the case manager's caseload and not the service.

Examples of Cost Measures Examples of Quality Measures NATIONAL QUALITY INDICATORS

 The focus on quality and outcomes is essential for the survival of healthcare providers, payers, and agencies in today's market. The pressures of consumers of healthcare services have led to a focus on national quality standards through the evaluation and measurement of certain outcome indicators of interest to consumers, payers, and providers. Different national organizations have advocated for national indicators. Although the indicators vary across the organizations, the focus is somewhat similar and to some degree the indicators are the same. The indicators basically focus on access to care and quality and cost of services. It is essential for case managers and administrators of case management programs to be knowledgeable about these national organizations so that they are better able to incorporate their recommended quality indicators in the evaluation process of the case management model.
      
The indicators of interest to the national organizations are usually related to performance of healthcare providers and agencies. Examples are nosocomial infection rate, fall-related injury rate, cost per episode of care, access to care, consumers' choice of services, patient satisfaction, and qualifications of healthcare providers. The national organizations most interested in quality indicators are the National Committee for Quality Assurance (NCQA); the Utilization Review Accreditation Commission (URAC); the Centers for Medicare and Medicaid Services (CMS), formerly known as the Health Care Financing Administration (HCFA); the Joint Commission on Accreditation of Healthcare Organizations (JCAHO); and the American Nurses Association (ANA).

National Committee for Quality Assurance
 The NCQA is a national organization comprised of healthcare quality management professionals. Originally established in 1976 to advance the profession of healthcare quality management, today its components include 48 state quality associations; an educational foundation; and an accreditation body for managed care health plans, specifically health maintenance organizations (HMOs), and for individual providers in healthcare quality management.
      
Additionally, NCQA offers a certification to health-care professionals. Individuals certified by NCQA receive certification as a Certified Professional in Healthcare Quality. This certification recognizes professional and academic achievement through participation in this voluntary certification program (Claflin and Hayden, 1998). The national indicators and performance measures identified by NCQA and used in the accreditation standards of managed healthcare plans are known as the Health Plan Employer Data and Information Set (HEDIS). HEDIS focuses on the quality of the systems, processes, care, and services a managed care plan delivers to its clients/enrollees. Information collected by NCQA from these health plans is made accessible to both consumers and employers who subscribe to managed care plans. Such information assists the consumer to choose the plan with the best quality performance. Further information about HEDIS is available at http://www.ncqa.org. HEDIS targets the following quality areas: Utilization Review Accreditation Commission
 The URAC, also known as the American Accreditation HealthCare Commission, was established in 1990 to evaluate the methods of utilization review employed by healthcare agencies, particularly the managed healthcare plans because at that time they were widely diverse. As a result of this examination, URAC made health utilization standards available for agencies to use in organizing their utilization review procedures. This acted as a step toward standardization in utilization review practices. In 1994 URAC developed its accreditation program for health plans that were not eligible for accreditation by NCQA, and in 1999 the program was expanded to highly integrated healthcare plans (Kongstvedt, 2001). URAC's accreditation program differs from NCQA's in that it is basically limited to utilization review programs.
      
URAC's accreditation program is also known today to accredit case management services. More detailed information is available at http://www.urac.org. The quality issues addressed in the accreditation standards of interest to case management are as follows: Outcome-Based Quality Improvement
 Outcome-based quality improvement (OBQI) is an approach used by homecare agencies to monitor and continuously improve their quality outcomes. The process relies on clinical, financial, and administrative outcomes data from the Outcome and Assessment Information Set (OASIS) (see Section 2, Chapter 1 for more information on OASIS) sponsored by the CMS. The goal of OASIS is to improve quality of care delivered to home healthcare patients and to provide data to CMS. Uniform data must be collected on all patients, or on selected patients, in an agency. The data are com pared with the same patient outcomes for the previous reporting period and against a reference sample of homecare patients. The data also are risk adjusted to control for any variable that might be affecting them. The outcomes are analyzed to determine whether they are inferior or superior to the previous data collection period. When found to be inferior, the data are then used to spearhead further investigation in the form of a continuous quality improvement (CQI) project(s). Further information about OBQI is available at http://www.chspr.org/obqi3.htm. OASIS focuses on the following areas of quality: ORYX—Joint Commission on Accreditation of Healthcare Organizations
 The ORYX initiative was implemented by the JCAHO in 1997 as a means to integrate outcomes and other performance measurement data into the JCAHO's accreditation process. The JCAHO's primary mission is to continuously improve the safety and quality of care provided to the public through the provision of healthcare accreditation and related services that support performance improvement in healthcare organizations. Today the ORYX project is known as the core measures.
      
During the initial phase of ORYX, accredited hospitals, long-term care organizations, home care organizations with an average monthly census of 10 or more patients, and behavioral healthcare organizations select two measures from a list of core measures provided by the JCAHO. The measures selected must relate to at least 20% of the patient population. The healthcare organization then collects the data on the selected measures and regularly submits it to the JCAHO. The JCAHO is responsible for analyzing the data and providing performance trends and patterns to the organization in advance of a JCAHO survey. The organization also is responsible for explaining its rationale for selecting the specific performance measures and how the data has been used to improve performance. The JCAHO plans to increase the number of measures and the percentage of patient population gradually each year. Further information about the ORYX project is available at http://www.jcaho.org/pms/core+measures/core+measures.htm.

American Nurses Association
 In 1995 the ANA undertook a project (i.e., the nursing report card for acute care settings) to explore the nature of the relationships and linkages between nursing care and patient outcomes in acute care settings. This project resulted from the implementation of organizational changes that decreased the numbers of registered professional nurses providing patient care and replaced them with unlicensed personnel. Another factor that led to the demand for such a project was the increasing numbers of reports about incidents of threats to quality and safe patient care delivery. A major purpose of this project was the provision of a framework for educating nurses, consumers, and policymakers about nursing's contribution to inpatient hospital care.
      
The strategy used to determine the nursing indicators was based on an examination of the nursing scientific literature, consultation with experts and nurse leaders in nursing quality and outcomes research, and conduct of focus groups of nurses from all levels of practice related to the acute care setting. This project identified three categories of indicators that are still in use today (ANA, 1995). Further information is available at http://www.nursingworld.org. The categories are as follows: REPORT CARDS

 Report cards, also known as scorecards, have become a common and popular practice in quality management during the past several years. They are used as a tool for displaying outcomes data in a meaningful, concise, clear, and easy-to-read manner. Report cards can be organized applying the outcomes classification system used by the healthcare organization or the department of case management such as the one discussed earlier in this chapter. They also may include benchmark data and/or targets, which makes the reports more meaningful and valuable (Figure 2-5). An advantage of applying an outcomes classification system to the report card is that it ensures focus on multiple elements of outcomes and not a single parameter such as cost. Adding elements such as consumer satisfaction to LOS and cost outcomes is essential.
Examples of the most common classification categories of outcome indicators are as follows: Each of these classifications may include several indicators that are measures of the activities, treatments, and outcomes of patient care practices.
      
Report cards provide healthcare organizations with a mechanism for data collection, aggregation, analysis and trending, benchmarking and comparing, and reporting. Standardizing the way outcomes data are displayed (by applying the above classification, for example) can be advantageous to many audiences, such as healthcare providers, administrators, payers, regulatory and accreditation agencies, and consumers. Defining the purpose of the report card and its audience is important because it influences its format, content, and level of sophistication. For example, designing a report card for a case management plan is different from designing a report card for a case management department. The case management plan's report card focuses on the topic of the plan; that is, a homogenous patient population, whereas the case management department is heterogeneous.
      
Report cards can be generated for a physician or other providers, such as the case manager, or for a disease entity, service or product line, case management plan, or department (i.e., case management department). Regardless of the type of audience the report card is intended for, Spath (1998) recommends that one first defines the purpose of the report card, delineate the learning opportunity intended by the card, and determine the extent of actions that need to be taken to improve outcomes and performance. It is also important to decide on the frequency of generating the report card and what is expected of those who receive a copy of it. Some organizations distribute the report card on a quarterly basis and as a part of their quarterly quality assurance/improvement procedures.
      
A report card for case management model evaluation or case management plan evaluation is necessary and it is desirable to be made available to key personnel (e.g., senior administrators, physician chairmen/chiefs, case managers, case management plan teams, and finance) in an organization on a quarterly basis. Such a report card may include the elements discussed earlier in the outcomes management and classification and the national quality indicators sections of this chapter. The previous and current improvement activities must also be included as a part of such report. These activities indicate who is responsible for what and indicate the timeframe for completing an improvement activity and its expected outcome. This aspect of the report card ensures the link between the report card and the quality management department or program and communication with the various key players. Report cards that apply this format are looked on favorably by accreditation and regulatory agencies.

KEY POINTS
  1. Outcomes can be used as indicators of quality.
  2. Outcomes are the goals of healthcare delivery and can be directly or indirectly related to the patient's health.
  3. Length of stay is an indicator of the success of a case management model and is often tracked to determine its effectiveness. However, it is not enough to examine it alone; it must be looked at in conjunction with other indicators such as cost of care and those shared in the report card section.
  4. Clinical outcome indicators give the case manager a guide for progressing the patient from day to day, episode to episode, or toward discharge.
  5. Variance data should be collected using variance tracking tools specific to the institution. Variances should also be classified and coded for ease of tracking. The same classifications should be used across the various care settings.
  6. Variance data collection may be made the responsibility of the interdisciplinary team members or the case manager only. Each institution may decide which mechanisms to be followed. Variance data should be tracked, collated, analyzed, and evaluated for improvement opportunities.
  7. Results of variance data analysis should be tied into a quality improvement system. Reporting variance data and providing feedback to clinicians and administrators are important and integral to case management systems.
  8. Indicators used to evaluate the case management model must be developed based on the recommendations of the national organizations interested in this area such as NCQA, URAC, CMS, ANA, and JCAHO.
REFERENCES

American Nurses Association: Summary of the Lewin-VHI, Inc., report: nursing report card for acute care settings, Washington, DC, 1995, The Association.

Bueno MM, Hwang RF: Understanding variances in hospital stay, Nurs Manage 24(14):51–57, 1993.

Claflin N, Hayden CT: Guide to quality management, ed 8, Glenview, Ill, 1998, National Association of Healthcare Quality.

Cohen EL: Case management: does it pay? J Nurs Adm 21(4):20–25, 1991.

Cohen EL, Cesta TG: Case management: from concept to evaluation, ed 2, St Louis, 1997, Mosby.

Crosson FJ: Why outcomes measurement must be the basis for the development of clinical guidelines, Manag Care Q 3(2):6–14, 1995.

Granger CV, Gresham GE: Functional assessment in rehabilitation medicine, Baltimore, 1984, Williams & Wilkins.

Hampton DC: Implementing a managed care framework through care maps, J Nurs Adm 23(5):21–27, 1993.

Hinshaw A, Atwood J: A patient satisfaction instrument: precision by replication, Nurs Res 31(1):170–175, 1982.

Hoffman PA: Critical path method: an important tool for coordinating clinical care, Joint Comm J Qual Improve 19:235–246, 1993.

Ignatavicius DD, Bayne MV: Medical-surgical nursing: a nursing process approach, Philadelphia, 1991, WB Saunders.

Ignatavicius DD, Hausman KA: Clinical pathways for collaborative practice, Philadelphia, 1995, WB Saunders.

Kongstvedt P: Essentials of managed health care, ed 4, Gaithersburg, Md, 2001, Aspen.

Mateo MA, Newton C: Managing variances in case management, Nurs Case Manag 1(1):45–51, 1996.

Pearson SD, Goulart-Fisher D, Lee TH: Critical pathways as a strategy for improving care: problems and potential, Ann Intern Med 123(12):941–948, 1995.

Schriefer J et al: Linking process improvement, critical paths, and outcomes data to increased profitability, Surg Serv Manage 2(6): 46–50, 1996.

Spath P: Nursing performance measures go public, Outcomes Manage Nurs Pract 2(3):124–128, 1998.

Strassner LF: The ABCs of case management: a review of the basics, Nurs Case Manag 1(1):22–30, 1996.

Tahan HA: Nurse case managers' responsibilities toward patient care outcomes, Semin Nurse Manag 6(3):100–103, 1998.

Tahan HA, Cesta TG: Evaluating the effectiveness of case management plans, J Nurs Adm 25(9):58–63, 1995.

Webster's third new international dictionary, vol III, Chicago, 1986, Encyclopedia Britannica.
Glossary      

Appendix A
Appendix B
Appendix C
Appendix D
Appendix E-1 Appendix E-2 Appendix E-3
Appendix F
Appendix G
Appendix H
Appendix I
Appendix J
Appendix K
Appendix L
Appendix M
Appendix N
Appendix O


Top