User login
STUDY DESIGN: Cross-sectional
POPULATION: We studied a total 266 randomly selected adult patient encounters representing 37 physicians.
OUTCOMES MEASURED: A problem was defined as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Visit duration and the number of billing diagnoses were also assessed.
RESULTS: On average, 2.7 problems and 8 physician actions were observed during an encounter. More than one problem was addressed during 73% of the encounters; 36% of these additional problems were raised by the physician and 58% by the patient. On average, each additional problem increased the length of the visit by 2.5 minutes (P <.001). The concordance between the number of problems observed and the number of problems on the billing sheet indicated a trend toward underbilling the number of problems addressed.
CONCLUSIONS: Multiple problems are commonly addressed during family practice outpatient visits and are raised by both the physicians and the patients. Our findings suggest that current views of physician productivity and the billing record are poor indicators of the reality of providing primary care.
Primary care disciplines continue to have a central role in the health care of Americans. They provide breadth of care within an ongoing relationship, bridging the boundaries between health and illness and guiding access to more narrowly focused care when needed.1 The ability to orchestrate a broad health agenda during a visit is central to primary care, but this ability is challenged by competing demands for time.2
Attempts to influence provision of care and treatment decisions by primary care physicians, such as financial incentives, administrative restrictions, and the implementation of evidence-based clinical guidelines add to the demands on physicians’ time and may affect how time is allocated during the day and with each patient. Within this context a primary care physician must prioritize the agenda for each patient visit. This may include providing services beyond the patient’s primary reason for the visit as time permits, such as including preventive services,3 follow-up of acute or chronic illnesses,1 mental health4 or family issues,5-7 or investigating “by the way” patient comments that may indicate serious medical issues.
The competing demands for time are compounded by patient requests during the visit. Based on an audiotape of 139 patient encounters, Kravitz and colleagues8 reported that on average a patient makes 5 requests for physician action or information per visit, and the number of unfulfilled requests was negatively associated with patient satisfaction. Such findings may fuel a sense of pressure to address patient requests. Also, another recent report indicates that the majority of patients do not have the opportunity to express all of their concerns before the physician redirects the interview; once redirected, additional patient concerns are rarely elicited.9 Fitting both the physician’s and patient’s agenda into the time allotted for an outpatient visit has important implications for the duration of the visit, physician productivity, and possibly patient outcomes.
Data on the number of problems raised and addressed have been limited by the lack of appropriate collection methods. Primarily audio and video technology have been used for the study of physician-patient communication.10-12 Direct observation of patient encounters12,13 and incorporation of ethnographic approaches have more recently been employed to fill a large void in the understanding of the content, context, and complexity of primary care.13-15 Findings from the Direct Observation of Primary Care study, which employed such methods, indicate that among 4454 patient visits care was provided to a secondary patient during 18% of the visits and preventive services were addressed during 32% of the illness visits.3 Data from that study provide a glimpse into some types of problems addressed in addition to the main reason for the visit; however, data about the number of problems addressed during patient encounters were not specifically collected by the nurse observer.
When additional issues are raised during a patient encounter, little is known about the nature of these problems, how additional problems affect the duration of the visit, and how well additional problems are reflected in the billing record. This led us to conduct an observational study to ask: How many problems are addressed during family practice outpatient visits, and who is raising additional problems? How much work and time is associated with addressing problems raised beyond the initial problem? How well does the billing list represent the number of problems addressed during the outpatient visit? Our study was designed to directly observe and record how many problems were raised and addressed during outpatient visits to family physicians.
Methods
Seven first-year medical students observed patient care provided by their summer fellowship family physician preceptor and other physicians in the preceptor’s practice from June through August 1999. Six of the sites were located in Northeast Ohio, and one was in Tulsa, Oklahoma.
Each student collected data on one randomly selected adult patient encounter for each half day of precepting. At the beginning of each half-day of patient care the student rolled a die to generate a random number to select a patient from the patient schedule. To ensure random selection of encounters within each half-day session, on alternating days the random number was counted from the beginning or the end of the half-day schedule. If the selected patient was aged younger than 18 years, the patient or physician preferred the encounter not be observed, or the patient did not show up for the scheduled appointment, the next scheduled appointment was selected as a replacement. Patient age and sex were collected for those who were no-shows or chose not to be observed, so they could be compared with those patients who were observed. Each student was to collect data on approximately 50 patient encounters during the 6-week summer fellowship. The physicians were blinded to the study purpose and were not told which patient encounter would be included in the study.
A problem was operationalized as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Each item was listed as it was raised, and the type of problem, who raised it, and what physician actions were involved to address it were coded. Each problem was coded as 1 of 14 categories: acute, acute follow-up, chronic, chronic follow-up, prevention, prevention follow-up, psychosocial, psychosocial follow-up, work-related administrative, health care system-related administrative, other family member’s problem, pregnancy, emergent, and other. The person who raised the problem was coded as 1 of 3 options: the physician, the patient or another person in the room. Multiple physician actions could be coded for how the problem was addressed. The 19 physician action categories included: question, reassurance, examination, procedure, referral, return visit, advice, review tests, order laboratory testing, prescription, provide written material, imaging, admits uncertainty, counseling, return to work/time off work letter, defer, complementary/alternative medicine, ignored or lost, and other.
Patient characteristics, the duration of the visit, and the billing diagnoses for each visit were also recorded on the data collection form. Videotaped encounters were used to pilot test the data collection form, to allow the observers to practice using the form in real time, and to calibrate the observers before data collection in the field.
We used descriptive statistics to address most research questions. Student t tests and chi-square tests were used to compare age and sex differences between participants and nonparticipants. We tested the association of the number of problems with the duration of the visit with analysis of variance and a test for linear trend. A difference score of the number of problems observed and the number of problems recorded on the billing sheet for the encounter was computed and summarized graphically.
Results
We collected usable data on 266 encounters representing 37 physicians. Patient and visit characteristics are displayed in Table 1. The patients had an average age of 48 years, and 69% were women. They were predominately white. A large proportion was observed visiting their regular primary care physician (83%), and 85% were established patients of the practice. Most of the observed patients had some kind of commercial health care insurance, 19% had Medicare, and a small proportion had Medicaid or no insurance. The visit duration ranged from 2 to 65 minutes; the median was 15 minutes with a mean of 19.3 (standard deviation [SD]=12.7). The first problem raised was most commonly an acute problem (49%); prevention and chronic illness were the first problem raised during 21% and 19% of encounters, respectively. Patients who were randomly selected but were not observed (n=52, primarily no-shows) were similar in sex (67% women, c2 =0.119, P=.73 ) but were younger than those patients who were observed (mean age=32.1 years, t=3.79, P=.001).
On average, 2.7 problems were raised during an encounter Table 2. Forty-four percent of all problems were classified as acute, 30% chronic, 14% prevention, 4% administrative, 2% psychosocial, and 6% were classified as other. Of the observed encounters, 73% had more than one problem addressed. The physician raised 36% of these additional problems, and patients raised 58%. The problems raised by physicians were most frequently pertaining to chronic illness, prevention, and follow-up issues. The problems raised by patients were most likely to be acute illness problems. Additional problems were least likely to arise when the first problem addressed was an acute problem (61%) compared with visits during which the first problem addressed was chronic or prevention focused, where 88% and 87%, respectively, included additional problems during the visit (c2=21.2, P <.001).
On average, 8 (SD=4.5) physician actions were observed per encounter Table 2. Physicians performed an average of 3.3 (SD=1.2) actions per problem. The most common physician actions were questioning (77%), physical examination (49%), prescription writing (32%), providing advice (31%), and reassurance (25%). Of the 452 additional problems raised, only 3% of problems were ignored, and 6% were deferred to another visit.
The association of the number of problems addressed with the duration of the visit was assessed by analysis of variance and a test for linear trend. As shown in Figure 1, the duration of the visit increased approximately 2.5 minutes for each additional problem addressed (P <.001 for linear trend). The visit duration within each of the number of problem groups varied greatly as indicated by the large range for each group; however, the SD for each of the groups as indicated by the shaded bars are a similar size for each of the groups (Levene’s test of equality of error variance=1.48, P=.195).
The concordance between the number of problems observed and the number of problems on the billing sheet was modest, with a trend toward billing for fewer problems than were observed. As shown in Figure 2, 29% of encounters represented a match between the number of problems observed and the number of problems on the billing sheet. Fifty-eight percent of the encounters had more problems observed than recorded on the billing sheet. A much smaller proportion of encounters recorded more problems on the billing sheet than were observed during the encounter.
Discussion
Our exploratory study suggests that it is common for multiple problems to be addressed during visits to a family physician regardless of the initial reason for the visit. Additional problems are raised by both physicians and patients and are rarely deferred or ignored by the physician. Although the phenomenon of integrating a broad health agenda and addressing multiple problems during a single outpatient visit may be well known by practicing community-based family physicians, it may not be recognized by policymakers or health services researchers whose window into the process of outpatient care is provided by the medical record and billing data.
Addressing the majority of a patient’s health care needs and providing comprehensive care is a core feature of quality primary care.16-20 Previous work has documented the wide range of diagnoses and clusters of diagnoses that family physicians commonly address during outpatient care.13,21 However, truly comprehensive care goes beyond providing a broad array of services; it also involves the integration of care in a physician-patient relationship context. Prioritizing, providing, and orchestrating care for acute and undifferentiated illness, chronic disease, preventive services, and mental health care represents a key feature of primary care practice such that the care is greater than the sum of its individual commodities.1 These data suggest that single visits often address a broad agenda of health care.
Overall, as the number of problems increase so does the length of the visit. Others have found that ordering or performing more tests, providing preventive services, and conducting ambulatory surgical procedures increase the length of the visit.22 It is not surprising that doing more is associated with a longer visit. However, the findings from our study suggest that longer visits and more physician actions are associated with addressing multiple unrelated problems during the patient encounter, which provides a different perspective on the intensity of the physician’s work.23-26
Factors that affect the duration of the visit are of interest to those who use physician productivity as a measure for making policy and management decisions. Primary care physician productivity is commonly defined as the number of patients seen per hour.27,28 Such indicators of productivity would rate a physician who saw many patients in a short time productive, while a physician who provided care to fewer patients but addressed multiple problems would be viewed as less productive. This viewpoint overlooks the cost savings that may result from the reduced number of future visits the patient may require to address these problems, the enhanced quality of care that may be attributable to follow-up of previously identified health concerns, and the enhanced patient satisfaction that may result from the physician’s expanded approach. The current measures of productivity are crude and possibly misleading indicators of the work involved with providing comprehensive primary care to patients. Perhaps health service researchers and policymakers should reconsider the definition of productivity in light of the number of problems addressed or the number of physician actions necessary to address the problems during a patient visit.
Our findings also have implications for evaluating the quality of care provided by family physicians. The current narrowly diseased-focused assessments of quality care are limited because they neglect to take into account the wide range of competing multiple illnesses, prevention, and psychosocial and family context issues confronting family physicians. Quality indicators for primary care should also assess the degree to which family physicians are making the right choices about how to prioritize among the multiple problems that could be addressed during an outpatient visit.
In combination with other reports,29 these data should caution the use of billing records as an indicator of the content of the visit. These data indicate that the billing record generally underrepresents the number of problems addressed during the visit. The lack of concordance between what was observed and what was billed may have several explanations. Underrecording on the billing sheet may be due to the lack of an adequate way to code some problems addressed. Some physicians may approach the completion of the billing sheet by documenting just enough to justify the time spent. Also, the mode of recording the billing (forms or computer programs) may limit the number of problems that can be recorded per visit. Nonconcordance may have also occurred if the physician made decisions about management of ongoing illnesses that were not overtly apparent to the observer.
Limitations
The generalizability of our findings is limited by the modest-sized convenience sample of physicians observed. The higher no-show rate by younger patients may have increased the number of problems seen per visit, since older patients tend to have more problems. However, the patient visits included in our study were randomly selected from all adult patient visits during the 6-week data collection period and were similar in sex to the few patients who were not observed and are likely to be reflective of the patients presenting for care. Although not assessed directly, inter-rater reliability among the 7 students was maximized through the use of videotaped patient encounters for practicing completing the data collection form and for calibrating the observers before data collection in the field.
Conclusions
Prioritizing and delivering a diverse array of services within a relationship context is a hallmark of family practice. Our data suggest that addressing multiple problems during a single outpatient visit is one important mechanism family physicians use to provide comprehensive care. The value of addressing multiple problems per visit in terms of patient satisfaction, cost, and quality of care deserves further investigation.
Acknowledgments
We are grateful to Catharine Symmonds, Catherine Bettcher, Elizabeth Welsh, Tracy Lemonovich, Robin Baines, and Sarah Younkin who contributed to the study design and data collection phase and without whose participation our study would not have been possible. William R. Phillips, MD, MPH, and Kurt C. Stange, MD, PhD, provided valuable suggestions on an earlier draft of this paper.
Related Resources
- Center for Research in Family Practice and Primary Care http://mediswww.cwru.edu/dept/CRFPPC.
- American Academy of Family Practice policy studies in family practice and primary care http://www.aafppolicy.org
1. Stange KC, Jaén CR, Flocke SA, Miller WL, Crabtree BF, Zyzanski SJ. The value of a family physician. J Fam Pract 1998;46:363-68.
2. Jaén CR, Stange KC, Nutting PA. The competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract 1994;38:166-71.
3. Stange KC, Flocke SA, Goodwin MA. Opportunistic preventive service delivery: are time limitations and patient satisfaction barriers? J Fam Pract 1998;46:419-24.
4. Callahan EJ, Jaén CR, Goodwin MA, Crabtree BF, Stange KC. The impact of recent emotional distress and diagnosis of depression or anxiety on the physician-patient encounter in family practice. J Fam Pract 1998;46:410-18.
5. Medalie JH, Zyzanski SJ, Goodwin MA, Stange KC. Two physician styles of focusing on the family. J Fam Pract 2000;49:209-15.
6. Medalie JH, Zyzanski SJ, Langa DM, Stange KC. The family in family practice: is it a reality? Results of a multi-faceted study. J Fam Pract 1998;46:390-96.
7. Flocke SA, Goodwin MA, Stange KC. The effect of a secondary patient on the family practice visit. J Fam Pract 1998;46:429-34.
8. Kravitz RL, Bell RA, Franz CE. A taxonomy of requests by patients (TORP): a new system for understanding clinical negotiation in office practice. J Fam Pract 1999;48:872-78.
9. Marvel MK, Epstein RM, Flowers K, Beckman HB. Soliciting the patient’s agenda: have we improved? JAMA 1999;281:283-87.
10. Korsch B, Putnam SM, Frankel R, Roter D. An overview of research on medical interviewing. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
11. Inui TS, Carter WB. A guide to the research literature on doctor/patient communication. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
12. Callahan EJ, Bertakis KD. Development and validation of the Davis Observation Code. Fam Med 1991;23:19-24.
13. Stange KC, Zyzanski SJ, Jaén CR, et al. Illuminating the black box: a description of 4454 patient visits to 138 family physicians. J Fam Pract 1998;46:377-89.
14. Crabtree BF, Miller WL, Aita V, Flocke SA, Stange KC. Primary care practice organization: a qualitative analysis. J Fam Pract 1998;46:403-09.
15. Miller WL, Crabtree BF. Clinical research: a multimethod typology and qualitative roadmap. In: Crabtree BF, Miler WL, eds. Doing qualitative research. 2nd ed. Thousand Oaks, Calif: Sage; 1999.
16. Institute of Medicine. Primary care: America’s health in a new era. Donaldson YK, Lohr KN, Vanselow NA, eds. Washington, DC: National Academy Press; 1996.
17. Institute of Medicine. Defining primary care: an interim report. Washington, DC: National Academy Press; 1994.
18. Institute of Medicine. Report of a study: a manpower policy for primary health care. Washington, DC: National Academy of Sciences, Institute of Medicine, Division of Health Manpower and Resource Development; 1978.
19. Starfield B. Primary care: concept, evaluation, and policy. New York, NY: Oxford University Press; 1992.
20. Starfield B. Primary care: balancing health needs, services and technology. New York, NY: Oxford University Press; 1998.
21. Rosenblatt RA, Cherkin DC, Schneeweiss R, Hart LG. The content of ambulatory medical care in the United States: an interspecialty comparison. N Engl J Med 1983;309:892-97.
22. Blumenthal D, Causino N, Chang Y, et al. The duration of ambulatory visits to physicians. J Fam Pract 1999;48:264-71.
23. Lasker RD, Marquis MS. The intensity of physicians’ work in patient visits. N Engl J Med 1999;341:337-41.
24. Iezzoni LI. The demand for documentation for Medicare payment. N Engl J Med 1999;341:365-67.
25. Braun P, Dunn DL. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1619-20.
26. Reynolds RD. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1621.
27. Hurdle S, Pope GC. Improving physician productivity. J Ambulatory Care Manage 1989;12:11-26.
28. Camasso MJ, Camasso AE. Practitioner productivity and the product content of medical care in publicly supported health centers. Soc Sci Med 1994;38:733-48.
29. Chao J, Gillanders WR, Flocke SA, Goodwin MA, Kikano GE, Stange KC. Billing for physician services: a comparison of actual billing with CPT codes assigned by direct observation. J Fam Pract 1998;47:28-32.
STUDY DESIGN: Cross-sectional
POPULATION: We studied a total 266 randomly selected adult patient encounters representing 37 physicians.
OUTCOMES MEASURED: A problem was defined as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Visit duration and the number of billing diagnoses were also assessed.
RESULTS: On average, 2.7 problems and 8 physician actions were observed during an encounter. More than one problem was addressed during 73% of the encounters; 36% of these additional problems were raised by the physician and 58% by the patient. On average, each additional problem increased the length of the visit by 2.5 minutes (P <.001). The concordance between the number of problems observed and the number of problems on the billing sheet indicated a trend toward underbilling the number of problems addressed.
CONCLUSIONS: Multiple problems are commonly addressed during family practice outpatient visits and are raised by both the physicians and the patients. Our findings suggest that current views of physician productivity and the billing record are poor indicators of the reality of providing primary care.
Primary care disciplines continue to have a central role in the health care of Americans. They provide breadth of care within an ongoing relationship, bridging the boundaries between health and illness and guiding access to more narrowly focused care when needed.1 The ability to orchestrate a broad health agenda during a visit is central to primary care, but this ability is challenged by competing demands for time.2
Attempts to influence provision of care and treatment decisions by primary care physicians, such as financial incentives, administrative restrictions, and the implementation of evidence-based clinical guidelines add to the demands on physicians’ time and may affect how time is allocated during the day and with each patient. Within this context a primary care physician must prioritize the agenda for each patient visit. This may include providing services beyond the patient’s primary reason for the visit as time permits, such as including preventive services,3 follow-up of acute or chronic illnesses,1 mental health4 or family issues,5-7 or investigating “by the way” patient comments that may indicate serious medical issues.
The competing demands for time are compounded by patient requests during the visit. Based on an audiotape of 139 patient encounters, Kravitz and colleagues8 reported that on average a patient makes 5 requests for physician action or information per visit, and the number of unfulfilled requests was negatively associated with patient satisfaction. Such findings may fuel a sense of pressure to address patient requests. Also, another recent report indicates that the majority of patients do not have the opportunity to express all of their concerns before the physician redirects the interview; once redirected, additional patient concerns are rarely elicited.9 Fitting both the physician’s and patient’s agenda into the time allotted for an outpatient visit has important implications for the duration of the visit, physician productivity, and possibly patient outcomes.
Data on the number of problems raised and addressed have been limited by the lack of appropriate collection methods. Primarily audio and video technology have been used for the study of physician-patient communication.10-12 Direct observation of patient encounters12,13 and incorporation of ethnographic approaches have more recently been employed to fill a large void in the understanding of the content, context, and complexity of primary care.13-15 Findings from the Direct Observation of Primary Care study, which employed such methods, indicate that among 4454 patient visits care was provided to a secondary patient during 18% of the visits and preventive services were addressed during 32% of the illness visits.3 Data from that study provide a glimpse into some types of problems addressed in addition to the main reason for the visit; however, data about the number of problems addressed during patient encounters were not specifically collected by the nurse observer.
When additional issues are raised during a patient encounter, little is known about the nature of these problems, how additional problems affect the duration of the visit, and how well additional problems are reflected in the billing record. This led us to conduct an observational study to ask: How many problems are addressed during family practice outpatient visits, and who is raising additional problems? How much work and time is associated with addressing problems raised beyond the initial problem? How well does the billing list represent the number of problems addressed during the outpatient visit? Our study was designed to directly observe and record how many problems were raised and addressed during outpatient visits to family physicians.
Methods
Seven first-year medical students observed patient care provided by their summer fellowship family physician preceptor and other physicians in the preceptor’s practice from June through August 1999. Six of the sites were located in Northeast Ohio, and one was in Tulsa, Oklahoma.
Each student collected data on one randomly selected adult patient encounter for each half day of precepting. At the beginning of each half-day of patient care the student rolled a die to generate a random number to select a patient from the patient schedule. To ensure random selection of encounters within each half-day session, on alternating days the random number was counted from the beginning or the end of the half-day schedule. If the selected patient was aged younger than 18 years, the patient or physician preferred the encounter not be observed, or the patient did not show up for the scheduled appointment, the next scheduled appointment was selected as a replacement. Patient age and sex were collected for those who were no-shows or chose not to be observed, so they could be compared with those patients who were observed. Each student was to collect data on approximately 50 patient encounters during the 6-week summer fellowship. The physicians were blinded to the study purpose and were not told which patient encounter would be included in the study.
A problem was operationalized as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Each item was listed as it was raised, and the type of problem, who raised it, and what physician actions were involved to address it were coded. Each problem was coded as 1 of 14 categories: acute, acute follow-up, chronic, chronic follow-up, prevention, prevention follow-up, psychosocial, psychosocial follow-up, work-related administrative, health care system-related administrative, other family member’s problem, pregnancy, emergent, and other. The person who raised the problem was coded as 1 of 3 options: the physician, the patient or another person in the room. Multiple physician actions could be coded for how the problem was addressed. The 19 physician action categories included: question, reassurance, examination, procedure, referral, return visit, advice, review tests, order laboratory testing, prescription, provide written material, imaging, admits uncertainty, counseling, return to work/time off work letter, defer, complementary/alternative medicine, ignored or lost, and other.
Patient characteristics, the duration of the visit, and the billing diagnoses for each visit were also recorded on the data collection form. Videotaped encounters were used to pilot test the data collection form, to allow the observers to practice using the form in real time, and to calibrate the observers before data collection in the field.
We used descriptive statistics to address most research questions. Student t tests and chi-square tests were used to compare age and sex differences between participants and nonparticipants. We tested the association of the number of problems with the duration of the visit with analysis of variance and a test for linear trend. A difference score of the number of problems observed and the number of problems recorded on the billing sheet for the encounter was computed and summarized graphically.
Results
We collected usable data on 266 encounters representing 37 physicians. Patient and visit characteristics are displayed in Table 1. The patients had an average age of 48 years, and 69% were women. They were predominately white. A large proportion was observed visiting their regular primary care physician (83%), and 85% were established patients of the practice. Most of the observed patients had some kind of commercial health care insurance, 19% had Medicare, and a small proportion had Medicaid or no insurance. The visit duration ranged from 2 to 65 minutes; the median was 15 minutes with a mean of 19.3 (standard deviation [SD]=12.7). The first problem raised was most commonly an acute problem (49%); prevention and chronic illness were the first problem raised during 21% and 19% of encounters, respectively. Patients who were randomly selected but were not observed (n=52, primarily no-shows) were similar in sex (67% women, c2 =0.119, P=.73 ) but were younger than those patients who were observed (mean age=32.1 years, t=3.79, P=.001).
On average, 2.7 problems were raised during an encounter Table 2. Forty-four percent of all problems were classified as acute, 30% chronic, 14% prevention, 4% administrative, 2% psychosocial, and 6% were classified as other. Of the observed encounters, 73% had more than one problem addressed. The physician raised 36% of these additional problems, and patients raised 58%. The problems raised by physicians were most frequently pertaining to chronic illness, prevention, and follow-up issues. The problems raised by patients were most likely to be acute illness problems. Additional problems were least likely to arise when the first problem addressed was an acute problem (61%) compared with visits during which the first problem addressed was chronic or prevention focused, where 88% and 87%, respectively, included additional problems during the visit (c2=21.2, P <.001).
On average, 8 (SD=4.5) physician actions were observed per encounter Table 2. Physicians performed an average of 3.3 (SD=1.2) actions per problem. The most common physician actions were questioning (77%), physical examination (49%), prescription writing (32%), providing advice (31%), and reassurance (25%). Of the 452 additional problems raised, only 3% of problems were ignored, and 6% were deferred to another visit.
The association of the number of problems addressed with the duration of the visit was assessed by analysis of variance and a test for linear trend. As shown in Figure 1, the duration of the visit increased approximately 2.5 minutes for each additional problem addressed (P <.001 for linear trend). The visit duration within each of the number of problem groups varied greatly as indicated by the large range for each group; however, the SD for each of the groups as indicated by the shaded bars are a similar size for each of the groups (Levene’s test of equality of error variance=1.48, P=.195).
The concordance between the number of problems observed and the number of problems on the billing sheet was modest, with a trend toward billing for fewer problems than were observed. As shown in Figure 2, 29% of encounters represented a match between the number of problems observed and the number of problems on the billing sheet. Fifty-eight percent of the encounters had more problems observed than recorded on the billing sheet. A much smaller proportion of encounters recorded more problems on the billing sheet than were observed during the encounter.
Discussion
Our exploratory study suggests that it is common for multiple problems to be addressed during visits to a family physician regardless of the initial reason for the visit. Additional problems are raised by both physicians and patients and are rarely deferred or ignored by the physician. Although the phenomenon of integrating a broad health agenda and addressing multiple problems during a single outpatient visit may be well known by practicing community-based family physicians, it may not be recognized by policymakers or health services researchers whose window into the process of outpatient care is provided by the medical record and billing data.
Addressing the majority of a patient’s health care needs and providing comprehensive care is a core feature of quality primary care.16-20 Previous work has documented the wide range of diagnoses and clusters of diagnoses that family physicians commonly address during outpatient care.13,21 However, truly comprehensive care goes beyond providing a broad array of services; it also involves the integration of care in a physician-patient relationship context. Prioritizing, providing, and orchestrating care for acute and undifferentiated illness, chronic disease, preventive services, and mental health care represents a key feature of primary care practice such that the care is greater than the sum of its individual commodities.1 These data suggest that single visits often address a broad agenda of health care.
Overall, as the number of problems increase so does the length of the visit. Others have found that ordering or performing more tests, providing preventive services, and conducting ambulatory surgical procedures increase the length of the visit.22 It is not surprising that doing more is associated with a longer visit. However, the findings from our study suggest that longer visits and more physician actions are associated with addressing multiple unrelated problems during the patient encounter, which provides a different perspective on the intensity of the physician’s work.23-26
Factors that affect the duration of the visit are of interest to those who use physician productivity as a measure for making policy and management decisions. Primary care physician productivity is commonly defined as the number of patients seen per hour.27,28 Such indicators of productivity would rate a physician who saw many patients in a short time productive, while a physician who provided care to fewer patients but addressed multiple problems would be viewed as less productive. This viewpoint overlooks the cost savings that may result from the reduced number of future visits the patient may require to address these problems, the enhanced quality of care that may be attributable to follow-up of previously identified health concerns, and the enhanced patient satisfaction that may result from the physician’s expanded approach. The current measures of productivity are crude and possibly misleading indicators of the work involved with providing comprehensive primary care to patients. Perhaps health service researchers and policymakers should reconsider the definition of productivity in light of the number of problems addressed or the number of physician actions necessary to address the problems during a patient visit.
Our findings also have implications for evaluating the quality of care provided by family physicians. The current narrowly diseased-focused assessments of quality care are limited because they neglect to take into account the wide range of competing multiple illnesses, prevention, and psychosocial and family context issues confronting family physicians. Quality indicators for primary care should also assess the degree to which family physicians are making the right choices about how to prioritize among the multiple problems that could be addressed during an outpatient visit.
In combination with other reports,29 these data should caution the use of billing records as an indicator of the content of the visit. These data indicate that the billing record generally underrepresents the number of problems addressed during the visit. The lack of concordance between what was observed and what was billed may have several explanations. Underrecording on the billing sheet may be due to the lack of an adequate way to code some problems addressed. Some physicians may approach the completion of the billing sheet by documenting just enough to justify the time spent. Also, the mode of recording the billing (forms or computer programs) may limit the number of problems that can be recorded per visit. Nonconcordance may have also occurred if the physician made decisions about management of ongoing illnesses that were not overtly apparent to the observer.
Limitations
The generalizability of our findings is limited by the modest-sized convenience sample of physicians observed. The higher no-show rate by younger patients may have increased the number of problems seen per visit, since older patients tend to have more problems. However, the patient visits included in our study were randomly selected from all adult patient visits during the 6-week data collection period and were similar in sex to the few patients who were not observed and are likely to be reflective of the patients presenting for care. Although not assessed directly, inter-rater reliability among the 7 students was maximized through the use of videotaped patient encounters for practicing completing the data collection form and for calibrating the observers before data collection in the field.
Conclusions
Prioritizing and delivering a diverse array of services within a relationship context is a hallmark of family practice. Our data suggest that addressing multiple problems during a single outpatient visit is one important mechanism family physicians use to provide comprehensive care. The value of addressing multiple problems per visit in terms of patient satisfaction, cost, and quality of care deserves further investigation.
Acknowledgments
We are grateful to Catharine Symmonds, Catherine Bettcher, Elizabeth Welsh, Tracy Lemonovich, Robin Baines, and Sarah Younkin who contributed to the study design and data collection phase and without whose participation our study would not have been possible. William R. Phillips, MD, MPH, and Kurt C. Stange, MD, PhD, provided valuable suggestions on an earlier draft of this paper.
Related Resources
- Center for Research in Family Practice and Primary Care http://mediswww.cwru.edu/dept/CRFPPC.
- American Academy of Family Practice policy studies in family practice and primary care http://www.aafppolicy.org
STUDY DESIGN: Cross-sectional
POPULATION: We studied a total 266 randomly selected adult patient encounters representing 37 physicians.
OUTCOMES MEASURED: A problem was defined as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Visit duration and the number of billing diagnoses were also assessed.
RESULTS: On average, 2.7 problems and 8 physician actions were observed during an encounter. More than one problem was addressed during 73% of the encounters; 36% of these additional problems were raised by the physician and 58% by the patient. On average, each additional problem increased the length of the visit by 2.5 minutes (P <.001). The concordance between the number of problems observed and the number of problems on the billing sheet indicated a trend toward underbilling the number of problems addressed.
CONCLUSIONS: Multiple problems are commonly addressed during family practice outpatient visits and are raised by both the physicians and the patients. Our findings suggest that current views of physician productivity and the billing record are poor indicators of the reality of providing primary care.
Primary care disciplines continue to have a central role in the health care of Americans. They provide breadth of care within an ongoing relationship, bridging the boundaries between health and illness and guiding access to more narrowly focused care when needed.1 The ability to orchestrate a broad health agenda during a visit is central to primary care, but this ability is challenged by competing demands for time.2
Attempts to influence provision of care and treatment decisions by primary care physicians, such as financial incentives, administrative restrictions, and the implementation of evidence-based clinical guidelines add to the demands on physicians’ time and may affect how time is allocated during the day and with each patient. Within this context a primary care physician must prioritize the agenda for each patient visit. This may include providing services beyond the patient’s primary reason for the visit as time permits, such as including preventive services,3 follow-up of acute or chronic illnesses,1 mental health4 or family issues,5-7 or investigating “by the way” patient comments that may indicate serious medical issues.
The competing demands for time are compounded by patient requests during the visit. Based on an audiotape of 139 patient encounters, Kravitz and colleagues8 reported that on average a patient makes 5 requests for physician action or information per visit, and the number of unfulfilled requests was negatively associated with patient satisfaction. Such findings may fuel a sense of pressure to address patient requests. Also, another recent report indicates that the majority of patients do not have the opportunity to express all of their concerns before the physician redirects the interview; once redirected, additional patient concerns are rarely elicited.9 Fitting both the physician’s and patient’s agenda into the time allotted for an outpatient visit has important implications for the duration of the visit, physician productivity, and possibly patient outcomes.
Data on the number of problems raised and addressed have been limited by the lack of appropriate collection methods. Primarily audio and video technology have been used for the study of physician-patient communication.10-12 Direct observation of patient encounters12,13 and incorporation of ethnographic approaches have more recently been employed to fill a large void in the understanding of the content, context, and complexity of primary care.13-15 Findings from the Direct Observation of Primary Care study, which employed such methods, indicate that among 4454 patient visits care was provided to a secondary patient during 18% of the visits and preventive services were addressed during 32% of the illness visits.3 Data from that study provide a glimpse into some types of problems addressed in addition to the main reason for the visit; however, data about the number of problems addressed during patient encounters were not specifically collected by the nurse observer.
When additional issues are raised during a patient encounter, little is known about the nature of these problems, how additional problems affect the duration of the visit, and how well additional problems are reflected in the billing record. This led us to conduct an observational study to ask: How many problems are addressed during family practice outpatient visits, and who is raising additional problems? How much work and time is associated with addressing problems raised beyond the initial problem? How well does the billing list represent the number of problems addressed during the outpatient visit? Our study was designed to directly observe and record how many problems were raised and addressed during outpatient visits to family physicians.
Methods
Seven first-year medical students observed patient care provided by their summer fellowship family physician preceptor and other physicians in the preceptor’s practice from June through August 1999. Six of the sites were located in Northeast Ohio, and one was in Tulsa, Oklahoma.
Each student collected data on one randomly selected adult patient encounter for each half day of precepting. At the beginning of each half-day of patient care the student rolled a die to generate a random number to select a patient from the patient schedule. To ensure random selection of encounters within each half-day session, on alternating days the random number was counted from the beginning or the end of the half-day schedule. If the selected patient was aged younger than 18 years, the patient or physician preferred the encounter not be observed, or the patient did not show up for the scheduled appointment, the next scheduled appointment was selected as a replacement. Patient age and sex were collected for those who were no-shows or chose not to be observed, so they could be compared with those patients who were observed. Each student was to collect data on approximately 50 patient encounters during the 6-week summer fellowship. The physicians were blinded to the study purpose and were not told which patient encounter would be included in the study.
A problem was operationalized as an issue requiring physician action in the form of a decision, diagnosis, treatment, or monitoring. Each item was listed as it was raised, and the type of problem, who raised it, and what physician actions were involved to address it were coded. Each problem was coded as 1 of 14 categories: acute, acute follow-up, chronic, chronic follow-up, prevention, prevention follow-up, psychosocial, psychosocial follow-up, work-related administrative, health care system-related administrative, other family member’s problem, pregnancy, emergent, and other. The person who raised the problem was coded as 1 of 3 options: the physician, the patient or another person in the room. Multiple physician actions could be coded for how the problem was addressed. The 19 physician action categories included: question, reassurance, examination, procedure, referral, return visit, advice, review tests, order laboratory testing, prescription, provide written material, imaging, admits uncertainty, counseling, return to work/time off work letter, defer, complementary/alternative medicine, ignored or lost, and other.
Patient characteristics, the duration of the visit, and the billing diagnoses for each visit were also recorded on the data collection form. Videotaped encounters were used to pilot test the data collection form, to allow the observers to practice using the form in real time, and to calibrate the observers before data collection in the field.
We used descriptive statistics to address most research questions. Student t tests and chi-square tests were used to compare age and sex differences between participants and nonparticipants. We tested the association of the number of problems with the duration of the visit with analysis of variance and a test for linear trend. A difference score of the number of problems observed and the number of problems recorded on the billing sheet for the encounter was computed and summarized graphically.
Results
We collected usable data on 266 encounters representing 37 physicians. Patient and visit characteristics are displayed in Table 1. The patients had an average age of 48 years, and 69% were women. They were predominately white. A large proportion was observed visiting their regular primary care physician (83%), and 85% were established patients of the practice. Most of the observed patients had some kind of commercial health care insurance, 19% had Medicare, and a small proportion had Medicaid or no insurance. The visit duration ranged from 2 to 65 minutes; the median was 15 minutes with a mean of 19.3 (standard deviation [SD]=12.7). The first problem raised was most commonly an acute problem (49%); prevention and chronic illness were the first problem raised during 21% and 19% of encounters, respectively. Patients who were randomly selected but were not observed (n=52, primarily no-shows) were similar in sex (67% women, c2 =0.119, P=.73 ) but were younger than those patients who were observed (mean age=32.1 years, t=3.79, P=.001).
On average, 2.7 problems were raised during an encounter Table 2. Forty-four percent of all problems were classified as acute, 30% chronic, 14% prevention, 4% administrative, 2% psychosocial, and 6% were classified as other. Of the observed encounters, 73% had more than one problem addressed. The physician raised 36% of these additional problems, and patients raised 58%. The problems raised by physicians were most frequently pertaining to chronic illness, prevention, and follow-up issues. The problems raised by patients were most likely to be acute illness problems. Additional problems were least likely to arise when the first problem addressed was an acute problem (61%) compared with visits during which the first problem addressed was chronic or prevention focused, where 88% and 87%, respectively, included additional problems during the visit (c2=21.2, P <.001).
On average, 8 (SD=4.5) physician actions were observed per encounter Table 2. Physicians performed an average of 3.3 (SD=1.2) actions per problem. The most common physician actions were questioning (77%), physical examination (49%), prescription writing (32%), providing advice (31%), and reassurance (25%). Of the 452 additional problems raised, only 3% of problems were ignored, and 6% were deferred to another visit.
The association of the number of problems addressed with the duration of the visit was assessed by analysis of variance and a test for linear trend. As shown in Figure 1, the duration of the visit increased approximately 2.5 minutes for each additional problem addressed (P <.001 for linear trend). The visit duration within each of the number of problem groups varied greatly as indicated by the large range for each group; however, the SD for each of the groups as indicated by the shaded bars are a similar size for each of the groups (Levene’s test of equality of error variance=1.48, P=.195).
The concordance between the number of problems observed and the number of problems on the billing sheet was modest, with a trend toward billing for fewer problems than were observed. As shown in Figure 2, 29% of encounters represented a match between the number of problems observed and the number of problems on the billing sheet. Fifty-eight percent of the encounters had more problems observed than recorded on the billing sheet. A much smaller proportion of encounters recorded more problems on the billing sheet than were observed during the encounter.
Discussion
Our exploratory study suggests that it is common for multiple problems to be addressed during visits to a family physician regardless of the initial reason for the visit. Additional problems are raised by both physicians and patients and are rarely deferred or ignored by the physician. Although the phenomenon of integrating a broad health agenda and addressing multiple problems during a single outpatient visit may be well known by practicing community-based family physicians, it may not be recognized by policymakers or health services researchers whose window into the process of outpatient care is provided by the medical record and billing data.
Addressing the majority of a patient’s health care needs and providing comprehensive care is a core feature of quality primary care.16-20 Previous work has documented the wide range of diagnoses and clusters of diagnoses that family physicians commonly address during outpatient care.13,21 However, truly comprehensive care goes beyond providing a broad array of services; it also involves the integration of care in a physician-patient relationship context. Prioritizing, providing, and orchestrating care for acute and undifferentiated illness, chronic disease, preventive services, and mental health care represents a key feature of primary care practice such that the care is greater than the sum of its individual commodities.1 These data suggest that single visits often address a broad agenda of health care.
Overall, as the number of problems increase so does the length of the visit. Others have found that ordering or performing more tests, providing preventive services, and conducting ambulatory surgical procedures increase the length of the visit.22 It is not surprising that doing more is associated with a longer visit. However, the findings from our study suggest that longer visits and more physician actions are associated with addressing multiple unrelated problems during the patient encounter, which provides a different perspective on the intensity of the physician’s work.23-26
Factors that affect the duration of the visit are of interest to those who use physician productivity as a measure for making policy and management decisions. Primary care physician productivity is commonly defined as the number of patients seen per hour.27,28 Such indicators of productivity would rate a physician who saw many patients in a short time productive, while a physician who provided care to fewer patients but addressed multiple problems would be viewed as less productive. This viewpoint overlooks the cost savings that may result from the reduced number of future visits the patient may require to address these problems, the enhanced quality of care that may be attributable to follow-up of previously identified health concerns, and the enhanced patient satisfaction that may result from the physician’s expanded approach. The current measures of productivity are crude and possibly misleading indicators of the work involved with providing comprehensive primary care to patients. Perhaps health service researchers and policymakers should reconsider the definition of productivity in light of the number of problems addressed or the number of physician actions necessary to address the problems during a patient visit.
Our findings also have implications for evaluating the quality of care provided by family physicians. The current narrowly diseased-focused assessments of quality care are limited because they neglect to take into account the wide range of competing multiple illnesses, prevention, and psychosocial and family context issues confronting family physicians. Quality indicators for primary care should also assess the degree to which family physicians are making the right choices about how to prioritize among the multiple problems that could be addressed during an outpatient visit.
In combination with other reports,29 these data should caution the use of billing records as an indicator of the content of the visit. These data indicate that the billing record generally underrepresents the number of problems addressed during the visit. The lack of concordance between what was observed and what was billed may have several explanations. Underrecording on the billing sheet may be due to the lack of an adequate way to code some problems addressed. Some physicians may approach the completion of the billing sheet by documenting just enough to justify the time spent. Also, the mode of recording the billing (forms or computer programs) may limit the number of problems that can be recorded per visit. Nonconcordance may have also occurred if the physician made decisions about management of ongoing illnesses that were not overtly apparent to the observer.
Limitations
The generalizability of our findings is limited by the modest-sized convenience sample of physicians observed. The higher no-show rate by younger patients may have increased the number of problems seen per visit, since older patients tend to have more problems. However, the patient visits included in our study were randomly selected from all adult patient visits during the 6-week data collection period and were similar in sex to the few patients who were not observed and are likely to be reflective of the patients presenting for care. Although not assessed directly, inter-rater reliability among the 7 students was maximized through the use of videotaped patient encounters for practicing completing the data collection form and for calibrating the observers before data collection in the field.
Conclusions
Prioritizing and delivering a diverse array of services within a relationship context is a hallmark of family practice. Our data suggest that addressing multiple problems during a single outpatient visit is one important mechanism family physicians use to provide comprehensive care. The value of addressing multiple problems per visit in terms of patient satisfaction, cost, and quality of care deserves further investigation.
Acknowledgments
We are grateful to Catharine Symmonds, Catherine Bettcher, Elizabeth Welsh, Tracy Lemonovich, Robin Baines, and Sarah Younkin who contributed to the study design and data collection phase and without whose participation our study would not have been possible. William R. Phillips, MD, MPH, and Kurt C. Stange, MD, PhD, provided valuable suggestions on an earlier draft of this paper.
Related Resources
- Center for Research in Family Practice and Primary Care http://mediswww.cwru.edu/dept/CRFPPC.
- American Academy of Family Practice policy studies in family practice and primary care http://www.aafppolicy.org
1. Stange KC, Jaén CR, Flocke SA, Miller WL, Crabtree BF, Zyzanski SJ. The value of a family physician. J Fam Pract 1998;46:363-68.
2. Jaén CR, Stange KC, Nutting PA. The competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract 1994;38:166-71.
3. Stange KC, Flocke SA, Goodwin MA. Opportunistic preventive service delivery: are time limitations and patient satisfaction barriers? J Fam Pract 1998;46:419-24.
4. Callahan EJ, Jaén CR, Goodwin MA, Crabtree BF, Stange KC. The impact of recent emotional distress and diagnosis of depression or anxiety on the physician-patient encounter in family practice. J Fam Pract 1998;46:410-18.
5. Medalie JH, Zyzanski SJ, Goodwin MA, Stange KC. Two physician styles of focusing on the family. J Fam Pract 2000;49:209-15.
6. Medalie JH, Zyzanski SJ, Langa DM, Stange KC. The family in family practice: is it a reality? Results of a multi-faceted study. J Fam Pract 1998;46:390-96.
7. Flocke SA, Goodwin MA, Stange KC. The effect of a secondary patient on the family practice visit. J Fam Pract 1998;46:429-34.
8. Kravitz RL, Bell RA, Franz CE. A taxonomy of requests by patients (TORP): a new system for understanding clinical negotiation in office practice. J Fam Pract 1999;48:872-78.
9. Marvel MK, Epstein RM, Flowers K, Beckman HB. Soliciting the patient’s agenda: have we improved? JAMA 1999;281:283-87.
10. Korsch B, Putnam SM, Frankel R, Roter D. An overview of research on medical interviewing. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
11. Inui TS, Carter WB. A guide to the research literature on doctor/patient communication. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
12. Callahan EJ, Bertakis KD. Development and validation of the Davis Observation Code. Fam Med 1991;23:19-24.
13. Stange KC, Zyzanski SJ, Jaén CR, et al. Illuminating the black box: a description of 4454 patient visits to 138 family physicians. J Fam Pract 1998;46:377-89.
14. Crabtree BF, Miller WL, Aita V, Flocke SA, Stange KC. Primary care practice organization: a qualitative analysis. J Fam Pract 1998;46:403-09.
15. Miller WL, Crabtree BF. Clinical research: a multimethod typology and qualitative roadmap. In: Crabtree BF, Miler WL, eds. Doing qualitative research. 2nd ed. Thousand Oaks, Calif: Sage; 1999.
16. Institute of Medicine. Primary care: America’s health in a new era. Donaldson YK, Lohr KN, Vanselow NA, eds. Washington, DC: National Academy Press; 1996.
17. Institute of Medicine. Defining primary care: an interim report. Washington, DC: National Academy Press; 1994.
18. Institute of Medicine. Report of a study: a manpower policy for primary health care. Washington, DC: National Academy of Sciences, Institute of Medicine, Division of Health Manpower and Resource Development; 1978.
19. Starfield B. Primary care: concept, evaluation, and policy. New York, NY: Oxford University Press; 1992.
20. Starfield B. Primary care: balancing health needs, services and technology. New York, NY: Oxford University Press; 1998.
21. Rosenblatt RA, Cherkin DC, Schneeweiss R, Hart LG. The content of ambulatory medical care in the United States: an interspecialty comparison. N Engl J Med 1983;309:892-97.
22. Blumenthal D, Causino N, Chang Y, et al. The duration of ambulatory visits to physicians. J Fam Pract 1999;48:264-71.
23. Lasker RD, Marquis MS. The intensity of physicians’ work in patient visits. N Engl J Med 1999;341:337-41.
24. Iezzoni LI. The demand for documentation for Medicare payment. N Engl J Med 1999;341:365-67.
25. Braun P, Dunn DL. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1619-20.
26. Reynolds RD. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1621.
27. Hurdle S, Pope GC. Improving physician productivity. J Ambulatory Care Manage 1989;12:11-26.
28. Camasso MJ, Camasso AE. Practitioner productivity and the product content of medical care in publicly supported health centers. Soc Sci Med 1994;38:733-48.
29. Chao J, Gillanders WR, Flocke SA, Goodwin MA, Kikano GE, Stange KC. Billing for physician services: a comparison of actual billing with CPT codes assigned by direct observation. J Fam Pract 1998;47:28-32.
1. Stange KC, Jaén CR, Flocke SA, Miller WL, Crabtree BF, Zyzanski SJ. The value of a family physician. J Fam Pract 1998;46:363-68.
2. Jaén CR, Stange KC, Nutting PA. The competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract 1994;38:166-71.
3. Stange KC, Flocke SA, Goodwin MA. Opportunistic preventive service delivery: are time limitations and patient satisfaction barriers? J Fam Pract 1998;46:419-24.
4. Callahan EJ, Jaén CR, Goodwin MA, Crabtree BF, Stange KC. The impact of recent emotional distress and diagnosis of depression or anxiety on the physician-patient encounter in family practice. J Fam Pract 1998;46:410-18.
5. Medalie JH, Zyzanski SJ, Goodwin MA, Stange KC. Two physician styles of focusing on the family. J Fam Pract 2000;49:209-15.
6. Medalie JH, Zyzanski SJ, Langa DM, Stange KC. The family in family practice: is it a reality? Results of a multi-faceted study. J Fam Pract 1998;46:390-96.
7. Flocke SA, Goodwin MA, Stange KC. The effect of a secondary patient on the family practice visit. J Fam Pract 1998;46:429-34.
8. Kravitz RL, Bell RA, Franz CE. A taxonomy of requests by patients (TORP): a new system for understanding clinical negotiation in office practice. J Fam Pract 1999;48:872-78.
9. Marvel MK, Epstein RM, Flowers K, Beckman HB. Soliciting the patient’s agenda: have we improved? JAMA 1999;281:283-87.
10. Korsch B, Putnam SM, Frankel R, Roter D. An overview of research on medical interviewing. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
11. Inui TS, Carter WB. A guide to the research literature on doctor/patient communication. In: Lipkin M, Putnam S, Lazare A, eds. The medical interview. New York, NY: Springer; 1995.
12. Callahan EJ, Bertakis KD. Development and validation of the Davis Observation Code. Fam Med 1991;23:19-24.
13. Stange KC, Zyzanski SJ, Jaén CR, et al. Illuminating the black box: a description of 4454 patient visits to 138 family physicians. J Fam Pract 1998;46:377-89.
14. Crabtree BF, Miller WL, Aita V, Flocke SA, Stange KC. Primary care practice organization: a qualitative analysis. J Fam Pract 1998;46:403-09.
15. Miller WL, Crabtree BF. Clinical research: a multimethod typology and qualitative roadmap. In: Crabtree BF, Miler WL, eds. Doing qualitative research. 2nd ed. Thousand Oaks, Calif: Sage; 1999.
16. Institute of Medicine. Primary care: America’s health in a new era. Donaldson YK, Lohr KN, Vanselow NA, eds. Washington, DC: National Academy Press; 1996.
17. Institute of Medicine. Defining primary care: an interim report. Washington, DC: National Academy Press; 1994.
18. Institute of Medicine. Report of a study: a manpower policy for primary health care. Washington, DC: National Academy of Sciences, Institute of Medicine, Division of Health Manpower and Resource Development; 1978.
19. Starfield B. Primary care: concept, evaluation, and policy. New York, NY: Oxford University Press; 1992.
20. Starfield B. Primary care: balancing health needs, services and technology. New York, NY: Oxford University Press; 1998.
21. Rosenblatt RA, Cherkin DC, Schneeweiss R, Hart LG. The content of ambulatory medical care in the United States: an interspecialty comparison. N Engl J Med 1983;309:892-97.
22. Blumenthal D, Causino N, Chang Y, et al. The duration of ambulatory visits to physicians. J Fam Pract 1999;48:264-71.
23. Lasker RD, Marquis MS. The intensity of physicians’ work in patient visits. N Engl J Med 1999;341:337-41.
24. Iezzoni LI. The demand for documentation for Medicare payment. N Engl J Med 1999;341:365-67.
25. Braun P, Dunn DL. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1619-20.
26. Reynolds RD. Reimbursement for evaluation and management services. N Engl J Med 1999;341:1621.
27. Hurdle S, Pope GC. Improving physician productivity. J Ambulatory Care Manage 1989;12:11-26.
28. Camasso MJ, Camasso AE. Practitioner productivity and the product content of medical care in publicly supported health centers. Soc Sci Med 1994;38:733-48.
29. Chao J, Gillanders WR, Flocke SA, Goodwin MA, Kikano GE, Stange KC. Billing for physician services: a comparison of actual billing with CPT codes assigned by direct observation. J Fam Pract 1998;47:28-32.