The questionnaire was composed of 2 parts. Part 1 contained 40 single-item questions on physicians’ perceptions of guideline recommendations for cancers of interest, and the perceived influence of various factors on their decision to order screening tests (all factors identified in the literature and in our previous study). The questionnaire also contained items on practice characteristics, demographics, and respondents’ personal experience with cancer or cancer screening tests.
Part 2 contained 6 clinical case vignettes; 2 for prostate specific antigen (PSA), 2 for mammography, and 2 for fecal occult blood testing (FOBT) and colonoscopy, for which recommendations can be unclear according to Canadian guidelines, or conflicting.2 As for screening for prostate cancer with PSA for men over age 50, there is fair evidence for when not to screen, but conflicting recommendations from at least 2 major organizations. Mammography for breast cancer screening in women age 40 to 49 has conflicting recommendations (different recommendations from at least 2 different organizations). FOBT and colonoscopy for colorectal cancer detection over age 40 are unclear C recommendations (insufficient evidence to either recommend or not).
The Clinical Case Vignettes
Clinical case vignettes have been shown to be a useful, inexpensive, and effective method for eliciting physicians’ decision-making behavior in a simulated situation.7 Case vignettes have been used to examine physicians’ practice behavior with cancer patients.8-11For research purposes, the usefulness of the clinical case vignette rests on the ability to vary specific factors (relevant independent variables under study) from one vignette to another, while keeping constant the surrounding factors of the case presented (the frame).
For each clinical case vignette, the dependent variable was the physician’s decision to order the screening test presented or not. The independent variables were the 4 most influential factors identified in the prior qualitative study,4 embedded within the description of each clinical case vignette. Each independent variable had 2 levels: presence or absence of patient anxiety, patient expectations for testing, and family history of cancer, and easy or difficult relationship. This enabled 16 different versions of each clinical case vignette frame, and 2 frames were developed for each cancer screening. The clinical case vignettes were developed and tested in 4 steps. First, 6 investigators (R.G., F.T., C.H., A.K., M.O., J.B.B.) generated case vignettes from their own clinical experience that reflected specified levels of the factors. Second, 12 family physician colleagues empirically validated the descriptions in the case vignettes. A minimum of 9 of these physicians had to correctly identify the intended level of each of the factors in question. Third, factors not attracting 75% agreement were corrected or replaced. Fourth, the modified clinical case vignettes were submitted to another group of 12 family physicians for their perceptions of the intended levels of the relevant factors. The final versions of the vignettes reflected concordance between the perceived and the intended levels in the factors for each case vignette. Figure 1.
Our design was “fractional” in the sense that we sampled from only a fraction of all possible combinations of independent variables. We had estimated that each family physician could respond to no more than 6 case vignettes (2 per cancer screening). The reduced set represented the vignettes that were clinically realistic. As a result, each physician received set of 6 vignettes offering a clinically meaningful spread of possible levels of the independent variables to maximize the opportunity to detect practice behavior variation. The clinical case vignettes were presented in random order to avoid sequence bias. This design ensured that each physician had 1 vignette with all independent factors absent, 1 with all factors present, and the remaining 4 with a diversity of the possible combinations of levels of patient factors.
Analyses
The analysis of the binary response for each test (order versus not order screening test) included the factors listed in the theoretical framework, with an additional random effect to take into account possible correlations among responses to 2 vignettes from the same physician. The estimation of each model’s parameters was performed using the Generalized Estimating Equation approach of SAS; this variant of logistic regression accounts for the non-independence of observations. We examined first the effect of each of the 4 principal factors-individually and together-on the decision to order a screening test. Then we looked for additional significant effects of physician demographics and perceptions of guidelines. We looked for interactions between the quality of the relationship with other factors. For each screening test, we developed a final parsimonious model which included all factors that were statistically significant at P = 0.05.
Results
Of the original 600 physicians, there were 351 respondents, 214 non-respondents, and 35 ineligibles (16 were not in full-time practice, defined as < 15 hours a week; 8 were not practicing; 6 were in another specialty; 4 had moved out of the jurisdiction; and 1 had died). The final response rate was 62.1% (351/565). The respondents’ demographic characteristics Table 1 reflected the Canadian family physician population, except that there were more certificants of the College of Family Physicians of Canada (akin to Board certification in the US) among the respondents.