Validating the Adult Primary Care Assessment Tool

,; ,; ,

Original Research

Validating the Adult Primary Care Assessment Tool

J Fam Pract. 2001 February;50(2):161

References

1. Starfield B. Balancing health needs, services, and technology. Oxford, England: Oxford University Press; 1998.

2. Bindman AB, Grumback K, Osmond D, et al. Primary care and receipt of preventive services. J Gen Intern Med 1996;11:269-76.

3. Roos N. Who should do the surgery? Tonsillectomy and ade-noidectomy in one Canadian province. Inquiry 1979;16:7383.-

4. Shi L. The relation between primary care and life chances. J Health Care Poor Underserved 1992;3:321-35.

5. Starfield B. Primary care: is it essential? Lancet 1994;344:1129-33.

6. Shi L. Primary care, specialty care, and life chances. Int J Health Serv 1994;24:431-58.

7. Greenfield S, Rogers W, Mangotich M, et al. Outcomes of patients with hypertension and non-insulin-dependent diabetes mellitus treated by different systems and specialties: results from the Medical Outcomes Study. JAMA 1995;274:1436.-

8. Lohr KN, Brooke RH, Kamberg CJ, et al. Use of medical care in the Rand health insurance experiment: diagnosis and service specific analyses in a randomized controlled trial. Med Care 1986;24:S1-87.

9. Goldberg GA, Newhouse JP. Effects of cost sharing on physiological health, health practices, and worry. Health Serv Res 1987;22:279-306.

10. Newhouse JP and the Health Insurance Group. Free for all? Lessons from the Rand Health Insurance Experiment. Cambridge, Mass: Harvard University Press; 1993.

11. Starfield B. Effectiveness of medical care: validating clinical wisdom. Baltimore, Md: the Johns Hopkins University Press; 1985.

12. Franks P, Fiscella K. Primary care physicians and specialists as personal physicians: health care expenditures and mortality experience. J Fam Pract 1998;47:105-09.

13. Farmer FL, Stokes CS, Fisher RH. Poverty, primary care and age-specific mortality. J Rural Health 1991;7:153-69.

14. Shi L, Starfield B, Kennedy B, Kawachi I. Income inequality, primary care, and health indicators. J Fam Pract 1999;48:275-84.

15. Shi L, Starfield B. Primary care, income inequality, and self-rated health in the US: mixed-level analysis. Int J Health Serv 2000;30:541-55.

16. Shi L, Starfield B. Income inequality, and racial mortality in US Metropolitan areas. Am J Public Health. In press.

17. Starfield B. Primary care: concept, evaluation, and policy. Oxford, England: Oxford University Press; 1992.

18. Flocke SA, Stange KC, Zyzanski S. The association of attributes of primary care with the delivery of clinical preventive services. Med Care 1998;36:AS21-30.

19. Starfield B, Cassady C, Nanda J, Forrest CB, Berk R. Consumer experiences and provider perceptions of the quality of primary care: implications for managed care. J Fam Pract 1998;46:216-26.

20. Safran DG, Kosinski M, Tarlov AR, et al. The primary care assessment survey: test of data quality and measurement performance. Med Care 1998;36:728-39.

21. Bindman AB, Grumback K, Osmond D, et al. Primary care and receipt of preventive services. J Gen Intern Med 1996;11:269-76.

22. Green LA. Science and the future of primary care. J Fam Pract 1996;42:119.-

23. Grumbach K. Separating fad from fact: family medicine, primary care, and the role of health services research. J Fam Pract 1996;43:30.-

24. Donaldson MS, Vanselow NA. The nature of primary care. J Fam Pract 1996;42:113.-

25. Safran DG, Tarlov AR, Rogers WH. Primary care performance in fee-for-service and prepaid health care systems: results from the Medical Outcomes Study. JAMA 1994;271:1579.-

26. Forrest CB, Starfield B. Entry into primary care and continuity: the effects of access. Am J Public Health 1998;88:1330-36.

27. Shi L. Experience of primary care by racial and ethnic groups in the US. Med Care 1999;37:1068-77.

28. Shi L. Type of health insurance and quality of primary care experience. Am J Public Health 2000;90:1848-55.

29. Flocke SA. Measuring attributes of primary care: development of a new instrument. J Fam Pract 1997;45:64-74.

30. Cassady C, Starfield B, Hurtado MP, Berk R, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. J Ambulatory Pediatric Assoc 2000;105:998-1003.

31. Shi L, Huang Y, Kelly K, Zhao M, Solomon SL. Gastrointestinal symptoms and use of medical care associated with child day care and health care plan among preschool children. Pediatr Infect Dis J 1999;18:596-603.

32. Shi L, Ning L, Huang Y, Kelly K, Zhao M. Respiratory symptoms and use of medical care associated with child day care and health care plan among preschool children. J SC Med Assoc. In press.

33. Institute of Medicine. Amanpower policy for primary health care. IOM publication 78-02. Washington, DC: National Academy of Sciences; 1978.

34. Institute of Medicine. Defining primary care: an interim report. Washington, DC: National Academy Press; 1994.

35. Fayers PM, Hard DJ. Factor analysis, causal indicators and quality of life. Quality Life Res 1997;6:139-50.

36. Norman GR, Streiner DL. Biostatistics: the bare essentials. St. Louis, Mo: Mosby; 1994.

37. Hatcher L. Astep-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute; 1994:57-127.

38. Devellis RF. Scale development: theory and applications. Newbury Park, Calif: Sage; 1991.

39. Likert R. Atechnique for the measurement of attitudes. Arch Psychol 1932;140:1.-

Analysis

The purpose of the validation was to assess the congruence between the theoretically derived measures and the empiric results in terms of the underlying structure of the principal primary care domains. Although conceptual framework was relied on in the construction of primary care measures, empiric validation was used to reduce the number of items so that the questionnaire became more concise.

The validation of PCAT-AE with the South Carolina sample involved several steps. First, principal component factor analysis was used to explore the structure of the PCAT-AE items and examine its construct validity by determining if the items fell into the hypothesized scales (factors; definitions of measurement-related concepts used in this paper can be found in the Appendix). Factor analysis was also used for item selection and placement into scales based on the pattern of the factor loadings.³⁵ Four criteria were used in deleting items and the determination of the final factors.^36-37 Afactor loading greater than 0.35 was considered meaningful and used as a criterion for retaining items. In addition, each retained factor should have at least 3 items with loadings greater than 0.35. All retained items should share the same conceptual meaning or construct. Also, all retained items should not have secondary loadings greater than 0.35.

Second, internal consistency reliability of the primary care scales was assessed by Cronbachs coefficient alpha (a)³⁸ and item-total correlation for items in each domain. Cronbachs coefficient alpha is based on the covariance among individual items in a scale and the number of items. It ranges from 0, indicating total lack of consistency, to 1, indicating complete internal consistency reliability. The item-total correlation is the correlation between an individual item and the sum of the remaining items that constitute the scale. If an item-total correlation is small, the item is not considered to be measuring the same construct that is measured by the other items in the scale. All the retained items exceeded the minimum acceptable item-total correlation of 0.30.³⁸

Third, the Likert scaling assumptions were tested for the final items related to the primary care scales. Likerts method of summated rating scales is based on the assumption that item responses in each scale can be summed without standardization or weighting.³⁹ The underlying assumptions that must be met include: (1) item-convergent validity (tested by item-scale correlations); (2) item-discriminant validity (tested using the scaling success rate, ie, correlation of each item with other items within the same scale is greater than with items from different scales); (3) equal item variance (tested by examining item means and standard deviations and the equivalence of the intraclass correlation and Scotts homogeneity ratio for each scale); (4) equal item-scale correlation (tested by examining the range of item-scale correlations); and (5) score reliability (tested by Cronbachs coefficient a.

Fourth, descriptive statistics were performed for the revised primary care scales, including mean, standard deviation, range, percentile, skewness, kurtosis, and interscale correlation. Since respondents who never saw a specialist did not answer the coordination questions, analyses were performed both with and without those questions, including the coordination domain.

Results

Subjects

For the HMO group, a total of 350 individuals responded after 3 mailings. Excluding the nonresponses due to wrong addresses and changed plans (n=340), the effective response rate was 53 percent (350/660). The respondents and nonrespondents were not significantly different in age, sex, race, and zip codes of mailing addresses. For the CHC group, a total of 1000 individuals were systematically selected and approached. Among them, 265 refused to be interviewed, 195 were not able to complete the interview prior to their appointment, and 540 completed the interview. Taking only refusal into account, the response rate was 67% (540/540+265). Men were more likely to refuse the interview than women. There were no significant differences in age and race between respondents and nonrespondents. All interviews were conducted by graduate public health students trained in interactive sessions and were completed in 1999.

The sample included 823 adults with an identified usual source of care. Among them, most (69% of HMO and 60% of CHC respondents) indicated a strong affiliation with their usual source of care (ie, all 3 doctors/places were the same). Very few (0.6% of HMO and 1.2% of CHC respondents) indicated the weakest affiliation with their usual source of care (ie, all 3 responses were different). Just over half of respondents (56%) were non-white (primarily black). Over half (55%) had an annual household income under $25,000. Most respondents (76%) had health insurance coverage all year and had been seeing their regular source of care for more than 1 year (82%). Sixty-three percent had seen their regular source of care for more than 2 years. The majority chose their own usual source of care (78%) and did not have trouble paying for their health care (74%). More than half of the respondents made at least 1 visit to a specialist (56%). This relatively high rate may be due to a somewhat elderly sample; more than 20% of the respondents were older than 65 years.