Abstract
Self-report and mono-method bias often threaten the validity of research conducted in business settings and thus hinder the development of theories of organizational behavior. This paper outlines a conceptual framework for understanding factors that influence the motivation of an employee to bias his or her responses to questions posed by organizational researchers. Using a longitudinal, multitrait-multimethod dataset, we illustrate various aspects of the problem and argue that traditional approaches for controlling self-report bias do not adequately prevent the problem. The results suggest the need for developing a theory of method effects and companion analytic techniques to improve the accuracy of psychological research in business settings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Becker, T. E., & Vance, R. J. (1992). Construct validity of three types of organizational citizenship behavior using a direct product model.Paper presented at the 1992 Academy of Management Meeting, Las Vegas (August).
Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. D. Dunnette & L. M. (Eds.), Handbook of industrial and organizational psychology (pp. 271–326). Newbury Park, CA: Sage.
Browne, M. W. (1984). The decomposition of multitrait-multimethod matrices.British Journal of Mathematical and Statistical Psychology, 37, 1–21.
Browne, M. W. (1993). Models for multitrait-multimethod matrices. In R. Steyer, K.F. Wender, & K.F. Widamen (Eds.), Psychometric Methodology (pp. 570–584). Stuttgart and New York: Gustav Fisher Verlag.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by multitrait-multimethod matrix.Psychological Bulletin,56,81–105.
Crowne, D. P., & Marlowe. D. (1964). The approval motive: Studies in evaluative dependence. New York: Wiley.
Donaldson, S. I. (1993). Effects of lifestyle and stress on the employee and organization: Implications for promoting health at work.Anxiety, Stress, and Coping,6,155–177. 259 STEWART I. DONALDSON AND ELISA J. GRANT-VALLONE
Donaldson, S. I. (1995). Worksite health promotion: A theory-driven, empirically based perspective. In L. R. Murphy, J. J. Hurrel, S. L. Sauter, & G. P. Keita (Eds.), Job stress interventions (pp. 73–90). Washington, DC: American Psychological Association.
Donaldson, S. I., & Blanchard, A. L. (1995). The seven health practices, well-being, and performance at work: Evidence for the value of reaching small and underserved worksites. Preventive Medicine,24,270–277.
Donaldson, S. I., Dent. C. W., Sussman, S., Stoddard, J. J., & Severson, H. H. (1996). The organizational implications of smokeless tobacco use in the Lumber Mill Industry. Addictive Behaviors,21,259–267.
Donaldson, S. I., Ensher, E. A., & Grant-Vallone, E. J. (2000). Longitudinal examination of mentoring relationships on organizational commitment and citizenship behavior. Journal of Career Development, 26(4),233–249.
Donaldson, S. I., Gooler, L. E., & Weiss, R. (1998). Promoting health and well-being through work: Science and practice. In X. B. Arriaga & S. Oskamp (Eds.), Addressing community problems: Psychological research and intervention (pp. 160–194).Newbury Park: Sage.
Donaldson, S. I., Graham, J. W., Piccinin, A. M., & Hansen, W. B. (1995). Resistance-skills training and onset of alcohol use: Evidence for beneficial and potentially harmful effects in public schools and in private catholic schools.Health Psychology,14, 291–300.
Donaldson, S. I., & Klein, D. (1997). Creating healthful work environments for ethnically diverse employees working in small and medium-sized businesses: A non-profit industry/ community/university collaboration model.Employee Assistance Quarterly,13, 17–32.
Donaldson, S. I., Thomas, C. W., & Graham, J. W. (2002). Understanding self-report bias in prevention research.Manuscript under review.
Donaldson, S. I., Thomas, C. W., Graham, Au, J., & Hansen, W. B. (2000). Verifying drug prevention program effects using reciprocal best friend reports.Journal of Behavioral Medicine,23,221–234.
Donaldson, S. I., & Weiss, R. (1998). Health, well-being, and organizational effectiveness in the virtual workplace. In M. Igbaria, & M. Tan,The virtual workplace (pp. 24–44). Harrisburg, PA: Idea Group Publishing.
Ensher, E. A., Grant-Vallone, E. J., & Donaldson, S. I. (2001). Effects of perceived discrimination on job satisfaction, organizational commitment, organizational citizenship behavior, and grievances. Human Resource Development Quarterly, 12, 53–72.
Fowler, F. Jr. (1995). Improving survey questions: Design and evaluation. Thousand Oaks, CA: Sage.
Graham, J. W., & Collins, N. L. (1991). Controlling correlational bias via confirmatory factor analysis of MTMM data. Multivariate Behavioral Research,26,501–523.
Graham, J.W., Collins, N.L., Donaldson, S.I., & Hansen, W.B. (1993). Understanding and controlling for response bias: Confirmatory factor analysis of multitrait-multimethod data. In R. Steyer, K.F. Wender, & K.F. Widamen (Eds.), Psychometric Methodology (pp. 585–590).Stuttgart and New York: Gustav Fisher Verlag.
Howard, H. (1994). Why do people say nasty things about self-reports? Journal of Organizational Behavior,15, 399–404.
Karasek, R. (1985). Job content questionnaire.Department of Industrial systems Engineering, University of Southern California, Los Angeles.
Karasek, R., & Theorell, T. (1990). Healthy work: Stress, productivity, and the reconstruction of working life.New York: Basic Books.
Lifestyle Assessment Questionnaire (1988). LAQ reliability/validity research efforts.Steven Point, WI: National Wellness Institute.
McCall, M. M., & Bobko, P. (1990). Research methods in the service of discovery. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 381–418). Palo Alto, CA: Consulting Psychologists Press, Inc.
McDowell, I. & Newell, C. (1987). Measuring health: A guide to rating scales and questionnaires. New York: Oxford University Press.
Mersman, J. L., & Donaldson, S. I. (2000). Factors affecting the convergence of self-peer ratings on contextual and task performance.Human Performance, 13(3),299–322
Moorman, R. H., & Podsakoff, P. M. (1992). A meta-analytic review and empirical test of 260 JOURNAL OF BUSINESS AND PSYCHOLOGY the potential confounding effects of social desirability response sets in organizational behavior research.Journal of Occupational and Organizational Psychology,65,131–149.
Podsakoff, P.M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects.Journal of Management,12,531–544.
Sacket, P. R., & Larson, J. R. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 419–489).Palo Alto, CA: Consulting Psychologists Press.
Schmitt, N. (1989). Editorial.Journal of Applied Psychology,74,843–845.
Schmitt, N. (1994). Method bias: the importance of theory and measurement.Journal of Organizational Behavior,15,393–398.
Schwartz, N. (1999). Self-reports: How the questions shape the answers.American Psychologist, 54, 93–105.
Shadish, W. R. (1993). Critical multiplism: A research strategy and its attendant tactics. New Directions for Program Evaluation, 60,13–57.
Smith, J. C. (1993). Understanding stress and coping.New York: Macmillan.
Spector, P. E. (1987). Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem? Journal of Applied Psychology,3,438–443.
Spector, P. E. (1994). Using self-report questionnaires in OB research: a comment on the use of a controversial method.Journal of Organizational Behavior,15,385–392.
Spector, P. E., & Brannick, M. T. (1995). The nature and effects of method variance in organizational research. In C. L. Cooper & I. T. Robertson (Eds.), International Review of Industrial and Organizational Psychology (pp. 210–245). New York: Wiley & Sons.
Stone, A.A., Turkkan, J.S., Bachrach, C.A., Jobe, J.B., Kurtzman, H.S. & Cain, V.S. (2000). The science of self-report: Implications for research and practice.Mahwah, NJ: Lawrence Erlbaum Associates.
Williams, L. J., & Brown, B. K. (1994). Method variance in organizational behavior and human resources research: Effects on correlations, path coefficients, and hypothesis testing.Organizational Behavior and Human Decision Processes,58,185–209.
Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in selfreported affect and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74,462–468.
Wyler, A. R., Masuda, M., & Holmes, T. H. (1968). The Seriousness of Illness Rating Scale. Journal of Psychosomatic Research, 11,363–374.
Zerbe, W. J., & Paulhus, D. L. (1987). Socially desirable responding in organizational behavior: A reconception.Academy of Management Review, 12,250–254.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Donaldson, S.I., Grant-Vallone, E.J. Understanding Self-Report Bias in Organizational Behavior Research. Journal of Business and Psychology 17, 245–260 (2002). https://doi.org/10.1023/A:1019637632584
Issue Date:
DOI: https://doi.org/10.1023/A:1019637632584