Discussion: View Thread

Dear Colleagues,

The use of control variables is widespread in entrepreneurship research-virtually every published article includes them. But, there are many gray areas, judgment calls, undisclosed practices, and overall lack of transparency in the inclusion/exclusion of controls and how their usage is reported in published articles. These idiosyncratic practices may account for, at least in part, lack of repeatability of empirical results in entrepreneurship and other fields (e.g., see article in press in SMJ titled "Creating repeatable cumulative knowledge in strategic management" by Bettis, Ethiraj, Gambardella, Helfat, and Mitchell).

To help address concerns about control variable usage, we are making the following article just published in Personnel Psychology available at http://mypage.iu.edu/~haguinis/pubs.html (item #125):

·       Bernerth, J. & Aguinis, H. 2016. A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69: 229-283.

We hope these recommendations will be useful for authors and also for journal editors/reviewers as they evaluate manuscripts reporting the use of control variables. The article's Abstract is below and we look forward to a continued dialogue about these and other important methodological issues with clear consequences for substantive conclusions.

All the best,

--Herman.

Abstract

The use of control variables plays a central role in organizational research due to practical difficulties associated with the implementation of experimental and quasi-experimental designs. As such, we conducted an in-depth review and content analysis of what variables and why such variables are controlled for in ten of the most popular research domains in organizational behavior/human resource management (OB/HRM) and applied psychology. Specifically, we examined 580 articles published from 2003 to 2012 in Academy of Management Journal, Administrative Science Quarterly, Journal of Applied Psychology, Journal of Management, and Personnel Psychology. Results indicate that, across research domains with clearly distinct theoretical bases, the overwhelming majority of the more than 3,500 controls identified in our review converge around the same simple demographic factors (i.e., gender, age, tenure), very little effort is made to explain why and how controls relate to focal variables of interest, and control variable practices have not changed much over the past decade. To address these results, we offer best-practice recommendations in the form of a sequence of questions and subsequent steps that can be followed to make decisions on the appropriateness of including a specific control variable within a particular theoretical framework, research domain, and empirical study. Our recommendations can be used by authors as well as journal editors and reviewers to improve the transparency and appropriateness of practices regarding control variable usage.

Herman Aguinis

John F. Mee Chair of Management

Professor of Organizational Behavior and Human Resources

Founding and Managing Director, Institute for Global Organizational Effectiveness

Indiana University

Kelley School of Business

http://mypage.iu.edu/~haguinis/

GO FROM MOMENT TO MOMENTUM

Hello Fellow Researchers,

As a follow up on Herman's email, I would like to draw your attention to two publications on the use of control variables in entrepreneurship research. These works also provide recommendations for improvement of the use of control variables specific to the current use of control variables in entrepreneurship research.

Schjoedt, L. & Bird, B. (2014). Control variables: use, misuse, and recommended use. In A. Carsrud & M. E. Brännback (Eds.), Handbook of Research Methods and Applications in Entrepreneurship and Small Business (pp. 136-155). Northampton, MA: Edward Elgar.

Schjoedt, L. & Sangboon, K. (2015). Room for improvement: an examination of the use of control variables. International Journal of Management and Business, 6(2), 57-73.

Like Herman, it is our hope that drawing attention to the use of control variables holds potential to improve the use of control variables in future research.

Happy Reading,

Leon

Dear Colleagues,

The use of control variables is widespread in entrepreneurship research-virtually every published article includes them. But, there are many gray areas, judgment calls, undisclosed practices, and overall lack of transparency in the inclusion/exclusion of controls and how their usage is reported in published articles. These idiosyncratic practices may account for, at least in part, lack of repeatability of empirical results in entrepreneurship and other fields (e.g., see article in press in SMJ titled "Creating repeatable cumulative knowledge in strategic management" by Bettis, Ethiraj, Gambardella, Helfat, and Mitchell).

To help address concerns about control variable usage, we are making the following article just published in Personnel Psychology available at http://mypage.iu.edu/~haguinis/pubs.html (item #125):

·       Bernerth, J. & Aguinis, H. 2016. A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69: 229-283.

We hope these recommendations will be useful for authors and also for journal editors/reviewers as they evaluate manuscripts reporting the use of control variables. The article's Abstract is below and we look forward to a continued dialogue about these and other important methodological issues with clear consequences for substantive conclusions.

All the best,

--Herman.

Abstract

The use of control variables plays a central role in organizational research due to practical difficulties associated with the implementation of experimental and quasi-experimental designs. As such, we conducted an in-depth review and content analysis of what variables and why such variables are controlled for in ten of the most popular research domains in organizational behavior/human resource management (OB/HRM) and applied psychology. Specifically, we examined 580 articles published from 2003 to 2012 in Academy of Management Journal, Administrative Science Quarterly, Journal of Applied Psychology, Journal of Management, and Personnel Psychology. Results indicate that, across research domains with clearly distinct theoretical bases, the overwhelming majority of the more than 3,500 controls identified in our review converge around the same simple demographic factors (i.e., gender, age, tenure), very little effort is made to explain why and how controls relate to focal variables of interest, and control variable practices have not changed much over the past decade. To address these results, we offer best-practice recommendations in the form of a sequence of questions and subsequent steps that can be followed to make decisions on the appropriateness of including a specific control variable within a particular theoretical framework, research domain, and empirical study. Our recommendations can be used by authors as well as journal editors and reviewers to improve the transparency and appropriateness of practices regarding control variable usage.

Herman Aguinis

John F. Mee Chair of Management

Professor of Organizational Behavior and Human Resources

Founding and Managing Director, Institute for Global Organizational Effectiveness

Indiana University

Kelley School of Business

http://mypage.iu.edu/~haguinis/

GO FROM MOMENT TO MOMENTUM

************************************** This message is from ENTREP which is sponsored by the Entrepreneurship Division of the Academy of Management. Please do not post messages with attached files. Commercial messages or spammed messages are not allowed on the list. The use of auto-responder "out-of-office" messages may also lead to your removal from the list. You can manage your subscription options, including joining or leaving the list here: http://aomlists.pace.edu/scripts/wa.exe?SUBED1=entrep&A=1 If you have questions or need help, please contact Jeff Pollack (jeff_pollack@ncsu.edu) or Kevin Cox (kcox24@my.fau.edu). Ventures HO!