New Papers and Our First Video Interviews on Experimental Design
Plus a donkey photo
We’re nearing the end of the semester now and I hope you all have the ability to take even a break. When I was an assistant professor (in the Pleistocene Era), teaching four sections of Freshman Management, I always had grand plans to turn in grades on December 23, take the following ten days off, then hit the ground running with around-the-clock research productivity. But usually a gray January came around with me resembling a zombie,1 so eventually I learned to focus on things I was really excited about and not even try the other stuff for those first few weeks of the new year. I’m sure many of you are able to bounce back quicker, and I hope those finishing spring teaching will recover quickly as well. And for those of you who are currently thinking “what are you talking about? My teaching semester is energizing!”—I have a bunch of papers you can review for me this week.
My family did a real spring break last month for the first time since I became EIC, so I was able to mostly step back from editing for a week. Surprisingly, there were no real crises during this period, so thank you all for the reprieve. Chief Editorial Cat Butterburger used his time at the neighbors exploring a potential new submission type for the journal but by the second page decided that “graphic novel papers” were not a good option—even for soccer (football) papers.
Today’s post is going to discuss experimental design, with our first ever video interviews of Hengchen Dai and Oliver Schilke. If you can remember back to December 2023 (Month 10 of the Eras Tour), we published a great Special Issue on Experimental Methods in Organization Theory, which included a paper by the editors presenting a primer on experimental methods. Experimental methods are powerful, but far more complicated than many know. Like great theory, a great experiment might seem elegantly simple after the fact, but the work necessary to get it there can be profound.
We’ve also got four great new papers to introduce here, all on important organizational challenges. Three cover differential career experiences of women: one on patent applications, one response to reports of workplace abuse, and one on how Black women executives navigate workplace relationships. A fourth paper examines the importance of good matches between startups and seed investors, which anyone involved with early-stage companies knows is a crucial yet underappreciated success factor. I hope you enjoy reading them below.
Newly Accepted Papers:
The Role of Reviewer Characteristics on the Diversity of Successful Applicants
Evelina Gavrilova-Zoutman, Steffen Juranek
Organizations rely on reviewers to assess diverse applications, but under high stress or workload, reviewers may rely on biases, potentially reducing diversity. Examining patent applications at the USPTO, we found that female-sounding names were 3.6 percentage points less likely to receive approval, especially under high examiner workload. This highlights the importance of managing reviewer stress to ensure fair and diverse outcomes. Female applicants do not profit from female reviewers in our setting. Hence, a more diverse reviewer pool does not reduce the female disadvantage.
Reject or Protect? Corrective Action in Response to Women’s vs. Men’s Reports of Workplace Abuse
Timothy G. Kundro, Alyssa Tedder-King, Olivia M. Walker, Marissa Shandell
Organizations encourage employees to report abusive behavior, assuming these reports will lead to corrective action. But does the gender of the person making the report influence whether action is taken? Research offers mixed findings: some studies suggest women’s reports are dismissed more often, while others indicate they are taken especially seriously.
Our research clarifies this debate by showing that corroboration—whether others back up the claim—plays a key role. When there’s little corroboration, reports from women are less likely to result in action compared to reports from men. However, when corroboration is strong, reports from women are just as likely—or even more likely—to lead to corrective measures. This pattern is particularly evident when the reporter’s general credibility is unclear.
Across six studies—including an analysis of U.S. government employee data and five experiments—we find that corroboration significantly impacts how abuse reports are handled, regardless of gender.
Jamie Jocelyn Ladge, Keimei Sugiyama, Alexis Nicole Smith, Marla Baskerville Watkins, Pamela Carlton
This study examines how Black women executives navigate workplace relationships. When experiencing relational triggers that emphasize their minority status, these executives engage in perspective-taking and status reflexivity to understand both their partners' views and their own position in interactions. Through introspection, they assess gaps between others' perceptions and their self-perception, choosing to either reduce or maintain these gaps based on the relationship's importance. This decision influences their subsequent emotional and behavioral responses toward interaction partners, shaping future interpretations of workplace interactions. The research advances workplace relationship literature by demonstrating how minority perspective-taking and status reflexivity serve as tools for navigating professional relationships across difference, particularly for those in underrepresented groups whose workplace advancement depends on successful relationship management.
Simone Santamaria, Stefano Breschi
In today's landscape, startups have a variety of funding options, including venture capitalists, business angels, seed incubators, and accelerators. But which is the best? This paper offers a framework to help startup teams identify their ideal seed investor match, based on an analysis of funding choices made by over 12,000 startups. The key insight is that choosing an investor solely based on availability or favorable terms is a mistake, as different investors provide different resources—some offer substantial financial support, while others provide mentorship and training. The optimal choice is to find partners who can create the most incremental value by leveraging each other's unique resources (resource complementarity), rather than simply seeking the most valuable or wealthy partners.
Experiments Are Harder Than You Think
I arrived at Carnegie Mellon in January 2005 with zero experimental training, but landed in the middle of an amazing set of experimental scholars. What I quickly learned was that getting experiments right is super hard. For someone like me trained in econometrics, it was tempting to look at t-tests and chi-squared tests and think experiments were easy. I was lucky to be quickly disabused of this fallacy. The design process is complex, even if the ultimate design might be elegantly simple. So many things can go wrong that destroy validity or identification. The ethical quagmires are many. There is a ton of tacit knowledge that is hard to learn without coursework, reading groups, and mentoring. Once you run the experiment, there’s no fixing any real faults in its design. Even beyond the typical errors of contamination, underpowered samples, weak manipulations, ect., a bunch of unexpected things happen. Field experiments in particular go wrong in the most unexpected ways. I’ve caught an RA faking data, had 5,000 euros stolen, had coauthors threatened by organized crime, and had data permanently embargoed by a foreign government. Most of my field experiments have failed for organizational reasons. Like many, I’ve had a coauthor accused of fraud.
We learn more and more each day that experiments we ran twenty years ago weren’t as scientifically strong as we thought—no different from all of us who ran wild with regressions. So a constant discussion about experimental best practices, principles, pitfalls, and methods. Fortunately, we published a large special issue in December 2023 on Experimental Methods in Organization Theory, and the editors wrote a beautiful primer on experiments that lays out many of these issues in a clear and elegant structure. I would strongly encourage all of you to read it, particularly if you’re considering integrating experimental methods into your research program for the first time. I’ve seen so many experimental papers in our field be DOA because of a fatal flaw in the design, many of them field experiments by successful scholars that cost tens of thousands of dollars and a year of the authors’ lives. Some things we can’t control, but guides like this one help minimize the ones we can.
I was fortunate to have one of the special issue editors, Oliver Schilke of University of Arizona, sit down with me to discuss the primer and experimental methods. The day before, I spoke with Senior Editor Hengchen Dai of UCLA, who uses a wide range of methods from lab and field experiments to regression discontinuity and machine learning. This is our first attempt here at the journal to use video on Substack, so if something doesn’t work right, I’m going to blame it on the cat. And yes, I’m no Barbara Walters (or Zach Galifianakis), but I’m hoping that by the end of year I’ll be able to quote one of my favorite Monte Python lines and say that “I got better.”
Hengchen Dai (UCLA)
Oliver Schilke (Arizona)
That’s it for today. I’ll leave you with the wisdom from this elderly wild donkey I met on vacation, who has cleverly discovered a way to ease his burden in the most subtle of ways.
-Lamar
More like a zombie villager (the librarian obviously) in Minecraft than the super fast ones in The Last of Us.