What Happened When a Retailer Randomized Stable Scheduling?

The experiment ran as a randomized controlled field experiment across 28 stores in the San Francisco and Chicago metros, 19 assigned to treatment and 9 to control, from November 2015 to August 2016 (Kesavan et al., 2022). It covered roughly 150,000 shifts of about 1,500 employees and was analyzed intent-to-treat with difference-in-differences, meaning stores were compared by the group they were assigned to rather than by how faithfully they carried the changes out. Stores that adopted the bundle loosely are averaged in with stores that adopted it well, which makes the headline number the estimate to quote for the experiment under its actual implementation conditions.

The treatment was a package, not a practice. Treatment stores received fixed shift times, consistent employee-to-shift assignment, core scheduling, part-time-plus with a 20-hour minimum, a shift-swap app, and targeted staffing (Kesavan et al., 2022). Every one of those levers moved at once, on purpose, because the researchers were testing whether a coherent scheduling regime pays rather than which knob is the profitable one. The design can show whether the regime paid, but it cannot isolate which practice did the work.

The peer-reviewed intent-to-treat estimate is a 5.1% gain in store productivity, decomposed into a 3.3% increase in sales and a 1.8% decrease in labor (Kesavan et al., 2022). It averages every assigned store, including those that implemented the bundle unevenly. The same analysis reported a 16.4% treatment-on-treated estimate adjusted for adherence, but that model-dependent estimate describes this experiment and is neither an expected upside nor an upper bound for another operation.

A separate practitioner report on the same experiment, by Williams et al. (2018), circulates far more widely than the journal article and was not peer reviewed. It reports productivity up 5%, equal to $6.20 more revenue per labor hour than control stores, and median sales up 7% in treatment stores. The 7% is a median across treatment stores, while the 3.3% is the peer-reviewed mean effect estimated intent-to-treat, so the figures cannot be swapped for one another or folded into the same productivity decomposition. When someone quotes a 7% sales lift from this experiment, they are quoting the practitioner report rather than the peer-reviewed finding.

Williams et al., 2018 states the result in store-level dollars: treatment stores averaged $4,363 more per week than controls, and median sales in treatment stores rose 7%. Across 19 treatment stores over 35 weeks the report puts added revenue at an estimated $2.9 million, against an out-of-pocket intervention cost of about $31,200. Those are report figures rather than peer-reviewed estimates, and the ratio between them is the reason this experiment gets cited in board decks.

What the experiment did not test is the story people attach to those dollars. The intuitive explanation, that the same people worked the same shifts often enough to know the floor and the stock and each other, is interpretation rather than a tested finding, and it should be labeled as such when it goes in front of a finance team.

The dollar figures themselves do not travel. The $6.20 more revenue per labor hour is denominated in Gap's 2015-16 revenue base, in two high-cost metros, in an apparel format with its own price points and margin structure, so a hospital, a contact center, or a distribution center has no basis for expecting that number. What does travel is the shape of the finding: productivity moved because sales rose while labor hours fell, and that ratio is measurable against any operation's own baseline.

Start with the design. Of the 28 stores, 25 were randomized, with 3 pretest stores carried into treatment by design, so assignment was not perfectly clean. More consequentially, the control group was not a no-intervention baseline: two-week advance notice and the elimination of on-call shifts were rolled out chain-wide to treatment and control stores alike from October 1 2015. The estimate is therefore the marginal effect of five additional practices layered on that base, not stable scheduling measured against chaotic scheduling. An operation still running on-call shifts and one-week notice sits further from the control group than it looks, and this experiment does not price what closing that gap is worth.

The remaining limits govern attribution. Free targeted additional staffing hours, funded centrally, went to 13 of 19 treatment stores, which confounds schedule stability with extra labor input, so some share of the sales gain may have been bought rather than organized. The treatment was a bundle, so no single practice can be credited, and adopting fixed shift times alone gives no claim on a proportional slice of 5.1%. And the setting was one retailer, one apparel format, two metros, nine months, so external validity to healthcare, contact centers, or manufacturing is unestablished rather than merely untested in public.