Seller Willingness to Disintermediate: Evidence from a Mystery Shopper Experiment on a Service Platform
Disintermediation poses a major threat to online platforms but remains understudied empirically due to measurement challenges. We address this gap by conducting a “mystery shopper” style field experiment to investigate sellers’ responses to buyer-initiated disintermediation on a service platform. Independent contractors acting as undercover buyers contacted sellers under experimentally varied conditions: (1) whether communication is monitored by the platform and (2) the complexity of the requested task. Relative to the baseline of monitored communication with low task complexity, unmonitored communication significantly increased seller willingness to disintermediate by up to 21.3 percentage points, while higher task complexity had no significant average effect. We then apply the causal forest algorithm to estimate the conditional average treatment effects for each individual observation. Communications being unmonitored universally induce higher levels of disintermediation for all sellers, albeit with varying magnitudes. However, higher task complexity produces opposing individual-level effects, motivating some sellers while deterring others, which reconciles the insignificant average effect. Second-stage analyses reveal that sellers with paid VIP membership or higher platform tiers are less responsive to the treatments, while sellers with longer tenure and more recent sales are more prone to disintermediate. Our results provide actionable insights for online platforms to develop targeted interventions to more effectively prevent disintermediation.
The Question: Will Sellers Go Off-Platform?
Platforms make money by matching buyers and sellers and taking a commission. Disintermediation happens when the two meet on the platform and then finish the deal somewhere else, cutting the platform out. It is one of the biggest threats platforms face.
It is also very hard to study. By definition, it happens outside the platform’s data. And even when a platform suspects it, observational data cannot tell whether the buyer or the seller pushed for it.
We ask: when a buyer proposes moving a deal off-platform, how willing are sellers to go along? And which sellers are most willing?
A Mystery Shopper Experiment
We ran a field experiment on one of the largest task-outsourcing platforms in China, similar to Fiverr or Upwork, in the logo-design category, where the platform charges a 20% commission.
Independent contractors posed as undercover buyers. Each one contacted a seller, discussed a logo-design job, and at the end of the conversation asked to complete the transaction off-platform. We recorded whether the seller agreed. No real transactions took place.
We varied two things in a 2 × 2 design:
| Simple task (text logo) | Complex task (cartoon logo) | |
|---|---|---|
| Monitored (platform’s own messaging) | Baseline | Higher complexity |
| Unmonitored (WeChat) | Unmonitored | Both |
The experiment ran over four weeks in April 2022.
What We Find
1. Unmonitored chats make sellers far more willing to go off-platform
Relative to the baseline (a monitored chat about a simple task), moving the conversation to an unmonitored channel raised sellers’ willingness to disintermediate by up to 21.3 percentage points. In raw terms, about 43% of sellers agreed in the baseline, compared with about 62–63% when the chat was unmonitored.
2. Task complexity cuts both ways
On average, a more complex task had no significant effect. But that average hides two opposing groups: complexity motivated some sellers and deterred others.
The causal forest makes this visible by estimating the effect for every observation:
Every seller responds positively to the absence of monitoring, though by different amounts. The effects of complexity, in contrast, fall on both sides of zero.
3. Who disintermediates?
- Sellers with paid VIP membership or higher platform tiers respond less. They have more to lose if caught.
- Sellers with longer tenure and more recent sales are more prone to disintermediate.
4. Existing detection misses it
The platform’s own detection algorithm flagged none of these conversations. A simple keyword rule would have flagged every conversation that moved to WeChat, even though only about 63% of those sellers actually agreed to go off-platform.
Why It Matters
Our results point to practical, targeted ways for platforms to fight disintermediation:
- Make on-platform communication the easiest option, for example by masking outside contact details and closing the feature gap with external messaging apps.
- Target monitoring where it matters. High-status sellers (VIP or tiered) respond least, so monitoring can focus on long-tenure, high-sales sellers without that status.
- Run regular mystery-shopper audits. Besides measuring the problem, they produce labelled data for training better detection algorithms.