Digital experimentation methods

1. Estimating Effects of Long-Term Treatments

Shan Huang*, Chen Wang, Yuan Yuan, Jinglong Zhao & Jingjing Zhang
Management Science, 2026 · ACM EC'23
Methods adopted by Tencent & ByteDance. Open-source on GitHub.

Introduces a longitudinal surrogate framework to estimate long-term treatment effects from short-term A/B test data. Validated on two real-world experiments with millions of WeChat users — reducing estimation bias by 59.8% on average over prevailing industry approaches.

2. Enhancing External Validity of Experiments with Ongoing Sampling

Chen Wang, Shan Huang & Shichao Han
ACM EC'24
Validated on 600 A/B tests on WeChat. Widely adopted in Tencent's daily experimentation operations.

A framework that detects shifts in participant composition during experiments and constructs stage-specific estimators. Improves true positive rates by 28–37% and reduces false positives by 17–29%.

AI for marketing decisions

1. LLM-Driven Causal Discovery and Inference: A Multi-Agent Bayesian Framework

Chen Wang , Shan Huang*, Shichao Han and Yong Wang*
Presented at MIT CODE 2025 · KDD 2026 Workshop on AI Data Scientist
Joint work with Tencent. Combines causal inference, Bayesian methods, and LLM reasoning in a closed-loop system.

A multi-agent Bayesian framework that uses LLMs to discover causal mechanisms from A/B tests and improve treatment-effect estimation across heterogeneous user groups. Validated on large-scale WeChat experiments.

2. LLMs in Product Selection for Small E-Commerce

Yi Ji & Shan Huang*
Designed for resource-constrained firms that lack data infrastructure.

An LLM-based framework that helps small businesses make product-selection decisions through comparative reasoning. Outperforms competitive ML models using Google Ads data.

Social media platforms

1. Social Advertising Effectiveness: Evidence from A Large-scale Field Experiment

Shan Huang, Sinan Aral, Yu Hu & Erik Brynjolfsson
Marketing Science, 39(6), 1142-1165, 2020
WeChat's first large-scale field experiment, along which WeChat's first A/B testing system was built.

A field experiment measuring social ad effectiveness across 71 products and 25 categories among 37M+ WeChat users. Status goods show stronger social advertising effects; experience goods do not outperform search goods.

2. Do More "Likes" Lead to More Clicks? Evidence from a Field Experiment on Social Advertising

Shan Huang & Song Lin
Journal of Marketing, 89(5), 88-110, 2024
Reveals the interplay between public and private consumer responses in social advertising.

A field experiment on WeChat Moments showing that displaying the first social cue (friend's like) boosts both liking and clicking, but additional cues increase likes without further increasing clicks — evidence of normative social influence crowding out informational influence.

3. Emotions in Online Content Diffusion

Yifan Yu, Shan Huang, Yuchen Liu & Yong Tan
Information Systems Research, 2025
Uses a newly generated domain-specific emotion lexicon. Spans diffusion structure, individual traits, and tie strength.

Analyzes 387K articles and 6M+ users on WeChat to show how discrete emotions (anxiety, love, surprise vs. anger, sadness, joy) drive differential diffusion patterns across social networks.

4. Algorithmic vs. Friend-based Recommendations in Shaping Novel Content Engagement

Shan Huang, Yi Ji & Leyu Lin
ACM EC'24
Large-scale experimental causal evidence (2M+ users) on how algorithms and peers differ in shaping content novelty.