Slide 8
Slide 8 text
Rademacher complexity
⊚ Consider a sample of 𝑛 instances: 𝑥1, … , 𝑥𝑛 drawn i.i.d. from 𝑃𝑋 .
⊚ Generate 𝑛 random variables 𝜎1, … , 𝜎𝑛 ∈ {−1, +1}.
Definition (Rademacher complexity)
For a set of real-valued functions F with input space X, a distribution 𝑃𝑋
on X, and sample size 𝑛,
the Rademacher complexity 𝑅(F, X, 𝑃𝑋
, 𝑛) is
𝑅(F, X, 𝑃𝑋
, 𝑛) = 𝔼 𝑥1
,…,𝑥𝑛
∼𝑃𝑋
𝜎1
,…,𝜎𝑛
∼𝐵𝑒𝑟(1/2)
[sup
𝑓 ∈F
|
2
𝑛
𝑛
∑
𝑖=1
𝜎𝑖
𝑓 (𝑥𝑖
)|] , (1)
where 𝜎1
… , 𝜎𝑛
∼ 𝐵𝑒𝑟(1/2) with values ±1.
5 ʢ 23