1 Applications of Jensen’s inequality

In this chapter, \(h\) denotes the function \(h(x) := x \log \frac{1}{x}\) for \(x \in [0,1]\).

Lemma 1.1 Concavity

✓

\(h\) is strictly concave on \([0,\infty )\).

Proof ▶

Check that \(h'\) is strictly monotone decreasing.

Lemma 1.2 log sum inequality

✓

If \(S\) is a finite set, and \(a_s,b_s\) are non-negative for \(s\in S\), then

\[ \sum _{s\in S} a_s \log \frac{a_s}{b_s}\ge \left(\sum _{s\in S}a_s\right)\log \frac{\sum _{s\in S} a_s}{\sum _{s\in S} b_s}, \]

with the convention \(0\log \frac{0}{b}=0\) for any \(b\ge 0\) and \(0\log \frac{a}{0}=\infty \) for any \(a{\gt}0\).

Proof ▶

Let \(B:=\sum _{s\in S} b_s\). Apply Jensen and Lemma 1.1 to show that \(\sum _{s\in S} \frac{b_s}{B} h(\frac{a_s}{b_s})\ge h(\frac{\sum _{s\in S} a_s}{B})\).

Lemma 1.3 converse log sum

✓

If equality holds in Lemma 1.2, then \(a_s=r\cdot b_s\) for every \(s\in S\), for some constant \(r\in \mathbb {R}\).

Proof ▶

By the fact that \(h\) is strictly concave and the equality condition of Jensen.