1 Applications of Jensen’s inequality

In this chapter, $h$ denotes the function $h (x) := x \log \frac{1}{x}$ for $x \in [0, 1]$ .

Lemma 1.1 Concavity

✓

$h$ is strictly concave on $[0, \infty)$ .

Proof ▶

Check that $h^{'}$ is strictly monotone decreasing.

Lemma 1.2 Jensen

✓

If $S$ is a finite set, and $\sum_{s \in S} w_{s} = 1$ for some non-negative $w_{s}$ , and $p_{s} \in [0, 1]$ for all $s \in S$ , then

\sum_{s \in S} w_{s} h (p_{s}) \leq h (\sum_{s \in S} w_{s} p_{s}) .

Proof ▶

Lemma 1.3 Converse Jensen

✓

If equality holds in the above lemma, then $p_{s} = \sum_{s \in S} w_{s} h (p_{s})$ whenever $w_{s} \neq 0$ .

Proof ▶

Lemma 1.4 log sum inequality

✓

If $S$ is a finite set, and $a_{s}, b_{s}$ are non-negative for $s \in S$ , then

\sum_{s \in S} a_{s} \log \frac{a_{s}}{b_{s}} \geq (\sum_{s \in S} a_{s}) \log \frac{\sum_{s \in S} a_{s}}{\sum_{s \in S} b_{s}},

with the convention $0 \log \frac{0}{b} = 0$ for any $b \geq 0$ and $0 \log \frac{a}{0} = \infty$ for any $a > 0$ .

Proof ▶

Let $B := \sum_{s \in S} b_{s}$ . Apply Jensen and Lemma 1.1 to show that $\sum_{s \in S} \frac{b_{s}}{B} h (\frac{a_{s}}{b_{s}}) \geq h (\frac{\sum_{s \in S} a_{s}}{B})$ .

Lemma 1.5 converse log sum

✓

If equality holds in Lemma 1.4, then $a_{s} = r \cdot b_{s}$ for every $s \in S$ , for some constant $r \in R$ .

Proof ▶

By the fact that $h$ is strictly concave and the equality condition of Jensen. □