Cauchy-Schwarz Inequality (and Amplification)

Problem: Prove that for vectors v, w in an inner product space, the inequality

\displaystyle |\left \langle v, w \right \rangle | \leq \| v \| \| w \|

Solution: There is an elementary proof of the Cauchy-Schwarz inequality (see the Wikipedia article), and this proof is essentially the same. What makes this proof stand out is its insightful technique, which I first read about on Terry Tao’s blog. He calls it “textbook,” and maybe it is for an analyst, but it’s still very elegant.

We start by observing another inequality we know to be true, that \| v - w \|^2 = \left \langle v - w, v - w \right \rangle \geq 0, since norms are by definition nonnegative. By the properties of a complex inner product we can expand to get

\displaystyle \| v \|^2 - 2 \textup{Re}(\left \langle v,w \right \rangle) + \| w \|^2 \geq 0

or equivalently

\displaystyle \textup{Re}(\left \langle v,w \right \rangle) \leq \frac{1}{2} \| v \|^2 + \frac{1}{2} \| w \|^2

This inequality is close to the one we’re looking for, but ‘weaker’ because the inequality we seek squeezes inside the inequality we have. That is,

\displaystyle \textup{Re}(\left \langle v,w \right \rangle) \leq |\left \langle v, w \right \rangle | \leq \| v \| \| w \| \leq \frac{1}{2} \| v \|^2 + \frac{1}{2} \| w \|^2

The first inequality is trivial (a complex number is always greater than its real part), the second is the inequality we seek to prove, and the third is a consequence of the arithmetic-geometric mean inequality. And so we have an inequality we’d like to “tighten” to get the true theorem. We do this by tightening each side of the inequality separately, and we do each by exploiting symmetries in the expressions involved.

First, we observe that norms of vectors are preserved by (complex) rotations v \mapsto e^{i \theta}v, but the real part is not. Since this inequality is true no matter which vectors we choose, we can choose \theta to our advantage. That is,

\displaystyle \textup{Re}(\left \langle e^{i \theta}v, w \right \rangle) \leq \frac{1}{2} \| e^{i \theta}v \|^2 + \frac{1}{2} \| w \|^2

And by properties of inner products and norms (pulling out scalars) and the fact that |e^{i\theta}| = 1, we can simplify to

\displaystyle \textup{Re}(e^{i \theta}\left \langle v,w \right \rangle) \leq \frac{1}{2}\| v \|^2 + \frac{1}{2} \| w \|^2

where \theta is arbitrary. Since we want to maximize the left hand side as much as possible, we can choose \theta to be whatever is required to make the number real. Then the real part is just the absolute value of the number itself, and we have

\displaystyle \left |\langle v,w \right \rangle | \leq \frac{1}{2} \| v \|^2 + \frac{1}{2} \| w \|^2

Now we tighten the right hand side by exploiting a symmetry in inner products: the transformation (v,w) \mapsto (\lambda v, \frac{1}{\lambda} w) preserves the left hand side (since |\lambda / \bar{\lambda}| = 1) but not the right. And so by the same reasoning, we can transform the above inequality into

\displaystyle \left |\langle v,w \right \rangle | \leq \frac{\lambda^2}{2} \| v \|^2 + \frac{1}{2 \lambda^2} \| w \|^2

And by plugging in \lambda = \sqrt{\| w \| / \| v \|} (indeed, this minimizes the expression for nonzero v,w) we get exactly the Cauchy-Schwarz inequality, as desired.

\square

This technique is termed “amplification” by Tao, and in his blog post he gives quite a few more advanced examples in harmonic and functional analysis (which are far beyond the scope of this blog).   The asymmetrical symmetry we took advantage of is a sort of “arbitrage” (again Terry’s clever choice of words) to take a weak fact and boost it to a stronger fact. And while the details of this proof are quite trivial, the technique of actively looking for one-sided symmetries is difficult to forget.

About these ads

8 thoughts on “Cauchy-Schwarz Inequality (and Amplification)

  1. The post by Tao showed up on my RSS a few days ago, but I was surprised to see that it was written in 2007. Did it pop up in your feed recently as well?

    Some typos:

    - In the paragraph beginning with “Now we tighten…”, the math mode doesn’t show up, and the end of the sentence should be “but not the *right*.”

    - The final choice of lambda should be the reciprocal of what you have written.

    Like

  2. Yeah, I was also surprised to see how long ago it was written (considering that I was not at all interested in math at that time). Thanks for catching those typos!

    Like

  3. typo in the e^iθ step. If you apply the same complex rotation to both v and w, their inner product remains the same. You should rotate just v so as to line up v and w.

    Another way of stating the amplification trick is that for complex numbers z

    Sup_Θ Re( e^iθ ) = abs(z)

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s