2D Fourier transform

(a) Consider two independent integer-valued random variables, X and Y. Variable X takes on only the values of the eight integers {1, 2,. .. , 8} and does so with uniform probability. Variable Y may take the value of any positive integer k, with probabilities P {Y = k} = 2 (i) Which random variable has greater uncertainty? Calculate both entropies H(X) and H(Y). [3 marks] (ii) What is the joint entropy H(X, Y) of these random variables, and what is their mutual information I(X; Y)? [2 marks] (b) What is the maximum possible entropy H of an alphabet consisting of N different letters? In such a maximum entropy alphabet, what is the probability of its most likely letter? What is the probability of its least likely letter? Why are fixed length codes inefficient for alphabets whose letters are not equiprobable? Discuss this in relation to Morse Code. [5 marks] (c) Explain why the real-part of a 2D Gabor wavelet has a 2D Fourier transform with two peaks, not just one, as shown in the right panel of the figure below. [4 marks] (d) Show that the set of all Gabor wavelets is closed under convolution, i.e. that the convolution of any two Gabor wavelets is just another Gabor wavelet. [Hint: This property relates to the fact that these wavelets are also closed under multiplication, and that they are also self-Fourier. You may address this question for just 1D wavelets if you wish.] [3 marks] (e) Show that the family of sinc functions used in the Nyquist Sampling Theorem, sinc(x) = sin(λx) λx is closed under convolution. Show further that when two different sinc functions are convolved together, the result is simply whichever one of them had the lower frequency, i.e. the smaller λ.