A CUSPIDALITY CRITERION FOR THE FUNCTORIAL PRODUCT ON GL(2) × GL(3), WITH A COHOMOLOGICAL

APPLICATION

DINAKAR RAMAKRISHNAN AND SONG WANG

1. Introduction

A strong impetus for this paper came, at least for the first author, from a question of Avner Ash, asking whether one can construct non-selfdual, nonmonomial cuspidal cohomology classes for suitable congruence subgroups Γ of SL(n, ℤ), say for n = 6. Such a construction, in special examples, has been known for some time for n = 3 ([AGG1984], [vGT1994], [vGKTV1997], [vGT2000]); it is of course not possible for n = 2. One can without trouble construct non–selfdual, monomial classes for any n = 2m with m ≥ 2, not just for constant coefficients (see the Appendix, Theorem E). In the Appendix we also construct non-monomial, non-selfdual classes for n = 4 using the automorphic induction to ℚ of suitable Hecke character twists of non-CM cusp forms of “weight 2” over imaginary quadratic fields, but they admit quadratic self-twists and are hence imprimitive. The tack pursued in the main body of this paper, and which is the natural thing to do, is to take a non-selfdual (non-monomial) n = 3 example π, and take its functorial product ⊠ with a cuspidal π′ on GL(2)/ℚ associated to a holomorphic newform of weight 4 for a congruence subgroup of SL(2, ℤ). The resulting (cohomological) n = 6 example can be shown to be non-selfdual for suitable π′. (This should be the case for all π′, but we cannot prove this with current technology – see Remark 4.1.) Given that, the main problem is that it is not easy to show that such an automorphic tensor product Π := π ⊠ π′, whose modularity was established in the recent deep work of H. Kim and F. Shahidi ([KSh2002-1]), is cuspidal. This has led us to prove a precise cuspidality criterion (Theorem A) for this product, not just for those of cohomological type, which hopefully justifies the existence of this paper. The second author earlier proved such a criterion when π is a twist of the symmetric square of a cusp form on GL(2) ([Wa2003]; such forms are essentially selfdual, however, and so do not help towards the problem of constructing non-selfdual classes. One of the reasons we are able to prove the criterion in general is the fact that the associated, degree 20 exterior cube L-function is nicely behaved and analyzable. This helps us rule out, when the forms on GL(2) and GL(3) are non-monomial, the possible decomposition of Π into an isobaric sum of two cusp forms on GL(3) (see section 7). This is the heart of the matter.

We will also give a criterion (Theorem B) as to when the base change of Π to a solvable Galois extension remains cuspidal. We will derive a stronger result for the cohomological examples (Theorem C), namely that each of them is primitive, i.e., not associated to a cusp form on GL(m)/K for any (possibly non-normal) extension K/ℚ of degree d > 1 with dm = 6. Furthermore, each of the three main non-selfdual GL(3) examples π of [vGT1994], [vGKTV1997] and [vGT2000] comes equipped, confirming a basic conjecture of Clozel ([C1988]), with a certain 3-dimensional ℓ-adic representation ρ_{ℓ} whose Frobenius traces a_{p}(ρ_{ℓ}) agree with the Hecke eigenvalues a_{p}(π) for small p. For π′ on GL(2)/ℚ defined by a suitable holomorphic newform of weight 4, with associated Galois representation ρ′_{ℓ}, we will show (Theorem D) that the six-dimensional R_{ℓ} := ρ_{ℓ} ⊗ ρ′_{ℓ}, which should correspond to Π, remains irreducible under restriction to any open subgroup of Gal(\overline{ℚ}/ℚ).

The first author would like to thank Avner Ash for his question and for making comments on a earlier version of this paper, Mahdi Asgari for initially kindling his interest (at Park City, UT) in the problem of establishing a precise cuspidality criterion for the Kim-Shahidi product, and the National Science Foundation for financial support through the grant DMS-0100372. The second author would like to thank James Cogdell and Henry Kim for their interest in his lecture on this work at the Fields Institute Workshop on Automorphic L-functions in May 2003.

2. The Cuspidality Criterion

Throughout this paper, by a cusp form on GL(n) (over a global field F) we will mean an irreducible, cuspidal automorphic representation π = π_{∞} ⊗ π_{f} of GL_{n}(𝔸_{F}). We will denote its central character by ω_{π}. One says that π is essentially self-dual iff its contragredient π^{∨} is isomorphic to π ⊗ ν for some character ν of (the idele classes of) F; when n = 2, one always has π^{∨} ≃ π ⊗ ω_{π}^{−1}. Note that π is unitary iff π^{∨} is the complex conjugate representation \overline{π}. Given any cusp form π, we can find a real number t such that π_{u} := π ⊗ |.|^{t} is unitary.

For any cusp form π′ on GL(2), put Ad(π′) = sym^{2}(π′)⊗ω_{π′}^{−1} and A^{4}(π′) = sym^{4}(π′) ⊗ ω_{π′}^{−2}. Recall that π′ is dihedral iff it admits a self-twist by a quadratic character; it is tetrahedral, resp. octahedral, iff sym^{2}(π′), resp. sym^{3}(π′), is cuspidal and admits a self-twist by a cubic, resp. quadratic, character. (The automorphy of sym^{3}(π′) was shown by Kim and Shahidi in [KSh2002-1].) We will say that π′ is of solvable polyhedral type iff it is dihedral, tetrahedral or octahedral.

Theorem A. Let π′, π be cusp forms on GL(2), GL(3) respectively over a number field F. Then the Kim–Shahidi transfer Π = π ⊠ π′ on GL(6)/F is cuspidal unless one of the following happens:

(a) π′ is not dihedral, and π is a twist of Ad(π′);

(b) π′ is dihedral, L(s, π) = L(s, χ) for an idele class character χ of a cubic, non-normal extension K of F, and the base change π_{K} is Eisensteinian.

Furthermore, when (a) (resp. (b)) happens, Π possesses an isobaric decomposition of type (2, 4) or (2, 2, 2) (resp. of type (3, 3)). More precisely, when we are in case (a), Π is of type (2, 2, 2) if π′ is tetrahedral, and (2, 4) otherwise.

Remark: By [KSh2002-1], Π = π ⊠ π′ is automorphic on GL(6)/F, and its L–function agrees with the Rankin–Selberg L–function L(s, π × π′). Theorem A implies in particular that Π is cuspidal if (i) π′ is not dihedral and (ii) π is not a twist of Ad(π′).

A partial cuspidality criterion was proved earlier by the second author in [Wa2003]; but he only treated the case when π is twist equivalent to the Gelbart–Jacquet symmetric square transfer of some cusp form on GL(2).

Theorem B. Let F be a number field and π′, π be cusp forms on GL(2)/F, GL(3)/F respectively. Put Π = π ⊠ π′. Assume that π′ is not of solvable polyhedral type, and π not essentially selfdual. Then we have the following:

(a) If π does not admit any self twist, Π is cuspidal without any self twist. Furthermore, if π is not monomial, then Π is not induced from any non-normal cubic extension.

(b) If π is not of solvable type, i.e., its base change to any solvable Galois extension is cuspidal, Π is cuspidal and not of solvable type; in particular, there is no solvable extension K/F of degree d > 1 dividing 6, and a cuspidal automorphic representation η of GL_{6/d}(𝔸_{F}), such that L(s, Π) = L(s, η).

Remark: If π is regular algebraic at infinity, and F is not totally imaginary, then π is not monomial (See Lemma 9.3).

We will prove Theorem A in sections 6 through 8, and Theorem B in section 9.

Before proceeding with the proofs of these theorems, we will digress and discuss the cohomological application.

3. Preliminaries on cuspidal cohomology

The experts can skip this section and go straight to the statement (in section 4) and the proof (in section 5) of Theorems C, D. Let

Γ ⊂ SL(n, ℤ),

be a congruence subgroup of G_{n}^{0} := SL(n, ℝ), which has finite covolume. Γ acts by left translation on the symmetric space X_{n}^{0} := SL(n, ℝ)/SO(n). The cohomology of Γ is the same as that of the locally symmetric orbifold Γ\X_{n}^{0}.

If H_{cont}^{∗} denotes the continuous group cohomology, a version of Shapiro’s lemma gives an isomorphism

H^{∗}(Γ, ℂ) ≃ H_{cont}^{∗}(G_{n}^{0}, C^{∞}(Γ\G_{n}^{0})).

The constant functions are in this space, and the contribution of H_{cont}^{∗}(G_{n}^{0}, ℂ) to H^{∗}(Γ, ℂ) is well understood and plays an important role in Borel’s interpretation of the values of the Riemann zeta function ζ(s) at negative integers.

We will be interested here in another, more mysterious, piece of H^{∗}(Γ, ℂ), namely its cuspidal part, denoted H_{cusp}^{∗}(Γ, ℂ), which injects into H^{∗}(Γ, ℂ) by a theorem of Borel. Furthermore, one knows by L. Clozel ([C1988]) that the cuspidal summand is defined over ℚ, preserved by the Hecke operators. The cuspidal cohomology is represented by cocycles defined by smooth cusp forms in L^{2}(Γ\G_{n}^{0}), i.e., one has

H_{cusp}^{∗}(Γ, ℂ) = H_{cont}^{∗}(G_{n}^{0}, L_{cusp}^{2}(Γ\G_{n}^{0})^{∞}),

where L_{cusp}^{2}(Γ\G_{n}^{0}) denotes the space of cusp forms, and the superscript ∞ signifies taking the subspace of smooth vectors. If 𝔊_{n} denotes the complexified Lie algebra of G_{n}, the passage from continuous cohomology to the relative Lie algebra cohomology ([BoW1980]) furnishes an isomorphism

H_{cusp}^{∗}(Γ, ℂ) ≃ H^{∗}(𝔊_{n}^{0}, K : L_{cusp}^{2}(Γ\G_{n}^{0})^{∞}).

It is a standard fact (see [BoJ1979], for example) that the right action of G_{n} on L_{cusp}^{2}(Γ\G_{n}^{0}) is completely reducible, and so we may write

L_{cusp}^{2}(Γ\G_{n}^{0}) ≃ ˆ⊕_{π} m_{π}H_{π},

where π runs over the irreducible unitary representations of G_{n}^{0} (up to equivalence), H_{π} denotes the space of π, ˆ⊕ signifies taking the Hilbert direct sum, and m_{π} is the multiplicity. Consequently,

H_{cusp}^{∗}(Γ, ℂ) ≃ ⊕_{π} H^{∗}(𝔊_{n,ℂ}^{0}, K; H_{π}^{∞})^{m_{π}}.

One knows completely which representations π of G_{n}^{0} have non-zero (𝔊_{n}^{0}, K)- cohomology ([VZ1984]; see also [Ku1980]). An immediate consequence (see [C1988], page 114) is the following (with [x] denoting, for any x ∈ ℝ, the integral part of x):

Theorem 3.1.

H_{cusp}^{i}(Γ, ℂ) = 0 unless d(n) ≤ i ≤ d(n) + [(n − 1)/2],

where

d(n) = m^{2} if n = 2m and d(n) = m(m + 1) if n = 2m + 1.

It will be necessary for us to work with the ℚ-group G_{n} := GL(n) with center Z_{n}, and also work adelically. Let 𝔸 = ℝ × 𝔸_{f} be the adele ring of ℚ, K_{∞} = O(n), and X_{n} = G_{n}(ℝ)/K_{n}, whose connected component is X_{n}^{0}. For any compact open subgroup K of G_{n}(𝔸_{f}), we have

(3.6)  S_{K} := G_{n}(ℚ)Z_{n}(ℝ)^{0}\G_{n}(𝔸)/K_{∞}K ≃ ∪_{j=1}^{r}Γ_{j}\X_{n}^{0},

where the Γ_{j} are congruence subgroups of SL(n, ℤ) and Z_{n}(ℝ)^{0} is the Euclidean connected component of Z_{n}(ℝ). We need the following, which follows easily from the discussion in section 3.5 of [C1988]:

Theorem 3.2. (i)

H_{cusp}^{∗}(S_{K}, ℂ) ≃ ⊕_{π∈Coh_{K}} H^{∗}(𝔊̃_{n,∞}, K_{∞}; π_{∞}) ⊗ π_{f}^{K},

where 𝔊̃_{n,∞} consists of matrices in M_{n}(ℂ) with purely imaginary trace, and Coh_{K} is the set of (equivalence classes) of cuspidal automorphic representations π = π_{∞} ⊗ π_{f} of G_{n}(𝔸) such that π_{f}^{K} ≠ 0, π_{∞} contributes to the relative Lie algebra cohomology, and (ω_{π})_{∞} is trivial on Z(ℝ)^{0}.

(ii) Suppose π = π_{∞} ⊗ π_{f} is a cuspidal automorphic representation of G_{n}(𝔸) with π_{f}^{K} ≠ 0 such that the restriction r_{∞} of the Langlands parameter of π_{∞} to ℂ^{∗} is given by the n-tuple

\{(z/|z|)^{n−1}, (\overline{z}/|z|)^{n−1}, (z/|z|)^{n−3}, (\overline{z}/|z|)^{n−3},
…,(z/|z|), (\overline{z}/|z|)\} ⊗ (z\overline{z})^{n−1}

if n is even, and

\{(z/|z|)^{n−1}, (\overline{z}/|z|)^{n−1}, (z/|z|)^{n−3}, (\overline{z}/|z|)^{n−3},
…,(z/|z|)^{2}, (\overline{z}/|z|)^{2}, 1\} ⊗ (z\overline{z})^{n−1}

if n is odd. Then π contributes to Coh_{K} in degree d(n).

□

Given any cohomological π as above, the fact that the cuspidal cohomology at any level K has a ℚ-structure ([C1988]) preserved by the action of the Hecke algebra H_{ℚ}(G_{f}, K) (consisting of ℚ-linear combinations of Kdouble cosets), implies that the G_{f}-module π_{f} is rational over a number field ℚ(π_{f}). When n = 2, such a π is defined by a holomorphic newform h of weight 2, and then ℚ(π_{f}) is none other than the field generated by the Fourier coefficients of h.

4. Non-selfdual, cuspidal classes for Γ ⊂ SL(6, ℤ)

The principle of functoriality predicts that given cuspidal automorphic representations π, π′ of G_{n}(𝔸), G_{m}(𝔸) respectively, there exists an isobaric automorphic representation π ⊠ π′ of G_{nm}(𝔸) such that for every place v of ℚ, one has

σ((π ⊠ π′)_{v}) ≃ σ(π_{v}) ⊗ σ_{v}(π′_{v}),

where σ is the map (up to isomorphism) given by the local Langlands correspondence ([HaT2000], [He2000]) from admissible irreducible representations of G_{r}(ℚ_{v}) to r-dimensional representations of W′_{v}, which is the real Weil group W_{ℝ} if v = ∞ and the extended Weil group W_{ℚ_{p}} × SL(2, ℂ) if v is defined by a prime number p.

This prediction is known to be true for n = m = 2 ([Ra2000]), More importantly for the matter at hand, it is also known for (n, m) = (3, 2) by a difficult theorem of H. Kim and F. Shahidi ([KSh2002-1]).

Put

T = T_{1} ∪ T_{2},

with

T_{1} = {53, 61, 79, 89} and T_{2} = {128, 160, 205}.

By the article [AGG1984] of Ash, Grayson and Green (for p ∈ T_{1}), and the works [vGT1994], [vGT2000], [vGKTV1997] of B. van Geemen, J. Top, et al (for p ∈ T_{2}), one knows that for every q ∈ T, there is a non-selfdual cusp form π(q) on GL(3)/ℚ of level q, contributing to the (cuspidal) cohomology (with constant coefficients) .

Theorem C. Let π′ be a cusp form on GL(2)/ℚ defined by a non-CM holomorphic newform g of weight 4, level N, trivial character, and field of coefficients K. Let π denote an arbitrary cusp form on GL(3)/ℚ contributing to the cuspidal cohomology in degree 2, and let π(q), q ∈ T, be one of the particular forms discussed above. Put Π = π ⊠ π′ and Π(q) = π(q) ⊠ π′. Then

(a) Π contributes to the cuspidal cohomology of GL(6).

(b) Π(q) is not essentially selfdual when N ≤ 23 and K = ℚ.

(c) If N is relatively prime to q, then the level of Π(q) is N^{3}q^{2}. Now let N ≤ 23 and K = ℚ. Then Π(q) does not admit any self-twist. Moreover, there is no cubic non-normal extension K/ℚ with a cusp form η on GL(2)/K such that L(s, Π(q)) = L(s, η), nor is there a sextic extension (normal or not) E/ℚ with a character λ of E such that L(s, Π(q)) = L(s, λ).

We note from the Modular forms database of William Stein ([WSt2003]) that there exist newforms g of weight 4 with ℚ-coefficients, for instance for the levels N = 5, 7, 13, 17, 19, 23.

Remark 4.1 Part (b) should be true for any π′. Suppose that for a given any cusp form π on GL(3), cohomological or not, the functorial product Π = π ⊠ π′ satisfies Π^{∨} ≃ Π ⊗ ν for a character ν. Then at any prime p where a_{p}(π′) ≠ 0, which happens for a set of density 1, we can of course conclude that a_{p}(π^{∨}) = a_{p}(π)ν(p). But this does not suffice, given the state of knowledge right now concerning the refinement of the strong multiplicity one theorem, to conclude that π^{∨} is isomorphic to a twist of π. In the case of the π(q), we have information at a small set of primes and we have to make sure that a_{p}(π′) ≠ 0 and \overline{a_{p}(π)} ≠ a_{p}(π) for one of those p. The hypothesis that K = ℚ is made for convenience, however, and the proof will extend to any totally real field.

In [vGT1994], [vGT2000], [vGKTV1997] one finds in fact an algebraic surface S(q) over ℚ for each q ∈ T_{2}, and a 3-dimensional ℓ-adic representation ρ(q) (for any prime ℓ), occurring in H_{et}^{2}(S(q)\overline{ℚ}, \overline{ℚ}_{ℓ}), such that

(∗)  L_{p}(s, ρ(q)) = L_{p}(s, π(q)),

for all odd primes p ≤ 173 not dividing q. Here is a conditional result.

Theorem D. Let π′ be a cusp form on GL(2)/ℚ defined by a non-CM holomorphic newform g of weight 4, level N, trivial character, and field of coefficients ℚ, with corresponding \overline{ℚ}_{ℓ}-representation ρ′. Let π(q), ρ(q), Π(q) be as above for q ∈ T_{2}. Put R(q) = ρ(q) ⊗ ρ′. Suppose (∗) holds at all the odd primes p not dividing qℓ. Then R(q) remains irreducible when restricted to any open subgroup of Gal(\overline{ℚ}/ℚ).

5. Proof of Theorems C, D modulo Theorems A, B

In this section we will assume the truth of Theorems A and B.

Proof of Theorem C: As π′ is associated to a holomorphic newform of weight 4, we have

σ(π′_{∞})|_{ℂ^{∗}} ≃ ((z/|z|)^{3} ⊕ (\overline{z}/|z|)^{3}) ⊗ (z\overline{z})^{3}.

And since π contributes to cohomology, we have (cf. part (ii) of Theorem 3.1)

σ(π_{∞})|_{ℂ^{∗}} ≃ ((z/|z|)^{2} ⊕ 1 ⊕ (\overline{z}/|z|)^{2}) ⊗ (z\overline{z})^{2}.

Since Π_{∞} corresponds to the tensor product σ(π_{∞}) ⊗ σ(π′_{∞}), we get part

(a) of Theorem C in view of Theorem A and part (ii) of Theorem 3.2.

Pick any q in T and denote by ℚ(π(q)) the field of rationality of the finite part π(q)_{f} of π(q). Then it is known by [AGG1984] that for q ∈ T_{1},

ℚ(π(53)) = ℚ(\sqrt{−11}), ℚ(π(61)) = ℚ(\sqrt{−3}), ℚ(π(79)) = ℚ(\sqrt{−15}), ℚ(π(89)) = ℚ(i),

while by [vGT1994], [vGKTV1997] and [vGT2000],

ℚ(π(q)) = ℚ(i), ∀ q ∈ T_{2}.

By hypothesis, π′ is non-CM, and by part (a), Π(q) is cuspidal. Suppose there exists a character ν such that for some q ∈ T,

Π(q)^{∨} ≃ Π(q) ⊗ ν.

Comparing central characters, we get ν^{6} = 1. We claim that ν^{2} = 1. Suppose not. Then there exists an element σ of Gal(\overline{ℚ}/ℚ) fixing ℚ(π(q)) such that ν ≠ ν^{σ}. Since π′ has ℚ-coefficients and π(q) has coefficients in ℚ(π(q)), we see that Π(q)_{f} must be isomorphic to the Galois conjugate Π(q)_{f}^{σ}, which exists because the cuspidal cohomology group has, by Clozel (see section 3), a ℚ-structure preserved by the Hecke operators. If we put μ = ν/ν^{σ} ≠ 1, we then see that Π(q) ≃ Π(q) ⊗ μ. But we will see below that Π(q) admits no non-trivial self-twist. This gives the desired contradiction, proving the claim. If ν is non-trivial, the quadratic extension F/ℚ it cuts out will need to have discriminant dividing q^{a}N^{b} for suitable integers a, b. For any prime p which is unramified in F, we will have

\overline{a}_{p}(π)a_{p}(π′) = ±a_{p}(π)a_{p}(π′).

For each j ≤ 3 and for each π′ with N ≤ 23 and K = ℚ, we can find, using the tables in [AGG1984], [vGT1994],[vGT2000], [vGKTV1997] and [WSt2003], a prime p such that a_{p}(π′) ≠ 0, ν(p) ≠ 0 and \overline{a}_{p}(π) ≠ a_{p}(π). This proves part (b) of Theorem C.

When N is relatively prime to q, the conductor of Π(q) must be N^{3}q^{2} as can be seen from the way epsilon factors change under twisting (see section 4 of [BaR1994] for example).

From now on, let N ≤ 23 and K = ℚ. One knows that as π′ is holomorphic and not dihedral, the associated Galois representation ρ′ remains irreducible when restricted to any open subgroup of Gal(\overline{ℚ}/ℚ). It follows that the base change of π′ to any solvable Galois extension remains cuspidal. In particular, it is not of solvable polyhedral type. We claim that π(q) is not monomial. Indeed, the infinite type of π(q) is regular algebraic [C1988], and to be monomial there needs to be a cubic, possibly non-normal, extension K/ℚ which can support an algebraic Hecke character which is not a finite order character times a power of the norm. By [We1955], for such a character to exist, K must contain a CM field, i.e., a totally imaginary quadratic extension of a totally real field, which forces K to be imaginary. But any cubic extension of ℚ has a real embedding, and this proves our claim. Note also that as π(q) is not essentially self-dual, it is not a twist of the symmetric square of any cusp form, in particular π′, on GL(2)/ℚ. Now it follows from Theorem B that Π(q) does not admit any self-twist.

Suppose K is a non-normal cubic field together with a cusp form η on GL(2)/K such that L(s, Π(q)) = L(s, η). Let L be the Galois closure of K (with Galois group S_{3}), and let E be the quadratic extension of ℚ contained in L. Then Π(q)_{E} will be cuspidal and automorphically induced by the cusp form η_{L} of GL(3, 𝔸_{L}). In other words, Π(q)_{E} admits a non-trivial self-twist. To contradict this, it suffices, in view of Theorem B, to show that π(q)_{E} admits no self-twist relative to L/E, i.e., that π(q)_{E} is not automorphically induced by a character μ of L. But as noted above, this forces L to be a totally imaginary number field containing a CM field L_{0}. Then either L = L_{0} or L_{0} = E. In the latter case, by [We1955], μ will be a finite order character times the pullback by norm of a character μ_{0} of E, forcing I_{L}^{E}(μ) to be not regular at infinity, and so this case cannot happen. So L itself must be a CM field, with its totally real subfield F. Then Gal(F/ℚ) would be cyclic of order 3 and a quotient of S_{3}, which is impossible. So this case does not arise either. So π(q)_{E} doe snot admit any self-twist, and Π(q) is not associated to any η as above.

Now suppose L(s, Π(q)) = L(s, λ) for a character λ of a sextic field L. If L contains a proper subfield M ≠ ℚ, then since m := [L : M] ≤ 3, one can induce λ to M and get an automorphic representation β of GL_{m}(𝔸_{M}) such that L(s, λ) = L(s, β) = L(s, Π(q)), which is impossible by what we have seen above. So L must not contain any such M. But on the other hand, since Π(q)_{∞} is algebraic and regular, we need L to contain, by [We1955], a CM subfield L_{0}, and hence also its totally real subfield F. Either F = ℚ, in which case L_{0} is imaginary quadratic, or F ≠ ℚ. Either way there will be a proper subfield M of degree ≤ 3, and so the purported equality L(s, Π(q)) = L(s, λ) cannot happen. We are now done with the proof of Theorem C.

Proof of Theorem D: By assumption, the ℓ-adic representation ρ is functorially associated to the cuspidal cohomological form π(q) on GL(3)/ℚ with q ∈ T_{2}.

Lemma 5.1. ρ is irreducible under restriction to any open subgroup.

Proof. Suffices to show that the restriction ρ_{E} to Gal(\overline{ℚ}/E) is irreducible for any finite Galois extension E/ℚ. Pick any such extension and write G =Gal(E/ℚ). Suppose ρ_{E} is reducible. Then we have either

(i) ρ_{E} ≃ τ ⊕ χ with τ irreducible of dimension 2 and χ of dimension 1; or

(ii) ρ_{E} ≃ χ_{1} ⊕ χ_{2} ⊕ χ_{3}, with each χ_{j} one-dimensional. Let V be the 3-dimensional \overline{ℚ}_{ℓ}-vector space on which Gal(\overline{ℚ}/ℚ) acts via ρ. Suppose we are in case(i), so that there is a line L in V preserved by Gal(\overline{ℚ}/E) and acted upon by χ. Note that G acts on \{τ, χ\} and, by the dimension consideration, it must preserve {χ}. Hence the line L is preserved by Gal(\overline{ℚ}/ℚ), which contradicts the fact that ρ is irreducible.

So we may assume that we are in case (ii). We claim that χ_{i} ≠ χ_{j} if i ≠ j. Indeed, since ρ arises as (the base change to \overline{ℚ}_{ℓ} of) a summand of the ℓ-adic cohomology of a smooth projective variety, it is Hodge-Tate, and so is each χ_{j}. So each χ_{j} is locally algebraic and corresponds to an algebraic Hecke character χ′_{j} of E. By the identity of the L-functions, we will have L^{S}(s, π) = ∏_{j} L^{S}(s, χ′_{j}) for a suitable finite set S of places S. By the regularity of π, each χ′_{j} must appear with multiplicity one, which proves the claim. Now let L_{j} denote, for each j ≤ 3, the (unique) line in V stable under Gal(\overline{ℚ}/E) and acted upon by χ_{i}. And G acts by permutations on the set {χ_{1}, χ_{2}, χ_{3}}. In other words, there is a representation r : G → S_{3} such that the G-action is via r. Put H = Ker(r), with corresponding intermediate field M. Then each L_{j} is stable under Gal(\overline{ℚ}/M), so that ρ_{M} ≃ ν_{1} ⊕ ν_{2} ⊕ ν_{3}, where each ν_{j} is a character of Gal(\overline{ℚ}/M). Also, M/ℚ is Galois with Gal(M/ℚ) ⊂ S_{3}. But from the proof of Theorem C that the base change π_{M} of π to any such M is cuspidal. However, if ν_{j}^{1} denotes the algebraic Hecke character of M defined by ν_{j}, the twisted L-function L^{S}(s, π_{M} ⊗ ν′_{j}^{−1}) will have a pole at s = 1, leading to a contradiction. We have now proved Lemma 5.1.

□

Note that Lemma 1 implies in particular that for any finite extension F/ℚ, ρ_{F} does not admit any self-twist.

Lemma 5.2. For any finite extension E/ℚ, the restriction ρ_{E} is not essentially self-dual.

Proof. Again we may assume that E/ℚ is Galois with group G. As before let V denote the space of ρ, and suppose that we have an isomorphism ρ ≃ ρ^{∨} ⊗ ν, for a character ν. Then there is a line L in V ⊗ V on which Gal(\overline{ℚ}/E) acts via ν. By Schur’s lemma (and this is why we have to work over \overline{ℚ}_{ℓ}), the trivial representation appears with multiplicity one in V ⊗V^{∨}. It implies that ν must appear with multiplicity one in V ⊗V. We claim that V ⊗ V contains no other character. Indeed, if we have another character ν′, we would have ρ ≃ ρ⊗μ, where μ = ν/ν′. But as noted above, ρ_{E} admits no self-twist, and so μ = 1, and the claim is proved. Consequently, the action of G on V ⊗V must preserve ν. In other words, the line L is stable under all of Gal(\overline{ℚ}/ℚ), contradicting the fact that ρ is not essentially self-dual. Done

□

Now consider R = ρ ⊗ ρ′. We know that both ρ and ρ′ remain irreducible upon restriction to any open subgroup and moreover, such a restriction of ρ is not essentially self-dual. It then follows easily that the restriction of R is irreducible.

This finishes the proof of Theorem D.

□

6. Proof of Theorem A, Part #1

By twisting we may assume that π, π′ are unitary, so that π^{∨} ≃ \overline{π} and π′^{∨} ≃ \overline{π′}, with respective central characters ω, ω′.

Now we proceed in several steps. Applying Langlands’s classification, ([La1979-1], [La1979-2], [JS1981]), we see that the Kim-Shahidi product Π = π ⊠ π′ must be an isobaric sum of cusp forms whose degrees add up to 6. Thanks to the Clebsch-Gordon decomposition

sym^{2}(π′) ⊠ π′ ≃ sym^{3}(π′) ⊞ (π′ ⊗ ω′),

Π is not cuspidal if π is a twist of sym^{2}(π′).

The list of all the cases when Π is not cuspidal is the following:

Case I: Π has a constituent of degree 1, i.e., Π = λ ⊞ Π′ for some idele class character λ and some automorphic representation Π′ of GL(5).

Case II: Π has a constituent of degree 2, i.e., Π = τ ⊞ Π′ for some cusp form τ on GL(2) and some automorphic representation Π′ of GL(4).

Case III: Π is an isobaric sum of two cusp forms σ_{1} and σ_{2} on GL(3).

We first deal with Cases I and II. We need some preliminaries. First comes the following basic result due to H. Jacquet and J.A. Shalika ([JS1981], [JS1990], and R. Langlands ([La1979-1], [La1979-2]).

Lemma 6.1. (i) Let Π, τ be isobaric automorphic representations of GL_{n}(𝔸_{F}), GL_{m}(𝔸_{F}) respectively. Assume that τ is cuspidal. Then the order of the pole of L(s, Π ⊗ \overline{τ}) at s = 1 is the same as the multiplicity of τ occurring in the isobaric sum decomposition of Π.

(ii) L(s, Π × \overline{Π}) has a pole at s = 1 of order m = ∑_{i} m_{i}^{2} if Π = ⊞_{i}m_{i}π_{i} is the isobaric decomposition of Π, with the π_{i} being inequivalent cuspidal representations of smaller degree. In particular, m = 1 if and only if Π is cuspidal.

An L–function L(s) is said to be nice if it converges on some right half plane, admits an Euler product of some degree m, say, and extends to a meromorphic function of finite order with no pole outside s = 1, together with a functional equation related to another L–function L^{∨}(s) given by

L(s) = W (d_{F}^{m}N)^{1/2−s}L^{∨}(1 − s),

where W is a non-zero scalar.

If π_{1}, π_{2} are automorphic forms on GL(m), GL(n) respectively, then the Rankin-Selberg L-function L(s, π_{1} × π_{2}) is known to be nice ([JPSS1983], [MW1995], [Sh1988]). Of course, the product of two nice L–functions is nice. Furthermore, we recall the following Tchebotarev-like result for nice L–functions ([JS1990]):

Lemma 6.2. Let L_{1}(s) = ∏_{v} L_{1,v}(s) and L_{2}(s) = ∏_{v} L_{2,v}(s) be two L– functions with Euler products, and suppose that they are both of exactly one of the following types:

(a) L_{i}(s) is an Artin L–function of some Galois extension;

(b) L_{i}(s) is attached to an isobaric automorphic representation;

(c) L_{i}(s) is a Rankin–Selberg L–function of two isobaric automorphic representations.

If L_{1,v}(s) = L_{2,v}(s) for all but finite places v of F, then L_{1}(s) = L_{2}(s).

Proof of Theorem A for Cases I and II.

Firstly, Case I can never happen. The reason is the following: If λ is a constituent of Π = π ⊠π′, then L(s, Π×λ̄) has a pole at s = 1 (Lemma 6.1), hence so does

L(s, π′ × π ⊗ λ̄) = L(s, π ⊠ π′ ⊗ λ̄).

However, π′ and π⊗λ̄ are cuspidal of different degrees, hence L(s, π′ ×π⊗λ̄) is entire, and we get the desired contradiction.

Now we treat Case II, where Π has a constituent τ of degree 2. We will show that this can happen IF AND ONLY IF π is twist equivalent to sym^{2}(π′) in which case τ is twist equivalent to π′.

In fact, for each finite v where π and π′ are unramified,

L_{v}(s, Π ⊗ τ̄) = L_{v}(s, π × (π′ ⊠ τ̄)),

where π′⊠τ̄ is the functorial product of π′ and τ̄ whose modularity (in GL(4)) was established in [Ra2000]. One may check the following: If π′_{v} = α_{v,1}⊞α_{v,2}, π_{v} = β_{v,1} ⊞ β_{v,2} ⊞ β_{v,3}, and τ′_{v} = γ_{v,1} ⊞ γ_{v,2}, then both sides of the equality is the same as ∏_{i,j,k} L(s, α_{v,i}β_{v,j}γ̄_{v,k}) where the product is over all i, j, k such that 1 ≤ i, k ≤ 2 and 1 ≤ j ≤ 3.

Hence by Lemma 6.2,

L(s, Π ⊗ τ̄) = L(s, π × (π′ ⊠ τ̄))

As τ is a constituent of Π, the L–functions on both sides above have a pole at s = 1. As π is cuspidal, this means by Lemma 6.1, π̄ is a constituent of π′ ⊠ τ̄. Hence π′ ⊠ τ̄ should possess a constituent of degree 1, namely a character μ.

Thus L(s, π′ × τ̄ ⊗ μ̄) = L(s, π′ ⊠ τ̄ ⊗ μ̄) has a pole at s = 1, implying that π′ is equivalent to τ ⊗ μ. Hence

π′ ⊠ τ̄ ≅ μ ⊞ Ad(τ) ⊗ μ,

which means that π ≅ Ad(τ) ⊗ μ ≅ Ad(π′) ⊗ μ.

Finally, it is clear that if Case II happens, then π′ cannot be dihedral. Furthermore, Π is Eisensteinian of type (2, 2, 2) if π′ is tetrahedral, and (2, 4) otherwise. we can see this by observing that

π′ ⊠ Ad(π′) ≅ sym^{3}(π′) ⊗ ω_{π′} ⊞ π′ ⊗ ω_{π′}^{2}

□

7. Proof of Theorem A, Part #2

It remains to treat Case III. Here again, π′ denotes a cusp form on GL(2) and π a cusp form on GL(3). Assume that Π = σ_{1} ⊞ σ_{2} where σ_{1} and σ_{2} are cusp forms on GL(3).

We will divide Case III into two subcases: In this section, we will assume that π′ is not dihedral. The (sub)case when π′ is dihedral will be treated in the next section.

The following equality is crucial, and it holds for all cusp forms π′ on GL(2) and π on GL(3):

Proposition 7.1.

L(s,π × π′; Λ^{3} ⊗ ω_{π}^{−1}χ)L(s, π′ ⊗ ω_{π′}χ)

= L(s, sym^{3}(π′) ⊗ χ)L(s, (π ⊠ π′) × π̃ ⊗ ω_{π′}χ) (1)

where ω_{π′} and ω_{π} are the respective central characters of π′ and π.

Proof of Proposition 7.1.

We claim that both sides of (1) are nice. Indeed, we see that formally, the admissible representation Λ^{3}(π ⊠ π′) ⊗ ω_{π}^{−1} is equivalent to sym^{3}(π′) ⊞ (Ad(π) ⊠ π′ ⊗ ω_{π′}). So the left hand side is nice. And the right hand side is nice by [KSh2002-1], whence the claim. So by Lemma 6.2 it suffices to prove this equality given by the Proposition locally at v for almost all v. It then suffices to prove the following identity (as admissible representations) for almost all v:

Λ^{3}(π′_{v} ⊠ π_{v}) ⊗ ω_{π_{v}}^{−1} ⊞ π′_{v} ⊗ ω_{π′_{v}}

= sym^{3}(π′_{v}) ⊞ π′_{v} ⊠ π_{v} ⊠ π̃_{v} ⊗ ω_{π′_{v}} (2)

Let v be any place where π′ and π are unramified. Say π′_{v} = α_{v,1} ⊞ α_{v,2}, π_{v} = β_{v,1} ⊞ β_{v,2} ⊞ β_{v,3}. Note that ω_{π′_{v}} = α_{v,1}α_{v,2} and ω_{π_{v}} = β_{v,1}β_{v,2}β_{v,3}. Then it is routine to check that the left and the right hand sides of (2) are equal to the sum of the following terms:

Terms A (2 terms): α_{v,1}^{3} ⊞ α_{v,2}^{3};

Terms B (2 × 4 = 8 terms): 4 copies of (α_{v,1} ⊞ α_{v,2}) ⊗ ω_{π′_{v}};

Terms C (12 terms): ⊞_{1≤i≤2,1≤j≠k≤3} α_{v,i}ω_{π′_{v}}β_{v,j}β_{v,k}^{−1}.

In fact, Terms A, B and C are obtained by expanding the right hand side of (2). Since

Ad(π_{v}) = 3 · \underline{1} ⊞ (⊞_{1≤j≠k≤3}, β_{v,j}β_{v,k}^{−1}),

the Terms C and (three of) the Terms B are obtained from π′_{v} ⊠π_{v} ⊠π̃_{v} ⊗ω_{π′}, and the Terms A and (one of) the Terms B arise from sym^{3}(π′).

The left hand side is easy to handle since we have the following:

Λ^{3}(π′_{v} ⊗ π_{v}) = ⊞_{1≤i≤2,1≤j,k≤3,j≠k} α_{v,i}ω_{π′_{v}}β_{v,j}^{2}β_{v,k}
⊞ ω_{π_{v}}α_{v,1}^{3} ⊞ ω_{π_{v}}α_{v,2}^{3}
⊞ 3ω_{π_{v}}α_{v,1}ω_{π′_{v}} ⊞ 3ω_{π_{v}}α_{v,2}ω_{π′_{v}}

In fact, the ω_{π_{v}}^{−1} twist of the thing above contributes the Terms C, Terms A and (three of) Terms B.

So we have proved (2), and hence (1). □

Let σ_{1} and σ_{2} be cusp forms on GL(3).

Lemma 7.2. Let η_{1} and η_{2} be the central characters of σ_{1} and σ_{2} respectively. Then

L(s, σ_{1} ⊞ σ_{2}; Λ^{3} ⊗ χ′) = L(s, η_{1}χ′)L(s, η_{2}χ′)

L(s, σ_{1} × σ̃_{2} ⊗ η_{2}χ′)L(s, σ_{2} × σ̃_{1} ⊗ η_{1}χ′) (3)

Proof. of Lemma 7.2. This is easy since at each place v where the σ_{i} are unramified,

Λ^{3}(σ_{1,v} ⊞ σ_{2,v}) = ⊞_{0≤i≤3} (Λ^{i}(σ_{1,v}) ⊠ Λ^{3−i}(σ_{2,v})) ,

Λ^{2}(σ_{i,v}) ≅ σ̃_{i,v} ⊗ η_{i}

and

Λ^{3}(σ_{i}) ≅ η_{i}.

Done by applying Lemma 6.2.

□

Before we apply Proposition 7.1 and Lemma 7.2, let us first investigate a special instance of Case III when σ_{1} and σ_{2} are both twists of π:

Lemma 7.3. If π ⊠ π′ = (π ⊗ χ_{1}) ⊞ (π ⊞ χ_{2}) then

sym^{3}(π′) ≅ (π′ ⊗ ω_{π′}) ⊞ χ_{1}^{3} ⊞ χ_{2}^{3}

Hence if π′ is not dihedral or tetrahedral, this cannot happen.

Proof of Lemma 7.3. Let v be any place where π′_{v} and π_{v} are unramified. Write

L_{v}(s, π′) = (1 − U_{v}(Nv)^{−s})^{−1}(1 − V_{v}(Nv)^{−s})^{−1}

and

L_{v}(s, π) = (1 − A_{v}(Nv)^{−s})^{−1}(1 − B_{v}(Nv)^{−s})^{−1}(1 − C_{v}(Nv)^{−s})^{−1}.

Then

L_{v}(s, π′ × π) =
(1 − U_{v}A_{v}(Nv)^{−s})^{−1}(1 − U_{v}B_{v}(Nv)^{−s})^{−1}(1 − U_{v}C_{v}(Nv)^{−s})^{−1}
(1 − V_{v}A_{v}(Nv)^{−s})^{−1}(1 − V_{v}B_{v}(Nv)^{−s})^{−1}(1 − V_{v}C_{v}(Nv)^{−s})^{−1}

Let X_{v} = χ_{1}(Frob_{v}) and Y_{v} = χ_{2}(Frob_{v}). Then

L_{v}(s, π ⊗ χ_{1})L_{v}(s, π ⊗ χ_{2}) =
(1 − X_{v}A_{v}(Nv)^{−s})^{−1}(1 − X_{v}B_{v}(Nv)^{−s})^{−1}(1 − X_{v}C_{v}(Nv)^{−s})^{−1}
(1 − Y_{v}A_{v}(Nv)^{−s})^{−1}(1 − Y_{v}B_{v}(Nv)^{−s})^{−1}(1 − Y_{v}C_{v}(Nv)^{−s})^{−1}

Consequently we have

{ U_{v}A_{v}, U_{v}B_{v}, U_{v}C_{v}, V_{v}A_{v}, V_{v}B_{v}, V_{v}C_{v} } =

{ X_{v}A_{v}, X_{v}B_{v}, X_{v}C_{v}, Y_{v}A_{v}, Y_{v}B_{v}, Y_{v}C_{v} }. (4)

In particular,

(U_{v}^{n} + V_{v}^{n})(A_{v}^{n} + B_{v}^{n} + C_{v}^{n}) = (X_{v}^{n} + Y_{v}^{n})(A_{v}^{n} + B_{v}^{n} + C_{v}^{n})

for each positive integer n. And besides, taking the products of the elements on each side of (4) and equating, we get

U_{v}^{3}V_{v}^{3} = X_{v}^{3}Y_{v}^{3}

Now we apply the following lemma.

Lemma 7.4. If X, Y, U, V, A, B, C are nonzero complex numbers such that (for all n > 0)

(U^{n} + V^{n})(A^{n} + B^{n} + C^{n}) = (X^{n} + Y^{n})(A^{n} + B^{n} + C^{n}),

and U^{3}V^{3} = X^{3}Y^{3}, then { U^{3}, V^{3} } = { X^{3}, Y^{3} }.

Proof of Lemma 7.4. If A^{3} + B^{3} + C^{3} ≠ 0, then U^{3} + V^{3} = X^{3} + Y^{3}. Hence { U^{3}, V^{3} } =

{ X^{3}, Y^{3} } as U^{3}V^{3} = X^{3}Y^{3}.

If A^{3} + B^{3} + C^{3} = 0, we claim that A + B + C ≠ 0. Otherwise,

−3ABC = 3AB(A + B) = (A + B)^{3} − (A^{3} + B^{3}) = −C^{3} + C^{3} = 0

Thus a, b or c is zero. This leads to a contradiction.

In fact we will prove the following statement: Claim: If a, b, c ≠ 0, then a + b + c or a^{3} + b^{3} + c^{3} is not zero.

So we claim also that A^{9}+B^{9}+C^{9} ≠ 0, and, A^{2}+B^{2}+C^{2} or A^{6}+B^{6}+C^{6} is not zero.

Hence

U^{n} + V^{n} = X^{n} + Y^{n}

holds for n = 1 and 9, and for one of 2 or 6.

If this equality holds for n = 1, and 2, then

UV = \frac{(U + V)^{2} − (U + V^{2})}{2} = \frac{(X + Y)^{2} − (X + Y^{2})}{2} = XY,

implying that { U, V } = { X, Y }, and the lemma will follow.

Now assume that U^{n} + V^{n} = X^{n} + Y^{n} holds for n = 1, 6 or 9. As we have already assumed that U^{3}V^{3} = X^{3}Y^{3}, U^{3n}V^{3n} = X^{3n}Y^{3n}. So we have

{ U^{n}, V^{n} } = { X^{n}, Y^{n} }

for n = 6 and 9.

Without loss of generality, assume that U^{9} = X^{9} and V^{9} = Y^{9}. If U^{6} = X^{6} and V^{6} = Y^{6}, then of course we have U^{3} = X^{3} and V^{3} = Y^{3} and the lemma follows. If U^{6} = Y^{6} and V^{6} = X^{6}, then U, V, X and Y have the same norm. However, since U + V = X + Y, the pairs {U, V } and {X, Y } are the same. hence implying the lemma. The reason for this comes from the following statement which is elementary: (Note that even when U + V = X + Y = 0, although we cannot directly apply this statement, we still have U^{3} + V^{3} = X^{3} + Y^{3} = 0, so that { U^{3}, V^{3} } = { X^{3}, Y^{3} }.)

Statement The pair (z_{1}, z_{2}) such that |z_{1}| = |z_{2}| = R and z_{1} + z_{2} = Z is uniquely determined by R > 0 and Z with 0 < |Z| < 2R.

So in all cases, { U^{3}, V^{3} } = { X^{3}, Y^{3} }.

□

Proof of Lemma 7.3 (contd.)

By the previous lemma,

{ U_{v}^{3}, V_{v}^{3} } = { X_{v}^{3}, Y_{v}^{3} }

at any unramified finite place v.

Hence

L(s, sym^{3}(π′_{v}))
= (1 − U_{v}^{3}(Nv)^{−s})^{−1}(1 − V_{v}^{3}(Nv)^{−s})^{−1}
(1 − V_{v}^{2}U_{v}(Nv)^{−s})^{−1}(1 − U_{v}^{2}V_{v}(Nv)^{−s})^{−1}
= (1 − X_{v}^{3}(Nv)^{−s})^{−1}(1 − Y_{v}^{3}(Nv)^{−s})^{−1}
(1 − V_{v}ω_{π′_{v}}(Frob_{v})(Nv)^{−s})^{−1}(1 − U_{v}ω_{π′_{v}}(Frob_{v})(Nv)^{−s})^{−1}
= L(s, χ_{1,v}^{3})L(s, χ_{2,v}^{3})L(s, π′ ⊗ ω_{π′_{v}})

Here we have used

U_{v}V_{v} = ω_{π′_{v}}(Frob_{v})

Hence

sym^{3}(π′_{v}) ≃ χ_{1,v}^{3} ⊞ χ_{2,v}^{3} ⊞ (π′_{v} ⊗ ω_{π′_{v}}),

and by Lemma 6.2 we get what we desire, namely,

sym^{3}(π′) ≃ χ_{1}^{3} ⊞ χ_{2}^{3} ⊞ (π′ ⊗ ω_{π′}).

□

Lemma 7.5. If π′ is tetrahedral with sym^{2}(π′) invariant under twisting by a cubic character χ, then

sym^{3}(π′) ≅ (π′ ⊗ ω_{π′}χ) ⊞ (π′ ⊗ ω_{π′}χ^{−1}).

Hence the situation of Lemma 7.3 will not happen if π′ is tetrahedral.

Proof of Lemma 7.5. Consider π′ ⊠ sym^{2}(π′) = sym^{3}(π′) ⊞ (π′ ⊗ ω_{π′}), which obviously contains

π′ ⊗ ω_{π′} as an isobaric constituent .

Since sym^{2}(π′) allows self twists by χ and χ^{−1}, the isobaric sum above should also contain π′ ⊗ ω_{π′}χ and π′ ⊗ ω_{π′}χ^{−1}. Together with π′ ⊗ ω_{π′}, they are pairwise inequivalent, the reason being that if a cusp form on GL(2) admits a self twist by a character, then such character has to be trivial or quadratic.

Thus, by the uniqueness of the isobaric decomposition, sym^{3}(π′) should have π′ ⊗ ω_{π′}χ and π′ ⊗ ω_{π′}χ^{−1} as its constituents, and there is no room for any other constituent.

□

Proof of Case III when π′ is not dihedral.

Assume that Π = π ⊠ π′ = σ_{1} ⊞ σ_{2} where σ_{1} and σ_{2} are cusp forms on GL(3) with central characters η_{1} and η_{2} respectively. Also, assume that π′ is not dihedral.

Subcase A: π does not allow a self twist by a nontrivial character.

From (1) (Proposition 7.1) and (3) (Lemma 7.2), we have

L(s, sym^{3}(π′) ⊗ χ)L(s, (π ⊠ π′) × π̃ ⊗ ω_{π′}χ)
= L(s, π′ × π; Λ^{3} ⊗ ω_{π}^{−1}χ)L(s, π′ ⊗ ω_{π′}χ)
= L(s, η_{1}ω_{π}^{−1}χ)L(s, η_{2}ω_{π}^{−1}χ)L(s, π′ ⊗ ω_{π′}χ)

L(s, σ_{1} × σ̃_{2} ⊗ η_{2}ω_{π}^{−1}χ)L(s, σ_{2} × σ̃_{1} ⊗ η_{1}ω_{π}^{−1}χ) (5)

Hence, take χ = ω_{π}η_{i}^{−1}, then the right hand side has a pole at s = 1 (as the remaining factors do not vanish st s = 1). Then the left hand side also has a pole at s = 1.

However, since π′ is not dihedral, then sym^{3}(π′) is either cuspidal or an isobaric sum of two cusp forms on GL(2) (Lemma 7.5), so any twisted L– function of sym^{3}(π′) has to be entire. So the only pole at s = 1 should come from L(s, (π ⊠ π′) × π̃ ⊗ ω_{π′}ω_{π}η_{i}^{−1}).

As π is cuspidal, Π = π ⊠ π′ should have both σ′_{i} = π ⊗ ω_{π′}^{−1}ω_{π}^{−1}η_{i} as constituents.

Since π is not monomial, it does not allow a self twist. Hence, if η_{1} ≠ η_{2} then σ′_{i} are different. Hence σ′_{1} and σ′_{2} are the only constituents of Π which are also twists of π. Furthermore, if η_{1} = η_{2}, then the order of the pole of both sides of (5), and hence also of L(s, (π ⊠π′)×π̃ ⊗ω_{π′}ω_{π}η_{i}^{−1}), is 2. Hence σ′_{1} = σ_{2} should be a constituent of Π with multiplicity 2.

Thus we get an isobaric decomposition of Π as a sum of two twists of π. Thus, from lemma 7.3 and Lemma 7.5, this cannot happen if π′ is not dihedral. This completes the treatment of Subcase A.

Subcase B: π admits a self twist by a nontrivial cubic character χ.

In this subcase, recall that we are assuming Π = π ⊠ π′ ≅ σ_{1} ⊞ σ_{2}, where σ_{1} and σ_{2} are cusp forms with respective central characters η_{1} and η_{2}.

We claim that σ_{1} and σ_{2} are also invariant when twisted by χ. Otherwise σ_{i} ⊗ χ and σ_{i} ⊗ χ^{−1} will be distinct from σ_{i}, while they should both be constituents of Π ≃ Π ⊗ χ ≃ Π. Hence Π has at least degree 3 × 3 = 9, which is impossible as it is automorphic on GL(6).

Moreover, let v be any place where π and π′ are unramified. Write (for i = 1, 2) π′_{v} = α_{v} ⊞ α′_{v}, π_{v} = β_{v} ⊞ β_{v}χ_{v} ⊞ β_{v}χ_{v}^{−1} (this form being implied by π ≅ π ⊗ χ), and σ_{i,v} = θ_{i,v} ⊞ θ_{i,v}χ_{v} ⊞ θ_{i,v}χ_{v}^{−1}.

Then we have

Π_{v} = (α_{v} ⊞ α′_{v}) ⊗ β_{v} ⊗ (1 ⊞ χ_{v} ⊞ χ_{v}^{−1})

and

σ_{1,v} ⊞ σ_{2,v} = (θ_{1,v} ⊞ θ_{2,v}) ⊗ (1 ⊞ χ_{v} ⊞ χ_{v}^{−1})

Since the sets of all cubes of characters occurring in the previous two isobaric decompositions should be the same, and since β_{v}^{3} = ω_{π_{v}} and θ_{v}^{3} = η_{i,v}, we must have

{ α_{v}^{3}ω_{π_{v}}, α′_{v}^{3}ω_{π_{v}} } = { α_{v}^{3}β_{v}^{3}, α′_{v}^{3}β_{v}^{3} }
= { θ_{1,v}^{3}, θ_{2,v}^{3} }
= { η_{1,v}, η_{2,v} }

So

sym^{3}(π′_{v}) ⊗ ω_{π_{v}} ≅ α_{v}^{3}ω_{π_{v}} ⊞ α′_{v}^{3}ω_{π_{v}} ⊞ (α_{v} ⊞ α′_{v}) ⊠ ω_{π′_{v}}ω_{π_{v}}
≃ η_{1,v} ⊞ η_{2,v} ⊞ π′_{v} ⊗ ω_{π′_{v}}ω_{π_{v}},

that is

sym^{3}(π′_{v}) ≅ η_{1,v}ω_{π_{v}}^{−1} ⊞ η_{2,v}ω_{π_{v}}^{−1} ⊞ π′_{v} ⊗ ω_{π′_{v}}

Thus by Lemma 6.2 or the strong multiplicity one theorem, we have

sym^{3}(π′) ≅ η_{1}ω_{π}^{−1} ⊞ η_{2}ω_{π}^{−1} ⊞ π′ ⊗ ω_{π′}

However, since π′ is not dihedral, sym^{3}(π′) is by Lemma 7.5 either cuspidal or an isobaric sum of two cusp forms on GL(2). This gives a contradiction.

This completes the treatment of Subcase B.

□

8. Proof of Theorem A, Part #3

In this part, we will treat the case when π′ is dihedral. After that, we will analyze precisely the cuspidality criterion when π is an adjoint of a form on GL(2). Again, F denotes a number field.

In fact, we will prove the following:

Theorem 8.1. Let π′, π be cusp forms on GL(2)/F, GL(3)/F respectively, with π′ dihedral.

Then Π = π ⊠π′ is cuspidal unless both the following two conditions hold:

(a) There is a non-normal cubic extension K′ of F such that π′_{K′} is Eisensteinian; equivalently, π′ is dihedral of type D_{6}.

(b) π is monomial and π = I_{K′}^{F}(χ′) for some idele class character χ′ of K.

If (a) and (b) both hold, then Π is an isobaric sum of two cuspidal representations of degree 3, which are both twist equivalent to π.

Before we prove this theorem, let us recall that a dihedral Galois representation ρ′ of G_{F} is said to be of type D_{2n} if its projective image is D_{2n}. It is clear that ρ′ is not irreducible if and only if n = 1 (Note that the projective image of D_{4n} must be a quotient of D_{2n} since D_{4n} has a nontrivial center). If 6|n, then D_{2n} has a unique cyclic subgroup with quotient isomorphic to D_{6} ≅ S_{3}. Suppose K′ is a non-normal cubic extension of F, and ρ′ restricted to G_{K′} is reducible. Then the projective image of G_{K′} should be a subgroup of that of G_{F} of index 3, hence is isomorphic to D_{2n/3}. Thus n = 3, and ρ must be dihedral of type D_{6}. Similarly, we conclude that if π′ is dihedral and π_{K′} is not cuspidal, then π′ is of type D_{6}.

Proof of Theorem 8.1.

First assume (a) and (b). Note that π′_{K′} = v_{1} ⊞ v_{2}, and π = I_{K′}^{F}(χ′). Then

Π = π ⊠ π′ ≅ π′ ⊠ I_{K′}^{F}(χ′)
≅ I_{K′}^{F}(π′_{K′} ⊗ χ′) ≅ I_{K′}^{F}(v_{1}χ′) ⊞ I_{K′}^{F}(v_{2}χ′)

Hence Π is not cuspidal.

Note that π_{i} = I_{K′}^{F}(v_{i}χ′) MUST be cuspidal as from Section 5, Π cannot have a character as its constituent.

Next, we prove that if Π is not cuspidal, then (a), (b) and the remaining statements of the theorem hold.

Step 1: π_{K} is cyclic cubic monomial.

Assume that π′ = I_{K}^{F}(τ) where τ is some idele class character of C_{K} with K a quadratic extension of F. and also assume that K/F is cut out by δ.

From Section 5, Case I and II cannot happen, so we are in Case III. Say Π = σ_{1} ⊞ σ_{2} where σ_{i} are some cusp forms on GL(3)/F. As π′ allows a self twist by δ, so does Π = π ⊠ π′. Thus σ_{1} ≅ σ_{2} ⊗ δ as the only possible characters that either σ_{i} allows (for self-twisting) should be trivial or cubic.

Let θ be the nontrivial Galois conjugation of K/F. Then π′_{K} ≅ τ ⊞ τ^{θ}. Hence the base change Π_{K} = π′_{K}⊠π_{K} is equivalent to π_{K}⊗τ plus π_{K}⊗(τ ∘θ). As Π = σ_{1} ⊞ σ_{1} ⊗ δ hence Π_{K} is equivalent to the isobaric sum of two copies of σ_{1}_{K}.

Thus π_{K} ⊗ τ ≅ π_{K} ⊗ (τ ∘ θ) ≅ σ_{1}. As π′ = I_{K}^{F}(τ) is cuspidal, τ ≠ τ ∘ θ. Hence π_{K} is forced to be cyclic monomial.

Step 2: π is non-normal cubic monomial.

By Step 1, μ = τ^{−1}(τ ∘ θ) is a cubic character of C_{K}. Let M be the cubic field extension of K cut out by μ. As μ ∘ θ = μ^{−1}, M^{θ} = M, thus M/F is normal, and Gal(M/F) ≅ S_{3}.

Furthermore, π_{K} = I_{M}^{K}(λ) for some character λ of C_{M}. And also, π_{M} is of the form λ ⊞ λ′ ⊞ λ′′.

Let K′ be a non-normal cubic extension of F contained in M. Then [M : K′] = 2 and π_{M} is a quadratic base change of π_{K′}.

We claim that π_{K′} is Eisensteinian, i.e., not cuspidal. The reason is that, if π_{K′} were cuspidal, then its quadratic base change π_{M} would be either cuspidal or the isobaric sum of two cusp forms of the same degree. Since π_{K′} is a cusp form on GL(3), we see from [AC1989] that this is impossible.

So π_{K′} must admit a character as an isobaric constituent, which means that π is induced from some character of C_{K′}.

Step 3: π′_{K′} is not cuspidal, hence π′ is dihedral of type D_{6}.

Recall that π′_{K} = τ ⊞ (τ ∘ θ) = τ ⊞ τμ, so that π′_{M} = τ_{M} ⊞ τ_{M} as M/K is cut out by μ.

Thus the projective image of ρ′_{M} is trivial, where ρ′ is the representation Ind_{K}^{F}(τ) of the Weil group W_{F}, and ρ′_{M} is the restriction of ρ to G_{M}. Hence the projective image of ρ must be D_{6}.

Remark: Even if π′ is selfdual, τ may be a character of order 3 or 6.

Step 4: σ_{1} and σ_{2} are all twist equivalent to π.

Observe that

(τμ^{−1}) ∘ θ = (τ ∘ θ)(μ ∘ θ)^{−1}
= τμμ = τμ^{−1}.

So τμ^{−1} is a base change of some character, say ν, of C_{F} to K/F.

So π′ = I_{K}^{F}(μ) ⊗ ν and π′_{K′} ≅ ν_{K′} ⊞ ν_{K′}δ_{K′}. By Step 2, we may assume that π = I_{K′}^{F}(λ) for a character λ of a non-normal cubic extension K′ of F. We get

Π = π ⊠ π′ = π′ ⊠ I_{K′}^{F}(λ)
= I_{K′}^{F}(π′_{K′} ⊗ λ) = I_{K′}^{F}(ν_{K′}λ ⊞ ν_{K′}λδ_{K′})
= I_{K′}^{F}(λ) ⊗ ν ⊞ I_{K′}^{F}(λ) ⊗ νδ

= (π ⊗ ν) ⊞ (π ⊗ νδ) (6)

Now the proof of Theorem 8.1 is completed. □

Remark. When π is twist equivalent to Ad(π_{0}), and π′ is dihedral, we claim that the only way Π = π ⊠ π′ can be not cuspidal is for Ad(π) to be non-normal cubic monomial, implying that π is of octahedral type. We get the following theorem which is more precise than the result in [Wa2003]:

Theorem 8.2. Let π′, and π′′ be two cusp form on GL(2)/F, and suppose that π′′ is not dihedral. then Π = π′ ⊠ Ad(π′′) is cuspidal unless one of the following happens:

(d) π′ and π′′ are twist equivalent.

(e) π′ and π′′ are octahedral attached with the same S̃_{4}-extension, and Ad(π′) and Ad(π′′) are twist equivalent.

(f) π′′ is octahedral, π′ = I_{K}^{F}(μ) ⊗ ν is dihedral, where μ is the cubic character which is allowed by Ad(π′′_{K}).

The proof of the theorem above uses the following proposition.

Proposition 8.3. Let π_{1} and π_{2} are two non-dihedral cusp forms on GL(2)/F, and Ad(π_{1}) and Ad(π_{2}) are twist equivalent. Then one of the following holds.

(g) π_{1} and π_{2} are twist equivalent (so that their adjoints are equivalent).

(h) π_{1} and π_{2} are octahedral attached with the same S̃_{4}-extension, and Ad(π_{1}) and Ad(π_{2}) are twist equivalent by a quadratic characters.

Proof of Proposition 8.3. (cf. [Ra2000]) It is clear that (g) and (h) imply that π_{1} and π_{2} are twist equivalent. So

it suffices to show the other side.

First assume that Ad(π_{1}) and Ad(π_{2}) are equivalent.

Consider Π = π_{1} ⊠ π_{2}. Note that

Π ⊠ \overline{Π} ≅ 1 ⊞ Ad(π_{1}) ⊞ Ad(π_{2}) ⊞ Ad(π_{1}) ⊠ Ad(π_{2})

admits two copies 1 as its constituents. Hence Π is not cuspidal.

If Π contains a character ν, then \overline{π}_{2} ≅ π_{1} ⊗ ν. If Π is an isobaric sum of two cusp forms σ_{1} and σ_{2} on GL(2), then check that Λ^{2}(π_{1}⊠π_{2}) is equivalent to

(Ad(π_{1}) ⊞ Ad(π_{2})) ⊗ ω_{π_{1}}ω_{π_{2}}

which does not contain any character constituent; However, Λ^{2}(σ_{1} ⊞ σ_{2}) is equivalent

ω_{σ_{1}} ⊞ ω_{σ_{1}} ⊞ (σ_{1} ⊠ σ_{2})

which contains two GL(1)-constituents. Thus we get a contradiction, and (g) holds.

Furthermore, assume that

Ad(π_{2}) ≅ Ad(π_{1}) ⊗ ϵ

where ϵ is a character. Then

Ad(π_{2}) ≅ Ad(π_{1}) ⊗ ϵ^{−1}

and hence

Ad(π_{1}) ⊠ Ad(π_{1}) ≅ Ad(π_{2}) ⊠ Ad(π_{2}).

However,

Ad(π_{i}) ⊠ Ad(π_{i}) ≅ 1 ⊞ Ad(π_{i}) ⊞ A^{4}(π_{i}).

Hence

Ad(π_{1}) ⊞ A^{4}(π_{1}) ≅ Ad(π_{2}) ⊞ A^{4}(π_{2}).

If Ad(π_{1}) and Ad(π_{2}) are equivalent, we get (g). Otherwise, Ad(π_{1}), which is a nontrivial twist of Ad(π_{2}), must be contained in A^{4}(π_{2}).

So π_{1} and π_{2} are of solvable polyhedral type.

If π_{2} is tetrahedral, then

A^{4}(π_{2}) ≅ Ad(π_{2}) ⊞ ω ⊞ ω^{2}

where ω is some cubic character that Ad(π_{2}) admits as as self-twist. So this cannot happen.

Thus π_{2} and π_{1} are octahedral, and

A^{4}(π_{2}) ≅ I_{K}^{F}(μ) ⊞ Ad(π_{2}) ⊗ ϵ

where K is a quadratic extension of F such that Ad(π_{2}_{K}) allows a self twist by μ, and ϵ is the quadratic character cuts out K.

So they must come from the same S̃_{4}-extension, and hence (h) holds. □

Proof of Theorem 8.2.

Set π = Ad(π′′) From what we have seen (including the proof of Theorem 8.1), Π = π′ ⊠ Ad(π′) is cuspidal unless (i) Ad(π′) and Ad(π′′) are twist equivalent; or (ii) π′ is dihedral of type D_{6}, π = Ad(π′′) is non-normal cubic monomial, and (a) and (b) of Theorem 8.1 hold.

If (ii) holds, then π′′ must be octahedral, and (f) must hold. If (1) holds, then Ad(π′) and Ad(π′′) are twist equivalent. Then part (g) of Proposition 8.3 implies (d), and part (h) of this proposition implies (e).

□

The proof of Theorem A is now complete.

9. Proof of Theorem B

In this part we deduce Theorem B from Theorem A. First we need some preliminaries.

Recall that a cusp form π on GL(n) over F is essentially selfdual if \overline{π} is twist equivalent to π. Throughout this section, π′ and π denote cusp forms on GL(2) and GL(3) over F. We assume that π′ is not dihedral, and π is not twist equivalent to Ad(π′′) for any cusp form π′′.

First, from Theorem A, we have the following:

Corollary 9.1. If π′ is not of solvable polyhedral type and π is not essentially selfdual, then Π = π ⊠ π′ is cuspidal.

Lemma 9.2. Let K be any solvable extension of F. If π is not essentially selfdual, and if π_{K} does not admit any self twist, then π_{K} is not essentially selfdual.

Proof of Lemma 9.2.

First, assume that [K : F] = l a prime so that K/F is cyclic. Let θ ≠ 1 be a Galois conjugation of K over F, and τ be a character cutting out K/F.

Assume that π_{K} is essentially selfdual. Say \overline{π}_{K} ≅ π_{K} ⊗ μ, for a character μ. Applying θ, and being aware of the fact that π_{K} is fixed by θ, we get

\overline{π}_{K} ≅ π_{K} ⊗ (μ ∘ θ)

Since π_{K} does not allow a self twist, then μ ∘ θ = μ, hence μ must be a base change of some character α of C_{F} to K.

Hence, \overline{π} and π ⊗ α have the same base change over K/F, and thus must be twist equivalent. This shows that π is also essentially selfdual.

In general case, let K_{0} = F ⊂ K_{1} ⊂ … ⊂ K_{n} = K be a tower of cyclic extensions of prime degree. Assume that π_{K_{n}} = π_{K} is essentially selfdual, then as π_{K_{n}} does not allow a self twist, neither does π_{K_{i}} for any smaller i, thus applying the arguments above inductively, we claim that π_{K_{i}} is essentially selfdual. In particular, π must be essentially selfdual.

□

Proof of Theorem B.

First prove (a). Π is cuspidal from Corollary 9.1. First, assume only that π is not essentially selfdual and does not allow a self twist. Assume that Π allows a self twist by some character ν. Without loss of generality, may assume that ν is of order 2 or 3. Let K/F be cut out by ν. Thus Π_{K} = I_{K}^{F}(η) is Eisensteinian of type (2, 2, 2) or (3, 3).

However, π_{K} is cuspidal as π does not allow a self twist. From Theorem A (and the remark at the end of Section 5), Π_{K} cannot be of type (3, 3) as π′_{K} is not dihedral, type (2, 2, 2) as π′_{K} is not tetrahedral (as π′ and hence π′_{K} is not of solvable polyhedral type). Thus Π_{K} must be cuspidal, and hence Π does not allow a self twist.

Moreover, assume that π is not monomial, in particular, π is not induced from a non-normal cubic extension. Want to prove that Π is not either.

Assume that Π ≅ I_{K′}^{F}(η) where η is some cusp form on GL(2) over K′ which is non-normal cubic over F. Let M be the normal closure of K′ over F and E be the unique quadratic subextension of M over F. Then π′_{E} is still not of solvable polyhedral type. And π_{E} is not cyclic monomial as π is not monomial. From Lemma 9.2, π_{E} is not essentially selfdual. Thus the first part of (a) implies that Π_{E} does not allow a self twist. Hence Π_{M} is still cuspidal. However, Π_{M} ≅ I_{K′}^{F}(η)_{M} = η_{M} ⊞ (η_{M} ∘ θ′) ⊞ (η_{M} ∘ θ^{′2}) where θ′ is the character cutting out M/E. We get a contradiction.

Thus (a) is proved.

Next prove (b). It suffices to prove that Π_{K} is cuspidal for any solvable extension K of F.

Since π′ is not of solvable polyhedral type, neither is π′_{K}. As π is not of solvable type, then π_{K} must be cuspidal. We claim that π_{K} cannot allow a self twist. Otherwise, say π_{K} ≅ π_{K} ⊗ ν. ν must be a cubic character. Let K_{1}/K be cut out by ν, then π_{K_{1}} should be Eisensteinian. However, K_{1}/F is contained in some solvable normal extension. Thus π is of solvable type. Contradiction.

Hence the claim. From Lemma 9.2, π_{K} is not essentially selfdual. Corollary 9.1 then implies that Π_{K} is cuspidal. Thus Π is not of solvable type.

□

Before we finish this section, we would like to prove the following lemma.

Lemma 9.3. Let π be a cusp form on GL(2m + 1) over F. Assume that π is regular algebraic at infinity, and F is not totally complex. Then π is not monomial.

Proof of Lemma 9.3.

Assume that π = I_{K′}^{F}(ν) where K′ is a field extension of F of degree 2m + 1, and ν is an algebraic character of C_{K}. As F is not totally complex, neither is K′ as [K′ : F] is odd. Thus from Weil ([We1955]), ν must be of the form ν_{0}∥ · ∥^{k} where ν_{0} is a character of finite order. Thus I_{K′}^{F}(ν) does not contain any nontrivial algebraic character at infinity, and hence cannot be regular at infinity. □

Appendix

The object here is to justify the statement made in the Introduction that it is possible to construct, for n > 2 even (resp. n = 4), non-selfdual, monomial (resp. non-monomial, but imprimitive), cuspidal cohomology classes for Γ ⊂ SL(n, ℤ).

Theorem E. (a) Fix any integer m > 1. Then there exists a cuspidal automorphic representation π of GL(2m, 𝔸_{ℚ}), which is not essentially self-dual, contributing to the cuspidal cohomology in degree m^{2}, and admitting a self-twist relative to a character of order 2m. In fact, this can be done for any, not necessarily constant, coefficient system.

(b) There exists a cusp form π on GL(4)/ℚ contributing to the cuspidal cohomology in degree 4, which is not essentially self-dual. It admits a self-twist relative to a quadratic character, but not relative to any quartic character.

In both cases it will be apparent from the proof below that there are infinitely many such examples, and by the discussion in section 3, they give rise, for arbitrary coefficient systems V, to non-selfual, cuspidal cohomology classes in H^{∗}(Γ, V) for suitable congruence subgroups Γ ⊂ SL(2m, ℤ) (m ≥ 2). Moreover there are naturally associated Galois representations, which are monomial in case (a), and are imprimitive but non-monomial in case (b).

Proof. (a) For any m > 1, fix a finite-dimensional coefficient system V. Then a cuspidal automorphic representation π of GL(2m, 𝔸_{F}) contributes to the cuspidal cohomology with coefficients in V iff it is algebraic with infinity type (in the unitary normalization): (cf. [C1988])

\{(z/|z|)^{k_{1}}, (\overline{z}/|z|)^{k_{1}}, (z/|z|)^{k_{2}}, (\overline{z}/|z|)^{k_{2}}, …, (z/|z|)^{k_{m}}, (\overline{z}/|z|)^{k_{m}}\},

where (k_{1}, k_{2}, …, k_{m}) is an ordered m-tuple of integers (determined by V) satisfying

k_{1} > k_{2} > ⋯ > k_{m}.

In particular, π_{∞} is regular. If V ≃ ℂ, then as seen in Theorem 3.2, k_{j} = 2(m−j)+1. Now pick any cyclic, totally real extension F of ℚ of degree m, and a totally imaginary quadratic extension K of F which is normal over ℚ. Let v_{1}, …, v_{m} denote the archimedean places of F, and for each j let w_{j} be a complex place of K above v_{j}. Choose an algebraic Hecke character χ of of K such that

(1)  χ_{w_{j}}(z) = (\frac{z}{|z|})^{2(m−j)+1} |z\overline{z}|^{2m−1} ∀ j ≤ m.

That such a character exists is a consequence of the discussion on page 3 of [We1955]. To elaborate a bit for the sake of the uninitiated, the necessary and sufficient condition for the existence of χ as above is that there be a positive integer M such that the following holds for all units u ∈ 𝔒_{K}^{∗} with components u_{j} at w_{j}:

(2)  ∏_{j≤m} (\frac{u_{j}}{\overline{u}_{j}} )^{Mk_{j}} = 1.

But the index of the real units 𝔒_{F}^{∗} in 𝔒_{K}^{∗} is finite by the Dirichlet unit theorem, and hence for a suitable M, u_{j}^{M} is real for all j. The desired identity (2) follows.

Next pick a finite order character ν of K and set

(3)  Ψ = χν.

Let τ be the non-trivial automorphism (complex conjugation) of K/F, and δ the quadratic character of F attached to K. Then we may, and we will, choose ν in such a way that

(4)  Ψ(Ψ ∘ τ) ≠ μ ∘ N_{K/ℚ}

for any character μ of ℚ, which is possible – and this is crucial – because [K : ℚ] ≥ 4 and so F = {x ∈ K | x^{τ} = x} ≠ ℚ. Put

π := I_{K}^{ℚ}(Ψ),

where I denotes automorphic induction ([AC1989]). Note that π makes sense because K/ℚ is solvable and normal, allowing automorphic induction to be defined. By looking at the infinity type (1) we see that π is regular and algebraic.

By construction, π_{∞} contributes to cohomology, and π is cuspidal because the infinity type of Ψ precludes it from being Ψ ∘ σ for any non-trivial automorphism σ of K/ℚ. To elaborate, note first that η := I_{K}^{F}(Ψ) is cuspidal and algebraic, corresponding to a Hilbert modular newform on GL(2)/F of the prescribed weight at infinity. Since the automorphic induction is compatible with doing it in stages, π is just I_{F}^{ℚ}(η), and since F/ℚ is cyclic, it suffices to check that for any automorphism τ of F, η and η ∘ τ are not isomorphic, which is clear from the properties of Ψ.

It remains to check that π is not essentially self-dual, which comes down to checking the same for the 2m-dimensional representation ρ of W_{ℚ} induced by the character Ψ of W_{K}. For this we need, by Mackey, to check that Ψ^{−1} ≠ (μ_{K})Ψ ∘ σ for any automorphism σ of K and any character μ of ℚ. By our choice of the infinity type, this is clear for any σ ≠ τ (and any μ).

For σ = τ, this is the content of (4). So we are done with the proof of part (a).

(b) Let K/ℚ be an imaginary quadratic field and β a non-dihedral cusp form of weight 2 over K with ℚ coefficients, such that no twist of β is a base change from ℚ. Here weight 2 signifies the fact that the Langlands parameter of β_{∞} is given by

σ(β_{∞}) = \{z/|z|, \overline{z}/|z|\} ⊗ (z\overline{z})

Here are two explicit (known) examples with these properties: First consider the (non-CM) elliptic curve

E_{1} : y^{2} + xy = x^{3} + (3 + \sqrt{−3})x^{2}/2 + (1 + \sqrt{−3})x/2

over K_{1} = ℚ(\sqrt{−3}). This was shown to be associated to a cusp form β_{1} of weight 2 and trivial central character on GL(2)/K_{1} by R. Taylor ([Ta1994]) such that a_{P} (E_{1}) = a_{P} (β_{1}) for a set of primes P of density 1. (In fact, recent results can be used to show that this equality holds outside a finite set of primes P.)

For the second example, set K_{2} = ℚ(i) and Q the prime ideal generated by 8+13i. Then there is a corresponding cusp form β_{2} of weight 2, conductor Q and trivial central character, as seen on page 344 of the book [EGM1998] by Elstrodt, Grunewald and Mennicke. Its conjugate by the non-trivial automorphism θ of K_{2} will have conductor Q^{θ} and so no twist of β_{2} can be a base change from ℚ. There is a corresponding elliptic curve

E_{2} : y^{2} + iy = x^{3} + (1 + i)x^{2} + ix

over K_{2} of conductor Q, and one knows for many P that a_{P} (E_{2}) = a_{P} (β_{2}). Next choose an algebraic Hecke character χ of K such that

χ_{∞}(z) = (z/|z|)^{2}|z\overline{z}|^{2}.

For example, we can choose χ to be the square of a Hecke character associated to a CM elliptic curve. Now consider, for j = 1, 2, the automorphic induction

π_{j} := I_{K}^{ℚ}(β_{j} ⊗ χ).

The infinity types chosen imply that either β_{j} ⊗ χ is not isomorphic to its transform by the non-trivial automorphism of K_{j}. So π_{j} is a cusp form on GL(4)/ℚ. It is cohomological, as easily seen by its archimedean parameter. That π_{j} is not essentially self-dual is an immediate consequence of the infinity types of χ and β. Finally, π_{j} admits a quadratic self-twist, namely by the character of ℚ associated to K_{j}, but it admits no quartic self-twist as β_{j} is non-dihedral. We are now done.

□

References

[AC1989] J. Arthur and L. Clozel, Base Change, and the Advanced Theory of the Trace Formula, Annals of Math. Studies 120, Princeton, 1989.

[AGG1984] A. Ash, D. Grayson, P. Green, Computations of cuspidal cohomology of congruence subgroups of SL(3, Z), Journal of Number Theory 19 (1984), no. 3, 412–436.

[BaR1994] L. Barthel and D. Ramakrishnan, A non-vanishing result for twists of Lfunctions of GL(n), Duke Math. Journal 74, No. 3 (1994), 681–700.

[BoJ1979] A. Borel and H. Jacquet, Automorphic forms and automorphic representations (with a supplement ”On the notion of an automorphic representation” by R. P. Langlands), Proc. Sympos. Pure Math., XXXIII, Automorphic forms, representations and L-functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 1, 189–207, Amer. Math. Soc., Providence, R.I., 1979.

[BoW1980] A. Borel and N.R. Wallach, Continuous cohomology, discrete subgroups, and representations of reductive groups, Annals of Mathematics Studies 94, Princeton University Press, Princeton, N.J.; University of Tokyo Press, Tokyo, 1980.

[C1988] L. Clozel, Motifs et formes automorphes: Applications du principe de fonctorialité, Automorphic forms, Shimura varieties, and L-functions, Vol. I (Ann Arbor, MI, 1988), 77–159.

[EGM1998] J. Elstrodt, F. Grunewald and J. Mennicke, Groups acting on hyperbolic space, Springer Monographs in Math., Berlin (1998).

[vGKTV1997] B. van Geemen, W. van der Kallen, J. Top and A. Verberkmoes, Hecke eigenforms in the cohomology of Congruence subgroups of SL(3, ℤ), Experimental Mathematics 6:2, 163–174 (1997).

[vGT1994] B. van Geemen and J. Top, A non-selfdual automorphic representation of GL_{3} and a Galois representation, Inventiones Math. 117 (1994), no.3, 391–401.

[vGT2000] B. van Geemen and J. Top, Modular forms for GL(3) and Galois representations, in Algorithmic number theory (Leiden, 2000), 333–346, Lecture Notes in Comput. Sci. 1838, Springer, Berlin (2000).

[HaT2000] M. Harris and R. Taylor, On the geometry and cohomology of some simple Shimura varieties, With an appendix by Vladimir G. Berkovich. Annals of Mathematics Studies, 151. Princeton University Press, Princeton, NJ, 2001.

[He2000] G. Henniart, Une preuve simple des conjectures de Langlands pour GL (n) sur un corps p–adique, Inventiones Math. 139 no. 2 (2000), 439–455 (French).

[JPSS1983] H. Jacquet, I. Piatetski-Shapiro and J. Shalika, Rankin-Selberg convolutions, Amer. J. Math.105, 367–464 (1983).

[JS1981] H. Jacquet and J.A. Shalika, Euler products and classification of automorphic forms I and II, Amer. J. of Math. 103 (1981), 499–558 & 777–815.

[JS1990] H. Jacquet and J.A. Shalika, Rankin–Selberg convolutions: Archimedean theory in Piatetski–Shapiro Festschrift, Isreal Math. conf. Proc., Part II, 125–207, The Weizmann Science Press of Isreal (1990).

[K2001] H. Kim, Functoriality of the exterior square of GL_{4} and the symmetric fourth power of GL_{2}, J. Amer. Math. Soc. 16 (2003), no.1, 139–183.

[KSh2002-1] H. Kim and F. Shahidi, Functorial products for GL(2) × GL(3) and the symmetric cube for GL(2) (with an appendix by C.J. Bushnell and G. Henniart), Annals of Math (2) 155, (2002), no.3, 837–893.

[KSh2002-2] H. Kim and F. Shahidi, Cuspidality of symmetric powers with applications,

Duke Journal of Math (112), no.1, 177–197.

[Ku1980] S. Kumaresan, On the canonical k-types in the irreducible unitary g-modules with nonzero relative cohomology, Invent. Math. 59 (1980), no. 1, 1–11.

[La1979-1] R.P. Langlands, On the notion of an Automorphic Representation. A Supplement to the preceeding paper, Proceedings of Symposia in Pure Mathematics (Corvallis), Vol 33 (1979), part 1, 203–207.

[La1979-2] R.P. Langlands, Automorphic Representations, Shimura Varieties, and Motives. Ein Märchen, Proceedings of Symposia in Pure Mathematics (Corvallis), Vol 33 (1979), part 2, 205–246.

[La1980] R.P. Langlands, Base change for GL(2), Annals of Math. Studies 96, Princeton, 1980.

[MW1995] C. Moeglin and J.-L. Waldspurger, Spectral decomposition and Eisenstein series: Une paraphrase de l’Écriture, Cambridge University Press (1995).

[Ra2000] D. Ramakrishnan, Modularity of the Rankin–Selberg L–series, and the multiplicity one for SL(2), Ann. Math. 150 (2000), 45–111.

[RaWa2001] D. Ramakrishnan and S. Wang, On the exceptional zeros of Rankin–Selberg L-Functions, Compositio Mathematica 135 (2003), 211–244.

[Sh1988] F. Shahidi, On the Ramanujan conjecture and finiteness of poles for certain L-functions. Ann. of Math. (2) 127 (1988), no. 3, 547–584.

[WSt2003] W. Stein, Modular forms database, see http://modular.fas.harvard.edu/Tables/index.html.

[Ta1994] R. Taylor, l-adic representations associated to modular forms over imaginary quadratic fields II, Inventiones Math. 116 (1994), no. 1-3, 619–643.

[VZ1984] D.A. Vogan, G.J. Zuckerman, Unitary representations with nonzero cohomology, Compositio Math. 53 (1984), no. 1, 51–90.

[Wa2003] Song Wang, On the Symmetric Powers of Cusp Forms on GL(2) of Icosahedral Type, IMRN 2003:44 (2003), 2373–2390.

[We1955] A. Weil, On a certain type of characters of the idle-class group of an algebraic number-field, Proceedings of the International Symposium on Algebraic Number Theory, Tokyo & Nikko, 1955, pp. 1–7, Science Council of Japan, Tokyo, 1956.

Dinakar Ramakrishnan 253-37 Caltech Pasadena, CA 91125, USA. dinakar@caltech.edu

Song Wang Department of Mathematics Yale University New Haven, CT 06520, USA. song.wang@yale.edu