KYUNGPOOK Math. J. 2019; 59(2): 225-231
Hyperinvariant Subspaces for Some 2×2 Operator Matrices, II
Il Bong Jung∗
Department of Mathematics, Kyungpook National University, Daegu 41566, Korea
e-mail : ibjung@knu.ac.kr

Eungil Ko
Department of Mathematics, Ewha Womans University, Seoul 03760, Korea
e-mail : eiko@ewha.ac.kr

Carl Pearcy Department of Mathematics, Texas A&M University, College Station, TX 77843, USA
e-mail : cpearcy@math.tamu.edu
* Corresponding Author.
Received: February 1, 2019; Revised: June 12, 2019; Accepted: June 14, 2019; Published online: June 23, 2019.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract

In a previous paper, the authors of this paper studied 2 × 2 matrices in upper triangular form, whose entries are operators on Hilbert spaces, and in which the the (1, 1) entry has a nontrivial hyperinvariant subspace. We were able to show, in certain cases, that the 2 × 2 matrix itself has a nontrivial hyperinvariant subspace. This generalized two earlier nice theorems of H. J. Kim from 2011 and 2012, and made some progress toward a solution of a problem that has been open for 45 years. In this paper we continue our investigation of such 2 × 2 operator matrices, and we improve our earlier results, perhaps bringing us closer to the resolution of the long-standing open problem, as mentioned above.

Keywords: invariant subspace, hyperinvariant subspace, compact operator.
1. Introduction

The notation and terminology herein are completely standard and exactly the same as in [5]; nevertheless, we briefly review the main definitions. Throughout this note ℋ will always denote a separable, infinite dimensional, complex, Hilbert space, and ℬ(ℋ) the algebra of all bounded linear operators on ℋ. The space of scalar multiples of the identity operator 1 is denoted, as usual, by ℂ1. For T in ℬ(ℋ) we write

${T}′={S∈B(H):ST=TS},$

for the commutant of T and σp(T) for the point spectrum of T. A subspace (i.e., a closed linear manifold) ℳ ⊂ ℋ is said to be a nontrivial invariant subspace (notation: n.i.s.) for an operator T in ℬ(ℋ) if (0) ≠ ℳ ≠ ℋ and Tℳ ⊂ ℳ. If ℳ is a n.i.s. for T and furthermore has the property that T′ℳ ⊂ ℳ for all T′ ∈ {T}′, then ℳ is said to be a nontrivial hyperinvariant subspace (notation: n.h.s.) for T. As is well-known, the problem of whether every T in ℬ(ℋ) has a n.i.s. (called the invariant subspace problem for operators on Hilbert space) remains unsolved, although many partial results are known. (For more information about this topic, the reader may wish to consult the excellent book [1]). It is also the case that there are two related problems whose answers are not known. The first is the question of whether every operator in ℬ(ℋ)ℂ1 has a n.h.s., called the hyperinvariant subspace problem for operators on Hilbert space. The second (called sometimes the hypertransitive operator problem for operators on Hilbert space) is the question of whether there exists an operator T in ℬ(ℋ) such that for every nonzero vector x in ℋ, the orbit of x under T, namely ${Tnx}n=0∞$, is dense in ℋ.

For the readers’ convenience we now restate [5, Theorem 2.1]:

### Theorem 1.1

Let A, B, and C be arbitrary operators in ℬ(ℋ), and define TC ∈ ℬ(ℋ ⊕ ℋ) matricially as

$TC:=(AC0B).$

If there exists a pair (X, ℳ), where X ∈ ℬ(ℋ) with AX = XB, andis a n.h.s. for A such that Xℋ ⊄ ℳ, then for every D in ℬ(ℋ), TD has a n.h.s.

Observe now that every operator S in ℬ(ℋ)ℂ1 that is known to have a n.i.s. but not known to have a n.h.s. is unitarily equivalent to some operator TC in ℬ(ℋ ⊕ ℋ) of the form (1.1) (but without the hypothesis that A has a n.h.s.). This follows from the fact that if either the known n.i.s. for S or its orthocomplement is finite dimensional, then S or S* has nonempty point spectrum, from which the existence of a n.h.s. for S follows trivially. Thus when studying operators like S, no generality is lost by instead considering operators of the form TC in (1.1). Moreover there are such operators for which the operator A in (1.1) is known to have a n.h.s., and it is this class of operators to be studied herein.

### Example 1.2

Let {en}n∈ℤ be an orthonormal basis for ℋ and let w = {wn}n∈ℤ be a bounded sequence of positive numbers that is also bounded away from 0. Define Ww ∈ ℬ(ℋ) by the equations

$Wwen=wnen-1, n∈ℤ.$

Obviously Ww is an invertible bilateral weighted shift, and with ℳ defined as

$M=𝖵n∈ℕ{e-n},$

one sees easily that ℳ is a n.i.s. for Ww and Ww| is unitarily equivalent to the forward weighted unilateral shift Vw defined by

$Vw-e-n=w-ne-(n+1), n∈ℕ.$

Moreover

$M⊥=𝖵n∈ℕ0{en},$

and if we define $Vw*+$ by the equations

$Vw*+e0=0, Vw*+en=wnen-1, n∈ℕ,$

then obviously $Vw*+$ is a backward weighted unilateral shift and Ww is unitarily equivalent to the operator TC in (1.1), where A is unitarily equivalent to Vw, B is unitarily equivalent to $Vw*+$, and C is unitarily equivalent to the operator of rank one R: ℋ → ℋ defined by

$Re0=w0e-1, Ren=0, n∈ℕ.$

Moreover, it is well-known that all forward weighted unilateral shifts have nontrivial hyperinvariant subspaces (cf., e.g., [10]). Thus if all operators of the form TC in (1.1), where A has a n.h.s., were known to have a n.h.s., then the longtime, still open problem of whether invertible weighted bilateral shifts have a n.h.s. would be solved.

On the basis of Example 1.2 the question of whether all operators of the form TC in (1.1) have a n.h.s. when A does is of considerable interest, and in this note we continue to make progress on this problem, improving some results in [5].

2. The Class (RIH)

We next define a (perhaps new) class of operators to which our main theorem below (Theorem 2.4) applies.

### Definition 2.1

An operator T in ℬ(ℋ) will be said to belong to the class (RIH) (or (RIH)(ℋ) if necessary to avoid confusion) if T satisfies the following three conditions:

• (a) neither T nor T* has nonempty point spectrum,

• (b) T has a n.h.s., and

• (c) for every n.h.s. of T, each of and has a n.h.s.

### Remark 2.2

The name (RIH) comes from the phrase “restrictions inherit nontrivial hyperinvariant subspaces”.

The interest in the class (RIH) arises from the fact that operators T in (RIH) have particularly nice hyperinvariant subspace lattices (notation: Hlat(T)).

### Proposition 2.3

Let T ∈ ℬ(ℋ) be an arbitrary operator in the class (RIH). Then Hlat(T) has the following properties.

• (I) T ∈ (RIH) if and only if T* ∈ (RIH).

• (II) For every ℳ ≠ (0) in Hlat(T) and everyin Hlat(T*), $dimM=dimN=ℵ0.$

• (III) ∩{ℳ ⊂ ℋ: ℳ ∈ Hlat(T) and ℳ ≠ (0)} = (0).

• (IV) ∨{ℳ ⊂ ℋ: ℳ ∈ Hlat(T) and ℳ ≠ ℋ} = ℋ.

Proof

All of I)–IV) follow easily from the definition of (RIH), the fact that ℳ is a n.h.s. for T if and only if ℳ is a n.h.s. for T*, and the fact that for any T in ℬ(ℋ), the intersection of any family of hyperinvariant subspaces for T is again a hyperinvariant subspace for T.

The principal result of this note is the following, which is of interest because of the important classes of operators in ℬ(ℋ) that are subsets of (RIH), as we shall see below.

### Theorem 2.4

Let A ∈ (RIH)(ℋ) and let B be an arbitrary operator in ℬ(ℋ). If there exists a nonzero X ∈ ℬ(ℋ) such that AX = XB, then for every C in ℬ(ℋ), the operator TC as in (1.1) has a n.h.s.

Proof

To apply Theorem 1.1, we observe that if X ≠ 0 and AX = XB, then it suffices to show that A has a n.h.s. ℳ such that Xℋ ⊄ ℳ. If (Xℋ) = ℋ, then every n.h.s. ℒ of A has this property, so we may suppose that (Xℋ) = ℒ ≠ ℋ. By III) of Proposition 2.3 we know that

$∩{N⊂H:N∈HlatA and N≠(0)}=(0),$

from which it follows trivially that we cannot have for every n.h.s. of A, and thus the proof is complete.

3. Applications

In this section we set forth some important classes of operators to which Theorem 2.4 applies, and thus we obtain new and better sufficient conditions on the operator A in the matrix in (1.1) under which the operator TC there has a n.h.s.

### Definition 3.1

An operator A in ℬ(ℋ) will be said to belong to the class (CK)(or (CK) (ℋ)) if there exists a (nonzero) compact operator K in ℬ(ℋ) such that

$σp(A)=σp(K)=σp(A*)=σp(K*)=∅$

and AK = KA. (The notation (CK) arises from the phrase “commutes with a compact operator”).

### Proposition 3.2

(CK) (ℋ) ⊂ (RIH) (ℋ).

Proof

Let A ∈ (CK) (ℋ) and commute with the (nonzero) compact operator K, where

$σp(A)=σp(K)=σp(A*)=σp(K*)=∅.$

Then A*K* = K*A* and by V. Lomonosov’s theorem ([9]), A has a n.h.s. ℳ. Now let be an arbitrary n.h.s. for A and note that and that commutes with . Moreover, is a nonzero compact operator, so has a n.h.s. It follows easily by checking the requirements that A ∈ (RIH).

### Corollary 3.3

Suppose A ∈ (CK) (ℋ) and B is an arbitrary operator in ℬ(ℋ). If there exists X ≠ 0 in ℬ(ℋ) such that AX = XB, then for every C ∈ ℬ(ℋ), the operator TC as in (1.1) has a n.h.s.

Proof

By Proposition 3.2, A ∈ (RIH), and the result is then immediate from Theorem 2.4.

### Corollary 3.4

Suppose A is any nonzero compact operator in ℬ(ℋ) and B is an arbitrary operator there. If there exists a nonzero operator X such that AX = XB, then for every C in ℬ(ℋ), the operator TC as in (1.1) has a n.h.s.

Proof

If A (or A*) has nonempty point spectrum, then the finite dimensional associated eigenspace is a n.h.s. for TC, whereas otherwise, since A ≠ 0, A ∈ (CK) and the result follows from Corollary 3.3.

### Remark 3.5

H.K. Kim in [6] raised the very interesting question of whether every operator TC in (1.1) such that A is a nonzero compact operator has a n.h.s. This problem remains open still, and Corollary 3.4 above seems presently to be the best result in the direction of showing that the answer may be “yes”. Note that if the answer eventually turns out to be “yes”, then that theorem would be a beautiful generalization of V. Lomonosov’s first theorem in [9], namely that every nonzero compact operator in ℬ(ℋ) has a n.h.s.

We now turn to another important class of operators pertinent to the operator in (1.1), the treatment of which is parallel to that of the class (CK).

### Definition 3.6

An operator A in ℬ(ℋ) will be said to belong to the class (CN) (or (CN)(ℋ)) if there exists a (nonzero) normal operator N in ℬ(ℋ) not of uniform multiplicity ℵ0 such that

$σp(A)=σp(N)=σp(A*)=σp(N*)=∅$

and AN = NA.

It is well-known from the multiplicity theory of normal operators (cf., e.g., [2]) that every operator A in the commutant of a normal operator N as in Definition 3.6 is an n-normal operator or a direct sum of operators at least one of which is an n-normal operator (for some n ∈ ℕ). And, via [4] and [3], all such operators are known to have a n.h.s. . Note that by Fuglede’s theorem, AN* = N*A, and therefore and . In other words, is a reducing subspace for N and is again a normal operator that commutes with and . But then, as above, has a n.h.s. These remarks are sufficient to constitute a proof of the following.

### Corollary 3.7

Suppose A ∈ (CN) (ℋ) and B is an arbitrary operator in ℬ(ℋ). If there exists a nonzero operator X such that AX = XB, then for every C ∈ ℬ(ℋ) the operator TC in (1.1) has a n.h.s.

We note in particular, that if A in Corollary 3.7 is a nonscalar normal operator, then the conclusions of that corollary remain true for all operators TC as in (1.1).

### Remark 3.8

H.J. Kim also studied in [7] matrices TC as in (1.1), where A is a normal operator, and in some cases he obtained the existence of a n.h.s. for TC. (This topic was also considered in the paper [8].)

We close this note by posing some unsolved problems concerning hyperinvariant subspaces for certain operators TC as in (1.1).

### Problem 3.9

Let TC be as in (1.1), where A and C are compact and nonzero, and B is an arbitrary operator. Does TC have a n.h.s.?

### Problem 3.10

Let TC be as in (1.1), where A has a n.h.s., B is an arbitrary operator, and C is nonzero and has finite rank. Does TC have a n.h.s.?

### Problem 3.11

Let TC be as in (1.1), where A has a n.h.s., B is an arbitrary operator, and C = 1. Does TC have a n.h.s.?

Acknowledgements

The first author was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government (MSIT) (2018R1A2B6003660). The second author was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2016R1D1A1B03931937).

References
1. I. Chalendar, and J. Partington. Modern approaches to the invariant-subspace problem. Cambridge Tracts in Math, 188, Cambridge Univ. Press, Cambridge, 2011.
2. J. Conway. Subnormal operators. Research Notes in Math, 51, Pitman Publ. Program, Boston, 1981.
3. R. Douglas, and C. Pearcy. Hyperinvariant subspaces and transitive algebras. Michigan Math J., 19(1972), 1-12.
4. T. Hoover. Hyperinvariant subspaces for n-normal operators. Acta Sci Math (Szeged)., 32(1971), 109-119.
5. IB. Jung, E. Ko, and C. Pearcy. Hyperinvariant subspaces for some 2 × 2 operator matrices. Kyungpook Math J., 58(2018), 489-494.
6. HJ. Kim. Hyperinvariant subspaces for operators having a compact part. J Math Anal Appl., 386(2012), 110-114.
7. HJ. Kim. Hyperinvariant subspaces for operators having a normal part. Oper Matrices., 5(2011), 487-494.
8. H. Kim, and C. Pearcy. Subnormal operators and hyperinvariant subspaces. Illinois J Math., 23(1979), 459-463.
9. V. Lomonosov. Invariant subspaces of the family of operators that commute with a completely continuous operator, (in Russian). Funktsional Anal i Prilozen., 7(1973), 55-56.
10. A. Shields. Weighted shift operators and analytic function theory. Topics in Operator Theory, Math Surveys and Monographs., 13(1974), 49-128.