Kyungpook Mathematical Journal 2018; 58(3): 489-494
Hyperinvariant Subspaces for Some 2×2 Operator Matrices
Il Bong Jung*
Department of Mathematics, Kyungpook National University, Daegu 41566, Korea
e-mail : ibjung@knu.ac.kr

Eungil Ko
Department of Mathematics, Ewha Womans University, Seoul 03760, Korea
e-mail : eiko@ewha.ac.kr

Carl Pearcy
Department of Mathematics, Texas A&M University, College Station, TX 77843, USA
e-mail : cpearcy@math.tamu.edu
*Corresponding Author.
Received: December 26, 2017; Accepted: February 20, 2018; Published online: September 23, 2018.

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract

The first purpose of this note is to generalize two nice theorems of H. J. Kim concerning hyperinvariant subspaces for certain classes of operators on Hilbert space, proved by him by using the technique of “extremal vectors”. Our generalization (Theorem 1.2) is obtained as a consequence of a new theorem of the present authors, and doesn’t utilize the technique of extremal vectors. The second purpose is to use this theorem to obtain the existence of hyperinvariant subspaces for a class of 2 × 2 operator matrices (Theorem 3.2).

Keywords: invariant subspace, hyperinvariant subspace, extremal vector, transitive algebra.
1. Introduction

Let ℋ be a separable, infinite dimensional, complex Hilbert space and ℬ(ℋ) the algebra of all bounded operators on ℋ. For T ∈ ℬ(ℋ), we write {T}′ for the commutant of T (i.e., for the algebra of all S ∈ ℬ(ℋ) such that TS = ST). A subspace ℳ ⊂ ℋ is invariant for T in ℬ(ℋ) if T ℳ ⊂ ℳ, and a subspace ℳ is hyperinvariant for T if it is an invariant subspace for all S in {T}′. The question whether every operator in ℬ(ℋ) has a nontrivial invariant subspace, which has been around since von Neumann studied it in the 1930’s, is still an open problem. Moreover the question of whether every operator in ℬ(ℋ) ℂ1 has a nontrivial hyperinvariant subspace is also open. The results in this note contribute to this circle of ideas.

In two recent papers [5] and [6], H. J. Kim, using the technique of “extremal vectors” introduced by Ansari and Enflo in [1] (for more information about this technique, see the book [3]), proved two nice theorems which we combine as

### Theorem 1.1

(H. J. Kim) Let T ∈ ℬ(ℋ ⊕ ℋ) be given matricially as

$T=(AC0B),$

where A, B, and C are arbitrary operators in ℬ(ℋ) such that A is either a nonzero compact operator or a nonscalar normal operator. Then at least one of T and

$T˜=(BD0A),$

where D is arbitrary operator in ℬ(ℋ), has a nontrivial hyperinvariant subspace (notation: n.h.s.).

The purpose of this note is first to give a short and simpler proof of a better theorem, and then to use this result to obtain the existence of n.h.s. for a class of 2×2 matrices with operator entries (Theorem 3.2). Our first result is the following.

### Theorem 1.2

Let A and B be operators in ℬ(ℋ) such that either A or B has a n.h.s. Then either

• for every operator C in ℬ(ℋ), the operator TC ∈ ℬ(ℋ ⊕ ℋ) given matricially as$TC=(AC0B),$

has a n.h.s., or

• for every operator D in ℬ(ℋ), the operator T̃D given matricially as$T˜D=(BD0A),$

has a n.h.s.

2. Preliminary Results

Before proving Theorem 1.2, we obtain two needed results. The first one is a new theorem of the present authors, and the second is an elementary proposition about transitive subalgebras of ℬ(ℋ ⊕ ℋ) (i.e., subalgebras with the property that has no nontrivial invariant subspace). The first of these two results generalizes theorems from [4].

### Theorem 2.1

Suppose T = TC is as in (1.1), where A, B, and C are arbitrary operators in ℬ(ℋ) and there exists X in ℬ(ℋ) satisfying AX = XB. If either

• there exists a n.h.s.for A such that X ℋ ⊂ ℳ, or

• there exists a n.h.s.for B such that ker X,

then T has a n.h.s.

Proof

We prove b); the proof of a) is quite similar and left to the reader. Thus we are given a n.h.s. for B and an operator X in ℬ(ℋ) such that AX = XB and ker X. With T as in (1.1), denote its commutant by

${T}′={(LσMσNσPσ):σ∈Σ}.$

Since every matrix in (2.1) commutes with T, upon doing the matrix multiplication, for each σ ∈ ∑ we obtain four equations, with the one corresponding to the (2, 1) entry of the product being

$BNσ=NσA, σ∈Σ.$

Multiplication of (2.2) on the right by X gives

$BNσX=NσAX=NσXB, σ∈Σ,$

so for each σ ∈ ∑, Nσ X commutes with B and therefore , σ ∈ ∑. By hypothesis there exists such that Xy ≠ 0. Finally, let us suppose, to obtain a contradiction, that {T}′ is transitive. It then follows easily from Proposition 2.2 below (which is completely independent of this theorem) that {Nσ Xy : σ ∈ ∑} = ℋ, which contradicts the fact that for all σ ∈ ∑. Thus {T}′ is not transitive and the proof is complete.

### Proposition 2.2

Suppose that

$S={(LσMσNσPσ):σ∈Σ}$

is a transitive subalgebra of ℬ(ℋ ⊕ ℋ), and let x, y ∈ ℋ with x ≠ 0. Then for every ɛ > 0, there exists Lσ1 [respectively, Mσ2, Nσ3, Pσ4 ] such that ||Lσ1xy|| < ɛ [respectively, ||Mσ2xy|| < ɛ, ||Nσ3xy|| < ɛ, ||Pσ4xy|| < ɛ].

Proof

It is well-known (cf., e.g., [8]) that every transitive subalgebra is 1-transitive, meaning that for every x ≠ 0 and y in ℋ and every ɛ > 0, there exists such that ||Aɛ xy|| < ɛ. If we apply this fact to the transitive subalgebra and the vectors (x, 0)t and (y, 0)t, we obtain an element

$(LσMσNσPσ)∈S$

such that

$ɛ>‖(LσMσNσPσ) (x0)-(y0)‖ =(‖Lσx-y‖2+‖Nσx‖2)1/2≥ ‖Lσx-y‖.$

By changing the positions of the vectors x and y in the direct sum ℋ ⊕ ℋ, the other three desired inequalities follow similarly.

Proof of Theorem 1.2

We shall prove i). The proof of ii) is essentially the same. To establish i) there are two cases - that in which A has a n.h.s. and that in which B has a n.h.s. Once again the proofs are virtually indistinguishable, so we shall content ourselves with proving i) under the condition that B has a n.h.s. . Thus, by virtue of Theorem 2.1 b), it is sufficient to exhibit an operator X in ℬ(ℋ) and a nonzero vector v such that AX = XB and Xv ≠ 0. We may and do suppose that ii) is false, that is, there exists D0 in ℬ(ℋ) such that D0 as in (1.2) has no n.h.s. We write the commutant of {D0} as

${T˜D0}′={(Lσ′Mσ′Nσ′Pσ′):σ∈Σ},$

and we are given that the algebra {D0}′ is transitive. Since D0 commutes with every matrix in (2.3), we obtain, for each σ ∈ ∑, four equations, one of which is

$ANσ′=Nσ′B, σ∈Σ.$

Let now v0 be an arbitrary nonzero vector in . It is an easy consequence of Proposition 2.2 that there exists σ0 ∈ ∑ such that $Nσ0′v0≠0$ (take x = v0, y a unit vector in ℋ, and ɛ = 1/2).

The proof is completed by setting v = v0 and $X=Nσ0′$.

Recall finally that for T ∈ ℬ(ℋ), then Lat(T) is by definition the lattice of all invariant subspaces of T and AlgLat (T) is the algebra of all operators A in ℬ(ℋ) such that Lat(T) ⊂ Lat(A). An operator T in ℬ(ℋ) is said to be reflexive if , the smallest unital subalgebra of ℬ(ℋ) that contains T and is closed in the weak operator topology on ℬ(ℋ).

### Corollary 2.3

Suppose that A is an arbitrary nonreflexive contraction in ℬ(ℋ) such that the spectrum of A contains the unit circle. Let also B, C, and D be arbitrary operators in ℬ(ℋ). Then either i) or ii) of Theorem 1.2 is valid.

Proof

One knows from [2, Corollary 7.3] that every such operator A has a n.h.s.

3. Applications

In this section we show that in certain situations Theorem 1.2 can be applied to yield a n.h.s. for the operator TC in (1.1).

### Proposition 3.1

Suppose A ∈ ℬ(ℋ) has a n.h.s., and B is an arbitrary operator in ℬ(ℋ) such that B commutes with A. Then the operator Q ∈ ℬ(ℋ ⊕ ℋ) given matricially by

$Q=(A1H0B)$

has a n.h.s.

Proof

We know from Theorem 1.2 that either Q or

$Q˜=(B1H0A)$

has a n.h.s., so it suffices to prove that Q and are similar, which we now do. It is well-known that the operator S ∈ ℬ(ℋ ⊕ ℋ) given by

$S=(1H0X1H),$

where X is arbitrary in ℬ(ℋ), is invertible, and

$S-1=(1H0-X1H).$

Therefore we calculate S−1QS, where X is yet to be chosen. Thus

$S-1QS=(1H0-X1H) (A1H0B) (1H0X1H) =(A+X1H-XA+(-X+B)X-X+B) =(B1H0A)=Q˜$

if X is chosen to be X = BA.

The following shows that Theorem 1.2 can be useful in obtaining a n.h.s. for certain classes of operators.

### Theorem 3.2

Suppose A ∈ ℒ(ℋ) has a n.h.s., and B and C are arbitrary operators in ℒ(ℋ) that commute with A, with C invertible. Then

$TC=(AC0B)$

has a n.h.s.

Proof

We know from Theorem 3.1 that

$Q=(A1H0B)$

has a n.h.s., and the calculation

$(C-1001H) (AC0B) (C001H)=(C-1AC1H0B)=(A1H0B)$

shows that TC and Q are similar.

### Remark 3.3

It would be interesting to show that Theorem 3.2 remains valid without the commutativity assumptions made there.

References
1. Ansari, S, and Enflo, P (1998). Extremal vectors and invariant subspaces. Trans Amer Math Soc. 350, 539-558.
2. Chevreau, B, Exner, G, and Pearcy, C (1989). On the structure of contraction operators III. Michigan Math J. 36, 29-62.
3. Chalendar, I, and Partington, J (2011). Modern approaches to the invariant-subspace problem. Cambridge Tracts in Math. Cambridge: Cambridge Univ. Press
4. Douglas, R, and Pearcy, C (1972). Hyperinvariant subspaces and transitive algebras. Michigan Math J. 19, 1-12.
5. Kim, HJ (2011). Hyperinvariant subspaces for operators having a normal part. Oper Matrices. 5, 487-494.
6. Kim, HJ (2012). Hyperinvariant subspaces for operators having a compact part. J Math Anal Appl. 386, 110-114.
7. Lomonosov, V (1973). Invariant subspaces of the family of operators that commute with a completely continuous operator. Funktsional Anal i Prilozen. 7, 55-56.
8. Radjavi, H, and Rosenthal, P (1973). Invariant subspaces. New York, NY: Springer-Verlag