Article Search
eISSN 0454-8124
pISSN 1225-6951

### Article

Kyungpook Mathematical Journal 2023; 63(2): 167-173

Published online June 30, 2023 https://doi.org/10.5666/KMJ.2023.63.2.167

### An Upper Bound for the Probability of Generating a Finite Nilpotent Group

Department of Mathematics, Miyaneh Branch, Islamic Azad University, Miyaneh, Iran

Department of Mathematics, Faculty of Sciences, University of Zanjan, P. O. Box 45371-38791, Zanjan, Iran
e-mail : sm_jafarian@znu.ac.ir

epartment of Mathematics, Farhangian University, Tehran, Iran
e-mail : h.rostami5991@gmail.com

Received: February 17, 2022; Revised: December 24, 2022; Accepted: February 22, 2023

### Abstract

Let G be a finite group and let ν(G) be the probability that two randomly selected elements of G produce a nilpotent group. In this article we show that for every positive integer n>0, there is a finite group G such that ν(G)=1n. We also classify all groups G with ν(G)=12. Further, we prove that if G is a solvable nonnilpotent group of even order, then ν(G)p+34p, where p is the smallest odd prime divisor of |G|, and that equality exists if and only if GZ(G) is isomorphic to the dihedral group of order 2p where Z(G) is the hypercenter of G. Finally we find an upper bound for ν(G) in terms of |G| where G ranges over all groups of odd square-free order.

Keywords: Soluble group, Nilpotent subgroup, Probability

### 1. Introduction

In the past 40 years, there has been a growing attention in the application of probability in finite groups (for example see [8, 16]). In this paper, we denote by ν(G) the probability that two randomly selected elements of G produce a nilpotent subgroup. In other words we have

ν(G)=|{(x,y)G×G:x,yisnilpotent}||G|2.

The notion ν(G) is introduced in [11] on the model of the commutativity degree, via

cp(G)=|{(x,y)G×G:x,yisabelian}||G|2.

Note that for x,yG, we have xy=yx if and only if x,y is abelian.

It is easy to see that cp(G)= xG|CG(x)||G|2 where CG(x) is the centralizer of x in G as CG(x) is a subgroup of G for any xG.

Similarly if

NilG(x)={yG|x,yisnilpotent},

then

ν(G)= xG|NilG(x)||G|2.

However, NilG(x) is not necessarily a subgroup of G, and so it is difficult to glean information about a group G from ν(G).

A finite group G is nilpotent if and only if ν(G)=1 (see Theorem 1 of [5]). On the other hand, Wilson [16] showed that in finite groups G the probability that two random elements of G produce a nilpotent group goes to 0 as the index of the Fitting subgroup of G goes to infinity.

Gustafson [8] proved that if G is a non-abelian group, then cp(G)58, and that equality holds if and only if GZ(G) is isomorphic to the Kelian four-group Z2×Z2. Several authors determined the structure of a finite group G when cp(G) is sufficiently large, see [2, 9, 12].

In [7] Guralnick and Wilson found that if G is a nonnilpotent group, then ν(G)12. In this paper we classify groups G with ν(G)=12 (see Proposition 2.6).

It is easy to see that cp(A5)=ν(A5)=112 where A5 is the alternating group of degree five. Dixon observed that cp(G)112 for any finite nonabelian simple group G. This was submitted by Dixon as a problem in Canadian Math. Bulletin, 13 (1970), with his own solution appearing in 1973. Guralnick and Robinson [6] extended this result to nonsolvable groups and determined precisely for which nonsolvable groups the equality happens. Recently in [10] the authors of the present paper showed that if G is a group such that NilG(x) is a subgroup of G for every x∈ G and ν(G)>112, then G is solvable.

Fulman et al. [5] proved that if G is a solvable nonnilpotent group and p is the smallest prime number that divides |G|, then ν(G)1p and equality holds if and only if p=2 and GZ(G) is isomorphic to the dihedral group of order 6 (see [5]). Here Z(G) is the hypercenter of G (i.e. the terminal term of the upper central series of G, see [3, 13]). In this article we improve this upper bound as follows.

Theorem 1.1. Suppose that G is a solvable nonnilpotent group of even order. Then ν(G)p+34p where p is the smallest odd prime number that divides |G|; equality holds if and only if GZ(G(G))D2p is the dihedral group of order 2p.

For a prime p we denote by Zpk the elementary abelian group of order pk. We propose the following conjecture for every nonnilpotent group of odd order.

Conjecture Let G be a finite solvable nonnilpotent group such that |G|=p1n1p2n2prnr where 2<p1<<pr are primes. Then

ν(G)pktk+pl21pktkpl2:=max{piti+pj21pitipj2:   pj|piti1,1j<ir,1tini}

for some 1l<kr and equality holds if and only if GZ(G)ZpktkZpl. We think that this conjecture is true for the class of N-groups, introduced by Abdollahi and Zarrin in [1], which are the groups in which NilG(x) is a nilpotent group for every xGZ(G). We feel that the method used in proof of main theorem of [15] may be useful in proving this.

In Section 2 we compute ν(G) for Frobenius groups and Dihedral groups. We also prove that for any positive integer n, there is a group G such that ν(G)=1n. Finally we classify all groups G with ν(G)=12. In Section 3 we verify Theorem 1.1 and, with Theorem 3.2, confirm the above conjecture for groups of square-free order.

In this article G is a finite group and Z(G) is its hypercenter. Most notation we use is standard and follows [14].

### 2. Computing ν(G) for Certain Groups

The following lemmas are very useful in the sequel.

Lemma 2.1. Suppose that G is a group. Then ν(G)=ν(GZ(G)).

Proof. See Corollary 3 of [5].

Lemma 2.2. Suppose that G and H are finite groups. Then ν(G×H)=ν(G)×ν(H).

Proof. The proof is not complicated.

Proposition 2.3. If G=HK is a Frobenius group where the Frobenius kernel is K and the complement is H, then ν(G)=1|H|2(11|K|)+ν(H)|K|.

Proof. By hypothesis, we have CG(h)H for each 1hH, CG(k)K for each 1kK and HHx=1 for each xGH. Now if h1k1,h2k2 is nilpotent such that h1,h2H and k1,k2K, then h1=h2=1 or k1=k2=1. On the other hand {K,(Hx1)|xK} is a partition of G and since K is nilpotent, we are done.

Corollary 2.4. Suppose that G is the dihedral group of order 2rn where r>1 and n is odd. Then ν(G)=n+34n.

Proof. Since D2rnZ(D2rn)D2r1n for r>1, by Lemma 2.1 we conclude that ν(D2rn)=ν(D2r1n)==ν(D2n). Since n is odd, D2n is a Frobenious group with the cyclic kernel of order n and so we are done by Proposition 2.3.

Corollary 2.5. For any integer n>0, there is a group G of even order such that ν(G)=1n.

Proof. Our proof is by induction on n. If n{1,2,3}, then the result holds since ν(D2)=1, ν(D6)=12 and ν(D18)=13. So assume that n4 and the that the result holds for all positive integers m<n. If n is even, then there is a group H where ν(H)=2n by induction hypothesis and so ν(H×D6)=1n by Lemma 2.2. Suppose that n is odd, then n=4m+1 or n=4m+3 for some positive integer m. It follows from Corollary 2.4 that ν(D2(4m+1))=m+1n and ν(D2(12m+9))=m+1n. Since m+1<n, we are done by induction hypothesis and Lemma 2.2. This completes the proof.

In the following we classify all groups G with ν(G)=12.

Proposition 2.6. Suppose that G is a finite group (not necessarily solvable). Then ν(G)=12 if and only if GZ(G)D6, the dihedral group of order 6.

Proof. We get necessity by By Lemma 2.1. Conversely if ν(G)=12, then the probability of solvability of G is equal or greater than 12 and so G is solvable by [7]. By Theorem 5 of [5], we conclude that ν(G)12 and equality holds when GZ(G)D6, as needed.

### 3. Upper Bound for ν(G)

S. Franciosi and F. Giovanni defined and studied a JNN group as a group all of whose proper quotients are nilpotent (see [4] and [5], Definition 1). It should be noted that a finite group G is a JNN group if and only if G=L⋉ A where A is an elementary abelian p-group and L is a nilpotent group such that p dose not divide the order of G and the action of L on A is faithful and irreducible (See Theorem 4 of [5] and what follows it).

Proof of Theorem 1.1.

If p=3, then ν(G)=1ν0(G)12 by Theorem 5 of [5] and equality holds if and only if GZ(G)D6. So we assume that the smallest odd prime divisor of |G| is greater than 3. It is enough to prove the result for JNN groups. For if G is a counterexample of minimal order, then there is a nontrivial normal subgroup K of G such that GK is nonnilpotent (since G is solvable). Suppose that r is the smallest odd prime that divides |GK|. If GK is of even order, then ν(G)ν(GK)r+34rp+34p because rp. Also if GK is of odd order, then by Theorem 5 of [5], ν(G)ν(GK)1r14p+34p which gives a contradiction. So let us assume that G is a JNN group. Then G=LA where LPk×Pk1××P1, Pi's are the unique Sylow pi-subgroups of L and A is an elementary abelian q-group. By setting N=Pk1(P1A) we have G=Pk⋉ N. We claim that if q=2 and 1xpPk, then |CG(xp)N||N4|.

Assume that |A|=2t(t2) and H=CG(xp)A. If |H|=2t1 and aAH, then axp=ah1 for some h1H. Hence axp2=axph1=ah12=a and so bxp2=b for all b∈ A. But Pk acts faithfully on A which implies that xp2=1, obviously absurd. Hence |H|2t2. If M:=Pk1×Pk2××P1, then CG(xp)N=M(CG(xp)A)=MH and so |CG(xp)N|=|M||H|=|N||A||H||N|4, as claimed.

Now we want to count the ordered pairs (x, y) in a fixed pair (a1N,a2N) for some a1,a2G where x,y is nilpotent. By page 14 of [5], the probability that a selected pair (x,y) from the coset pair (xpN,ypN) generates a nilpotent subgroup is not greater than |CG(xp)CG(yp)N||N| and by our claim this probability is equal or less than 14.

Now we continue by induction on the number k of prime divisors of |L|. Here our aim is showing that if the upper bound mentioned in the assertion is correct for N, it is correct for G too. As mentioned above if q=2, then there is nothing to prove. So assume that q≠ 2. Since the action of Pk on A is faithful, in a similar way it can be seen that |CG(xp)N||N|q. If q≥ 5, then this probability is equal or less than 1514p+34p and by the assumption on N, we conclude that ν(G)p+34p. Also it is not hard to see that if k≥ 2, then the equality does not hold, since in this case N is not an elementary group and as mentioned above above in both cases, whether q is equal to 2 or not, the probability is less than 14<p+34p. So it is enough to prove it for the base step of the induction. Assume that G=RA where A=(Zq)n, R is a Sylow r-subgroup and |R|=rm. Then we investigate two cases:

Case 1: Assume that q=2. Then ν(G)22n+(22nr2m22n)×1422nr2m. So ν(G)r2m+34r2m<r+34r. As one can see, equality cannot hold in this case.

Case 2: Suppose that r=2. Then ν(G)q2n+(q2n22mq2n)×1qq2n22m=22m+q1q22m and since q≠ 3, we have 22m+q1q22mq+34q and equality holds if and only if m=1 and hence GZ2(Zq)n. Now we claim that n=1.

Let 1aA and 1xR. If ax=a, then ar=a and since the action of R on A is irreducible, we have a=A. Henceforth GZ2ZqD2q. Otherwise, it can be assumed that CG(R)A=1, which results that GZ2(Zq)n is a Frobenius group. It follows that ν(G)=qn+34qn (see Proposition 2.1). This implies that the equality exists in our assertion if and only if n=1 and GD2q, while G is a JNN group.

Now if G is not a JNN group, so there is a normal subgroup N of G such that GN is a JNN because G is solvable. Let ν(G)=p+34p where p is the smallest odd prime that divides |G| and GN is of even order and ps be the smallest odd prime that divides the order of GN. Then p+34p=ν(G)ν(GN)ps+34ps which implies that p=ps and GND2p. Now by an argument similar to that on page 16 of [5] it can be proved that GZ(G)D2p. Let GN be of odd order and ps>3 be its smallest prime divisor. Then p+34p=ν(G)ν(GN)1ps, our final contradiction.

For an odd prime p, we denote by Gp the set of all solvable nonnilpotent groups G of even order such that p is the smallest odd prime that divides the order of G.

Corollary 3.1. Suppose that GGp where p is an odd prime . Then ν(G) is the largest value of ν on Gp if and only if GZ(G)D2p.

Theorem 3.2. Suppose that G is a finite group of odd order and |G|=p1p2pr where p1<<pr are primes. Then we have

ν(G)pk+pl21pkpl2:=max{pi+pj21pipj2:  pj|pi1,1j<ir}

for some 1l<kr and the equality holds if and only if GZ(G)ZpkZpl.

Proof. Similar to the proof of Theorem 1.1 we prove it for JNN groups. Let G=LA be a JNN group and let p1<p2<<pr. Then AZpr and p1p2pr1|pr1 since L acts faithfully on A. We proceed by induction as it was done in the Theorem 1.1. Thus set NG such that |N|=p2p3pr. It follows that G=P1N. We claim that if the assertion is correct for N it will be correct for G too. It is not hard to see that the probability that a pair selected from the coset pair (xpN,ypN) for some xp,ypP1 generates a nilpotent subgroup of G is bounded by |CG(xp)CG(yp)N||N|. But the action of P1 on A is faithful and then if both xp and yp are not identity, then |CG(xp)CG(yp)N||N|1pr. Now since 1pr<max{pi+pj21pipj2|pj|pi1,1i,jr}, one can conclude that the bound is right and the equality does not hold when r3. Coming back to the base of induction, let |G|=p1p2 with p1<p2. Then G=Zp1Zp2 and ν(G)=p2+p121p2p12, as wanted.

### References

1. A. Abdollahi and M. Zarrin, Non-nilpoten graph of a group, Comm. Algebra, 38(12)(2010), 4390-4403.
2. F. Barry, D. MacHale and A. N She, Some supersolvability conditions for finite groups, Math. Proc. Royal Irish Acad., 106A(2)(2006), 163-177.
3. S. Dixmier, Exposants des quotients des suites centrales descendantes et ascendantes d'un groupe, C. R. Acad. Sci. Paris, 259(1964), 2751-2753.
4. S. Franciosi and F. Giovanni, Soluble groups with many nilpotent quotients, Proc. Royal Irish Acad. Sect. A., 89(1989), 43-52.
5. J. E. Fulman, M. D. Galloy, G. J. Sherman and J. M. Vanderkam, Counting nilpotent pairs in nite groups, Ars Combin., 54(2000), 161-178.
6. R. M. Guralnick and G. R. Robinson, On the commuting probability in nite groups, J. Algebra, 300(2)(2006), 509-528.
7. R. M. Guralnick and J. S. Wilson, The probability of generating a nite soluble group, Proc. London Math. Soc., 81(3)(2000), 405-427.
8. W. H. Gustafson, What is the probability that two group elements commute?, Amer. Math. Monthly, 80(9)(1973), 1031-1034.
9. R. Heffernan, D. MacHale and A. N She, Restrictions on commutativity ratios in finite groups, Int. J. Group Theory, 3(4)(2014), 1-12.
10. S. M. Jafarian Amiri, H. Madadi and H. Rostami, On the probability of generating nilpotent subgroups in a nite group, Bull. Aust. Math. Soc., 93(3)(2016), 447-453.
11. E. Khamseh, M. R. R. Moghaddam and F. G. Russo, Some Restrictions on the Prob- ability of Generating Nilpotent Subgroups, Southeast Asian Bull. Math., 37(4)(2013), 537-545.
12. P. Lescot, H. N. Nguyen and Y. Yang, On the commuting probability and supersolv-ability of finite groups, Monatsh. Math., 174(2014), 567-576.
13. D. H. McLain, Remarks on the upper central series of a group, Proc. Glasgow Math. Assoc., 3(1956), 38-44.
14. D. J. S. Robinson. A course in the theory of groups. New York: Springer-Verlag; 1996.
15. H. Rostami, The Commutativity Degree in the Class of Nonabelian Groups of Same Order, Kyungpook Math. J., 59(2019), 203-207.
16. J. Wilson, The probability of generating a nilpotent subgroup of a nite group, Bull. London Math. Soc., 40(2008), 568-580.