Let us denote the class of all normalized analytic functions as A. These functions, represented as f(z), have the form

where D is the set of complex numbers z such that |z|<1. We define S as the subclass of A that consists of univalent functions.

If h1(z) and h2(z) belong to A, we say that h1(z) is subordinate to h2(z) if there exists a function ζ(z) with ζ(0)=0 and |ζ(z)|<1 in D such that h1(z)=h2(ζ(z)). We denote this as h1(z)≺h2(z).

A function f(z) in S is considered bi-univalent if its inverse, f-1(w), has an analytic continuation to |w|<1 in the w-plane. We denote σ as the class of all bi-univalent functions in D.

If f-1(w)=g(w) is of the form

then we have

The class of bi-univalent functions was introduced by Lewin [4] in 1967, who provided an estimate for the second coefficient for functions in this class as |a2|<1.51. This result was later improved by Brannan and Clunie [2] to |a2|≤2. There is extensive literature on the estimates of the initial coefficients of bi-univalent functions (see [1, 6, 7]).

The Fekete-Szegö problem involves finding sharp bounds for |a3-ϱa22| of any compact family of functions. When ϱ=1, the functional represents the Schwarzian derivative, which plays a significant role in the theory of Geometric functions.

For η∈R and η≠0, the generating function of Gegenbauer polynomials is defined as

where r∈[-1,1] and z∈D. The function Gη, for a fixed r, is analytic in D and can be written as

where Qkη(r) is the Gegenbauer polynomial of degree k. When η=0, Gη does not exist. Thus, the Gegenbauer polynomial for η=0 is generated by the following function [3, 5]

The function Gη(r,z) gets normalized when η>-1/2. The Gegenbauer polynomials are defined by the following recurrence relation [1]

with initial conditions

For k=2, we have

Table 1 shows the special cases of Qkη(r).

Table 1 . Special cases of the Gegenbauer polynomials.

S. No.	Parameter	Special Cases
1	η=0.5	Legendre polynomials
2	η=1	Chebyshev polynomials

Figure 1 shows the images of D under Gη(r,z).

Figure 1. Image of D under Gη(r,z).

Figure 2 shows the Graphs of Qkη(r).

Figure 2. Graph of Qkη(r).

Unless otherwise mentioned, we assume in this paper that

where v≥0 and t∈[0,1]. It is evident that δn is a real number and δn≥1, and

For every h∈A, we define

If h∈A is of the form h(z)=z+∑j=2∞ujzj, we have

With the aid of Gegenbauer polynomials, we define subclasses of σ using the notion of subordination.

Definition 1.1. A function f∈σ is said to be in the class Aσ(v,t r),

if

and

where g(w)=f-1(w)=w+∑j=2∞bjwj.

Example 1.1. For t=0 and v≥1, f∈σ is in the class Aσ(v,0 r)=Dσ(v r), if

and

where g=f-1.

Example 1.2.

For t=0 and v=1, f∈σ is in the class Aσ(1,0 r)=Dσ(1 r)=Hσ(r), if

and

where g=f-1.

Theorem 2.1. If f(z), given by (1.1), is in Aσ(v,t r), then

and

where D=83η21+ηr3+8η(1+η)r2+23η-6η2+4r-4η.

Proof. Since f∈Aσ(v,t r), there exist two analytic functions α,β:D→D given by

and

with α(0)=β(0)=0,|α(z)|<1,|β(w)|<1 for all z,w∈D such that

and

Or equivalently

and

Since |α(z)|<1 and |β(w)|<1, it is clear that

for j=1,2,.... From (2.6) and (2.7), we have

and

From (2.10) and (2.12), we can easily see that

and

Upon adding (2.11) and (2.13), we get

By using (2.17) in (2.18), we have

which implies

Upon subtracting (2.13) from (2.11) and using (2.16), we get

Then, in aid of (2.17), we get

Thus

By using (2.10), (2.11), (2.12) and (2.14), we get

which implies

Corollary 2.1. If f(z), given by (1.1), is in Dσ(v r), then

and

where D is as in Theorem 2.1.

Corollary 2.2. If f(z), given by (1.1), is in Hσ(r), then

and

where D is as in Theorem 2.1.

Corollary 2.3. If f(z), given by (1.1), is in Aσ(v,t r) with η=1, then

and

Corollary 2.4. If f(z), given by (1.1), is in Aσ(v,t r) with η=12, then

and

Remark 2.1. When v=0,t=1,η=1 and |r|≤0.75490..., we obtain Brannan and Cliunie's [<xrefrid="ref2"ref-type="bibr">2</xref>] conjecture |a2|≤2.

2.1. Visualization of Certain Class-Specific Functions and Co-efficient bounds

Results need to be clear and recognized. Geometrical visualization is the use of visualization to comprehend and investigate mathematical processes. Some of the diagrams that assist us in seeing, comprehending, and analyzing the nature of the functions and co-efficient bounds for a2 which holds Brannan and Clunie's conjecture in Aσ(v,t r) with a suitable choice of parameters, are presented in this article.

Theorem 3.1. If f(z), given by (1.1), is in Aσ(v,t r) and ϱ∈R, then

Proof. For ϱ∈R and from (2.20), we have

By using (2.19), we get

where Υ(v,t)=(1-ϱ)[Q1η(r)]22δ3[Q1η(r)]2-δ22Q2η(r). Thus

Corollary 3.1. If f(z), given by (1.1), is in Dσ(v r) and ϱ∈R, then

Corollary 3.2. If f(z), given by (1.1), is in Hσ(r) and ϱ∈R, then

Corollary 3.3. If f(z), given by (1.1), is in Aσ(v,t r) with η=1 and ϱ∈R, then

Corollary 3.4. If f(z), given by (1.1), is in Aσ(v,t r) with η=1/2 and ϱ∈R, then

Figure 3. Example for class and co-efficient bounds a2 for class Aσ(v,t r) with a suitable choice of parameter.