### Original Article

Kyungpook Mathematical Journal 2010; 50(1): 37-47

**Published online** March 23, 2010

Copyright © Kyungpook Mathematical Journal.

### The Fekete-Szegö Problem for a Generalized Subclass of Analytic Functions

Erhan Deniz, Halit Orhan

Department of Mathematics, Faculty of Science, Ataturk University, 25240 Erzu- rum, Turkey

**Received**: March 23, 2010; **Revised**: March 23, 2010; **Accepted**: March 23, 2010

In this present work, the authors obtain Fekete-Szegö inequality for certain normalized analytic function $f(z)$ defined on the open unit disk for which $frac{(1-alpha )z(D_{lambda, mu }^{m}f(z))^{prime }+alpha z(D_{lambda, mu }^{m+1}f(z))^{prime }}{% (1-alpha )D_{lambda, mu }^{m}f(z)+alpha D_{lambda, mu }^{m+1}f(z)}$ $(lambda geq mu geq 0,$ $min {mathbb{N}}_{0},$} $alpha geq 0) $ lies in a region starlike with respect to 1 and is symmetric with respect to the real axis. Also certain applications of the main result for a class of functions defined by Hadamard product (or convolution) are given. As a special case of this result, Fekete-Szegö inequality for a class of functions defined through fractional derivatives is obtained. The motivation of this paper is to generalize the Fekete-Szegö inequalities obtained by Srivastava *et al*., Orhan *et al*. and Shanmugam *et al*, by making use of the generalized differential operator $D_{lambda, mu }^{m}.$

**Keywords**: Fekete-Szegö, problem, Analytic functions, Hadamard product, Starlike functions