In the present work, the shape of a mixing vane in a pressurized water reactor fuel assembly has been optimized numerically using three-dimensional Reynolds-averaged Navier-Stokes analysis of the flow and heat transfer to find the effects of flow characteristics downstream of the mixing vane on heat transfer augmentation and also to optimize the shape of the mixing vane to increase the resistance to departure from nucleate boiling by enhancing the heat transfer without excessive pressure loss. The response surface method is employed as an optimization technique. The objective function is defined as a combination of the heat transfer rate and the inverse of friction loss with weighting factor. The bend angle and base length of the mixing vane are selected as design variables. In most of the numerical experiments, both the heat transfer and friction loss are enhanced as the bend angle and base length increase. The swirl and cross-flow characteristics and thermal-hydraulic performances of different shapes of mixing vane are discussed. From the results, the close relationship between the swirl factor and the heat transfer rate has been found. In the specified ranges of the design variables, the sensitivity of the objective function to the base length is only about one-tenth of the sensitivity to the bend angle. Nine points for numerical experiments were sufficient for construction of a reliable response surface. The optimum shape has been obtained as a function of the weighting factor in the objective function.