In the Heliotron J device, the configuration effects on the particle confinement are studied experimentally with tangentially injected neutral beams and a charge-exchange (CX) neutral particle analyzer (NPA) system. The hydrogen neutral beam are co-injected into deuterium plasmas heated by electron cyclotron heating. The detected CX flux increases, as the CX-NPA is oriented to the beam-facing direction. The behavior of the CX flux is studied by changing one of the Fourier components in the magnetic field, the bumpiness component, B04/B00, from 0.01 to 0.15. Here, Bmn is the Fourier component of the magnetic field strength in the Boozer coordinates where the subscript m/n denotes poloidal/toroidal mode numbers. The dependence of the CX flux on the configurations and pitch angle, which represents the change of the loss cone shape predicted by noncollisional orbit calculation, is observed. The bulk deuterium temperature slightly increases with increasing the bumpiness component. The decay time of the CX flux just after the neutral beam is turned off becomes longer with increasing the bumpiness component. By comparison of observation and calculation of the Fokker-Planck equation, the loss time of fast ions in the high-bumpiness configuration is longer than that of the standard configuration in Heliotron J.