The Yb3+ doped Ba2YB'O6 (B'= Ta5+, Nb5+) were prepared by high temperature solid-state reaction method, their structures were determined by x-ray diffraction and refined by Rietveld method. The diffuse reflection absorption, excitation and emission spectra of yb3+:Ba2YB'O6 (B'= Ta5+, Nb5+) were measured at room temperature. Under the excitation of ultraviolet light, these phosphors exhibit broad charge transfer band emissions of TaO6 or NbO6 centre with large Stokes shift. The Yb3+ doped into these hosts are situated at y3+ sites of cubic symmetry (Oh). The experimental energy levels of Yb3+ in Ba2YTaO6 and Ba2YNbO6 were determined by photoluminescence and diffuse reflection absorption spectra. Their wavefunctions and theoretical energy levels were obtained by diagonalising the Hamiltonian matrix. The experimental energy levels were fitted by Levenberg-Marquardt iteration algorithm to determine crystal field parameters. Then, the magnetic-pole transition line strengths of yb3+:Ba2YB'O6 (B'=Ta5+, Nb5+) from (2F5/2)Г8-to the low-energy states were calculated.