The gear transmission system has been widely applied in mechanical systems, and many high-performance applications of these systems require low weight. With the aid of establishing the optimization model of the gear transmission system that consists of an objective function and some constraints (for example, the bending stress, the contact stress, the torsional strength, etc.), the optimal weight design of the gear transmission system can be transformed into the optimization problem for the objective function under the constraints. Moreover, both the shaft and the gear of the gear transmission system are considered simultaneously in our design. The hybrid Taguchi-genetic algorithm (HTGA) is employed to find the optimal design variables and the optimal weight of the system. An illustrated example for the single spur gear reducer is given to show that the optimal weight design problem can be successfully solved using the proposed design scheme. It also proves the high efficiency and feasibility of the algorithm in the gear design.