Temporal and spatial variations of concentrations of heavy metals including mercury (Hg), zinc (Zn), lead (Pb), arsenic (As), copper (Cu), and cadmium (Cd) in the sediments of the Yangshan Deepwater Harbor were determined based on 6 cruises in autumn and winter, respectively, from 2010 to 2013. The results demonstrated that the overall concentrations of heavy metals were low and distributed in uniform patterns. The concentrations of Hg, Zn, Pb, and Cd in autumn were significantly higher than those in winter with small fluctuations for As and Cu in terms of seasonal variations. Results of factor analysis showed that Pb, Cd, and Zn were derived from inland industrial and shipping discharges as well as the degradation of organic pollutants in marine environment. While agricultural pollutions, cargo shifting and construction debris from reclamation projects contributed to the sources of Cu, As, and Hg. Ecological risk assessment by Mean Sediment Quality Guideline Quotient (SQG-Q) revealed that the degree for eco-risk of the sediments was low-and-moderate in autumn, higher than that in winter. Hg and Cu were the dominant eco-risk factors. The results of Index of geoaccumulation (Igeo) showed that the whole sites of the sea area were barely influenced by Hg, As, Zn, and Pb, and the extents of Cd and Cu contaminations were in low grade. Contamination degree of all the six heavy metals could be ranked as the following: Cd〉Pb〉Zn〉Hg〉As. According to the results of integrated score of factor analysis, the contamination degree for heavy metals in sediments of the Yangshan Deepwater Harbor was low, despite sites No.5, No.4 and No.3, which were heavily contaminated compared with others.