Conversion of cropland into perennial vegetation land can increase soil organic carbon (SOC) accumulation, which might be an important mitigation measure to sequester carbon dioxide from the atmosphere. The “Grain for Green” project, one of the most ambitious ecological programmes launched in modern China, aims at transforming the low-yield slope cropland into grassland and woodland. The Loess Plateau in China is the most important target of this project due to its serious soil erosion. The objectives of this study are to answer three questions: (1) what is the rate of the SOC accumulation for this “Grain for Green” project in Loess Plateau? (2) Is there a difference in SOC sequestration among different restoration types, including grassland, shrub and forest? (3) Is the effect of restoration types on SOC accumulation different among northern, middle and southern regions of the Loess Plateau? Based on analysis of the data collected from the literature conducted in the Loess Plateau, we found that SOC increased at a rate of 0.712 TgC/year in the top 20 cm soil layer for 60 years under this project across the entire Loess Plateau. This was a relatively reliable estimation based on current data, although there were some uncertainties. Compared to grassland, forest had a significantly greater effect on SOC accumulation in middle and southern Loess Plateau but had a weaker effect in the northern Loess Plateau. There were no differences found in SOC sequestration between shrub and grassland across the entire Loess Plateau. Grassland had a stronger effect on SOC sequestration in the northern Loess Plateau than in the middle and southern regions. In contrast, forest could increase more SOC in the middle and southern Loess Plateau than in the northern Loess Plateau, whereas shrub had a similar effect on SOC sequestration across the Loess Plateau. Our results suggest that the “Grain for Green” project can significantly increase the SOC storage in Loess Plateau, and it is recommended to expand grassland and shrub areas in the northern Loess Plateau and forest in the middle and southern Loess Plateau to enhance the SOC sequestration in this area.