The cemetery was established in the years of World War I as the cemetery of the Tallinn garrison. The oldest grave dates back to 1916 and holds Russian, Estonian, and German soldiers killed during World War I.

The graves from 1918–1944, the gravestones of the Estonian soldiers and the monuments of the Estonian War of Independence were largely destroyed by the Soviet authorities and the graveyard was taken over by the Red Army for use by the Soviet occupation forces after World War II.
The graves of fifteen British servicemen killed in the Estonian War of Independence between 1918–1920 were repaired in 1994. Her Majesty Queen Elizabeth II awarded Linda Soomre honorary Membership of the Order of the British Empire for dedication and bravery in protecting the British graves during the years of the Soviet rule. Linda Soomre was in charge of the Tallinn City Centre Cemetery for 35 years. After the destruction of the gravestones she had made the ground overnight a maintenance area saving the remains of the British soldiers from being violated. Linda Soomre also saved the graves of two Estonian generals, Johan Unt and Ernst Põdder, by keeping the burial sites covered with dirt. The monument for the generals, originally opened in 1933, was restored on 22 February 1998.
The graves of the Estonian Soldiers and the demolished structure of the Estonian War of Independence monument in the graveyard are not restored. The registration book of people buried at this cemetery between years 1918–1944, with over 1,150 names, is maintained in Tallinn city central archives.