In 1861 or 1862, Francis Walkins, stonemason of Stawell discovered an outcrop of freestone in the Mount Difficult range while on a shooting trip and realised that the stone was extremely durable. By the 1870s the stone was being quarried for use in Stawell. But, such was the potential of the quarry that a tramway was built to link it to the main railway at Stawell, and a township was gazetted adjacent to the quarry. In 1879, the Stawell freestone was unanimously recommended for the construction of Parliament House. The stone was also used for the Melbourne town hall and various other important buildings. The construction of the tramway link from Stawell to the quarry had facilitated the use of the stone in areas other than just locally.
The quarry is significant because of its association with several important Melbourne buildings. It is also of local importance in that the stone from the quarry was extensively used in the rebuilding of Main Street in Stawell after a major fire in the 1870s. The quarry is also important because there remains a fund of tangible evidence of the operations of the quarry in the form of buildings, machinery and worked rock faces.