A church was built in about 1200 near the Tyburn Tree, on the bank of the Tyburn River. In 1400 this was demolished and replaced with a church, St Marys, at the location of the plaque. The river, or bourne, ran through the parish and so it and its church became known as St Mary le bourne. This church was demolished and rebuilt on the same site in 1740-2. Demolished in 1949.
In 1813 a replacement church was built, and still stands, on Marylebone Road opposite York Gate. When this was built the old church became a chapel of ease for the parish.
There must be pictures, and photographs, of the 1740s church but we can't find them. Here is a rather nice aquatint of the 1813 one from 1829.