Not far from the taxi rank, set back off a little corner of the Market Place in Wantage, lies the parish church.

It is a large building, subject to some significant architectural changes in its centuries as a place of worship.

Unlike many English churches, the sightlines are not that great, surrounded as it is at close quarters by local shops, pubs and restaurants. But it is high up and its tower can be glimpsed from unexpected viewpoints, as I go about my business as a licensed Hackney Carriage driver.

Colloquially referred to as Wantage Parish Church, rather than by its dedication to Saint Peter and Saint Paul, John Betjeman, whose statue overlooks the churchyard, once wrote about it: “It is the oldest and grandest building in the town and soaked in the prayers of many generations.”

At least three times this year, I have given way to a cortège emerging from the narrow one-way street leading from the church, a priest walking slowly in front.

And, though Wantage has expanded outwards as far as the turning for Lockinge, the centre of the town around the Market Place remains almost unchanged, at least in spirit, since Betjeman’s day.

The builders’ lorries plying to and from these building sites daily spray my taxi with dirty water when it rains.

Herald Series:

But it was to the church that I returned on Monday night, for the installation of the new vicar, the first such service held there for more than 26 years.

Part of the charm of the Church of England is that it has many peculiar and eccentric traditions. This service was no exception, laden with symbolism and all diligently explained in notes printed before the Liturgy in the Order of Service.

There was the tolling of the bell by the newly-instituted, installed and inducted vicar, for the number of times they hope to remain in office.

Then there were the presentation of symbols, such as the keys to the church, the Book of Common Prayer and the Bible.

And, just as important as the rarely spoken liturgy, there was the after church party, with a buffet as gloriously eclectic and diverse as the Church of England itself. As a longstanding parishioner tweeted after the service: “We like a good party in Wantage. Hospitality is at the heart of the gospel.”

Wantage Parish Church is quite different from my own place of worship. Somewhat unusually, the church choir sings from the nave in modern choir stalls, located behind the congregation. It not only makes them lucidly audible, but gives a real lift to communal singing.

As one of the anthems sung there says " I was glad when they said unto me, Let us go into the house of the Lord.” And so was I, on Monday night, in Wantage.