Avoiding VAT when buying Pub for Residential Purpose

I am buying a Pub to convert it to flats .
The ground floor of the building is the pub and first floor is already a residential flat .
The entire building is for sale .

The flat (first floor) and the pub (ground floor) are on the same title on land registry though they pay separate council tax .

The entire building is owned by a big pub chain company and they are selling it.
The buyer is supposed to pay VAT on top of the purchase price.

But, I heard if the property is purchased for residential purpose you can avoid VAT .

I am **buying it with my wife ** and our aim is to convert the pub area (ground floor) also to one or two flats (need planning permission).
If I purchase, will I be able to avoid VAT ? Is there any form which we need to submit to HMRC to avoid VAT?