I think you've answered your own question. As I understand it, the VOS finish does not have a final buffing stage, so it's matt/non-gloss and has 'character'. Any polishing or buffing you do will of course gloss up the polished area.
The whole point of the VOS finish is that it doesn't look new, but like an aged instrument. If you want the new look then you'll need to polish up the guitar you have all over.
I'd leave it as it is, play it, and never polish it. Let it gain some mojo!