thing is that between 50 and 100 watts there are only a few dB difference, so if you want power amp saturation, it won't make a big difference between 50 and 100W.
Usually 100W tops sound more articulated than 50W but it depends, but that's a sound difference more than a volume difference.
I've been playing 50W and 100W amps my whole live - until recently I got myself a THD Univalve, where I have full control on how much power or what kind of sound I want. Limit is the max power of around 18W (single tube class A), but that's much louder as it seems.
The sound of this amp is fantastic. Think again whether you really need a 100W amp. If not, sell the Rivera and get something smaller, you won't regret it.