Interesting question. 3 things:
1. The Legacy Valk has rake/trail values of 32 degrees and 6". Rather cruiser-ish values. Which means they are generally more stable than sport/sport tourer types of bikes with trail values of 4" or so.
A 2" diameter smaller front tire is enough to change your trail by about 1" - that is significant.
Here's a fun little calculator:
http://www.datamc.org/downloads/motorcycle-rake-trail-calculator/Racer/track guys adjust trail in 1/4" increments, or smaller... Changing back to the stock tire size should slow down handling; I.e., a little more effort to get it started turning. Think of reducing the trail as slightly negating the effects of gyroscopic action.
2. Relative to rolling over/into things, Tony Foale
https://books.google.com/books?id=84hF-qoR5I8C&pg=PT90&lpg=PT90&dq=gyroscopic+larger+tire+size+motorcycle&source=web&ots=FYv2xIP8Mf&sig=mujCZQGyMvkBlyMXSIXFZj6uy4o&hl=en#v=onepage&q&f=falsehas arguably the best treatise on bike handling ever done. From page 3-18:

Which also addresses the un-answered (but related) question of why dirt bikes have 21" front wheels...

That said, I am not sure how noticable this would be on our bikes...
3. Last (and probably most obvious) is the reduction in ground clearance with the smaller diameter tire. Depending upon where and how you ride this may or may not be important.
Prolly more info than you wanted - but without a definitive answer

So.
I'd say that if the bike seems...skittish (or you just want it to handle slower), and/or you ride where there are numerous potholes that seem to affect the bike more than you think it should, and/or the bike bottoms out on speedbumps..., going back to the stock size tire would help all those maladies.