..... hit a really rough spot and the downward force of my weight jerked the handle bars way down. The handlebars have factory grooves where the clamps go. They are now smooth
The Valk handlebars are serrated at the mating surfaces, or were serrated. Perhaps, whatever the textured surface on the 'bar backs' weren't the best match.
Imho, I don't think shimming the handlbars is the safest thing to do. Under stress, if it gives you might be screwed. I had the handlebars rotate on me one day while sitting on
the Valk in the garage. I last worked on the handlebars months earlier. Until then never felt anything wrong with the handlebars. Turned out that I had torqued all four bolt wrong. All four, so I musta used the wrong ft lbs. The feeling that you get is well I leave it at that.

Myself, I've always torqued the front bolt first. Things end up somewhat like this (exaggerated).

Just for conversation, the tightness doesn't rely 100% on textured surfaces. The clamps aren't full hemispheres. A segment is shaved off or excluded (red, below).

This insures the top clamp never bottoms out however minutely before it's full torque (A). I think it's the same with the brake lever & clutch lever perch.
True, something you couldn't miss.
