John - not seeing it, it's hard to give a definitive answer, but as a rule, you have to figure out what the roughness is, and how deep it is...
Two different types of "roughness", and different "cures" - there's "orange peel", which is a roughness in the coat itself, and then there's contamination - either actual dust, or overspray from painting.
Is it definitely from the clear coat, or is it possibly from deeper in the color layer? It sounds like it's from the clear, since you say you sanded the black. 800 grit's a good choice. That would eliminate any problems from the color coat.
Did you clear coat fairly soon after finishing the color coat? Sooner is better, as it allows the clear to bond chemically with the color coat, as well as decrease the possibility of dry overspray.
Did you possibly spray "dry"? That is, when the clear dries almost before going on the lid. That can cause a roughness, really easily. Painting in the wind, or a really warm shop, can do that. The clear should look like glass, when you spray it on. A problem a lot of people do, is go back over it "one more time", and since they are going both faster, and trying to put on less coat, it tends to go on dry, causing roughness. That you mentioned that the top is fine, but sides are rough, it leads me to believe that it went on "dry" on the sides, either with overspray, or going faster.
Did you use a tack cloth, before the clear coats? that's a MUST, even if it "LOOKS" fine, it usually isn't.
Depending on how rough it is, the 1500 grit before buffing, should GREATLY improve it. Just be sure you don't go through into the color coat, as then you might be re-doing it all.
Again, I'm not a professional painter, but I DID spend the night in a Holiday Express...
Let us know how it turns out - you are correct, use a slow speed buffer. And you definitely want to get your technique ironed out, before you go to the next piece - it's a LOT easier to do the lid two or three times, than the entire trunk!
You'll get it - suddenly it will just "click" and you'll amaze yourself!!
The big thing is, have fun.
On another note, you might take it to a pro, and just ask them - most pro's I have ever known, have been more than happy to give hints and tips to someone - they seem to KNOW that if you don't get it right, that you'll be back to pay them to do it. They'll also be able to tell you exactly what you have.
Keep us posted, please! We all like to learn from other's efforts.

EDIT - on a side note, I'm getting ready to paint my saddlebags - my "pro" quoted me $200, for a custom design, and two-color, with pinstripe, to match my tank. It would be hard for me to buy all the paints and other chemicals, to do it, for that! It will probably be one of the very first times, I haven't done my own painting, but I KNOW his will be perfect, the first time. He has done all my insurance work, in the past, and has always done EXCELLENT work.
My bags are going to look similar to this one, but the white will be black, and really lends itself to the tank lines...
