Found the problem. It turned out my rear light (green -) wire was not properly grounded somewhere further down the road. I grounded the green wire separately to the frame as well and now everything is working OK. Even better, the light is shining much brighter than it used to do. Since it was working and I did not notice the light to be less bright, it got me.

Connecting the LED was a piece of cake now.
Thanks everyone, for your constructive advice!