Originally, overclocking was done to snag more performance out of the system. While that is done to some effect today, it's not always as potent due to the fact that the CPU is no longer the deciding factor in a PC's performance. Today it's more for the hell of it, fun, and/or because you can.
Edjis provided some good links to help in safe overclocking. Simply jacking up the voltage as high as you can go and cranking the FSB to the max usually doesn't work, at least very well, so overclocking does require some patience.