OC put simply?
Overclocking is running a CPU, video card, RAM, bus, or whatever else has a clock-generated speed, and running it faster than the speed. How much faster depends on the part, the voltage, cooling, and luck. You're only guanteed the speed that the manufacturer says, nothing more. As a general rule, overclocking does void your warranty on whatever is overclocked, so understand that before overclocking anything. Having said that, there are companies that provide the means to overclock, even though they don't come out and endorse it. So far, only DFI has really been publically gung-ho about it, although Abit, Asus, Epox, Chaintech, MSI and Soltek are a few companies that tend to have good overclocking options as a whole as far as motherboards go.
Overclocking the CPU and RAM are done with the motherboard, usually in BIOS. Some companies are also providing a utility to allow overclocking in Windows. They're fine, but in the past Windows-based OC tools tended to be rather flaky. While that part is resolved, you still have to worry about it like any other software, as in crashing and bugs are still a possibility. Most of it is just the old school overclockers, like myself, that love the BIOS settings over the OS-based software. With BIOS, you set it, and it stays that way. While, yes, you can change some stuff and not boot up, you can reset BIOS and start fresh---although any non-OC'd settings you have made will also be reset to their defaults. If you take it slow, and test your OC in increments, you'll do alright. It's doing stupid stuff, like pumping in excessive voltage and/or using poor cooling.
Most of what I said regarding CPU overclocking applies to the video card, but you don't have a BIOS that you can get into on the video card. It does have a BIOS, but I haven't heard of any video card that allows you to OC through it's own BIOS. You do have programs, like Riva Tuner or Rage3D Tweaker, that you can OC your card through Windows. Outside of physically modifying the video card's circuit board (also PCB: Printed Circuit Board) and/or flashing a modified BIOS for your video card, you cannot change the voltage.