Pay close attention to the minimum requirements for the software you want to use to make sure you computer and operating system can run the software.
Find a quiet room where noise is at a minimum.
Consider using the Center For New Media where you will find all the software and hardware needed to create podcasts.
Microphones
Microphones can cost anywhere from a few dollars to thousands of dollars. For effective podcasting, you won't need to spend a lot, $15-$40 for the most basic headset mic, and on the high end $200 for studio quality recording gear.
The quickest and easiest way to get started is to use either a built-in mic or a USB mic.
Built-in Mics
Recent iMacs have built-in iSight cameras that have a decent mic appropriate for podcast recordings
Recent Mac laptops have either a built-in iSight camera with microphone, or a built-in microphone
Some PCs (especially laptops) have built-in mics, look in the documentation of your computer to see if you have one
USB Headset Microphones
Cheapest choice
Easy to use, just plug and play
Connects to the computer by USB
Most get their power from the computer through USB
Following is a list of manufacturers who make a number of economic headset USB mics and some links to vendors to see prices.
M-Audio Nova - will need a cable adapter to connect by USB
Digital Audio Recorders
Portable audio devices that record digital audio onto digital media cards such as the SD cards (also used for digital cameras)
No need to be connected to a computer to record
Good for recording in the field, interviews where 2 or more people are talking
Until recently, a high quality digital recorder would cost you in the $400 range, the Zoom H2 can be had for around $200.
Other Audio Gear
Pop Filter - to help prevent popping p's and hot s's. See also: How To Make Your Own Microphone Pop-Filter Mic Stand - economic desktop mic stand, if your mic does not come with one Preamp - for recording with two mics at once, necessary for some higher level mics to get powered and connected to the computer, comes with USB or Firewire interface for connecting to computer