COMPUTER
The major investment you need is a PC, and I recommend a Windows Laptop (XP or Win 7) if you want to hustle around and gig. Most of the Freeware on this site is Windows-only, there's just more of it baby. To run high-end commercial software, or software with lots of samples, you need a fairly strong PC. But using modelling software that avoids samples you can go much lower. My system has just 1 gig of memory and it works fine. You won't need much disk space either. You can squeeze some EPs, and organ, speakers, and effects into 10 meg, and you can get pretty much everything on this site in under 100 meg.
KEYBOARD CONTROLLER
You can't play keyboards without a keyboard. It just ain't right baby. What you want is a MIDI keyboard. critical thing is to enjoy the action of the keys, weighted and semi-weighted cost more but they can have a nicer feel to play. Also important is having buttons, switches, dials, and eight or more faders, that you can map to the software you going to use, to things such as tremolo, or organ drawbars.
There are a lot of keyboards to choose from, and I'm guessing if you had thousands to spend you'd go ahead and get a Nord Electro/Stage. Really classy MIDI controllers with faders, dials, and buttons galore include:
● Fatar Studiologic VMK-161 Plus Hammer/Organ [Review]
● M-Audio Axiom Pro 61 [Review]
● Novation SL61 MK1/MK2 [Review]
● Roland Cakewalk A-800 Pro 61 [Review]
There are a number of cheaper controllers in the "plastic toy" range, but many - such as the Behringers - don't have faders to use as organ drawbars. One does though, and don't let the "plastic toy" put you off, if you can play you can play, check out the crazy organist examples.
● M-Audio Oxygen 61 [Review] [example] [example]
Many of the reviews are for versions with fewer keys but don't let that put you off your stride. For sequencer-input by a chubby white guy in his bedroom making dance music for muscular guys in black string vests to smile too much to, 25 keys is plenty. But for proper Organ/Electric Piano jamming by The Man With The Plan (and that's you, baby!) you want 61 keys or more.
If all that is just too much dough remember any MIDI keyboard is gonna do. Check your local classifieds such as Gumtree. Lots of people buy keyboards, get bored, and sell them for next to nothing, others want to upgrade their set-up to a more expensive rig. Whatever keyboard you do get, RTFM - read the funking manual. You can almost certainly download the manual if you have to. It will tell you how to set your system up for transmitting MIDI.
EXTRA CONTROLLER
If your keyboard doesn't have much the way of dials or faders, don't fret, you can add more to your set-up. The higher-end units are so expensive you'd probably be better off just getting one of the keyboards above, but there is a cheap and cheerful way to get sliders and dials onto your keyboard:
● Korg nanoKontrol [Review]
SOUNDCARD
If your computer has a high-end audio soundcard, cool. The chances are it doesn't. You should get a good external soundcard, if you amplify most laptop sound (e.g. from the headphone socket) the chances are you'll hear a lot of background hum and hiss.
There are two basic types of output, balanced and unbalanced. You can read up on the details elsewhere [here] but the summary is unbalanced is what you find at home on your CD player and if you're using it on stage you want very short cables, if you want to use longer cables like the pros do, you need balanced.
The unbalanced units are 16-bit, but dirt-cheap:
● Behringer UCA [202 ] [222]
● Numark Sterio|iO [site]
Balanced units are feature-rich 24-bit rigs, but cost quite a bit more:
● Lexicon Alpha Studio [review]
● Cakewalk UA-25EX [site]
SUMMARY
That's about it. You might need some cables. I use cheap MIDI cables but use good quality audio cables. Note some of the sound-cards come with MIDI ports too, so you can plug your equipment into them rather than run separate MIDI-USB cables.
That's the hardware sorted baby - you're almost ready to have fun!