The Rpi is a full fledged ARM computer, so it should be able to run about everything compiled for ARM (within system requirements). The boards do not have any on board storage, so everything is on the SD card. If you just want a working system, buy a preformatted SD card from the Foundation, or give PiCard a try, a GUI SD preparation tool made by Liam Frazer.
The boards do not include NAND or NOR storage - everything is on the SD card, which has a FAT32 partition with GPU firmware and a kernel image, and an EXT2 partition with the rootfs.
We're not currently using a bootloader - we actually boot via the GPU, which contains a proprietary RISC core (wacky architecture ;) . The GPU mounts the SD card, loads GPU firmware and brings up display/video/3d, loads a kernel image, resets the SD card host and starts the ARM.
You could replace the kernel image with a bootloader image, and that would work fine.
Source code and binaries for Raspberry Pi will be available at various places from launch, including pre-built Linux distributions.
Ubuntu is currently listed as the default distribution on the Raspberry Pi website, but the Ubuntu developers have now stated they will not be supporting the ARMv6 architecture, so Ubuntu is likely to be dropped.
Eben says (regarding default distribution): "Either Ubuntu or Fedora; the main point in Fedora’s favour is their ongoing support for ARMv6 architectures."
Main page: RaspberryPiBoardDistributions
Instead of just using compiler + editor, you can use "development tool chains" which integrate compiler, build system, packaging tools etc. in one tool chain.
The initial patches (Media:Rpi-linux-patches-3.1.9.tgz) are released against the 3.1.9 kernel - but may also work against later kernels.
The kernel is now also available on https://github.com/raspberrypi/linux
See rpi kernel compilation (unfinished) for how to compile the kernel.
(left here until new page is created:) TODO: Explain how to apply these patches and correctly obtain a default configured kernel.
The Broadcom processor on Raspberry Pi contains an ARM v6 general purpose processor and a Broadcom VideoCore IV GPU. No data is currently available on other cores (if any) available in the BCM2835.
There is broad compiler support including gcc - please see ARM Compilers
The GPU provides APIs for Open GL ES 2.0, hardware-accelerated OpenVG, and 1080p30 H.264 high-profile decode.
The GPU is capable of 1Gpixel/s, 1.5Gtexel/s or 24 GFLOPs of general purpose compute and features a bunch of texture filtering and DMA infrastructure - the Raspberry Pi team are looking at how they can make this available to application programmers.
The GPU blob is an 18MB as an elf file, plus libraries. It does an awful lot. 
There is a DSP, but there isn't currently a public API (Liz thinks the BC team are keen to make one available at some point).
From Eben: "We'll have to take a look. We support hardware-accelerated OpenVG, and have had Flash Lite running incredibly fast. Personally, I'd like to get the official hardware-accelerated Flash 10 going on there, running against OpenGL ES 2.0, but that's something to think about after the launch."
This section collects hints, tips & tricks for various software components.
Raspberry Pi plans to support Python and C as primary teaching languages, but expect to have some sort of BASIC on there too. Perhaps even BBC BASIC or SuperBASIC depending on copyright issues.
See main page RPi Programming for more detail.