Most of the software on the AGC (Apollo Guidance Computer) is stored in a special read-only memory known as core rope memory, fashioned by weaving wires through and around magnetic cores, though a small amount of read/write core memory is available.
Bonus Nerd Link: the Virtual AGC Project
Especially when you consider computers at the time were roughly the size of shipping containers, and the hard drive was a whole other box.
http://factmyth.com/factoids/the-first-hard-drive-was-announced-in-1956/