As long as computers have been around, there has been a need for some sort of storage device. Storage started with punch cards and vacuum tubes. It moved on to reels of tape and floppy discs. Then it moved on to internal magnetic media in the form of hard disk drives (HDDs). We went from vacuum tubes to solid state memory chips. Optical media was invented. After this it was just a matter of miniaturization of either the medium, or the means of storing data on the medium.
External Magnetic Media
External magnetic media comprised a large portion of basic portable storage for computer system for a long time. Floppies were not named for their external appearance, but the for the media itself, which was a flimsy Mylar disc. The first floppy discs were 8" in size. They eventually came down to 5 1/4", and then 3 1/2". In the mid-1990s, high-capacity floppy formats were released in the form of ZIP and SuperDisk drives, and later JAZ drives. The largest regular floppy held 1.44 MB (megabytes) of data. ZIP drives held 100 MB, SuperDisks held 120MB and JAZ drives held 1 GB (gigabyte). A 2 GB JAZ drive was released a few years after the first model.
Tape storage has been in use since the 1950s and is still used today for system backups. Much like audio cassette tapes, tape storage uses a long thin tape coated with magnetic particles that pass over a head which aligns the particles to form binary bits. Current tape systems can store terabytes of information.
Internal Magnetic Media
The first hard disk drives were introduced in the 1950s. They were the size of refrigerators and had very low capacities by today's standards. Over time, components shrank, and magnetic particle densities went up, allowing for greater capacities. In the early- to mid-1980s, HDDs were down to the size of about two red bricks, and weighed about as much. The last 5 1/4" form factor HDD was made in the late 1990s, at which point 3 1/2" HDDs became the norm. Current desktop HDDs can hold up to 2 TB (terabytes) of data.
CD-ROM technology was introduced in 1985. It only started to became prevalent on PCs in the early 1990s. It remained the de facto optical storage medium until DVD-ROM drives became cheap. DVD-ROM drives did not reach the computer market until the late-90s.
Early in the 2000s, compact memory formats like Compact Flash came out. Rivals to the format such as SD memory, Multi-Media Card, xD picture card and others came out, which were even smaller. Soon flash memory with a USB connector came out, and gave people a non-proprietary means to store and transport large files. SD cards and USB flash drives of 16 GB and 32 GB are not uncommon.