Monday, May 2, 2011

The Scourge of Preservation : Disk-Based Copy-Protection

In collecting and playing floppy disk IBM PC compatible games from the DOS and pre-DOS era, there are two obstacles, copy-protection and media decay.  I would like to concentrate on the first in this entry.  Let me start by describing the physical workings of a floppy disk, then an overview of copy protection methods, then finally solutions for them, both old and new.

1.  Structure of a Regular Floppy Disk

A floppy disk, whether on a 5.25" or a 3.5" disk, is divided up into sides, tracks and sectors.  Except for the earliest programs, all regular IBM floppy disks use two sides.  The earliest drives that came with the IBM PC in 1981 were also single sided, but these were replaced by dual sided drives within a year.  All  compatibles released after 1982, even the PCjr., come with double sided drives.  These are the standard formats which DOS games used :

Floppy     # of       # of       # of      Total           Media
Type       Sides    Tracks  Sectors Capacity       Name
5.25"          1          40          8      160KB     Double Density
5.25"          1          40          9      180KB     Double Density
5.25"          2          40          8      320KB     Double Density
5.25"          2          40          9      360KB     Double Density
3.5"            2          80          9      720KB     Double Density
5.25"          2          80        15       1.2MB     High Density
3.5"            2          80        18     1.44MB     High Density

Each sector within a track stores 512 bytes of data.  Thus, for example, the capacity of a 360KB = 2 x 40 x 9 x 512.  An .img/.ima file is a raw dump of each sector in order from Track 0/Sector 0 to  Track 80/Sector 18 (assuming a 1.44MB disk).  The first track or so contains the DOS information, including the directory structure, file and attributes list, disk label (if one).  If the disk does not contain disk-based protection, an ordinary dump will be all that is needed to preserve the disk, assuming the contents are undamaged.  Regular DOS game disks are not bootable, DOS will need to be loaded from a hard disk drive or floppy disk before the game will work.

IMG/IMA files are raw dumps of a disk and should have file sizes equal to the total capacity of the disk, regardless of how much space on the disk is actually used.  A raw dump will preserve the disk label (used by some installation programs) and file date and time stamps.  While there is no difference between the two file extensions, DOSBox will only boot images with the IMG extension.  Thus there is an unofficial distinction whereby floppy images which can boot on a PC use the img extension and ordinary DOS disks the IMA extension.  An IMZ file has been compressed and should be extracted before use.

The idea behind copy protection is to prevent a game from working without the original floppy.  This means that the original floppy is non-standard in some way.  Before we discuss the ways in which a floppy can be non-standard, first lets categorize our IBM PC Games.

2.  Types of IBM PC Compatible Games

A.  PC Booter Games (Non-DOS Booters)

Officially, hard disks were not available for the IBM PC platform until the arrival of the IBM PC XT in 1983.  Even then they were extremely expensive, well out of the budget of your average home user.  Hard drives were not commonplace on consumer PCs until the late 1980s.  DOS was often used for two purposes, for running BASIC and saving and cataloging BASIC programs and for formatting disks for programs to save user files.  Most programs released during the early years were self-booting, you inserted the program disk in the drive, turned the computer on, and it would load your program.  When you were done you removed the disk from the drive and turned the machine off or inserted another program disk and did a hard Ctrl-Alt-Del reset.  The BIOS of the IBM PC and Compatibles would check the the disk in the first drive and see whether there was a boot loader on Track 0.  If there was, whether DOS or not, it would allow the program on the disk to control the system.  If not, IBM's PCs would go to the BASIC stored in ROM and clones and compatibles would usually ask the user to insert a bootable disk into the drive.

Games from these early years were frequently bootable and many times used their own custom and faster disk access routines, not using DOS's disk routines.  Since loading DOS first served no purpose, these games were free to structure themselves on a disk as they pleased.  Generally speaking, DOS could not read these disks.  Nor did these disks have to use the standard DOS formats described above, although most did.    Electronic Arts pushed the limits of the 5.25" Double Density drives by having 10 sectors per track, or 200/400K disks.  Bootable games tended to reside on two 5.25" double density or one 3.5" double density disks maximum and usually only supported CGA cards and Tandy/PCjr. graphics.  A few will support MDA, Hercules or EGA and Microsoft Serial Mice and IBM Graphics Printers.  Many arcade ports and clones can be found on PC Booters, but important series like King's Quest and Microsoft Fight Simulator first were released as booters.  Infocom's classic text adventures may have been released as booters.  Some small or early booters used single sided disks.  The first commercial IBM PC game, Microsoft Adventure, is a booter.

B.  DOS Floppy Only Protected Games

Not every game publisher felt the need to write custom disk routines when the ones from DOS worked well enough and were free if the user loaded DOS before starting the game.  Unlike the booter, these disks were generally not bootable unless the user copied the system files from a DOS disk to make the disk bootable.  These disk were readable in DOS and usually only the first disk had any protection.  However, early games were not intended to be installed onto hard drives (which the user or the programmers may not have been able to afford) and no benefit would result since the game would seek the files off the floppy disks.  Even though these disks were readable in DOS, they were still non-standard in some way so that they could not be copied using standard disk copying programs.  Many of these games still had no quit/exit to DOS capabilities,  requiring Ctrl-Alt-Del or a system shutdown to use another program.  Ultima II for the PC is a good example of this type.

C.  DOS Hard Drive Installable Protected Games

Later games realized that reading files off a hard disk, even slow ones, was faster than off floppies.  Moreover, a multiple floppy game could be installed to a hard drive, eliminating disk swapping.  Some games came on three or more 5.25" disks, and not everyone could afford or had room for two floppy drives.  These games would have programs on one of the disks to copy themselves to the hard disk drive or would give the user instructions on how to copy the files using DOS.  Still, the first disk had to be in the drive when the game was being used to access the protection.  Most games have quit commands in them by this time.  Frequently these games supported EGA cards, but some VGA games originally came on protected disks.  Lemmings the its Add-On Oh No! More Lemmings from 1991 was among the last-known of major IBM PC Compatible game with disk-based copy protection.

D.  Document Based Copy Protected Games

Due to the eventual proliferation of hard drives and annoyance with games that could not be backed up if the original got damaged or destroyed and had to be kept in the drive, companies moved away from protected disks.  However, to discourage people from sharing their games with every friend in their neighborhood, the companies used document checks instead.  The disks themselves would be ordinary DOS disks and full or partial hard drive installation would be encouraged.  So, upon startup or some point within a game, the game would ask a question which would require the user to refer to his documentation to answer it.  Sometimes the user would have to enter a word from a particular line of a particular paragraph from a particular page of the manual.  Sometimes he would have to enter a symbolic code from a codebook.  There may have been a codewheel included which the player would have to align as instructed.  Sometimes the player could only see the code with the help of a red filter gel.  Sometimes the game would refer the player to a paragraph in a book containing descriptions and dialog.  The idea was that the codes would be hard to photocopy or transmit through the BBSes of the day.  CDs, which were difficult to copy for several years, put an end of document checks.

3.  Methods of Floppy-Based Copy Protection

I do not pretend to be an expert on this subject, methods of floppy disk copy protection include :

Write Protection - The original disk was either physically write protected or not write protected, and if the user failed to duplicate the physical protection on the copy, the game would detect it and fail to work or trash the disk.

Fat or Thin Tracks - Normal double density disks had tracks of 8 or 9 sectors.  However, some games used 10 sectors per track or fewer than 8 sectors, which the disk controller could read.  Do not expect DOS to write non-standard tracks.

Oversized or Undersized Sectors - Normal sectors are 512 bytes, but valid sector sizes for the disk controller range from 128 bytes to 8,192 bytes, even though a double density track generally cannot store more than 6K of information.  DOS always assumes 512 byte sectors for standard floppy disks.

Extra Tracks - Data can be stored on Track 41 or above on a 5.25" disk or Track 81 or above on a 3.5" disk or 5.25" high density disk.  Something else DOS does not support.

Unformatted Tracks - The game can check for the existence of unformatted Track(s), a Track with no sectors defined, somewhere on the disk. If tracks are unformatted above a certain number, a game may fail if the copy formats those tracks.  DOS formats only all tracks on a disk.

Weak Bits - Some area of the disk could have information written onto it that cannot be reliably read.  A game can refuse to work if several reads to an area with weak bits protection produces the same result, the game expects randomness, but a standard PC floppy controller cannot write unreliable data.   

CRC Errors - Each sector after the data contains CRC values for error checking for the data in the sector.  If the CRC check fails, an error will be produced.  Some games have deliberate CRC errors which can be read, and if not correct, can cause the game to fail.  A standard PC floppy controller cannot write bad CRCs.

Sync Byte/Gap Byte/Sector ID Manipulation - Before the data in each sector, there are sync bytes, gap bytes and a sector ID to organize and separate the sectors on a Track.  A game could manipulate this area and if the manipulation on the copy is not the same as the original, fail.

Burn Hole - Machines exist that could precisely burn a hole in the disk with a laser (not the sector 0 index hole).  A program could check for the existence of the hole and refuse to work if not encountered.  Typically used only on very expensive software, no games have been known to use it.

These methods can and have been combined.  Like with today's LaserLock, SafeDisc, SecuROM, and StarForce, certain protection methods were marketed under the names ProLok, SuperLock, EverLock, and Cops Copylock II.  An example of a famous protection scheme can be found here :

http://www.sierrahelp.com/GeneralHelp/FloppyDiskBackupProblems.html

4.  Ways to Defeat Disk Based Copy Protection

A.  Software Solutions

Some companies understood that there was a market for programs for users who wanted to protect their expensive software.  The most famous back in the day was Central Point Software, who made a software product called CopyIIPC.  CopyIIPC was designed to copy floppies only, not to preserve them in an image file.  As new protections were released, updated versions of CopyIIPC were released to handle new protections.  CopyIIPC allowed for better copies through better manipulation of the PC disk controller.  Pressure from some software copies eventually persuaded Central Point Software to remove certain functionality in later versions of the software, so a user may need an earlier version to break a particular protection scheme.  If CopyIIPC could not duplicate a protection on a copied disk, the program would have the program to remove or alter the protection so it would work with the copied disk.  Development stopped after v6.00 in 1990.

CopyIIPC also came with the nokey and noguard programs.  Nokey would allow users of DOS Protected Floppies to use their programs without having a keydisk inserted in the floppy disk drive while the program was running.  Noguard allowed otherwise non-hard drive installable programs to be installed onto the hard drive and even in later versions eliminated document checks for some games.  CopyIIPC has no innate capabilities of making images for later backup, so some enterprising programmer wrote a tool called Snatchit to allow backing up and restoring the image to a floppy.  These files use the .cp2 extension but no emulators are known support them.  CopyIIPC + Snatchit are speed sensitive and getting the combo to work in a fast 386 or better may not be possible.

Teledisk is a competitor to CopyIIPC and can handle many types of protections.  It can also archive protected disks into an image format without a Snatchit type program.  It will fail if it encounters an unformatted track, however.  Unlike CopyIIPC, it will run in any speed of PC, and its .td0 disk images are supported in the MESS and PCE emulators.

Other programs include Rawcopy, Diskdupe, Anadisk, Locksmith, Neverlock, CrackAid and The Patcher. Some will copy disks, others only remove protection.  Disk2FDI is a more recent, trialware program that is capable of reading but not writing disks, but requires two floppy drives (trial version) or a special floppy-to-parallel cable (registered version).  Its .fdi images are not widely supported in emulators.

B.  Hardware Solutions

The PC Disk Controller could read more of a disk than write it, so if software failed, then a hardware solution was needed.  Central Point Software released the CopyIIPC Option Board, an ISA card that was connected between the disk drive and the regular disk controller.  This card could read the magnetic flux transitions on the disk and write them too, essentially capable of making bit perfect copies of a disk.  The software, called Transcopy, was updated regularly to handle new protections.

The original Option Board was released in three variations, the long TTL board, the short TTL board, and the VLSI "Transcopy" chip board.  The long TTL board does not have a jumper to select DMA1, which is required for Tandy 1000 computers (and probably would not fit in them anyways).  All otherwise have the same capabilities, but the VLSI board is impossible to fix if the main chip goes bad.  There was an Enhanced Option Board, which was a regular Option Board with circuitry to emulate Burn Hole protection on disks and a switch to turn that functionality on and off.  Finally, there was a Deluxe Option Board that could copy some Macintosh disks and use high density drives to some degree.  No Option Board can copy High Density protected disks, but no games are known to come on disk-protected High Density disks.

Transcopy is an easy to use program, and it can archive images to a file for later recording or multiple copies.  The resulting files use the .img format, but are far larger and quite incompatible with the later .ima/.img format. Transcopy went up to version 5.4, but it seems only the Deluxe Option Board can use versions 5.x and below, while the original Option Board can seemingly only use versions 4.x and lower.  So if a disk image is made with version 5.x, it may not be writable with 4.x software.  Normally, it is best to use the latest version of Transcopy and go down if there is a problem (due to the pressure put on Central Point Software by software publishers), but it may be necessary to stick with version 4.x or below for widest distribution.

The Option Boards are 8-bit ISA cards and work in IBM PCs, XTs, ATs, Tandy 1000s, PS/2s (with ISA) and many compatibles.  Computers in the late 1980s (Tandy 1000 TX, IBM PS/2 Model 30) that supported 3.5" drives frequently provided power to the drives on the drive data cable, so custom cables will have to be made to get these machines working with the Option Board.  An end user can do this if he has the pinouts for the cable at hand.  Tandy machines require floppy cables without a twist in them.

Using Transcopy and the Option Board does not seem particularly speed-dependent.  I have heard users write that they were able to use the hardware and software in a Pentium II or III computer, which were among the last to regularly feature ISA slots.  Pentium IV and socketed Athlon boards with ISA slots tended to have incomplete ISA implementations.  However, there is a disadvantage in that the hardware is ISA based, so a vintage machine of some nature is required to use it.  The resulting .img format is not supported by any emulator I know.  The last Option Boards and the last update to the software was about 1990.

More recently, the Catweasel board was created that can read and write protected disk formats in a PC drive.  The Catweasel originally came in ISA (MK1) and Amiga Zorro III (MK2) but soon was released for PCI (MK3 & MK4 & MK4plus).  It usually has sockets for SID chips to be used with Commodore 64 emulators.  It was first released in 1996 and the last version of the card was put out in 2004.  Last drivers date from early 2007.  Even more recently, the Kyroflux board was released for USB.  Both are capable of imaging and (very recently for Kyroflux) writing to disks.  These products should work with PC protections, but can also image disks from the Commodore PET, VIC-20, 64 & Amiga, the Atari 8-bit and ST computers, the Apple II and Macintosh, the Tandy TRS-80 and Color Computers, MSX.   These boards are still in production, unlike the Option Boards.  Their availability tends to be iffy, so if you see one for sale, you should grab one if you can afford it.  Write support for protected PC disks has yet to be implemented.