Friday, May 18, 2018

The Search for Artifact Color on the Commodore 64

NTSC composite artifact color is something typically relegated to computers with off-the-shelf graphics hardware.  We associate it with the Apple II mainly, which used it in its high resolution modes.  TRS-80 Color Computer fans also know it very well, because it was the only color available in that computer line's graphics until the Model 3.  IBM PCs also used with more frequency than was commonly known in the early days in CGA graphics cards.  But Commodore didn't rely on off-the-shelf 74-series logic to drive its home computers' displays.  It had bought the MOS company and all its chip fabrication expertise.  Its computers used real graphics chips and they displayed real color.  They didn't need the composite tricks to get their graphics working and they didn't need boards devoted to graphics either.  But I have come across some information which suggests that the assumption that the Commodore 64 did not support composite artifact color may not be supportable.




The Commodore VIC-20 used a graphics and sound chip called the VIC (Video Interface Chip), MOS 6560.  It could output 16 colors and supported a 176x184 resolution.  A large single-color border surrounded the active display.  This resolution was low enough not to cause much color artifacts.  Then the C64 used the VIC-II chip, the MOS 6567.  This chip was much more advanced and could display 16 colors at 160x200 or 320x200 modes with borders.  Each tile in the 160-pixel mode could use up to four colors, but each tile in the 320-color mode could only use two colors.

Now most games used the 160-pixel color mode.  With only 160-pixels, the pixel clock and the NTSC clock were more-or-less synchronous with each other, so each color had its own distinctiveness and did not clash with the next color.  But with 320-pixels being displayed, the 3.58MHz NTSC clock cannot keep up with the pixel clock. While Apple and IBM used an integer multiple of 3.58MHz to ensure artifact colors would be produced, Commodore used a pixel clock of 8.18MHz to try to make sure that the pixel clock would not fall neatly between the NTSC color clock, thereby reducing the reliability of artifacts.  It also generated colors directly, having the circuitry to display preset combinations of eight different "angles" on the NTSC color wheel and eight different luminance values combined for the official 16-color palette.

So you might think that artifact colors are too unreliable to be useful in the C64, but I was unsure.  Consider the following screenshots :



Those screenshots are of the dungeons of Ultima III and Ultima IV, respectively, that I took (and submitted to MobyGames) via WinVICE.  Note how the dungeon walls have a pattern of alternating vertical white and black lines.  As you may know Ultima III and Ultima IV were developed for the Apple II computers and use the alternating white and black pattern to show single-color walls in the dungeons on a color display using artifact colors.  On a monochrome display which suppresses the NTSC color carrier, the walls would use the pattern shown above on an Apple II.  The versions for the IBM PC (with CGA) and Atari 8-bit home computers rely on artifact colors and show either solid color walls or striped walls depending on the type of display to which you are connecting these computers.

But the C64 was not intended to support composite artifact colors.  It's high resolution mode could easily use solid colors for these walls given that you can choose two colors in each 8x8 tile.  Why did the programmer choose to use striped wall tiles instead of solid wall tiles?  And why use white?Unless the C64 could produce composite artifact colors, there was no good reason other than a programmer who never saw how the port would look on the C64.

A couple of years ago, I asked on the NESdev forum why the NES did not show composite color artifacts and the conversation eventually turned to a comparison between the NES and the C64.  The conclusion from the first round of discussion was that the C64 displayed a fully compliant NTSC picture and thus should show those stripey walls even with a composite signal : http://forums.nesdev.com/viewtopic.php?f=10&t=14394&hilit=c64&start=15#p174784

For quite a while I could not prove or disprove my suspicions that there was some support for composite artifact color in these games.  The C64 emulators were all PAL-centric and PAL is a more sophisticated color decoding system that does not show composite color artifacts.  Any kind of CRT filter in emulators like WinVICE do not try to properly generate NTSC color.  While I had a C64, until I had a way to play games on it, there was nothing I could do.  Last year I acquired a 1641 Ultimate II+ flash cart and floppy drive emulator and could finally start some testing.  S-Video capture of the C64 showed no color artifacts as I expected.  The walls were white and stripey, although not quite as sharp and distinct as the ideal captures shown above.  I know I tried my capture card with the composite connection and saw more blending of the lines than S-Video but no color to speak of.  I concluded that the programmer must have intended white.  At this time, which was in mid-2017, the one thing I did not do was to connect my C64 to my TV to see what my TV actually displayed.  Here is what my capture cards show with the R8 VIC-II :

VIC-II 6567R8 S-Video

VIC-II 6567R8 Composite
A few months ago the subject was brought up again on NESdev, and this time the claim was that the old revision of the C64's VIC-II graphics chip should show reliable artifact colors : http://forums.nesdev.com/viewtopic.php?f=2&t=17061&hilit=c64&start=45#p214470  The 6567 VIC-II came in many revisions and the revision is marked on the chip's package.  Known revisions are the R56A, R7, R8 & R9.  Between the R56A and the R7, Commodore made slight timing tweaks to the VIC-II which caused color artifacts to be much more suppressed than on the R56A.  The oldest C64s with the 5-pin video output DINs use the R56A.  Later C64s with the 8-pin port will likely use an R7 or above.

The timing differences between the R56A and the R7-9s are slight.  The R56A outputs 262 vertical lines and the R7-9 output 263 (Canonical NTSC outputs 262.5 lines)  The R56A displays each line in  64 machine cycles vs. 65 machine cycles for R7-9.  See here : http://www.zimmers.net/cbmpics/cbm/c64/vic-ii.txt  Apparently the even values are more composite artifact color friendly than the odd values used in the later revisions.

On the strength of the discussion and the expertise of the individuals leading the discussion in NESdev, I decided to take a second look at the composite graphics of Ultimas III & IV.  Unfortunately my C64 has an 8-pin DIN with a 6567 R8 inside it, so I knew going into it that I would probably not see ideal composite colors.  This is what I saw with my C64 :



The above colors were encouraging, it seems that Commodore's tweaks reduced the composite color artifacts to some extent, enough to discourage people from exploring the capabilities.  But what about the R56A?  That was the chip which was supposedly composite artifact color friendly, but I did not have one.  Fortunately my friend Cloudschatze had a "parts C64" that used the 5-pin DIN and could lend to me.

However, when I received his C64, the system would turn on to a black screen.  There was no SID chip in the system, but that would not cause the C64 to fail to boot.  I noticed that one of the 6526s got burning hot to the touch within 30 seconds, so at some point it would have to be replaced but even that would not cause the system to fail to boot.

The systems using the 5-pin video output DINs use a PCB designated as Assembly # 326289.  The schematic covering this system is 326106.  My 8-pin video output DIN system used the next major PCB revision, Assembly # 250407, schematic 251138.  Given that the chips used in each PCB were identical and use the same U## designations, I decided to remove the R56A from the 5-pin system and put it the 8-pin system.  Even though Commodore did not always socket its chips, they always socketed the VIC-IIs in the older PCBs.  The old VIC-II worked just fine in the newer PCB.  (Intros and Demos like the R56A even less than the R7-9s, Ultima IV Remastered Edition refuses to work with an R56A.)  Firing up Ultima IV on my TV gave me this image :


I was rather surprised, I expected solid color walls, not what I saw in that photo.  But you can see solid bands of red, blue and green.  At some angles there are hints of brown and purple.  When I showed my picture to Cloudschatze, he commented "More like a circus fun house than a dungeon. Guess that could be scary."  Both C64 ports were done by Charles "Chuckles" Bueche.  He also did the Atari 8-bit port of Ultima II, which also used artifact color graphics, and did other programming work for the Apple II and Atari 8-bit computers.  Given that "Chuckles" often found his way into the Ultima games as a seemingly witless court jester to Lord British, the above screenshot is likely a sample of a wicked sense of humor. I didn't take a picture of Ultima III's dungeon because its dungeon walls would look exactly like Ultima IV's. Unlike the 6567R8 captures, my capture card this time did not lead mislead me :

VIC-II 6567R56A Composite 
VIC-II 6567R56A S-Video
I cannot find many other C64 games where this kind of graphical effect is likely to have been intentionally used.  Wrath of Denethenor by Sierra Online might show some interesting patterns on its world map screens.  Origin's Moebius and Windtalker might also show some interesting colors as might be the title screen to Pinball Construction Set.  Ironically, Ultimas I, II and V for the C64 will not.  The first two games use pure wire-frame dungeons and Ultima V has far more detailed dungeon graphics than any of the previous games.  Here are a few more screenshots showing other games which use it, either intentionally or accidentally :

Seven Cities of Gold (edges of box)

Rad Warrior (status area)

Pinball Construction Set
One caveat about these findings is that I was using the R56A VIC-II in a later PCB than may have been used by the original programmer.  The original R56As have a reputation of showing washed out color and trailing ghost images, but when I put my R56A into Assembly 250407, the colors I saw were just as bold and sharp as my R8 VIC-II.  I would suggest therefore that the original C64 PCB, Assembly 326289 lacked good filtering and saturation circuitry to make the VIC-II shine.  The walls might be slightly fuzzier with Assembly 326289 as a result but I do not believe that any serious differences would be apparent.  There are a pair of passive component swaps that should be done when using an R56A in an Assembly 250407 PCB according to Schematic 251138, but my board already had the R56A suitable components installed even though it came with an R8.  Schematic 251138 is proof that using an R56A is an approved Commodore configuration and could have existed on systems that came from the factory.

Here are a pair of video captures I took of the composite video output.  The first shows the output from the 6567R8 and the second the 6567R56A :





A special thanks to my buddy Cloudschatze for providing the hardware that made this blog entry possible and the guys on NESDev who gave me the information I needed to establish this phenomenon. 

8 comments:

  1. I have written an emulator that models the Vic-20 (amongst others) with a genuine NTSC decoding attached. Not an NTSC-esque shader, not anything subjective, a genuine handing off of composite information with a proper knowledge of phase. So e.g. the Atari 2600 and ColecoVision show solid vertical rainbows because they're both in-phase, the Apple's artefacts display as colours despite the decoder having no knowledge that an Apple is sending it that particular square wave, etc. Or any system-specific knowledge being in the decoder.

    The NTSC Vic-20 is not in-phase, it produces lines at the proper NTSC rate so phase is half a cycle different on each line, and there are an odd number of lines in each field so (ignoring discussion of interlaced mode) each line has the opposing phase from frame to frame.

    From your depiction of the C64, it looks like it has the Atari/TMS-esque property of using a slightly miscorrect line length in order to remain in phase. That's a simplification even if you haven't aligned your pixels as an integral divisor of the colour subcarrier because it means you don't have to check any state when generating your colour burst.

    The effect is then exactly what you'd expect from drawing the columns of pixels you show in monochrome. You would see exactly the same thing (well, possibly at a different density) on a TMS machine such as the ColecoVision or an MSX. If you check Karl Guttag's stash of memos on the TMS you'll see that they called this "the rainbow effect" and that engineers were directed to look into it as a bug. A fix was tested, without all of the modes working, but I guess the imprudent wasn't worth it. Commodore must have formed a different conclusion.

    I'll fire up my emulator this evening and plot some vertical lines with NTSC decoding in MSX BASIC. Hopefully I'll get a chance to come back with a screenshot.

    ReplyDelete
  2. Although the non-NTSC pixel clock means that an alternating pattern just gives those rainbows, it seems like a different pattern might make a consistent hue (albeit probably with varying luminance). Probably not very useful for games, but it'd be interesting to see what kind of images you could get from an image converter which is aware of the number of pixels per chroma carrier cycle.

    ReplyDelete
  3. I demonstrate something like how the "rainbow effect" can be seen in the TMS-derived Sega Master System in these videos : https://www.youtube.com/watch?v=creDKP43oCI
    https://www.youtube.com/watch?v=Au-9yyrxQx4

    ReplyDelete
  4. For the record, https://postimg.cc/image/41rplf5if/ is the rainbow effect in full swing on an NTSC MSX. The listing at the bottom is the code I used to generate the image at the top. As a bluffer's guide, the colour statement specifies white as the colour to paint in, then black for both the background and border colours, and in each iteration of the loop the pset moves the current graphics coordinate to the one listed and the draw means "draw downward for 192 pixels".

    So it's 128 vertical white lines on a black background. As above, the TMS is in-phase with the colour subcarrier so each line begins at the same phase. Then each pixel is 2/3rds of a colour clock, hence the repetition of colours.

    ReplyDelete
  5. The pattern appears to repeat every six pixels, pretty neat effect but somewhat tricky to use. Of course if you use a base color other than white, the possibilities multiply by fourteen.

    ReplyDelete
  6. I'm sure there's at least one full-colour C64 platform game I've seen demonstration of that looks much nicer on an NTSC TV than either PAL or in an emulator, because the programmers deliberately took advantage of artefacting (and the "multiplied possibilities" hinted at by Great Hierophant). Can't remember the name of it, but it was fairly late-period and cartoonish with a (yellow and pink?) dinosaur-like creature as the main character... Obviously it must be a pretty difficult thing to pull off as it's not particularly common, even amongst the demoscene...

    FWIW, the non-rainbowed TV screenshots look like the "stripey" walls are a sort of brownish grey colour to me, which might actually have been what was intended, given that RF or composite was the main, expected connection method rather than S-video, much less any kind of sharp-pixeled RGB... And textures like that remained common, especially for 3D rendering, into the 16-bit era, because of how they would blend on the cheaper screens used for gaming...

    ReplyDelete
    Replies
    1. Sorry to be a bit of a downer but the game you're referring to is almost certainly Mayhem in Monsterland, a PAL-only release which uses the much simpler palette switching trick to generate colours beyond the standard 16. Nothing to do with NTSC artifact colour, common as mud in demos, but rarely seen in games.

      Delete
    2. Mayhem in Monsterland has a very strong demoscene connection and takes many influences from it. They really put their best effort out for a platform that was thoroughly obsolete in 1992. The original game really does not work on NTSC machines or intended for them. It took some heroic hacking efforts to fit it into the much less generous VBlank time of the 60Hz standard.

      Delete