Famous graphics chips: Nintendo 64

Silicon Graphics had been a leader and highly respected workstation developer that rose to fame and fortune based on its introduction of a VLSI geometry processor in 1981. In the ensuing years, it developed leading graphics technologies at the high end. A high-end super high-performance workstation could cost over $100,000. Therefore, the idea of adapting such state of the art technology to a consumer product like a game console that sold for a few hundred dollars was considered bold, challenging, and crazy.

Nonetheless, in 1992 and early 1993, Silicon Graphics (SGI) founder and CEO Jim Clark met with Nintendo CEO Hiroshi Yamauchi to discuss just that—squeezing an SGI graphics system into a console. And thus, was born the idea of the Nintendo 64.

Even the number, 64, was outrageous. Most consoles of the time were struggling to get from 8-bits to 32-bit; 64-bits was considered science fiction.

Even so, they did it and on November 24, 1995, at Nintendo's 7th Annual Shoshinkai trade show, the company unveiled the Nintendo 64 console. Later in May 1996, at the E3 conference in L.A., Nintendo showed the Nintendo 64 and announced it would be available in the U.S. in September.

It was an amazing amount of technology crammed into a ridiculously small package, and at a crazy low price of $250 ($420 today).

Nintendo 64 motherboard, CPU, and Reality Co-Processor, with RMEM below the processors. (Source: Nintendo)

 

The little super-computer would be considered feature-rich today, and other than the clock speeds would be a competitive device.

Nintendo 64 features:

 

 

• 64-bit custom MIPS R4300 CPU, clock speed of 93.75 MHz
• Rambus DRAM (4 Mbytes) with a maximum bandwidth of 4,500 Mbits/s
• Sound and graphics, and pixel drawing coprocessors, clock speed of 62.5 MHz
• Resolutions range from 256 × 224 to 640 × 480. Normal resolution is 320 × 240, 24bpp
• 32-bit RGBA frame buffer, with 21-bit color video output
• Graphics processor (RPC) includes:
  • Z-buffer
  • Anti-aliasing
  • Texture mapping: tri-linear interpolated with MIP maps, environmental mapping, and perspective correction
• Size: 10.23 inch × 7.48 inch × 2.87 inch; weight: 2.42 pounds
• The system came with a multifunction 2D and 3D game controller, including digital and analog joysticks, many buttons
• MIPS and RPC processors were 0.35µ silicon manufactured by NEC for Nintendo.

The organization of the system consisted of two major chips, the main CPU and the Reality Coprocessor (RCP) designed by SGI. The following diagram shows the overall arrangement.

Nintendo 64 block diagram

 

The VR4300 main CPU was a 64-bit microprocessor that ran at 93.75 MHz with 64-bit registers, data paths, and buffers to insure high-speed data movement within the chip. The wide data paths were particularly important for operations such as bit-stream decoding and matrix manipulation, core features in video and graphics processing. The VR4300 device also supported double-precision floating-point operation for high-performance graphics. Large on-chip caches (16 Kbytes instruction and 8 Kbytes data) provided higher performance for interactive applications by reducing the need for frequent memory accesses. It was built using 0.35-micron drawn (0.27-micron effective) NEC process technology.

Within the RCP were two major sub-systems, the Reality Signal Processor (RSP) and the Reality Display Processor (RDP).

Nintendo 64 Reality Signal Processor block diagram

 

Also known as RSP, it contained:

• The Scalar Unit: A MIPS R400-based CPU which implemented a subset of the R400 instruction set.
• The Vector Unit: A co-processor that performed vector operations with thirty-two 128-bit registers. Each register was sliced in eight parts to operate eight 16-bit vectors at once (just like SIMD instructions on conventional GPUs).
• The System Control: Another co-processor that provided DMA functionality and controls the neighbor display processor module.

In order to operate the RSP, the CPU stored in RAM a series of commands called a display list along with the data that would be manipulated. The RSP would read the list and apply the required operations on it. The available features included:

• Geometry transformations.
• Clipping & Culling (removing unnecessary & unseen polygons).
• Lighting.

The RSP fed the Reality Display Processor (RDP), which is illustrated in the following block diagram.

Nintendo 64 Reality Display Processor (RDP) block diagram

 

After the RSP finished processing polygon data, it sent rasterization commands to the RDP. These commands are either sent using a dedicated bus called XBUS or through main RAM.
The RDP was just another processor with fixed functionality that included multiple engines used to apply textures over polygons and project them on a 2D bitmap. It could process either triangles or rectangles as primitives, the latter is useful for drawing sprites. The RDP’s rasterization pipeline contained the following blocks:

• A rasterizer that allocated the initial bitmap that served as a frame-buffer.
• A texture unit that applied textures to polygons using 4 KB of dedicated memory (called ‘TMEM’) allowing up to eight tiles to be used for texturing. It could also perform the following operations on them:
  • 4-to-1 bilinear filtering for smoothing out textures.
  • Perspective correction to improve the coordinate precision of the textures.
• A color combiner that mixed and interpolated multiples layers of colors (for instance, to apply shaders).
• A blender that mixed pixels against the current frame-buffer in order to apply translucency, anti-aliasing, fog, dithering, and z-buffering, the latter being critical to efficiently cull unseen polygons from the camera viewpoint (replacing software-based polygon sorting methods which could drain a lot of CPU resources).
• A memory interface used by multiple blocks to read and write the current frame-buffer in RAM and/or fill the TMEM.

The RDP provided four modes of functioning; each mode combined these blocks differently in order to optimize specific operations.
The RDP supported 16.8 million colors. The system could display resolutions from 320 × 240 up to 640 × 480 pixels. Most games that made use of the system's higher resolution 640 × 480 mode required use of the Expansion Pak RAM upgrade.

The system had several advanced, high-end graphics capabilities such as:

• Realtime anti-aliasing—removes jagged edges from the objects, creating a smooth and realistic view as the player moves through a scene. 
• Advanced texture mapping techniques—generate high-quality textures and retain the natural texture of every object in the scene, independent of how close the player is to the object. 
• Realtime depth buffering—removes hidden surfaces during the realtime rendering process of a scene, allowing game developers to efficiently create 3D environments. 
• Automatic load management—enables the objects in the scene to move smoothly and realistically, by automatically tuning the graphics processing. 

 

The console came with a new three-grip controller that allowed 360-degree precision movement. A “3D stick” enabled players to identify any angle in 360 degrees, as well as control the speed of a character’s movement. Other new additions include “C Buttons,” which changed a player’s perspective, and a “Z Trigger,” for shooting games. In addition, the controller featured a memory pack accessory which allowed players to use a special memory card to save gameplay information on their controller. This enabled players to take their gameplay data with them and play on other Nintendo 64 systems. 

Over 350,000 Nintendo 64s sold within days of its release.  

Console Wars. The console market was highly contested then (as now), and new companies were entering the market as older ones were being driven out. As a result, the suppliers started a price war that almost ruined them all.

In August 1996, Nintendo announced plans to drop the price to $199.50 just before it launched in the U.S. That was done to match the 32-bit systems from Sony and Sega head to head. Sony, in turn, reacted by reducing prices on many of its titles to $39.99. Sega said it didn’t plan to reduce its console price.

Then six months later, in March 1997, Nintendo announced a new, lower price of $149.95 for Nintendo 64. At the time, the company cited production efficiencies resulting from the planned global sales increase to 12 million units for the 12-month period starting April 1, 1997, and favorable foreign exchange rates. With Sony, Sega, and Nintendo all at below $150, it was not unreasonable to expect a sub-$100 sales bonanza for Christmas 1997.

By May 1998, there were really only two players in the console market, the Sony PlayStation and the Nintendo 64. The Saturn system could still be found, but not for long.

Then in August 1999, Nintendo did drop the price of the Nintendo 64 to under $100 as fall and holiday market heated up.

Conclusion
A lot of advanced techniques and solutions were incorporated into the Nintendo 64 which have become the basis for modern 3D gaming. The hardware itself had a list of major features:
Here are some things developers made use of with the software end that paved the way for modern game engines.

• Trilinear mipmapping, the most often touted one
• Edge-based anti-aliasing (which we have today as FXAA and MLAA)
• Basic realtime lighting (which implies the N64’s GPU is a hardware T&L GPU)
• The one thing that stands out in my mind though when it comes to the Reality Coprocessor (RCP) is that it’s probably one of the first, if not the first, GPUs that was fully programmable. The processor ran on microcode which developers could tweak to suit their needs. The problem was Nintendo didn’t release the tools on tweaking those codes until late in the N64’s life span. But once they did, a few companies (notably Rare and Factor5) pushed the system to its limits.

 

Other features that were exploited by game studios included:

• Smart use of clipping. Nintendo had at least some merit when using cartridges for their speed. This allowed sections of the game world that are not visible to not be rendered until the player gets very close to them. This YouTube video (you’ll have to skip ahead a bit) shows this off.
• Banjo Kazooie had a novel way to push out large textures for detailed environments. One of the issues was that it caused memory fragmentation, which would mean that even though technically enough memory was around, there wasn’t enough contiguous memory to store something in. So they had a realtime memory defragger run during the game.
• Texture streaming. This was done by game developer Factor5 (though I’m sure others did something similar) for Indiana Jones and the Infernal Machine. This allowed them to stream textures being rendered, thus overcoming the 4KB texture memory limit.
• Frame-buffer effects. This is used for effects like motion blur, shadow mapping, “cloaking,” and something that still amazes me: render to texture (Textures that are created and updated at run time).
• Level of detail (LoD). This is a trick where if a model is sufficiently far away, it gets swapped for a low-poly model.

 

The Nintendo 64 was truly ahead of its time, but unlike other pioneers, the company didn’t suffer for being so advanced. The Nintendo 64 did very well and drove the industry forward toward more realistic and high-performance computer graphics.

Epilog
Nintendo may have spurred the creation of the PlayStation. In the early 1990s, Nintendo partnered with Sony to develop a new CD-ROM console and attachment for the Super Nintendo system, resulting in a prototype called the Nintendo PlayStation.
However, Sony’s deal with Nintendo fell through. Sony ultimately decided to ditch Nintendo and launch the PlayStation on its own — a decision that would completely change the course of the video game industry. That ultimately led to the birth of Sony's massive PlayStation brand, and a major and long-term competitor to Nintendo.

In May 1999, Nintendo decided to use IBM’s 400 MHz, 128-bit PowerPC chip called Gekko in Nintendo’s new Dolphin game console 

Nintendo also announced that the new system would be using a new graphics chip designed by ArtX Inc. ArtX was formed in 1998 by ex-SGI/MIPS employees who developed the Nintendo 64 graphics processor. 

And then in May 2001, Nintendo announced the GameCube scheduled for launch in Japan on September 14th, in North American November 5th, and in Europe early 2002. The GameCube was featured the Flipper Chip from ATI (through ATI’s acquisition of ArtX). The integrated processor included a 2D and 3D graphics engine, a DSP for audio processing from Macronix, and all system I/O functions including CPU, system memory, joystick, optical disk, flash card, modem, and video interfaces and an on-chip high bandwidth frame buffer. IBM supplied the 485 MHz Gekko microprocessor.