Better plotting modes, text anywhere on the screen and double-height characters for Sega Master System BBC BASIC

Monday, 9th August 2021

I've made quite a few changes to the way colours and plotting modes are handled in the Sega Master System version of BBC BASIC; previously this was handled on a per-graphics-mode driver basis, but this resulted in a lot of duplicated code and some inconsistencies. The new code is a bit more straightforward but also adds quite a few new features, most obviously different plotting modes:

Screenshot showing three colours blended together with an OR plotting mode
Plotting with the OR mode

The above screenshot shows the result of filling three circles with the OR plotting mode. The red circle is colour 9, the blue one is colour 10 and the green one is colour 12. Where the colours overlap they are ORed together when drawing, so for example red OR blue is 9 OR 10 which is 11, the colour code for magenta, hence the colour between the two is magenta. Where all three colours mix, that's 9 OR 10 OR 12 which is 15, the colour code for white.

It's the logical colours (the palette indices) that are combined by these plotting operations, rather than the RGB values, it just happens that the stock palette lines up neatly with the RGB values!

Another addition I've made is support for VDU 5. In normal operation, text sent to the display is displayed at the text cursor position. In VDU 5 mode the text is instead drawn at the graphics cursor position, which allows text to be drawn anywhere on the screen, outside the confines of its usual grid. It also takes into account the graphics plotting modes mentioned above and so you can blend text with other graphics operations. Here's an example:

Screenshot showing text drawn in a wavy fashion on the screen

The somewhat blurry text has been drawn in two passes slightly offset from one another, once with colour 1 and again with colour 2, ORed together so that where the text overlaps you get colour 3, which is white. The program for this is as follows:

 10 REM Draw wavy text in a watery style.
 20 MODE 2
 30 REM Set up a suitably watery palette.
 40 COLOUR 0,0,0,64
 50 COLOUR 1,0,64,128
 60 COLOUR 2,0,128,255
 70 COLOUR 3,255,255,255
 80 REM Draw text at the graphics cursor position.
 90 VDU 5
100 REM How many lines of text will we draw?
110 RESTORE
120 L%=-1
130 REPEAT READ L$
140   L%=L%+1
150 UNTIL L$=""
160 REM Draw in two passes, slightly offset, in OR mode for a blurry effect
170 FOR C%=1 TO 2
180   GCOL 1,C%
190   RESTORE
200   Y%=(960+L%*50)/2
210   REPEAT READ L$
220     IF LEN(L$)>0 AND L$<>"-" PROCWAVES(L$, C%*4, Y%-C%*4)
230     Y%=Y%-50
240   UNTIL L$=""
250 NEXT C%
260 REM Restore normal text operation.
270 VDU 4
280 REM Wait for a key then exit.
290 REPEAT UNTIL INKEY(0)=TRUE
300 REPEAT UNTIL INKEY(0)<>TRUE
310 END
320 REM Draw some wavy text at X%,Y%
330 DEFPROCWAVES(TEXT$,X%,Y%)
340 LOCAL I%
350 X%=X%+(1280-(LEN(TEXT$))*35)/2
360 FOR I%=1 TO LEN(TEXT$)
370   MOVE X%,Y%+20*SIN(X%/80+Y%/160)
380   X%=X%+35
390   PRINT MID$(TEXT$,I%,1);
400 NEXT I%
410 ENDPROC
420 REM Text strings to display.
430 DATA "VDU 5 can be used to place text"
440 DATA "anywhere you like on the screen,"
450 DATA "not just aligned to the main"
460 DATA "text grid."
470 DATA "-"
480 DATA "This is fun, isn't it?"
490 DATA

When you're finished you can use VDU 4 to return to the normal text mode. In the video modes where I've had enough free VRAM to spare for it I've also added support for user-defined characters. You can send 23, then a character number, then eight bytes of pixel data for the 8×8 character bitmap to the VDU and then use that data for subsequent text operations (the characters from &80..&FF can be redefined). These user-defined characters can also be used as 8×8 sprites when combined with VDU 5.

I'm having quite a lot of fun writing my own test programs but it's also been useful to be able to run existing software as a test. I've tried to improve my sound handling and it now sounds a bit more faithful to the BBC Micro (though the Master System is a slower machine, so it can't quite keep up with more adventurous pieces of music).

Screenshot showing the BBC Disc System Welcome screen     Screenshot showing the 'Bones' demo
The BBC Disc System "Welcome" screen and the Bones demo

With the new graphics plotting modes and VDU 5 support the BBC Disc System "Welcome" program runs, for example, though it looks a bit rough around the edges due to mismatches in the video modes. Bones also runs, however the mismatch is even more severe here and half way through the skeleton turns red as it's running in the four-colour video mode and COLOUR 1 is mapped to red, not the white in the monochrome mode the demo is expecting. The misalignment comes from differences in the number of rows and columns of text that are available; the demo mixes graphics commands (which are mapped reasonably closely to the BBC Micro's screen) and text; if the demo expects the screen to be 40 columns wide and the best I can offer with this hardware is 32 columns, it won't line up! Of course, the BASIC program can be tweaked to get it working properly, but it's fun to see just how close it gets without any tweaking...

One recurring issue, however, is with programs that expect to be able to use Teletext features. The BBC Micro's default video mode used a Teletext character generator, which allows for bright colourful text and some limited graphics support in very little memory. I had previously mapped this mode to the TMS9918A text mode, which allows for 40 characters in 24 rows, just one row short of the BBC Micro's 40×25. It's a fast mode and is good for program editing due to the large number of text characters on screen at a time (all other modes are 32×24) however it is very limited - there's no support for colour, for example, other than the global foreground and background colours.

To provide some colour support I loaded the character set into memory twice, once normally and once inverted, so via the COLOUR statements it's possible to inject some variety into this mode. The BBC Micro's Teletext MODE 7 doesn't support the COLOUR statement, it relies on embedding special control codes in character positions on the screen that affects the rendering of the rest of the text in the line – for example, a command might change the colour, or cause the rest of the line to flash. As I can't possibly support such codes, I just ignore them, and the programs run fine but look a bit bland.

Screenshot showing double-height text being duplicated

The main exception to this, I've found, is the double-height text mode. You can switch this on within a line by storing the value 141 in a character cell and then off again with 140 afterwards. This will only display the top halves of the characters, so to display the bottom halves you need to output the same content on the line below. A lot of BBC Micro programs seems to take advantage of this feature, so most programs I'd tried ended up having doubled-up text like you can see in the screenshot above.

The character set I'm using is only 96 characters long (32-127) but I had avoided investigating this as I assumed I'd need to triple the number of characters - the regular-height ones, plus the top and bottom halves of the doubled characters. 96×3=288 which is more than the 256 characters we have available, so I didn't think this would fit.

However, I did think that maybe some characters shared parts - for example, the top halves of the colon and semicolon are the same, and the bottom halves of the semicolon and comma are the same, so I wrote a quick program to count how many unique patterns would be needed and found that it would only take 225 patterns in total, well within our 256 pattern budget!

I set up a new video mode based on the original TMS9918 text mode. This generates these 129 additional patterns for the double-height characters when initialised, and also checks the nametable when outputting characters to see if there's a "double height" control character in the same line to determine whether it should pick one of the stretched characters (picking a top or bottom half depending on whether there are an even or odd number of lines above it that also contain the double height control character).

Screenshot showing double-height text being stretched

This new mode does lose the inverted text, but I think the double-height text will be more useful in practice – or at least is better at making programs look less broken!

A four colour any-pixel-addressable video mode for the Master System

Wednesday, 4th August 2021

The video display processor in the Sega Master System is derived from the TMS9918A and provides a number of different screen modes. These modes are mostly character-based and generate the picture that is displayed on your TV based on two blocks of data in video RAM: the name table, which specifies which character appears in each cell on the screen and the pattern generator which contains the pixel data for each character.

Example of TMS9918A Text output
TMS9918A "Text" mode

The "Text" mode is probably the most straightforward example. Only two colours can be displayed on the screen; a foreground and a background colour, both selected from a fixed palette of 15 colours. The name table is 40 cells wide and 24 tall. Each cell, or text position, is a single byte that refers to one of 256 patterns in the pattern generator, so the complete name table is 960 bytes long. Each character is displayed as 6×8 pixels but the data in the pattern generator is eight pixels wide, and each row is stored as a single byte with a set bit selecting the foreground colour and a cleared bit selecting the background colour (the two least-significant bits are ignored). Each character pattern therefore takes up 8 bytes of VRAM, or 2,048 bytes (2KB) for a complete set of 256.

The "Graphics I" mode changes the character size to be the full 8×8 pixels at the expense of the number of characters that can be displayed per line; this is reduced from 40 to 32, so the name table is now 32×24 and 768 bytes long. The pattern generator is the same size as before and you can still only display 256 unique patterns on-screen at a time, but there is at least some provision for colour. The top five bits of the character/pattern number are used as an index into a 32-byte colour table, where each colour table entry contains two four-bit values corresponding to the foreground and background colours for the pattern. This means that patterns 0-7 share the same two colours, patterns 8-15 have the next set of two colours, 16-23 after that and so on; 256 patterns, in groups of eight with a different pair of colours assigned to each of those 32 groups.

Example of TMS9918A Graphics II output
TMS9918A "Graphics II" mode

Fortunately, the TMS9918A is a revised version of the original TMS9918 which adds the much more useful "Graphics II" mode. The name table is the same as before, 32×24 cells of one byte each, and the patterns are still 8×8 pixel bitmaps. However the pattern generator table is now three times the size, storing 768 unique 8×8 patterns in 6KB. To be able to address all 768 patterns when you can only store values from 0-255 in the name table the screen is divided into three sections; the top third references the first 2KB of the pattern generator (patterns 0-255), the middle third references the second 2KB of the pattern generator (patterns 256-511) and the bottom third references the last 2KB of the pattern generator (patterns 512-767). This allows every single character position in the nametable to reference a unique pattern, and therefore have a bitmap that covers the entire screen area.

The colour table has also been significantly improved. Instead of one pair of colours for each group of eight patterns, each pattern now gets its own table of colour pairs, one per row. This doesn't allow every pixel on the screen to have its own colour, but you can at least give each 8×1 region its own pair of colours. This does lead to attribute clash when drawing; you can see this in the screenshot of the cones above, where the colour from some lines bleeds into the colour set for nearby lines if they happen to pass through the same 8×1 pixel block.

Very few Master System games use the TMS9918A video modes (F16 Fighting Falcon seems to be about the only concrete example), instead using a new video mode added specifically for the Master System. This has a number of nice new features (such as hardware scrolling and two user-definable 16-colour palettes instead of a fixed 15-colour one), but it's the name table and pattern generator that's of main interest. Instead of having a separate bitmap for the pattern and its colour table, each pattern is now a 32 byte object where each 8 pixel row is encoded as four bytes, one byte for each bit of the four-bit colour palette index. That is, if you wanted to store the top row of a pattern where the leftmost pixel was set to palette index 1, the rightmost pixel was set to palette index 15 and everything else was left as 0 it would be stored like this:

.db %10000001
.db %00000001
.db %00000001
.db %00000001

The second row would then follow with another four bytes, all the way down to the bottom row for the complete 32 byte pattern. This scheme allows you to set each pixel in every pattern to any of the sixteen colours per palette, so if you could fill the screen with such patterns then it would be possible to set any pixel on the screen to any of sixteen colours, solving the attribute clash problem. Unfortunately, to do so would take 768 patterns (32×24), and if each pattern is 32 bytes then that would take up 24KB of video RAM. The Master System only has 16KB of video RAM, which would limit you to 512 patterns, so it's not possible to fill the screen with unique patterns. In practice you can't even have 512 patterns, as you still need to store the name table in video RAM, and as each name table entry has been inflated to two bytes (which does at least let you index pattern numbers above 255 without splitting the screen into thirds as per the Graphics II scheme) that leaves even less room for patterns.

In practice you are limited to 448 patterns, which is not enough to cover the screen. As you will generally start from a blank screen and then add to it, the name table could be filled with blanks by default and then as you draw over the screen of it new patterns could be allocated as required. This is how the graphics currently work in the 16-colour Master System video mode, and it allows for colourful drawings with no colour clash as long as you don't fill too much of the screen.

Example of 16-colour Master System output
Cautious use of the 16-colour Master System mode

That's the mode the sphere above is rendered in; you can see there's fine detail at its North pole without becoming subject to colour clash and it doesn't fill so much of the screen that we ran out of unique patterns to allocate. As it would be very bad to run out of patterns when attempting to draw text the character set (the 96 characters from 32-127) are permanently allocated as unique patterns which further reduces us to 352 tiles for graphics output.

It's possible to constrain the graphics viewport via VDU 24 to a smaller region which would ensure you never needed more than 352 tiles, as long as your new viewport was properly aligned to the 8×8 grid. For example, 22×16=352 which means that you could have a 176×128 pixel graphics window that you could draw to without risking running out of patterns.

Screenshot of colour clash in Graphics II     Screenshot of running out of patterns in Master System mode
Colour clash in Graphics II (left) versus running out of tiles in Master System Mode (right)

The two screenshots above show the trade-off between colour clash in Graphics II and corruption due to running out of patterns in the 16-colour Master System graphics mode, both caused by running this program:

 10 INPUT "Mode",M%
 20 MODE M%
 30 FOR I%=0 TO 47
 40   GCOL 0,I%
 50   X%=(I%/47)*1279
 60   Y%=(I%/47)*959
 70   MOVE X%,0
 80   DRAW 1279,Y%
 90   DRAW 1279-X%,959
100   DRAW 0,959-Y%
110   DRAW X%,0
120 NEXT I%
130 REPEAT:UNTIL INKEY(0)=-1
140 REPEAT:UNTIL INKEY(0)<>-1

Having to worry about your pattern budget isn't really in the spirit of fun and experimentation. If we're up against hardware and memory limitations I'd rather have a slightly less capable video mode that does at least generate correct output however you use it.

It is possible to completely avoid colour clash if you use the Graphics II mode and simply never use any colour commands, but only having two colours is a bit drab. With a bit of tinkering I came up with another graphics mode that is still using the Master System mode, but reduces the number of colours down to 4. Quite fittingly, a lot of BBC Micro screen modes only supported four physical colours, so I set this new mode to use the same black/red/yellow/white palette by default:

Example of 4-colour Master System output
Four colours on-screen at once, no clash and no corruption

In fact, the palette is the whole key to this solution. You may recall that the Master System mode supports two 16-colour palettes. Usually one is assigned as the "background" palette and the other as the "sprite" palette, however each entry in the name table contains a flag that lets you specify which palette the selected pattern should be shown with. In this new video mode, the top half of the screen is assigned to patterns 0-383 using the "background" palette, and the bottom half of the screen is assigned to the same set of patterns 0-383 but to use the "sprite" palette. The palette is then filled with four colours, like this:

Illustration of the reduced colour palette
Using all 32 palette entries to display four colours on screen simultaneously

The top row is the "background" palette and shows the four desired colours repeating in order four times. The bottom row is the "sprite" palette, and shows that each of the four colours is stretched to fill four consecutive palette entries. This use of repetition effectively turns two bits in each pattern's bitplanes into "don't care" values, depending on which palette is selected.

For example, assume we're in a pattern assigned to the "background" (top) palette and we want to display red. This is colour %01, but as the red entry is repeated we could use use any of colours %0001, %0101, %1001 or %1101 and they'll all come out red – we don't care about the top two bits.

Alternatively, we could be in a pattern assigned to the "sprite" (bottom) palette and want to display yellow. This is colour %10, but it fills the entire palette block from %1000 to %1011, so again we don't care what the lower two bits are, just what the top two bits are.

Using this somewhat redundant palette, we can pack two different four-colour patterns for different parts of the screen into a single sixteen-colour pattern. It's a shame that a halving in bit count ends up quartering the number of colours but I think it's an acceptable solution considering the hardware limitations.

Better drawing, editing, memory and emulation for BBC BASIC on the Sega Master System

Tuesday, 3rd August 2021

I have continued to work on the Sega Master System version of BBC BASIC, and it's feeling much more like a practical version of BASIC than something that was just holding together to run one specific program!

One of the key improvements is to standardise the handling of the different graphics modes. Previously I was using a coordinate system where (0,0) is in the top left corner and all drawing operations were carried out using physical device coordinates (so the screen was treated as being 256 pixels wide and 192 pixels tall). I have now moved the origin (0,0) into the bottom left corner with the Y axis pointing up and scale down all coordinates by 5, effectively making the screen 1280 logical units wide and 960 units tall. This isn't 100% compliant, as the BBC Micro and other versions of BASIC treat the screen as being 1024 units tall, but dividing by 5⅓ is considerably trickier and it would result in the graphics being squished further vertically, so I think using a logical resolution of 1280×960 is an acceptable compromise. I've added some VDU statements to allow you to move the graphics origin as well as define custom graphics and text viewports, so you can control where on the screen graphics and text appear and how they are clipped/scroll.

Screenshot of the Mandelbaum program output
The output of the "Mandelbaum" program following these changes

I have also changed the default palette to more closely match the one used by the BBC Micro and other versions of BBC BASIC. This isn't too difficult when using the Master System's "Mode 4" as that has a user-definable palette, but the legacy TMS9918A modes have a fixed palette so I've tried to match the default palette as sensibly as I can to the TMS9918A palette. It's possible to change the logical to physical palette mappings under Master System "Mode 4" via a VDU command which writes directly to the CRAM (you can either remap one of the 16 logical entries to one of the stock 16 colours, or supply an RGB value directy to select additional colours) which allows for neat tricks like palette cycling, but the TMS9918A modes currently only let you change the current text/drawing colour, not amend the palette as that's fixed in hardware.

I've also added filled/outlined circle and axis-aligned ellipse PLOT commands using some code written by Darren "qarnos" Cubitt which I originally used in the TI-83 Plus version of BBC BASIC. This code is very capable and fully accepted 16-bit coordinates for its inputs, however it was also originally designed to output to a 96×64 pixel screen so the final plotting was done with 8-bit coordinates ranging from -128..127. Fortunately the Master System's screen also fits in 8-bit coordinates at 256 pixels wide but that's not quite enough information as you also need to be able to tell if a particular point is off-screen (less than zero or greater than 255); simply clipping it against those boundaries will result in a vertical smear on the left or right edge of the screen when drawing outlines. Fortunately I was able to figure out how to modify his code to add some extra clipping status flags to ensure that ellipses were clipped and displayed correctly on any part of the screen.

Screenshot showing filled circles     Screenshot showing filled cones
Filled circles and filled/outlined ellipses make these drawings possible

The only graphics operation exposed by the mode-specific drivers before was a simple "set pixel" routine. This is fine for lines but quite slow for filling shapes so graphics mode drivers can now supply a "fill horizontal span" routine for faster shape-filling. If the routine is left unimplemented a stub routine is provided that fills the span using the "set pixel" routine.

I also added a rectangle-filling PLOT command, which is perhaps not the most exciting-sounding graphics operation but it is used to clear the screen so it is at least useful. More interesting is a triangle-filling routine, something I've never enjoyed writing!

Usually I get very bogged down in the idea that the pixels affected when you fill a triangle should exactly fit within the pixels that are outlined by drawing lines between each of the triangle's points, no more and no less. This can be a bit difficult when the triangle is filled by tracing its edges from top to bottom and drawing horizontal spans between them. If the edge is "steep" (its overall height is greater than or equal to its width) then this isn't too bad, as there's only one X coordinate for each Y coordinate where a pixel would have been plotted. However, when the edge is "shallow" (its overall width is greater than its width) there are going to be certain Y coordinates where the line drawn would have had multiple pixels plotted. In that case, where is the boundary of the horizontal span?

The cop-out answer I've used in the past has been to set up three buffers the total height of the screen and to "draw" the three lines first using the same line-drawing algorithm as the line PLOTting command, keeping track of the minimum and maximum X coordinate for each Y coordinate. When it's time to fill the triangle the minimum and maximum X coordinate for each edge can be determined based on the current Y coordinate and a span drawn between them for perfect triangles. On the TI-83 Plus this takes up four bytes per line (minimum and maximum 16-bit values) for a 64 pixel tall screen, with three buffers for the three lines that comes to 4×64×3=768 bytes, pretty bad. On the Sega Master System that would be 4×192×3=2304 bytes, totally unacceptable on a machine with only 8KB total work RAM!

Screenshot of a 3D sphere rendered by BBC BASIC
Each face of this 3D sphere is filled by two triangles.

I've instead simply done my best to interpolate from one X coordinate to the other when working my way down the triangle and filling scanlines, doing a bit of extra pre-incrementing and fudging of initial error values depending on whether it's the top half or bottom half of the triangle. My test was to draw the outline of a triangle in bright white and then to fill a dark blue triangle over the top, if any bright white pixels were visible around the outside this indicated a gap. I mostly got it filled, but I then tried my test program on a BBC Micro emulator and found the BBC Micro exhibited similar gaps so I don't think I'm doing too badly! The above screenshot of the 3D sphere was rendered using this triangle-filling code.

Screenshot showing buggy BASIC code with a PRONT instead of a PRINT statement

I've also been working on improving the line editor. This is called by BASIC when it's asking for a line of text input from you. Previously I'd only implemented adding characters to the end of the line and pressing backspace to remove characters from the end of the line; if you'd typed in a long piece of code and made a mistake at the start you'd need to backspace all the way to the mistake, correct it, then re-type the rest of the line. Now you can use the cursor keys (or home/end) to move around within the line, insert or overwrite new characters (toggled by pressing the insert key on the keyboard) at the cursor position or backspace/delete characters mid-line as required. It sounds like a small thing but it was quite a lot of code to get right and makes a big difference for usability!

Another feature I've added to aid modifying lines of code is the *EDIT command. This takes a line number as an argument and then brings up the program line you've requested in the line editor, ready for editing. The way this works is a bit sneaky, as it's not natively implemented by BBC BASIC! The trick that makes it work is the ability to override two routines, OSLINE (which BASIC calls when it wants to display the line editor) and OSWRCH (which BASIC calls when it wants to output a character).

When a valid *EDIT <line> command is entered, OSLINE is overridden with the first custom routine. This routine doesn't ask for a line of input, but instead copies L.<line> to the input buffer, overrides OSWRCH with a routine that captures data to RAM, overrides OSLINE with a second custom routine, then returns. BASIC therefore thinks you've typed in a LIST statement so it dutifully starts outputting the line via OSWRCH, but this has been overridden to write the characters to RAM rather than the screen. When it's done this BASIC then calls OSLINE again for the next line of input, which brings up the second custom OSLINE handler. This pulls the data from RAM previously captured by the OSWRCH handler, copies it to the OSLINE buffer, and dispays it on-screen as if you'd just typed it in. It then restores the original OSLINE and OSWRCH handlers before jumping back into the stock OSLINE routine, so you can continue editing the line that you'd requested via *EDIT.

A modified Monopoly cartridge, used to run BBC BASIC with an 8KB RAM expansion.

All of this hopefully makes entering programs via the keyboard less cumbersome. Of course, not having to type in programs in full every time you wanted to run them would be even better, and an attempt to give BASIC more memory provides another way to load programs in a somewhat roundabout manner.

The photograph above shows the modified Monopoly cartridge that I'm now using to test BBC BASIC on real hardware instead of the modified After Burner cartridge I was using before. The advantage of Monopoly is that it has an additional 8KB RAM on board, which is used to save games in progress. Sega's cartridge mapper allows for on-cartridge RAM to be mapped into the address range $8000..$BFFF, immediately below the main work RAM which is at $C000..$DFFF. If present, then, BASIC's memory range (which runs from PAGE to HIMEM) can be extended by enabling the save RAM and moving PAGE down.

$8000..$BFFF is a 16KB range, though, and I mentioned that Monopoly has an 8KB RAM. This is true, and what it means is that the 8KB cartridge RAM is accessible from $8000..$9FFF but is then repeated ("mirrored") from $A000..$BFFF. The cartridge RAM detection therefore has to check two things: firstly that there is RAM in the first place (which can be verified by modifying memory at $8000 and seeing if those values stick) and how big it is (which can be checked by writing to $8000 and seeing if that has also modified the data at $8000+<RAM size>). Once presence of any RAM has been determined during startup, RAM mirroring is checked in 1KB, 2KB, 4KB and 8KB offsets. If any RAM mirroring is detected, then it's assumed the RAM is the size that was being checked at the time, however if not it's assumed that the RAM is the full 16KB. At this point, PAGE (which is $C000 in a stock machine with no cartridge RAM) is moved backwards by the size of the detected cartridge RAM, e.g. to $A000 for an 8KB RAM and $8000 for a 16KB RAM. This results in 16KB or 24KB total available to BBC BASIC, a considerable upgrade from the plain 8KB work RAM!

An added bonus of this is that when you type in programs they grow upwards in memory from PAGE. As PAGE now starts within your cartridge memory, and that cartridge memory retains its contents courtesy of a backup battery, it means that if your entered program is smaller than the size of your cartridge memory you can restore it when you switch the console back on by typing OLD.

That's very well for life on real hardware, but I continue to do most of my development testing in an emulator. I'm having to use my own emulator as there aren't any other Master System emulators that also include a PS/2 keyboard emulator, but I did end up running into a very weird bug. Certain trigonometric functions in BBC BASIC were producing very wrong values, resulting in very odd-looking output.

Screenshot showing wonkily-rendered cubeScreenshot showing wonkily-rendered Mandelbaum setScreenshot showing wonkily-plotted sphere
These shapes are on the wonk

Even though the Z80 emulator at the heart of the program passed ZEXDOC (an instruction tester that checks documented functionality) I remembered that it had failed some aspect of ZEXALL which checks the undocumented flags too. I re-ran ZEXALL and found the problem was with my implementation of the bit instruction, so I worked on fixing that including emulation of the Z80's internal temporary memptr/WZ register to ensure that bit n,(hl) set bits 3 and 5 of the flag register appropriately. ZEXALL now passes, but unsurprisingly it didn't fix my problem (as I didn't really think that BBC BASIC would rely on undocumented functionality!)

I ended up isolating certain exact values that when plugged into SIN() or COS() would produce incorrect results. I then dug out my old CP/M emulator and tried plugging those values into the generic CP/M version of BBC BASIC (which has none of my code in it!) and that produced the same, incorrect, results confirming that the issue was definitely in my Z80 emulation and not in some flaw of the BBC BASIC port.

After tracing through the code and dumping out debug values at certain points and seeing where it differed to a known good reference emulator I found that the fault occurred in FMUL, and from there I found that at some point the carry flag (which is either rotated or added to registers with RR or ADC instructions) was being lost. RR looked fine but digging into my implementation for the 16-bit ADC I found the culprit: the code is the same for ADD and ADC, but in ADC if the carry flag is set then the second operand is incremented by one first. This produces the correct numeric answer, but if the second operand was $FFFF on entry to the function then adding a carry of 1 to it would cause it to overflow back to 0. As this happens before any of the rest of the calculations are made it means that the final value of the carry flag was calculated based on op1+0 instead of op1+$10000, hence the loss of the carry flag.

Fortunately, fixing this fixed the wonky output demonstrated above and I feel slightly more confident in my emulation! Now I can continue working on BBC BASIC, I've had an idea for a screen mode that has a reduced number of colours but will hopefully let you draw anywhere on the screen without running out of tiles causing odd corruption (as happens in the usual 16-colour mode 4) and without attribute clash (as happens in the TMS9918A Graphics II mode)...

Running BBC BASIC on the Sega Master System

Thursday, 22nd July 2021

I've recently been spending some time finding a way to run BBC BASIC on the Sega Master System, inspired by BASIC Month 6: The Mandelbaum Set on the RetroBattlestations Reddit community.

Photograph of the final setup with a Sega Master System running BBC BASIC and a Z88 computer acting as file store.

When this month's program was first announced I tried running it on my only unarguably retrobattlestation, my Cambridge Z88, but the screen's low (vertical) resolution didn't do the program much justice.

I thought this gave me two options:

  1. Control some external piece of retro tech to produce higher-resolution output (e.g. a printer or a plotter.
  2. Port a BASIC interpreter to another retro system with a higher resolution display and run the program on that.

Unfortunately, I don't own any old printers or plotters (or a Logo-like turtle!) so option 2 seemed my best option. I had some experience adapting Richard Russell's BBC BASIC (Z80) to run on the TI-83 Plus calculator so I thought I should pick the Sega Master System as that also has a Z80 CPU in it. Here are some rough specs:

  • 3.58(ish)MHz Z80 CPU.
  • 8KB work RAM.
  • TMS9918A-derived VDP for video with 16KB dedicated VRAM accessed via I/O port.
  • SN76489-derived PSG for sound.
  • Two controller ports with six input pins and two pins that could be configured as inputs or outputs.
  • Software loaded from ROM cartridge or card slot with a small BIOS ROM that detects whether a cartridge or card is inserted (and if not, runs its own built-in game).

There is some precedence to this endeavour with the computer version of Sega's SG-1000, the SC-3000, which came with a keyboard and had BASIC ROM cartridges.

An RS-232 serial adaptor and PS/2 keyboard adaptor for the Master System.

I wanted to try to keep this project as retro as possible, so no modern microcontrollers as I've been accused of cheating by using them in the past. I did have to make a couple of adaptors to allow me to plug in a keyboard and to give the Master System a serial port to load or save programs over – more about these later!

Loading BASIC onto the Master System

The first problem was getting BASIC onto the Master System at all. As my Master System doesn't have BASIC in ROM (it comes with Hang-On, which is perhaps more fun but less likely to handle the Mandelbaum set) I'd need to load the program onto a cartridge or card. Master System cartridges usually contain a mapper circuit to handle bank switching in the lower 48KB of the Z80's address space (the upper 16KB contains the 8KB work RAM, appearing twice) which is normally integrated directly into the ROM chip for the game, however there are a handful of games that have separate mapper chips and ROM chips and the ROM chips that Sega used have a pinout that is extremely close to that used by common EEPROMs (e.g. 29F010,
49F040) with only a couple of pins needing to be swapped around. After Burner is one such cartridge, so I've modified an old copy to let me plug in flash memory chips that I can program with the version of BASIC I'm working on. A switch at the top of the cartridge lets me switch back to the original pin configuration if I want to play the original copy of After Burner.

Typing commands into BASIC

With a flashable cartridge to hand I was able to assemble a version of Richard Russell's BBC BASIC (Z80) with some simple code stubs in place to direct text output to the screen. Output is useful but only half the story, we still need to be send commands to BASIC!

The Master System doesn't have its own keyboard, so I'd need to find some way to interface one to it. I have previously used PS/2 keyboards in a number of projects, as they are pretty simple to deal with. Electrically, they use two open collector I/O lines for bidirectional data transfer (one as clock, one as data), and fortunately each controller port of the Master System has two pins that can be configured as outputs which can therefore be used to interface with a keyboard (either left as inputs and pulled weakly high in their idle state, or driven low as outputs for their active state).

Showing the internal wiring of the PS/2 keyboard adaptor and controller passthrough.

It acts as a pass-through cable so you can still have a regular controller plugged in when using the keyboard.

The two pins on the Master System control port that can act as outputs are TH and TR; TH is normally used by light guns to latch the horizontal counter in the video chip so is unused with normal controllers so it's no great loss here, but TR is the right action button (marked "2") so by using this pass-through adaptor you do unfortunately lose one of the controller buttons. However, you do gain a hundred or so keyboard keys, so I don't think it's too bad a compromise...

For the software side of things I adapted my Emerson AT device library, previously written for the TI-83 Plus calculators, to the Sega Master System hardware. This library handles the low-level AT device protocol and also translates the raw keyboard scancode values to the corresponding characters.

Saving and loading programs

At this point I was able to type in BASIC programs and run them on the Master System, which was pretty neat! However, I was still working on adding new features (e.g. drawing commands for graphics) and having to type in the entire Mandelbaum program after every change was going to get pretty exhausting. I could bake it into the ROM but that seemed like cheating, so I thought I should try to find a way to load the file from an external source. A floppy disk or tape cassette would seem authentically retro but adding a floppy drive controller to the Master System would be a fairly complicated task and I don't have a suitable data cassette recorder to even attempt loading from tape so I thought some sort of file store accessible over a serial port would be a good option.

The Z88 has a serial port and can act as a remotely-controlled file store when running the PC Link software (with the protocol documented here. This seemed like a good choice, if not for the fact that the Sega Master System doesn't have a serial port of its own. To get around this, I added one, using a MAX232 chip to adapt the Master System's 5V logic levels to RS-232 compatible ones so I could plug in a null modem cable from the Z88 or my PC without accidentally frying the Master System with -12V.

Showing the insides of the RS-232 serial port adaptor.

I wrote some code that bit-bangs the serial data over the controller port lines using the timing loop code I wrote for a previous BASIC Month (Crisps Tunes) to support rates between 19200 and 300 baud. 19200 baud is somewhat unreliable but that's OK because the Z88's 19200 baud is unreliable too, so the default 9600 baud speed does a good job. RTS/CTS handshaking has to be implemented, as there is no hardware serial support on the Master System and it needs to be actively polling the port to receive any data. In doing so I noticed one awkward fact about my PC's serial port - if you de-assert RTS it will continue sending data until its buffer is empty, presumably only checking the RTS line when it's about to top up the buffer. In practice this means that even if I change RTS virtually as soon as the start bit for the first byte is received, the PC will continue to send up to 256 bytes before stopping. To get around this I added a serial receive buffer that immediately checks for the next byte even after asserting RTS, and this seems to have done the trick.

The protocol used by PC Link requires acknowledgement after every single byte so is very slow but at least it's reliable. I plumbed the PC Link code into BASIC's LOAD and SAVE which makes loading and saving programs as transparent and easy as if you had a floppy disk in the system instead!

Pressed for size

The Master System has 8KB of RAM. Of this, 16 bytes are mapped to special hardware functions and BBC BASIC reserves 768 bytes for itself, so we're already down to 7,408 bytes. I initially reserved 256 bytes for my own needs (display settings, VDU command buffer, serial port status, keyboard status etc) bringing it down to 7,152 bytes. The 16KB of display memory is not directly accessible to the CPU, so it can't be used for additional work RAM, and having to access it indirectly via I/O ports is very slow but I can't afford to mirror parts of it in RAM for speed.

Initially the Mandelbaum program ran well enough by stripping out comments, but I then added sound support (with eight 13-byte envelope definitions, and four channels with their own state, copy of their active envelope and a command queue for 32 bytes per channel, adding around another 140 bytes of memory usage) and the program stopped running with a "No room" error during execution (performing a square root operation, of all things!) so I guess the tolerances were very tight. I went and combine more lines of code into single lines and replaced two-letter variable names with single-letter ones (no, really) and it was able to run again but I don't think 8KB is a particularly comfortable amount of RAM for a BASIC computer!

Further design considerations

The version of the BASIC host interface used here is very much a work-in-progress. I would need to extend it considerably to be useful, including:

  • Fuller support of different VDU commands, e.g. redefining character shapes, changing the text and graphics viewports, better colour handling (differentiating between logical palettes and physical palettes).
  • Better support of other modes (so far only TMS9918A "Text" and "Graphics II" modes are used, there is a Master System-specific "mode 4" but that lacks graphics support and only takes advantage of hardware scrolling for extremely fast program LISTing).
  • Implementation of more graphics commands - so far only PLOT 4 (MOVE) and PLOT 5 (DRAW) are implemented. They also use a non-standard coordinate system with (0,0) in the top left of the screen and a screen resolution of 256x192, whereas for standardisation with other BBC BASIC implementations this should move (0,0) to the bottom left and use a logical resolution of 1280x1024 or similar.
  • Support of other file systems rather than rely on a Z88 running PC Link, e.g. using I²C EEPROMs (as they use two open collector pins, so could be plugged into a controller port via a passive adaptor).
  • Support of extra RAM, either integrated directly on a custom cartridge or using the battery-backed SRAM supported by some other cartridge types.
  • Native controller support via BASIC ADVAL command (at the moment you can access the controller ports directly with GET(&DC))

This is before even getting into adding anything machine-specific (e.g. to take advantage of scrolling tilemaps or hardware sprites), but getting the BASICs down (and consistent!) is a very important starting point. Consistency is useful, after all I was able to get the original program running under BBC BASIC with very minimal changes to it.

But, in the short term, I have at least succeeded in what I set out to do, which was to run the Mandelbaum program on a retro "computer" wholly unsuited to it!

The above video provides another demonstration of the setup – playing the Cold Tea music demo, albeit with a heavily stripped-down version of the visuals as the Master System doesn't have anything that can match the capabilities of the BBC Micro's teletext mode 7.

Migrating SVN repositories to GitHub, maintaining history even on addition/removal of a trunk folder

Friday, 16th July 2021

Years ago I had a few projects hosted in Google Code's SVN repositories, but Google closed Code down in 2015 and though I backed the files up locally via svnsync to carry on working on the projects the code was never made publicly available again.

I decided it would be a good idea to move these repositories to GitHub, but quickly ran into the problem that during the initial migration from my local repository to Google Code's repository I had been forced to move all of the code into a trunk folder (I do not generally use SVN's branching or tagging features) and attempting to synchronise from SVN to Git lost all revision history prior to that change.

After scouring the Internet and many dead ends I came up with the below solution which has worked for me. As far as I can tell the Git tools used are designed for regular synchronisation of repositories and not a one-shot bulk migration. There are separate tools that claim to do a better job with less effort but getting them up and running on my Windows system probably takes longer than figuring out how to do it with Git's own tools...

First of all, you'll need to create a user list that maps your SVN user name(s) (left) to your name and email address (right) used on GitHub, like this:

Ben = Ben Ryves <benryves@benryves.com>
Benjamin = Ben Ryves <benryves@benryves.com>
benryves = Ben Ryves <benryves@benryves.com>
benryves@benryves.com = Ben Ryves <benryves@benryves.com>

Save this list in a text file in your current directory (e.g. users.txt). Next you'll need to need to find the revision number where the move to the trunk folder happened, e.g. by checking the SVN logs. For the sake of this example, let's pretend it was in revision 100. You can now use git svn clone to create a temp copy of the SVN repository. To continue the example, suppose the SVN repository was backed up to D:\SVN\Google\brass-assembler, then the command would look like this:
git svn clone --no-metadata --authors-file=users.txt -r 1:99 file:///D/SVN/Google/brass-assembler temp

There are some notable differences from the usual here, which I'll try to explain:
  • There is no --stdlayout parameter. This is because this makes the assumption that the repository follows the conventional trunk/tag/branch folder used in some SVN repositories, but this repository doesn't make use of such a structure (at least until revision 100!)
  • The -r 1:99 argument limits the operation to be between the revisions 1 and 99. This is when all the files were in the root of the repository, before they were moved in revision 100.
  • The SVN path looks like it's missing a colon, but this is intentional – at least on Windows the Git tools fail if the path contains a colon like it would when using the SVN tools for local paths, so remove that colon.

This may take a while to get started but eventually it should start synchronising the SVN repository to the new Git one in the temp folder up to (and including) the last revision where everything was in the root (99).

Once that has happened, you can perform the trick that makes this work. Open the file temp\.git\config in a text editor and in the [svn-remote "svn"] section you should find the line fetch = :refs/remotes/git-svn. As revisions after this point were moved into a trunk directory, we need to change this to fetch from trunk instead, so change this line to fetch = trunk:refs/remotes/git-svn:

[svn-remote "svn"]
	noMetadata = 1
	url = file:///D/SVN/Google/brass-assembler
	fetch = trunk:refs/remotes/git-svn
[svn]
	authorsfile = C:/Users/Ben/users.txt

Save the file, and then change your current working directory to your temporary repository, fetch the revisions from after the one that moved everything to trunk up to HEAD, then merge the changes (without the merge you'll only see up to the previously-cloned revision 99):
cd temp
git svn fetch -r 101:HEAD
git merge remotes/git-svn

At this point you can clone the temporary repository into a final one, so go up a level, clone the temp repository and delete it:
cd ..
git clone temp Brass3
rmdir /s /q temp

Finally, you can push this local repository to GitHub. When you add a new repository GitHub will provide a clone URL (in the form https://github.com/<username>/<reponame>.git) so change the remote origin for the local repository that was just created and push your changes:
cd Brass3
git remote rm origin
git remote add origin https://github.com/benryves/Brass3.git
git push origin master

Once that has completed you should be able to view your code on the GitHub site with its full history. The last thing I do is to delete the local Git repository and clone again from the remote one on GitHub using the GitHub Desktop program as I am more comfortable using a GUI tool to keep an eye on changes (and I find I'm less likely to mess something up by accidentally mistyping something!)

In case you couldn't tell from the above examples I've also been looking at the source code for my old Brass 3 assembler project. I've been working on a little Z80 assembly project: running BBC BASIC on the Sega Master System, which has involved a lot of my old projects – Brass 3 to assemble the code, Cogwheel to test it, Emerson to handle PS/2 keyboard input and of course my experiences with running BBC BASIC on the TI-83 Plus calculator.

One thing I realised during all this was that Brass 3 had some problems running on 64-bit versions of Windows – the help application is completely non-functional, for example, and crashes silently to desktop. I dug into the code to fix it, only to find out that I'd already done so two years ago, and even got as far as rebuilding the installer package but then just forgot to upload it to the Internet. So, I'm very sorry for the delay, but I have now uploaded "Beta 14" to the Brass 3 page.

Page 2 of 49 1 2 3 4 5 649

Older postsNewer postsLatest posts RSSSearchBrowse by dateIndexTags