8 Bit binary relocation format - working draft

V2 as of 01 apr 2005

(c) André Fachat (afachat@gmx.de)

Foreword

This file format stems from the previously developed o65 file format, which has proven to be valuable for a lot of purposes in the 6502 world. However, there have been requests to include additional features into o65 to support other CPUs and larger systems. The beauty of o65 is its simplicity, and implementing more features in a compatible way would lead to a more complicated format, which would loose this beauty. For this reason, I have decided to define a new file format, o65-v2 that incorporates more features for an extended set of requirements. Nevertheless this format is based on o65 (in its version 1.3 and tries to keep the beauty of it - although in an incompatible way.

Here are the changes to the o65 v1.3 version.

1. A New header word for CPU type, removing the old CPU2 bits from the mode header.
2. A New header word for OS type.
3. A New header word for chain section identification.
4. A New header mode bit for byte order.
5. New relocation table entry types for 32 bit addresses.

1) Preface

With some new 8 bit operating systems comes the need for a new binary format. In multitasking operating systems like Lunix, SMOS, or OS/A65, a binary file cannot be loaded to a fixed location that might already be used by another program. Therefore it must be possible to relocate the program to an arbitrary address at least at load time. In addition to that, more specific information might be stored in a binary executable file, like interrupt vectors for example.

This text gives a good solution to this problem for the 6502 CPU and an assembler source format to use this format in a general manner. The file format can even be used as an object file format, i.e. a format a linker can use as an input file.

2) binary format

2.1) General

The file differs from simple file formats, in that a lot more information is stored in the file. First the data is structured in separate segments to allow different handling of text (program code), data (like tables) and bss (uninitialized data).

Also tables are included to allow late binding, i.e. linking the file with other files at load time, and relocation, i.e. executing the file at different addresses in the CPU's address space.

2.2) Segments

As already used in other formats, the assembler uses three different segment types, i.e. text (the actual program code, read only and executable), data (initialized variables, read write and executable), Ed. Note: for self modifying code? and bss (uninitialized variables, read write and not executable). To have these different segments seems to be 'overdesigned', but they actually make memory handling easier in more complex operating systems or systems with virtual addresses (OS/A65, for example).

The text segment is defined to be read-only memory. This doesn't allow self-modifying code in this segment, but allows memory sharing in virtual memory architectures. The data segment actually is like the text segment, only it is allocated writable. This segment might not be shared between different processes. The contents of these two segments are loaded from the file. The bss segment is uninitialized data, i.e. upon program start, it is not defined - and not loaded from the file. This area is read-write and can be used during program execution. It is also not shared between processes. In addition to these segments, the file format also includes a zeropage segment type, to allow zeropage variables to be relocated. The "zeropage" is a segment that allows "short" addressing modes and its meaning varies from CPU type to CPU type. In a 6502 for example this is the zeropage, in a 65816 this is the bank zero segment. This zeropage segment is like a bss segment, in that only the length, but not the data is saved.

The different segments hold different type of data and can be located anywhere in memory (except zero segment, which has to be in the zeropage resp. bank zero). The program must therefore not assume anything about the relative addresses between different segments.

2.3) Relocation

In general, there are three ways to handle the relocation problem so far:

1. Tables: have a relocation table for a text segment. The table can be located in front of the code, but then the relocation table must be stored in a side-storage to be used when the code is being loaded.
   If the table is located behind the code, the code must be loaded first and then relocated when the table is loaded.
2. Deassembling: The loader goes through the code, "deassembles" it and changes all absolute addresses. The Problem here is that the loaded needs to know or have hints about where data is in the code.
3. Relocation info in the code: here each address is preceeded with an 'escape' code and is relocated when loading. This approach requires that the code is loaded bytewise to be relocated on the fly. So it disallows block oriented transfer from storage media to memory.

2.4) External References & Exported Globals

As this file format should not only be used as an executable format, but also as object file format, it must provide a way to define references - references exported from this object and labels referenced in this object. The external references list (also called 'undefined list') lists the addresses where labels not defined in this object are referenced. The exported globals list lists the addresses that are available for other objects. The labels are named by null-terminated strings.

Even an executable file can have non-empty globals and externals lists, but only if the operating system allows this. In this case, so called 'late binding' is used to link the object with some global libraries at link time.

2.5) File extension

The proposed standard extension for the described format is ".o65" when used as an object file.

2.6) Format description

The binary format is the following:

   (
	header

	text segment

	data segment

	external references list

	relocation table for text segment

	relocation table for data segment

	exported globals list
   )

2.6.1) Header

The header contains the minimum needed data in a fixed struct. The rest of the necessary information is put into the header options. [Note: .word is a 16 bit value, .byt is a simple byte. .long is a 32 bit value. .size is a 16 or 32 bit value according to .word and .long, depending on the size bit in the mode field. Byte order for all fields according to the order bit in the mode field. ]

This is the fixed struct:

   (
	.byt $01,$00		; non-C64 marker

        .byt $6f, $36, $35      ; "o65" MAGIC number!
	.byt 1			; version

	.byt mode0, mode1	; mode word
	.byt cpu0, cpu1		; CPU type
	.byt os0, os1		; operating system id
	.byt sect0, sect1	; operating system specific section marker

	.size tbase		; address to which text is assembled to 
				; originally
	.size tlen		; length of text segment
	.size dbase		; originating address for data segment
	.size dlen		; length of data segment
	.size bbase		; originating address for bss segment
	.size blen		; length of bss segment
	.size zbase		; originating address for zero segment
	.size zlen		; length of zero segment
	.size stack		; minimum needed stack size, 0= not known.
				; the OS should add reasonable values for
				; interrupt handling before allocating
				; stack space
   )

	mode1.7 :	order	0= LSB first	1= MSB first byte order
	mode1.6	:	reloc	0= bytewise... 	1= page(256byte)wise relocation
						   allowed
	mode1.5	:	size	0= size=16 bit,	1= size=32 bit
	mode1.4 :	obj	0= executable	1= object file
	mode1.3 :	simple	0= (ignored)	1= simple file addresses
	mode1.2 :	chain	0= (ignored)	1= another file follows this one
	mode1.1 :	bsszero 0= (ignored)	1= the bss segment must be zeroed out for this file

	mode0.0-1:	align	0= byte align,	
				1= word (i.e. 2 byte) align
				2= long (4 byte) align
				3= block (256 byte) align

• The order bit tells the loader the byte order of the fields in the header that depend on the size bit. Zero means low byte first, one means high byte first. If size is 32, then zero means the order 2^0, 2^8, 2^16, 2^24, while one means the order 2^24, 2^16, 2^8, 2^0.
• The reloc bit defines if an object file can be relocated bytewise, or if it must be page-aligned. A page has 256 bytes. The restriction to pagewise relocation simplifies the relocation table and also allows simpler compilers/assemblers.
• The size bit determines the size of the segment base address and length entries. Currently the 16 bit size (size bit = 0) works for 6502 and 65816 CPUs.
• The obj bit distinguishes between object files and executables. An object file is used as assembler output that can be linked with other object files to build an executable or an object library.
• The simple bit signals the loader that the load addresses have a specific form. This form fulfills the following conditions:

          dbase   = tbase + tlen
          bbase   = dbase + dlen
  

  This condition ensures that the loader can actually load the text and data segments in one block, and can then use the same base address for the relocation of all three, the text, data and bss segments. The simple mode bit is optional, in that when it is set the conditions must be fulfilled, but if not set the conditions may or may not be fulfilled.
• The chain bit signals the loader that after the current o65 "file" there is another "file" appended to the actual file on disk. This way "multi-o65" files can be built. An "o65" file in a multi-o65 file is here now called "section". Chaining allows the following scenarios:
  1. Init code in a separate segment - the chain contains a first o65 section with the code to run the program, and a second o65 with initialization code that can be thrown away after init. As the init code may just as any program need zero-, data- and bss segments, a full o65 file structure is provided in the section.
  2. Larger systems have mapped memory. The chain bit allows to provide different sections to be loaded in different memory mappings in a single file.
  3. Fat binaries: A single file could hold different o65 sections, one for each different type of CPU. The loader could ignore the parts that do not fit the CPU that it is running on.
  The loader may support binding undefined references in a later section to global labels exported from an earlier section. Otherwise the operating system should provide calls to access the separate sections, e.g. when they are loaded into different memory mappings. The next o65 section starts again with the header (including non-C64 marker and magic number), so sections with different characteristics may be chained. The last section must have chain=0. The chain bit is optional, if it is set and a loader does not support it, the file may be rejected right away. For these purposes the loader should have a means of identifying different sections and their purposes. This can be done using the sect header field.
• The bsszero bit tells the loader that the executable to be loaded requires the bss segment to be zeroed out. If it is not set, then the code must not assume any special value in the bss segment (which is the default behaviour for o65 version 1.2 and below). A loader that does not support zeroing out the bss segment must reject a file with this bit set.
• The two align bits give the address boundary the segments can be placed. Even the 6502 needs this, as, for example, "jmp ($xxFF)" is broken. The align bits are valid for all of the segments. [Note: if reloc=1, then align should be 3. But if align=3, reloc need not be 1, because reloc switches to a simpler version of the relocation table. The reloc bit might be obsoleted in newer versions of this format. Though it should be set, if necessary.]

All unused bits in the mode field must be zero. A loader should reject files with bits set that it does not understand.

The cpu field determines the type of CPU. It consists of two bytes that determine CPU family and version respectively. See the appendix for the definition of currently known CPUs.

The os field determines the type of Operating System. It consists of two bytes that determine OS family and version respectively. See the appendix for the definition of currently known OSes.

The sect field determines the type of section (in a chained file). It is operating system specific and consists of two bytes that could for example determine which sections must be loaded into the same memory mapping or other specific requirements for that specific section. Must be zero if not used.

Note that the header size is 32 if the size bit is zero and 50 if the size bit is one.

The fixed sized struct is immediately followed by a list of header options. Each header option consists of a single byte total length, a type byte and some data bytes if needed. A single length byte of $00 ends the header option list.

   (
  	{			; optional options, more than one allowed
	   .byt olen		; overall length (including length and type
				; byte
	   .byt otype		; option type
	   [ .byt option_bytes ]
	}
	.byt $00		; end of options marker (i.e. option len=0)
   )

- Filename:
  type=0; len=strlen(filename_in_ascii)+3; content="filename_in_ascii",0
  The string contains the name of the object.

- Operating System Header
  type=1; len=?
  the first data byte is the OS type:
    	1 	OSA/65 header supplement
	2	Lunix header supplement
	3	CC65 generic module (new in v1.3)
	4	opencbm floppy modules (new in v1.3)
	[others to follow?]
  the following data contains OS specific information.
  A suggested data byte is the OS version as second byte.

- Assemblerprogram:
  type=2; len=strlen(ass)+3; content="ass",0
  The string contains the name of the assembler resp. linker that produced 
  this file/object.
  For example (syntax see below)
     .fopt 2, "xa 2.1.1g",0
  becomes
     0c 02 78 61 20 32 2e 31 2e 31 67 00
  in the file.

- Author:
  type=3; len=strlen(author)+3; content="author",0
  The string contains the author of the file. 

- Creation data:
  type=4; len=strlen(date)+3; content="date_string",0
  The string contains the creation date in format like:
  "Sat Dec 21 14:00:23 MET 1996", where we have the day, Month, date,
  time, timezone and year. See output of `date`...

2.6.2) text and data segments

The text and data segments are just the assembled code. The only difference between text and data segments is the read/write mode of the two segments. Therefore, to be compliant to this file format, self-modifying code goes into the data segment.

2.6.3) Undefined references list

The next list is an ASCII list of labels that are referenced in this file but not defined. The lists is preceeded with the number of undefined labels (16 or 32 bits, according to the mode.size bit).

undef_list:	number_of_undefined_labels.s
		"undefined_label1",0
		"undefined_label2",0
		...

The character encoding and length of the names of the undefined labels should be appropriate for the target platform, that may define additional constraints. The encoding must allow zero-terminated byte arrays as string representations. To allow short loading times, the names should not be exceedingly long.

2.6.4) Relocation tables

The relocation tables are the same format for the two segments, text and data. In general a relocation entry consists of the offset from the previous relocation address to the next one, the type of the relocation and additional info. Relocation not only defines the relocation when moving object code to a different address, but also filling in the undefined references.

Each table starts at relocation address = segment base address -1. I.e. if the segment base address is $1000 for example, the first entry has an offset computed from base address-1 = $0fff. The offset to the next relocation address is the first byte of each entry. If the offset is larger than 254 (i.e. 255 or above), than a 255 is set as offset byte, the offset is decremented by 254 (note the difference) and the entry is started again.

{ [255,...,255,] offset of next relocation (b), typebyte|segmentID [, low_byte] }+

WORD	$80	2 byte address (or low word of a 4 byte address)
HIGH	$40	high byte of a 2 byte address
LOW	$20	low byte of a 2 byte address
SEGADR	$c0	3 byte address (65816)
SEG	$a0	segment byte of 3 byte address
LONG	$60	4 byte address
HWORD	$10	high word of a 4 byte address

0		undefined
1		absolute value
2		text segment
3		data segment
4		bss segment
5		zero segment

Please note that although the byte order may be MSB or LSB, the byte order in the relocation table is fixed as described here. As the relocation table will most probably be read bytewise, this should not matter. In the code, however, the addresses will be handled according to the header mode bit order.

If the type is HIGH, the low byte of the value is stored behind the relocation table entry, if bytewise relocation is allowed (header mode field bit 14). If only pagewise relocation is allowed and the low byte is implicitely set zero (i.e. it is _not_ saved in the relocation table).

If the type is HWORD, then the lower word of the address is stored in the relocation table, lower byte first.

If the type is SEG, then the two lower bytes of the three byte segment address are stored behind the entry in the relocation table, lower byte first.

If the segment is "undefined", the typebyte is immediately followed by the two (mode size=0) or four (mode size=1) byte value index in the undefined references list. If it is a high byte relocation, the low byte is saved behind the index value (Note: the relocation value need not be automatically zero, as it can be an offset to the value of the undefined reference). The index value determines the undefined reference, which must be looked up by the loader. When the value of the undefined reference is found it is added to the the value of the relocation address in the segment.

The value taken from the relocation address in the segment, together with the low byte/lower bytes from the relocation table (if HIGH/SEG/HWORD entry) form the address (or the lower parts of it in case of WORD for 4 byte address) used if the segment would be used unrelocated. To relocate the segment, the difference between the relocated segment base address and the segment base address from the file is then added to the above address. The result is again saved in the segment.

A zero offset byte ends the relocation table. The first offset is computed from the segment base address-1, to avoid a 0 value in the first entry.

[Ed. Note: needs to go into appendix to CPU codes/specifics] Note that direct addressing modes do not generate entries in the relocation table. instead it is assumed that the 65816 direct register holds the correct value (i.e. zero segment base address) when running this program.

Example (for file contents see appendix B.1):

Segment Base address in file (header.tbase) is $1000. The start address of the text segment after relocation is real.tbase = $1234.

Now the first (unrelocated) address at which a relocation should take place is here:

$1222	A9 23 		lda #>vector

Now we reference the high byte of an address, lets say vector=$23d0 (not relocated), in the text segment. Therefore the relocation type becomes 'HIGH | text_segmentID = $42', which is the next byte. Because we are referencing a high byte of an address, the low byte of the unrelocated address is saved behind the typebyte in the relocation entry. This byte is missing when referencing a low byte or address.

The relocation table entry is now:

$ff, $ff, $28, $42, $d0.

vector_file = ($23 << 8) + $d0 = $23d0

vector_relocated = vector_file + (real.tbase - header.tbase)
		 = $23d0 + ($1234 - $1000)
		 = $23d0 + $234
		 = $2604

Had "vector" now been an undefined reference, and "vector" would be the second label in the undefined references list, we would get the following relocation table entry (assuming mode.size=0):

$ff, $ff, $28, $40, $02, $00, $00

2.6.5) exported globals list

The global list is a list of names, together with the target segment and the offset in the segment for each name. It is preceeded with the number of exported labels. This allows the loader to allocate a table large enough, if needed. The number of labels and the offset value are 16 bit or 32 bit values according to the size bit in the header mode field. The segmentID is a byte value and the same as in the relocation table entry (see section 2.6.3).

	number_of_exported_labels.s
        "global_label_name_in_asc1",0, segmentID.b, value.s
	...

Note: an undefined reference can not be exported. Doing this would lead to circular references for example when linking multiple object files, therefor it is not allowed.

The character encoding and length of the names of the undefined labels should be appropriate for the target platform, that may define additional constraints. The encoding must allow zero-terminated byte arrays as string representations. To allow short loading times, the names should not be exceedingly long.

3) assembler source format

The assembler source format is a suggestion only. It will be implemented in xa65, a cross assembler for 6502 CPUs running on Unix/Atari ST/Amiga as a reference platform.

The assembler provides a way to embed absolute address code in relocatable code. This is needed when code should be copied to a specific location known at assemble time. There also is a way to make a file 'romable'. You can give the start address of the _file_ in ROM, and the assembler automatically sets the text segment start address to where the code will be in the ROM. Of course, the other segments must be taken care of with -b? command line parameter, that set the segment start address.

3.1) embed absolute code in relocatable files

When the assembler is started in relocatable mode, everything is put into a .o65 relocatable file. All address references generate relocation table entries. If a "*= value" pseudo opcode is encountered, then the assembler switches to absolute mode. The following opcodes don't generate relocation table entries. If a "*=" without a value is read, then the assembler switches back to relocatable mode. The relocation program counter is increased with the length of the absolute part and the absolute code is embedded between the relocatable parts.

3.2) embed relocatable code in absolute files

This is dropped - too complicated. Should better be done with some objdump or linker programs or so.

3.2) Header options

Before any opcode (after starting in relocatable mode, or after a .reloc opcode), a header option can be set by:

	.fopt byte1, byte2, ...

	.fopt 3, "Andre Fachat",0

3.3) allocation of data segment/zeropage segment address space

The assembler switches between the different segments by the means of ".text", ".data", ".bss" and ".zero" pseudo opcodes. After starting in relocatable mode, the assembler is in the text segment.

The text segment contains the program code. Data holds the initialized data, while bss and zero segments contain uninitialized data for normal/zeropage address space. Everything that is between one of these segment opcodes and the next segment opcode gets into the corresponding segment, i.e. labels, assembled code etc. The text and data segments are saved in the file, while for the bss and zero segments only the length is saved in the file.

The assembler should issue a warning when a direct addressing mode is used without a zero segment address and vice versa for 65816 CPUs.

3.4) referencing data/bss/zeropage addresses

One problem with the 6502 is, that it cannot load an address within one step or assembler opcode. So an address is loaded with standard byte opcodes, like "lda #<label". But how do we decide, whether "label" is an address or not, and what do we if we get something like "lda #zp_label + 12 * label2"?

The assembler is now intelligent enough to evaluate such expressions and check for:

- no address label			: ok, absolute
- one address label, only add to label	: ok, relocate
- difference between two addresses 	: If addresses in same segment, compute
					  diff and set absolute, otherwise bail
- everything else			: warning

3.5) aligning code

The 6502 has the problem that some opcodes (e.g. "JMP ($xxFF)" are broken, if the address given is at some (odd) address. But when loading a relocatable file, one cannot know if an address will be odd or even. Therefore there is a new opcode,

	.align 2

4) Additional Notes

4.1 Clearance

This file is surely not the optimum and could be improved. Also the header option "assigned numbers" should be added here.

For this reason the author, André Fachat, will function as a clearing point, where problems can be discussed and numbers can be assigned.

4.2 Character Sets

Appendix

A) CPU codes & specifics

6502 CPU family

The stack space is also very limited. A binary format has to provide a measure of how much stack space is needed for the application.

Such limits should be defined as 2 byte values, even if the 6502 only has a range of 8 address bits for zeropage and stack.

Another problem is, that an address can be 'split', i.e. you can just use the high byte or the low byte separately in an opcode. This gives need to a special relocation table format, that can cope with half-address references.

65816 CPU family

B) OS codes

A) Late binding

Late binding means that during the assembler run the values of some variables are not known. Instead these variable values are filled in when the program file is loaded into the system.

As an example let's discuss an example for a program that needs to access some hardware at the expansion port of the C64. The hardware is located either at IO1 ($de00) or IO2 ($df00) depending on some hardware switch. To allow to use only one executable for the program, it uses a variable "IOPORT" that is not defined in the program itself, but set by the o65 loader using late binding.

The program accesses the io port is using the variable:

	lda IOPORT

	xa -R -LIOPORT -o program.o65 program.a65 

The resulting file looks like: Ed. Note: needs to be redone

00000000  01 00 6f 36 35 00 00 00  00 10 03 00 00 04 00 00  |..o65...........|
00000010  00 40 00 00 04 00 00 00  00 00 00 ad 00 00 01 00  |.@..............|
00000020  49 4f 50 4f 52 54 00 02  80 00 00 00 00 00 00     |IOPORT.........|

When loading the file, the loader must know in advance what value IOPORT should be assigned. This is not further discussed here. When the loader loads the file, and recognizes the name IOPORT in the undefined references table, it remembers the index of this name in the table. Then, when the relocation table contains a reference to the undefined label with the index value for IOPORT, the value for that variable is then used in the relocation.

If, for example the loader knows that IOPORT=$de00, then the text segment is relocated to

	$ad $00 $de

	lda IOPORT+1

00000010  00 40 00 00 04 00 00 00  00 00 00 ad 01 00 01 00  |.@..............|
                                               ^^

	$ad $01 $de

B) File examples

B.1 Example from section 2.6.4

The example source file is

.text
        .dsb $222,$aa

        lda #>vector

        .dsb $23d0-$1224,$55

        vector = *;

	xa -R -o test2.o65 test2.a65

00000000  01 00 6f 36 35 00 00 00  00 10 d0 13 00 04 00 00  |..o65...........|
00000010  00 40 00 00 04 00 00 00  00 00 00 aa aa aa aa aa  |.@..............|
00000020  aa aa aa aa aa aa aa aa  aa aa aa aa aa aa aa aa  |................|
*
00000230  aa aa aa aa aa aa aa aa  aa aa aa aa aa a9 23 55  |..............#U|
00000240  55 55 55 55 55 55 55 55  55 55 55 55 55 55 55 55  |UUUUUUUUUUUUUUUU|
*
000013e0  55 55 55 55 55 55 55 55  55 55 55 00 00 ff ff 28  |UUUUUUUUUUU....(|
000013f0  42 d0 00 00 01 00 76 65  63 74 6f 72 00 82 d0 23  |B.....vector...#|
00001400

	ld65 -bt 4660 test2.o65

00000000  01 00 6f 36 35 00 00 00  34 12 d0 13 00 10 00 00  |..o65...4.......|
00000010  00 40 00 00 02 00 00 00  00 00 00 aa aa aa aa aa  |.@..............|
00000020  aa aa aa aa aa aa aa aa  aa aa aa aa aa aa aa aa  |................|
*
00000230  aa aa aa aa aa aa aa aa  aa aa aa aa aa a9 26 55  |..............&U|
00000240  55 55 55 55 55 55 55 55  55 55 55 55 55 55 55 55  |UUUUUUUUUUUUUUUU|
*
000013e0  55 55 55 55 55 55 55 55  55 55 55 00 00 ff ff 28  |UUUUUUUUUUU....(|
000013f0  42 04 00 00 01 00 76 65  63 74 6f 72 00 82 d0 23  |B.....vector...#|

B.x more examples

(to be done with reference assembler)