Forth Programmer’s Handbook

3.4 BLOCK-BASED DISK ACCESS

Forth systems provide access to mass storage using either a block-based or a filebased method (and occasionally both). In a block-based system, mass storage is partitioned into some number of blocks, each 1024 bytes long. These blocks may be in files, depending on the underlying operating system (if any). In a filebased system, a host operating system is required. It provides and manages files of variable length, which Forth uses directly for mass storage.

This section discusses the words used to access and manage disk blocks and block buffers in Forth. Section 3.5 discusses the words used to access mass storage using files. One of these sections (and sometimes both) will be relevant to a particular Forth system.

3.4.1 Overview

The block-based disk access method is intended to be simple and to require a minimum of effort to use. The disk driver makes data on disk directly accessible to other Forth words by copying disk data into a buffer in memory, and placing on the stack the buffer’s address. Thus, Forth routines access disk data using the same techniques it uses for other memory accesses. Because disk data always appears to be in memory, this scheme is a form of virtual memory for program source and data storage.

Another consideration in the design of the disk driver is to make disk access as fast as possible. Because disk operations are very slow, compared to memory operations, data is read from disk or written to disk only when necessary.

The disk is partitioned into 1024-byte data areas called blocks. This standard unit has proven to be a useful increment of mass storage. As a unit of source text, for example, it contains an amount of source which can be comfortably

86 System Functions

Forth Programmer’s Handbook

displayed on a CRT screen; as the basis for a database system, it is a convenient, common multiple of typical record sizes.

Each block is addressed by a block number. On native Forth systems, the block number is a fixed function of the block’s physical position on the disk. Absolute addressing of the disk both speeds the driver’s execution and eliminates most of the need for disk directories and indexes. On OS-hosted Forth systems, the blocks may be located in one or more files, each an integral multiple of the block size, and an internal table maps OS files to block space.

3.4.2 Block-Management Fundamentals

A program ensures that a block is in memory in a block-buffer by executing the word BLOCK. BLOCK uses a block number from the stack and returns the address of the first byte of that block in memory. For example:

9 BLOCK U.

will return an address such as:

46844 ok

where 46844 is the address of the first byte of the buffer containing block 9. If a block is already in memory, BLOCK will not re-read it from disk.

Although BLOCK uses a disk read to get data if it is not already in memory, BLOCK is not merely a read command. If BLOCK must read a requested block from disk, it uses BUFFER to select a buffer to put it in. BUFFER frees a block buffer, writing the buffer’s previous contents to disk if it is marked (by UPDATE, see below) as having been changed since the block was read into memory.

BUFFER expects a block number on the stack, and returns the address of the first byte of the available block buffer it assigns to this block. For example:

127 BUFFER U.

will get a block buffer, assign block number 127 to the buffer, and then type the address of the buffer’s first byte:

36084 ok

System Functions 87

Figure 8. Block handling in a file-based Forth system

Although BUFFER may write a block, if necessary, it will not read data from disk. When BUFFER is called by BLOCK to assign a buffer, BLOCK will follow the selection of a buffer by actually reading the requested block from disk into the buffer.

The following example displays an array of the first 100 cells in block 1000, shown with five numbers per line:

88 System Functions

Forth Programmer’s Handbook

The phrase I CELLS + converts the loop counter from cells to bytes (because internal addresses are always byte addresses), and adds the resulting byte offset to the address of the block buffer returned by BLOCK. The word ? fetches and types the cell at that address.

BUFFER may be used directly (i.e., without being called by BLOCK) in situations where no data needs to be read from the disk. Examples include initializing a region of disk to a default value such as zero, or a high-speed data acquisition routine writing incoming values directly to disk from a memory array 1024 bytes at a time.

Forth systems will have at least one, but usually many, block buffers. The number of buffers may be changed easily. Applications with several users using disk heavily may run slightly faster with more buffers. Your product documentation will give details on changing the size of the buffer pool.

The command UPDATE marks the data in a buffer as having been changed, so that it will be written to disk when the buffer must be used for another block. UPDATE works on the most recently referenced buffer, so it must be used immediately after any operation that modifies the data.

The following example uses BUFFER to clear a range of blocks to zero:

: ZEROS ( first last -- ) 1+ SWAP DO

I BUFFER 1024 ERASE UPDATE LOOP ;

As another example, assume that an application has defined A/D to read a value from an A/D converter. To record up to 512 samples in block 700, use:

In this example, the phrase 512 MIN “clips” the specified number of samples at 512. As in the example of SHOW above, the phrase I CELLS converts the loop counter (in samples) into a byte offset to be added to the address of the start of the block, returned by BLOCK. BUFFER cannot be used in this case,

System Functions 89

Forth Programmer’s Handbook

because we are adding samples one at a time and must preserve previous samples written in the block.

Because BLOCK maps disk contents into memory, virtual memory applications are simple. The first step is to write a word to transform an application address into a physical address, consisting of a block number and an offset within that block. For a virtual byte array, such a definition is:

Here, 1024 is the number of bytes per disk block and 250 is the block number where the virtual array starts. The array may occupy any number of blocks, limited only by physical mass storage constraints.

Fetch and store operations for this virtual memory scheme are defined as:

BLOCK does not normally perform any error checking or retries at the primitive level, because an appropriate error response is fundamentally applicationdependent. Some applications processing critical data in non-real-time (e.g., accounting applications) should attempt retries* and, if these fail, stop with an error message identifying bad data. Other applications running continuously at a constant sampling rate (e.g., data loggers) cannot afford to wait, and should simply log errors.

*Most disk controllers and all OSs perform retries automatically. On these, there is nothing to be gained by attempting retries from within a Forth application.

90 System Functions