Unify SEND/RECEIVE and Streams with a buffer-oriented communications API, that specifically allows for shared memory streams and optimizes the number of kernel calls.
These concepts are preliminary and the discussion documented here is basically a scratchpad of ideas that will be further refined, and at some point implemented.
The Buffers interface replaces the SEND/RECEIVE and Streams APIs at the same time.
The main concept of the API is the Buffer. A buffer is a memory area that is defined by a start address, a length, and a type that defines how read and write pointers, as well as changes to start address and length are handled.
The following types are defined: 1. message - this buffer will be atomically sent as one set of bytes, and receives sets of bytes atomically. This replaces the SEND/RECEIVE API 2. ring - this buffer works as a ring buffer, similarly to the kernel buffers used in V2. This replaces the current Streams API 3. rwbuf - a memory buffer that is being read from or written to. The extends to Streams API so that file servers can directly write to target environment instead of going through the kernel with every byte (zero-copy load) 4. mmap - a shared memory area between two tasks/environments. This allows for disk servers to provide shared buffers to file servers.
|
Note
|
a file server is what is today implemented in e.g. FSDEV and FSIEC, providing file system services to other processes. A disk server is a new type of process, that provides block-oriented access to partitions, disk images etc. A FAT file server could for example use an SCSI disk server to provide file systems from a SCSI disk partition. |
As the 6502 does not have enough and not large enough registers to give address and length of a buffer and further parameters in a single call, a separate buffer definition block is used to define a buffer:
-
0/1: address of buffer (2 bytes)
-
2/3: length of buffer (2 bytes)
-
4: type of buffer (1 byte)
-
5: buffer flags (1 byte)
-
(6/7: amount of data stored - see API)
This buffer definition is given to the kernel once, and is then used throughout the communication in the channel (see below).
A buffer can be - if appropriate flags are given - re-defined by the kernel. This allows the kernel for example to map buffers in two tasks into the same memory area - so no copying is required for the actual data transfer.
Channels replace streams. Once a communication is established and the buffer is given to the kernel, only the channel number is used for further calls.
The kernel maintains two buffers for each channel - one for the sender, one for the receiver. The sender writes data into its own buffer, then tells the kernel how many bytes are written. The kernel then copies the data over to the receiver buffer. When the receiver calls the kernel, it gets the amount of data that it can read from the buffer.
If specific flags are given (tbd), the kernel can re-map the buffer, and overwrite the buffer definition block address and length. The reason is that re-mapping read- and write buffers to the same address allows for zero-copy loading or saving of data from/to devices.
The streams API - GETSTR, GETC, PUTC, FRESTR, UNGETC - will be re-created for buffers to allow simple migration.
The SEND/RECEIVE API will be replaced with a new API.
- PCONNECT
-
send a message to another task - initiate a SEND/RECEIVE channel. The (best/only?) supported buffer type is the Message buffer. PCONNECT blocks until the message has been fully taken out of the buffer. Further communication on that channel is then done by using CPUSH and CPULL.
- Input
-
-
AC/YR - address of buffer definition block
-
XR - target port number
-
- Output
-
-
C=1 - error
-
AC - error code
-
E_ILLPAR - not a valid port number (not open for listening)
-
E_INT - the call has been interrupted by a signal (semantics / preserved registers to be defined)
-
-
-
C=0 - success
-
XR - channel number to use for further communication
-
-
- PLISTEN
-
register a port
- Input
-
-
none
-
- Output
-
-
C=1 - error
-
AC - error code
-
-
C=0 - success
-
XR - port number to use for further communication
-
-
- PACCEPT
-
bind a buffer to an incoming port send (PCONNECT) request. Copy the initial message to the receiver buffer. The (best/only?) supported buffer type is the Message buffer. Further communication on that channel is then done by using CPUSH and CPULL.
- Input
-
-
AC/YR - address of buffer definition block
-
XR - port number
-
- Output
-
-
C=1 - error
-
AC - error code
-
E_ILLPAR - not a valid port number or port already in use (another process waiting)
-
E_INT - the call has been interrupted by a signal (semantics / preserved registers to be defined)
-
-
-
C=0 - success
-
XR - channel number to use for further communication
-
-
- PCLOSE
-
close a port
- Input
-
-
XR - port number
-
C=1 - error
-
AC - error code
-
E_ILLPAR - not a valid port number or port already closed
-
-
-
- Output
-
-
C=0 - success
-
- GETCHAN
-
allocate a new channel Replacement for GETSTR
- Input
-
-
n/a
-
- Output
-
-
C=0 - success
-
XR - new channel number
-
-
C=1
-
AC - error code
-
-
- CPUT
-
send a byte to a channel
- Input
-
-
XR - channel number
-
AC - byte to send
-
- Output
-
-
C=0 - success
-
C=1
-
AC - error code
-
-
- CGET
-
get a byte from a channel
- Input
-
-
XR - channel number
-
- Output
-
-
C=0 - success
-
AC - byte taken from channel
-
-
C=1
-
AC - error code
-
-
- FRECHAN
-
release a channel Replacement for FRESTR
- Input
-
-
XR - channel number
-
- Output
-
-
C=0 - success
-
C=1
-
AC - error code
-
-
- CHANCMD
-
modify a channel Replacement for STDCMD
- CBUF
-
set the read or write buffer for a channel
- Input
-
-
C - 0=read buffer, 1=write buffer
-
XR - channel number
-
AC/YR - address of buffer definition block
-
- Output
-
-
C=0 - success
-
Iff the buffer given as input to CBUF is re-definable, the buffer information can be replaced with the existing buffer of the channel.
-
Iff data is already available, the kernel does a CPUSH/CPULL, transferring the data.
-
-
C=1
-
AC - error code Note: if input channel is STDNUL, a new channel is allocated as in GETCHAN, and the buffer is then set. If the buffer given to CBUF is re-definable, the call could map an existing buffer appropriately and return the mapped buffer information in the struct. If a read buffer is defined, and the stream already contains data, it is automatically transferred to the given buffer. If a write buffer is defined that already contains data (using the extended 8-byte buffer definition), the data is automatically transferred.
-
-
- CPUSH
-
send bytes in buffer to other end
- Input
-
-
XR - channel number
-
AC - number of bytes to be sent
-
- Output
-
-
C=0 - success
-
AC - number of bytes the receiver has already taken from the local buffer since the last call
-
-
C=1
-
AC - error code
-
-
- CPULL
-
get bytes from other end into buffer
- Input
-
-
XR - channel number
-
AC - number of bytes to acknowledge (that have been read) since the last call
-
- Output
-
-
C=0 - success
-
AC - number of bytes that have been transferred to the local buffer
-
-
C=1
-
AC - error code
-
-
Four different buffer types can be used. Each buffer type works differently. The behaviour of each buffer type is described below
The Message buffer is a buffer that does an "all or nothing" approach when sending (CPUSH) or receiving (CPULL). It is used to replace the "PCBUF" used for sending and receiving in V1 and V2.
A Ring buffer allows continuous streaming, using a read and a write pointer that wrap around at the end of the buffer’s size. (This is the default implementation for a stream buffer the current kernel)
An Uni buffer is a unidirectional, single use buffer that is read or written once. After that the channel is either closed, or the buffer needs to be re-defined with CBUF. This is used for example to load data from a file or store data in a file.
To allow for optimized, if possible even zero-copy loads across environments, if a process is prepared for it, the channel buffer can be re-mapped by the kernel.
1) flag to be set in CBUF to enable this feature
2) specific return value for CPUSH/CPULL to then call CBUF again
In V1 and V2, the kernel manages a single, internal ring buffer to transfer bytes with streams. In V3, the kernel manages multiple buffer definition blocks - one for reading and one or potentially more for writing.
When the channel is created, an internal ring buffer is allocated and set as read buffers. This channel is being used by CPUT and CGET.
There are a couple of other kernel calls that use the single PCBUF/SENDBUF to exchange data:
-
DEVCMD: uses PCBUF to either take a device name to be translated into a device number or vice versa, return a device name for a device number
-
FORK: Necessary information to fork a new task is given to the kernel in PCBUF
-
GETINFO: the kernel returns task and thread information in PCBUF
Note, that for DEVCMD, FORK, and GETINFO a stream as replacement would be difficult to use, as they are expected to work in an atomic matter in the kernel. I.e. the information or space in the buffer must be available all-or-nothing. So, for example, replacing the buffer with a (char-based) stream is not feasible, as the kernel may block if the stream gets emtpy (on reading) or full (on writing).
If these calls could allocate or map the buffer space from a stream, it could be possible to replace the PCBUF in these calls as well.