INTRODUCING LXM BRIEF HISTORY OF XMODEM During the glory days of CP/M, the computer industry was plagued with a plethora of diskette formats. Since there was no single standard around, particularly when the 5" format was introduced, data interchange between computers was difficult. Several manufacturers attempted to provide a solution by giving their users the ability to at least read multiple diskette formats. Rather than promoting a standard format, this approach encouraged manufacturers to adopt proprietary formats as a marketing tool. Partially as a response to this situation, and partially out of a need to provide data transmission capabilities, Ward Christiansen developed the original specification for the XModem file transfer protocol. The XModem protocol, elegant in its simplicity and effectiveness, has transcended the decline of CP/M-based systems, and the ascendency of MS-DOS to become a de facto standard for data transfer between remote computers. This fact is true, even in the light of the near-universal acceptance of another de facto standard for diskette formats, as popularized by the IBM-PC family. With the advent of the new PS/2 with its incompatible micro- diskettes, the XModem protocol's place, at least for the foreseeable future, seems assured. PROTOCOL FUNDAMENTALS THE TRANSMISSION BLOCK The fundamental building block of XModem is an 8-bit byte, no parity please, arranged into transmission blocks of exactly 132 characters, never more nor less. In one extension to XModem, commonly called CRC-XModem, the transmission blocks are exactly 133 characters. We will discuss this extension , and a second known as YModem, later. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C As you might guess, not only is the length of a transmission block fixed, but so too is the format. The XModem block format looks like this: <~REC> <...128 Data Bytes...> where: SOH - 0x01 Signals the start of a block REC - Is the sequential block number reduced modulo 256 ~REC - The ones-compliment of REC checksum - an 8-bit sum of the preceding 128 data bytes, formed by adding each byte to an accumulator, then dropping all but the low order 8 bits of the result. The intent of the additional characters is to facilitate error checking and recovery, making the XModem protocol an "error-free" means of data transfers under otherwise hostile conditions. The block number and its compliment are included to insure that no blocks are accidentally lost during transmission, and to prevent the accidental duplication of a data block which might be caused by spurious line noise. The checksum seeks to insure the validity of the actual data which might become garbled by line noise. The fixed length block format tends to simplify the logic required to implement the protocol in the first place. THE INTER-COMPUTER DIALOG Xmodem is essentially uni-directional in nature. That is, the actual flow of meaningful information occurs in one direction at a time, from the sending computer to the receiving computer. This is not meant to imply, however, that only one computer sends and the computer receives. To the contrary, there is a constant "dialog" going on between the two computers during the course of the transmission. It is this dialog that permits XModem to be true, "error-free" transfer method. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C Xmodem is a receiver-driven protocol. The receiving computer initiates and almost exclusively controls the transmissions, through a series of pre-planned responses. A typical, though brief dialog might look like this: RECEIVER SENDER MEANING --------------> READY TO RECEIVE <------------- FIRST BLOCK SENT --------------> BLOCK RECEIVED OK <------------- SECOND BLOCK --------------> ERROR, RESEND <------------- SECOND BLOCK --------------> BLOCK RECEIVED OK <------------- END OF FILE --------------> E-O-F UNDERSTOOD As you can see, it is through the responses to the sending computer that the receiver controls the link. Xmodem also makes provision for unusual circumstances by providing both time-out and cancellation mechanisms. The rules for time-outs, time periods which may elapse before a disruption in transmission occurs are as follows: 1. Waiting for SOH - 10 seconds, Resend last acknowledgement on time-out. 2. Waiting for other block characters - 1 second, replace expected character with some pre-defined value upon time-out. 3. Waiting for a reply to a block - 10 seconds, Resend last block upon time-out. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C The original protocol description also makes it possible for either the sender or receiver to cancel the transmission. This is of particular value when too many transmission errors occur, or, in rare cases, when the physical connection is broken. The threshold beyond which either side may request cancellation, although specified as 10 attempts in the original description of the protocol, has become somewhat arbitrary over time. The cancellation code, or 0x10, may be sent by either the sender or receiver in place of the or normal acknowledgment character respectively. ENHANCEMENTS TO XMODEM Over the years, several significant enhancements have been made to the protocol as originally defined by Ward Christiansen. The intent of these enhancements have generally been to improve upon the error-handling capability of the protocol, and to make the protocol more amenable to some less time-efficient communications environments, particularly those presented by for-pay services such as CompuServe. To make the protocol more error-free, the checksum employed in the original design has been replaced by a 16 bit calculation, called a Cyclical Redundancy Check or CRC. Based upon a polynomial equation, the CRC method can be mathematically demonstrated to be sensitive to all but about .03 per cent of errors. This error detection rate, about 99.97 percent accurate, is far superior to the approximately 90 to 92 percent accuracy of the checksum approach. The overhead of an additional 8 bits per message seems to make this enhancement well worth while. The second enhancement seeks to improve communication over packet-switched networks, like those employed by CompuServe and other national services. Basically this enhancement introduces a relaxed time-out between characters in a block, allowing additional time before the protocol recognizes a time-out condition. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C A third enhancement, variant of the original XModem protocol seeks to improve overall efficiency by expanding the block size from 128 characters to 1024 characters. This variant, commonly call ymodem, reduces the number of times that an acknowledgment sequence must be sent, thereby increasing the efficiency of actual data transfer. The LXM XModem engine supports all of the enhancements mentioned above. THE LXM ENGINE OVERVIEW One of the reasons for the popularity of the XModem protocol over time is the simplicity of the implementation. Given a reasonable communications package upon which to build, developing a basic implementation of XModem is relatively simple, although under certain operating systems, such as the Unix family, the attempt of the OS to be all things to all people may actually become an impediment. The intent of the LXM engine is to 1) provide XModem capability, 2) in a flexible, easy to understand package, 3) built upon a sound foundation, the LiteComm communications ToolBox. Because of the manner in which the LXM engine was designed virtually any application can have access to XModem's error-free protocol, without being relegated to simply transferring files. MAJOR ENGINE COMPONENTS The LXM engine consists, primarily of two functions, lcxrrec and lcxtrec, receive a record and transmit a record respectively. Because these two functions 'understand' the XModem protocol internally, they assume full responsibility for all of the necessary housekeeping, relying upon the host program for only the most basic of functions. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C The lcxrrec function receives one or more records from a companion system. The host program controls the checking method, CRC or checksum, the time-out method, normal or relaxed, handles, in what ever way appropriate, blocks of data that have been received, and can optionally monitor the flow of data from the companion system. Lcxrrec assumes the responsibility for synchronizing with the companion system and for correctly maintaining the flow of information. Lcxtrec performs in a like fashion when the host program wants to send one or more records. The host program has only to present the record to be transmitted, and lcxtrec does the rest. The lcxtrec module assumes responsibility for establishing synchronization with the receiving computer, and for recognizing the checking method, CRC or checksum, that the receiver wants to use. Lcxtrec also terminates the transmission, when told to do so by the host program, and permits the host to optionally monitor the data flow. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C NOTES AND WARNINGS MODIFICATIONS The LXM engine is closely integrated with the LiteComm ToolBox. While there is every likelihood that the engine can be modified to function with other similar packages, Information Technology, Ltd., can only support the LXM engine when used with either the LiteComm ToolBox. PARITY AND DATA BITS The XModem protocol is an 8-bit protocol. That is to say it neither recognizes nor tolerates parity checking, i.e. 7 data bits plus a specified parity. Since the LXM engine cannot determine the current settings for parity and number of data bits, the responsibility for controlling these settings rests with the host program. Before using either of the key functions lcxrrec or lcxtrec, the host program must insure that the settings are no parity, 8 data bits, using the comm_setup function. Upon final termination of the function, the host program must reset these values to their original settings, if necessary. INTERNAL TIMER FUNCTION Both lcxrrec and lcxtrec employ a hardware-based timing function that connects directly to the normal real-time clock vector 0x1C; The Microsoft and Turbo C version of this function also establishes an special routine, using the atexit() function, to insure that this vector is re- established at is original setting when program termination occurs. This is not true of the Datalight version however. The Datalight library does not, at present, have an like atexit(). CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C While every effort has been made, for Datalight users, to insure that the vector is properly reset, this plan may be thwarted by abnormal termination of the host program resulting in a subsequent system crash. The safest method to Datalight user's to employ would be to use the lc_clrclock() function before host program termination. As an alternate approach, Datalight users may want to investigate STEVE'S LIBRARY designed for Datalight C users's. This excellent library does have an equivalent function to the atexit(). BUFFER SIZE RESTRICTION The LiteComm ToolBox permits you a great deal of freedom in tailoring the communications handler to meet your specific requirements. We must caution you, however, that when you are using the LXM engine, the minimum buffer sizes required by the comm_opn function are not adequate to support LXM, particularly when transmitting records at either low baud rates, or on the new generation of high-speed (above 6 MHz) processors. During the development of LXM, it was quickly discovered that too small a transmit buffer quickly resulted in a buffer overflow condition. We recommend that, when using the LXM engine, the transmit buffer be set at a minimum 256 bytes to avoid the overflow condition. If you intended to employ the YModem variant, then the minimum buffer sizes should be set to at least 1030. XON-XOFF CONTROL If your host program employs the XON-XOFF functions in LiteComm, you must be sure that the automatic XON-XOFF handing is disabled before you attempt to transmit or receive records using LXM. Failure to observe this caution may result in a system hang, requiring that the system be re-booted. USER-ACCESSIBLE DATA CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C Within LXM, certain variables have been defined as being globally available. The host program may examine the contents of these variables at any time to determine the current state of the LXM engine. The correct definitions of these variable is contained in litexm.h. Except as noted below, the host program must NOT alter the contents of these variables. abort_flag - This is the only variable of the group that may be altered by the host program. The flag is checked periodically by the engine functions. If _abort_flag has a value of 1, the function in progress will be cancelled automatically by sending a CAN instruction to the companion system. See the TTL.C sample program for an example of how this flag may be set. crc - If this variable has a non-zero value, then the LXM engine is currently using the CRC error- checking method. A zero value indicates that the original checksum method is being used. checksum and crcaccum - These variables hold the current/last calculated check value. In a practical sense, they are of no value to the host program. rec - This variable contains the current record (block) number being processed. In the event of an uncorrectable error, rec would contain the number of the expected block. In the case of a successful transmission or reception, rec-1 is the number of the block sent or received. The value contained in this variable, reduced modulo 256, provides the block number required by the XModem protocol and must NEVER be disturbed. ymodem - This variable is set to TRUE if the host application wants to transmit a block in YModem mode (1024 characters). See the discussion of lcxtrec, below. PROGRAMMERS REFERENCE CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C ENGINE-RELATED FUNCTIONS The following pages document the LXM engine functions as currently implemented. They follow, in style and content, the documentation for the LiteComm ToolBox itself. Use the following pages as an addendum to your LiteComm documentation. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C lcxrrec _______________________________________________________________ SUMMARY #include #include int lcxrrec(port,buff,rbsize,hmode,hdshk) unsigned port; unsigned char *buff; int *rbsize; int hmode; unsigned char *hdshk; DESCRIPTION Receive a block of either 128 or 1024 bytes from the companion system, using the checking method specified in hdshk, and the time-out value specified in hmode. If necessary, establish synchronization with the companion system. The port parameter is the same as used throughout the LiteComm ToolBox. Buff should be a pointer to an array of 128 bytes in when using XModem mode, 1024 bytes for YModem mode. All record characters are received into this area, and, if the area is too small, LXM will overwrite adjacent data without warning. Rbsize is a POINTER to an integer variable. Lcxrrec returns the actual length of the received block to this variable. This approach is needed since, using YModem, blocks may be either 1024 bytes or 128 bytes at the discretion of the SENDER. Lcxrrec automatically detects the block length and returns it to the caller so that the caller can process the block correctly. The value of hmode determines whether normal or relaxed time-out values are used. Please use the constants provided in the litexm header file to insure a proper setting. The value contained initially in hdshk must be either CRC or NAK, defined in litexm.h. The former specifies the use of CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C the 16 bit CRC checking mode, while the latter specifies the use of the original checksum method. Note - If you intend to operate in YModem mode, hdshk must specify CRC. Note that hdshk is a pointer to a character. Lcxrrec uses this area to store its reply to the last received block. Once you have set this value, and have started to receive, DO NOT alter its value under any circumstances. Doing so may cause unpredictable results. EXAMPLE See the accompanying program TTL for an example of lcxrrec usage. RETURN VALUES Lcxrrec may return several values, as defined in the litexm.h file. Each return value should cause the host program to respond in specific ways. SUCCESS - A record has been successfully received into the buff area. Host program process the record and calls lcxrrec again. RETRIES - The maximum number of attempts to receive a single record has been exceeded. Lcxrrec automatically cancels the session. Host program should handle in an appropriate manner, e.g. issue error message. ERR (-1) - Lcxrrec has detected that the host program has requested cancellation of transmissions through the _abort_flag setting (see below) or a fatal record sequence has occurred, e.g. a block number has been skipped. Lcxrrec automatically cancels the session. CAN - The sending program has requested cancellation. Host program should handle in a fashion similar to RETRIES. EOT - The sending program has no more data to transmit. Lcxrrec acknowledges the EOT message automatically. Host program handles like an end-of-file condition. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C TOUT - Lcxrrec failed to establish synchronization with the sending program while waiting to receive the SOH (or STX for YModem) character at the start of the block. The session is automatically cancelled. DUPSEQ - The block just received is a duplicate of the preceding block. The host program should ignore the data contained within the block, the call lcxrrec to proceed with data transfer. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C lcxtrec _______________________________________________________________ SUMMARY #include #include int lcxtrec(port, buff) unsigned port; unsigned char *buff; DESCRIPTION Transmit a block of 128 or 1024 bytes to the companion system. The checking protocol, CRC or checksum, is detected automatically when the receiving station issues its initial handshaking sequence. If necessary, establish synchronization with the companion system before transmitting. The port parameter is the same as used throughout the LiteComm ToolBox. Buff should be a pointer to a 128 or 1024 byte record to be sent to the companion system. This is not a typical, null- terminated string as usually found in C. All bytes, starting at the pointer will be transmitted. It is the responsibility of the host program to provide any padding that might be required to satisfy the length requirement. Generally, short records are padded with NULLS or ^Z (0x1A). Lcxtrec determines the block size to be transmitted based upon the current value of the global variable ymodem (see the introduction). If ymodem has a value of TRUE, then buff is assumed to point to a 1024 byte record. If ymodem is FALSE, then a 128 byte record is assumed. This feature has particular value when operating in YModem mode. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C When using YModem protocol, block sizes may be either 128 or 1024 bytes, at the senders discretion. This serves two purposes. In the case of a large number of errors, many YModem implementations will automatically switch to the smaller block size if too many errors occur using the larger block size, in an effort to get the data through, then revert back to the normal block of 1024 characters. Lcxtrec implements this feature automatically. Generally, at the end of a file, the final block to be sent will not exactly match the transmission block size. Under XModem, this means that as many as 127 wasted characters might be transmitted. Under YModem, as many as 1023 wasted characters can be transmitted in the final block. However, the host program that calls lcxtrec can determine if the block to be sent is less than 1024 bytes, set the ymodem variable to FALSE, and send multiple 128 byte blocks. Using this approach, the amount of wasted characters will be no worse than the original XModem. The TTL sample program, included in the package, shows one way of approaching this problem. EXAMPLE See the accompanying program TTL for an example of lcxtrec usage. RETURN VALUES Lcxtrec may return several values, as defined in the litexm.h file. Each return value should cause the host program to respond in specific ways. SUCCESS - The record has been successfully sent from the buff area. Host program either calls lcxtrec with the next record to transmit, or lcxteot to indicate End of Transmission to the companion system. RETRIES - The maximum number of attempts to send a single record has been exceeded. Lcxtrec automatically cancels the session. Host program should handle in an appropriate manner, e.g. issue an error message. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C ERR (-1) - Lcxtrec has detected that the host program has requested cancellation of transmissions through the _abort_flag setting (see below). CAN - The receiving program has requested cancellation. Host program should handle in a fashion similar to RETRIES. CopyRight (c) 1987, 1988 Information Technology, Ltd. LXM(tm) - LITECOMM (tm) XMODEM ENGINE for Microsoft Datalight and Turbo C lcxteot _______________________________________________________________ SUMMARY #include #include int lcxtrec(port) unsigned port; DESCRIPTION Send and End of Transmission (End of File) to the receiving system. This function must be called to successfully close out the transmission session. EXAMPLE See the accompanying program TTL for an example of lcxtrec usage. RETURN VALUES Lcxteot returns one of two values, as defined in the litexm.h file. SUCCESS - The receiving station has correctly acknowledged the EOT. The host program terminates transmission mode normally. CAN - Either the receiving system has responded to the EOT message with a CAN code, or has failed to respond correctly and the lcxteot function has sent the CAN code to the receiving system. Host program should handle in an appropriate manner, e.g. issue an appropriate error message. Regardless of the value returned, invoking lcxteot always terminates the current transmission session. CopyRight (c) 1987, 1988 Information Technology, Ltd.