Windows via C/C++: Synchronous and Asynchronous Device I/O

I can’t stress enough the importance of this chapter, which covers the Microsoft Windows technologies that enable you to design high-performance, scalable, responsive, and robust applications. A scalable application handles a large number of concurrent operations as efficiently as it handles a small number of concurrent operations. For a service application, typically these operations are processing client requests that arrive at unpredictable times and require an unpredictable amount of processing power. These requests usually arrive from I/O devices such as network adapters; processing the requests frequently requires additional I/O devices such as disk files.

In Microsoft Windows applications, threads are the best facility available to help you partition work. Each thread is assigned to a processor, which allows a multiprocessor machine to execute multiple operations simultaneously, increasing throughput. When a thread issues a synchronous device I/O request, the thread is temporarily suspended until the device completes the I/O request. This suspension hurts performance because the thread is unable to do useful work, such as initiate another client’s request for processing. So, in short, you want to keep your threads doing useful work all the time and avoid having them block.

To help keep threads busy, you need to make your threads communicate with one another about the operations they will perform. Microsoft has spent years researching and testing in this area and has developed a finely tuned mechanism to create this communication. This mechanism, called the I/O completion port, can help you create high-performance, scalable applications. By using the I/O completion port, you can make your application’s threads achieve phenomenal throughput by reading and writing to devices without waiting for the devices to respond.

The I/O completion port was originally designed to handle device I/O, but over the years, Microsoft has architected more and more operating system facilities that fit seamlessly into the I/O completion port model. One example is the job kernel object, which monitors its processes and sends event notifications to an I/O completion port. The Job Lab sample application detailed in Chapter 5, “Jobs,” demonstrates how I/O completion ports and job objects work together.

Throughout my many years as a Windows developer, I have found more and more uses for the I/O completion port, and I feel that every Windows developer must fully understand how the I/O completion port works. Even though I present the I/O completion port in this chapter about device I/O, be aware that the I/O completion port doesn’t have to be used with device I/O at all–simply put, it is an awesome interthread communication mechanism with an infinite number of uses.

From this fanfare, you can probably tell that I’m a huge fan of the I/O completion port. My hope is that by the end of this chapter, you will be too. But instead of jumping right into the details of the I/O completion port, I’m going to explain what Windows originally offered developers for device I/O. This will give you a much greater appreciation for the I/O completion port. In “I/O Completion Ports” on page 320 I’ll discuss the I/O completion port.

Opening and Closing Devices

One of the strengths of Windows is the sheer number of devices that it supports. In the context of this discussion, I define a device to be anything that allows communication. Table 10-1 lists some devices and their most common uses.

Table 10-1 Various Devices and Their Common Uses

Device	Most Common Use
File	Persistent storage of arbitrary data
Directory	Attribute and file compression settings
Logical disk drive	Drive formatting
Physical disk drive	Partition table access
Serial port	Data transmission over a phone line
Parallel port	Data transmission to a printer
Mailslot	One-to-many transmission of data, usually over a network to a machine running Windows
Named pipe	One-to-one transmission of data, usually over a network to a machine running Windows
Anonymous pipe	One-to-one transmission of data on a single machine (never over the network)
Socket	Datagram or stream transmission of data, usually over a network to any machine supporting sockets (The machine need not be running Windows.)
Console	A text window screen buffer

This chapter discusses how an application’s threads communicate with these devices without waiting for the devices to respond. Windows tries to hide device differences from the software developer as much as possible. That is, once you open a device, the Windows functions that allow you to read and write data to the device are the same no matter what device you are communicating with. Although only a few functions are available for reading and writing data regardless of the device, devices are certainly different from one another. For example, it makes sense to set a baud rate for a serial port, but a baud rate has no meaning when using a named pipe to communicate over a network (or over the local machine). Devices are subtly different from one another, and I will not attempt to address all their nuances. However, I will spend some time addressing files because files are so common. To perform any type of I/O, you must first open the desired device and get a handle to it. The way you get the handle to a device depends on the particular device. Table 10-2 lists various devices and the functions you should call to open them.

Table 10-2 Functions for Opening Various Devices

Device	Function Used to Open the Device
File	CreateFile (pszName is pathname or UNC pathname).
Directory	CreateFile (pszName is directory name or UNC directory name). Windows allows you to open a directory if you specify the FILE_FLAG_BACKUP_SEMANTICS flag in the call to CreateFile. Opening the directory allows you to change the directory’s attributes (to normal, hidden, and so on) and its time stamp.
Logical disk drive	CreateFile (pszName is "\\.\x:"). Windows allows you to open a logical drive if you specify a string in the form of "\\.\x:" where x is a drive letter. For example, to open drive A, you specify "\\.\A:". Opening a drive allows you to format the drive or determine the media size of the drive.
Physical disk drive	CreateFile (pszName is "\\.\PHYSICALDRIVEx"). Windows allows you to open a physical drive if you specify a string in the form of "\\.\PHYSICALDRIVEx" where x is a physical drive number. For example, to read or write to physical sectors on the user’s first physical hard disk, you specify "\\.\PHYSICALDRIVE0". Opening a physical drive allows you to access the hard drive’s partition tables directly. Opening the physical drive is potentially dangerous; an incorrect write to the drive could make the disk’s contents inaccessible by the operating system’s file system.
Serial port	CreateFile (pszName is "COMx").
Parallel port	CreateFile (pszName is "LPTx").
Mailslot server	CreateMailslot (pszName is "\\.\mailslot\mailslotname").
Mailslot client	CreateFile (pszName is "\\servername\mailslot\mailslotname").
Named pipe server	CreateNamedPipe (pszName is "\\.\pipe\pipename").
Named pipe client	CreateFile (pszName is "\\servername\pipe\pipename").
Anonymous pipe	CreatePipe client and server.
Socket	socket, accept, or AcceptEx.
Console	CreateConsoleScreenBuffer or GetStdHandle.

Each function in Table 10-2 returns a handle that identifies the device. You can pass the handle to various functions to communicate with the device. For example, you call SetCommConfig to set the baud rate of a serial port:

BOOL SetCommConfig(
   HANDLE       hCommDev,
   LPCOMMCONFIG pCC,
   DWORD        dwSize);

And you use SetMailslotInfo to set the time-out value when waiting to read data:

BOOL SetMailslotInfo(
   HANDLE hMailslot,
   DWORD  dwReadTimeout);

As you can see, these functions require a handle to a device for their first argument.

When you are finished manipulating a device, you must close it. For most devices, you do this by calling the very popular CloseHandle function:

BOOL CloseHandle(HANDLE hObject);

However, if the device is a socket, you must call closesocket instead:

int closesocket(SOCKET s);

Also, if you have a handle to a device, you can find out what type of device it is by calling GetFileType:

DWORD GetFileType(HANDLE hDevice);

All you do is pass to the GetFileType function the handle to a device, and the function returns one of the values listed in Table 10-3.

Table 10-3 Values Returned by the GetFileType Function

Value	Description
FILE_TYPE_UNKNOWN	The type of the specified file is unknown.
FILE_TYPE_DISK	The specified file is a disk file.
FILE_TYPE_CHAR	The specified file is a character file, typically an LPT device or a console.
FILE_TYPE_PIPE	The specified file is either a named pipe or an anonymous pipe.

A Detailed Look at CreateFile

The CreateFile function, of course, creates and opens disk files, but don’t let the name fool you–it opens lots of other devices as well:

HANDLE CreateFile(
   PCTSTR pszName,
   DWORD  dwDesiredAccess,
   DWORD  dwShareMode,
   PSECURITY_ATTRIBUTES psa,
   DWORD  dwCreationDisposition,
   DWORD  dwFlagsAndAttributes,
   HANDLE hFileTemplate);

As you can see, CreateFile requires quite a few parameters, allowing for a great deal of flexibility when opening a device. At this point, I’ll discuss all these parameters in detail.

When you call CreateFile, the pszName parameter identifies the device type as well as a specific instance of the device.

The dwDesiredAccess parameter specifies how you want to transmit data to and from the device. You can pass these four generic values, which are described in Table 10-4. Certain devices allow for additional access control flags. For example, when opening a file, you can specify access flags such as FILE_READ_ATTRIBUTES. See the Platform SDK documentation for more information about these flags.

Table 10-4 Generic Values That Can Be Passed for CreateFile’s dwDesiredAccess Parameter

Value	Meaning
0	You do not intend to read or write data to the device. Pass 0 when you just want to change the device’s configuration settings–for example, if you want to change only a file’s time stamp.
GENERIC_READ	Allows read-only access from the device.
GENERIC_WRITE	Allows write-only access to the device. For example, this value can be used to send data to a printer and by backup software. Note that GENERIC_WRITE does not imply GENERIC_READ.
GENERIC_READ | GENERIC_WRITE	Allows both read and write access to the device. This value is the most common because it allows the free exchange of data.

The dwShareMode parameter specifies device-sharing privileges. It controls how the device can be opened by additional calls to CreateFile while you still have the device opened yourself (that is, you haven’t closed the device yet by calling CloseHandle). Table 10-5 describes the possible values that can be passed for the dwShareMode parameter.

Table 10-5 Values Related to I/O That Can Be Passed for CreateFile’s dwShareMode Parameter

Value	Meaning
0	You require exclusive access to the device. If the device is already opened, your call to CreateFile fails. If you successfully open the device, future calls to CreateFile fail.
FILE_SHARE_READ	You require that the data maintained by the device can’t be changed by any other kernel object referring to this device. If the device is already opened for write or exclusive access, your call to CreateFile fails. If you successfully open the device, future calls to CreateFile fail if GENERIC_WRITE access is requested.
FILE_SHARE_WRITE	You require that the data maintained by the device can’t be read by any other kernel object referring to this device. If the device is already opened for read or exclusive access, your call to CreateFile fails. If you successfully open the device, future calls to CreateFile fail if GENERIC_READ access is requested.
FILE_SHARE_READ | FILE_SHARE_WRITE	You don’t care if the data maintained by the device is read or written to by any other kernel object referring to this device. If the device is already opened for exclusive access, your call to CreateFile fails. If you successfully open the device, future calls to CreateFile fail when exclusive read, exclusive write, or exclusive read/write access is requested.
FILE_SHARE_DELETE	You don’t care if the file is logically deleted or moved while you are working with the file. Internally, Windows marks a file for deletion and deletes it when all open handles to the file are closed.

The psa parameter points to a SECURITY_ATTRIBUTES structure that allows you to specify security information and whether or not you’d like CreateFile’s returned handle to be inheritable. The security descriptor inside this structure is used only if you are creating a file on a secure file system such as NTFS; the security descriptor is ignored in all other cases. Usually, you just pass NULL for the psa parameter, indicating that the file is created with default security and that the returned handle is noninheritable.

The dwCreationDisposition parameter is most meaningful when CreateFile is being called to open a file as opposed to another type of device. Table 10-6 lists the possible values that you can pass for this parameter.

Table 10-6 Values That Can Be Passed for CreateFile’s dwCreationDisposition Parameter

Value	Meaning
CREATE_NEW	Tells CreateFile to create a new file and to fail if a file with the same name already exists.
CREATE_ALWAYS	Tells CreateFile to create a new file regardless of whether a file with the same name already exists. If a file with the same name already exists, CreateFile overwrites the existing file.
OPEN_EXISTING	Tells CreateFile to open an existing file or device and to fail if the file or device doesn’t exist.
OPEN_ALWAYS	Tells CreateFile to open the file if it exists and to create a new file if it doesn’t exist.
TRUNCATE_EXISTING	Tells CreateFile to open an existing file, truncate its size to 0 bytes, and fail if the file doesn’t already exist.

CreateFile’s dwFlagsAndAttributes parameter has two purposes: it allows you to set flags that fine-tune the communication with the device, and if the device is a file, you also get to set the file’s attributes. Most of these communication flags are signals that tell the system how you intend to access the device. The system can then optimize its caching algorithms to help your application work more efficiently. I’ll describe the communication flags first and then discuss the file attributes.

CreateFile Cache Flags

This section describes the various CreateFile cache flags, focusing on file system objects. For other kernel objects such as mailslots, you should refer to the MSDN documentation to get more specific details.

FILE_FLAG_NO_BUFFERING

This flag indicates not to use any data buffering when accessing a file. To improve performance, the system caches data to and from disk drives. Normally, you do not specify this flag, and the cache manager keeps recently accessed portions of the file system in memory. This way, if you read a couple of bytes from a file and then read a few more bytes, the file’s data is most likely loaded in memory and the disk has to be accessed only once instead of twice, greatly improving performance. However, this process does mean that portions of the file’s data are in memory twice: the cache manager has a buffer, and you called some function (such as ReadFile) that copied some of the data from the cache manager’s buffer into your own buffer.

When the cache manager is buffering data, it might also read ahead so that the next bytes you’re likely to read are already in memory. Again, speed is improved by reading more bytes than necessary from the file. Memory is potentially wasted if you never attempt to read further in the file. (See the FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_RANDOM_ACCESS flags, discussed next, for more about reading ahead.)

By specifying the FILE_FLAG_NO_BUFFERING flag, you tell the cache manager that you do not want it to buffer any data–you take on this responsibility yourself! Depending on what you’re doing, this flag can improve your application’s speed and memory usage. Because the file system’s device driver is writing the file’s data directly into the buffers that you supply, you must follow certain rules:

• You must always access the file by using offsets that are exact multiples of the disk volume’s sector size. (Use the GetDiskFreeSpace function to determine the disk volume’s sector size.)

• You must always read/write a number of bytes that is an exact multiple of the sector size.

• You must make sure that the buffer in your process’ address space begins on an address that is integrally divisible by the sector size.

FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_RANDOM_ACCESS

These flags are useful only if you allow the system to buffer the file data for you. If you specify the FILE_FLAG_NO_BUFFERING flag, both of these flags are ignored.

If you specify the FILE_FLAG_SEQUENTIAL_SCAN flag, the system thinks you are accessing the file sequentially. When you read some data from the file, the system actually reads more of the file’s data than the amount you requested. This process reduces the number of hits to the hard disk and improves the speed of your application. If you perform any direct seeks on the file, the system has spent a little extra time and memory caching data that you are not accessing. This is perfectly OK, but if you do it often, you’d be better off specifying the FILE_FLAG_RANDOM_ACCESS flag. This flag tells the system not to pre-read file data.

To manage a file, the cache manager must maintain some internal data structures for the file–the larger the file, the more data structures required. When working with extremely large files, the cache manager might not be able to allocate the internal data structures it requires and will fail to open the file. To access extremely large files, you must open the file using the FILE_FLAG_NO_BUFFERING flag.

FILE_FLAG_WRITE_THROUGH

This is the last cache-related flag. It disables intermediate caching of file-write operations to reduce the potential for data loss. When you specify this flag, the system writes all file modifications directly to the disk. However, the system still maintains an internal cache of the file’s data, and file-read operations use the cached data (if available) instead of reading data directly from the disk. When this flag is used to open a file on a network server, the Windows file-write functions do not return to the calling thread until the data is written to the server’s disk drive.

That’s it for the buffer-related communication flags. Now let’s discuss the remaining communication flags.

Miscellaneous CreateFile Flags

This section describes the other flags that exist to customize CreateFile behaviors outside of caching.

FILE_FLAG_DELETE_ON_CLOSE

Use this flag to have the file system delete the file after all handles to it are closed. This flag is most frequently used with the FILE_ATTRIBUTE_TEMPORARY attribute. When these two flags are used together, your application can create a temporary file, write to it, read from it, and close it. When the file is closed, the system automatically deletes the file–what a convenience!

FILE_FLAG_BACKUP_SEMANTICS

Use this flag in backup and restore software. Before opening or creating any files, the system normally performs security checks to be sure that the process trying to open or create a file has the requisite access privileges. However, backup and restore software is special in that it can override certain file security checks. When you specify the FILE_FLAG_BACKUP_SEMANTICS flag, the system checks the caller’s access token to see whether the Backup/Restore File and Directories privileges are enabled. If the appropriate privileges are enabled, the system allows the file to be opened. You can also use the FILE_FLAG_BACKUP_SEMANTICS flag to open a handle to a directory.

FILE_FLAG_POSIX_SEMANTICS

In Windows, filenames are case-preserved, whereas filename searches are case-insensitive. However, the POSIX subsystem requires that filename searches be case-sensitive. The FILE_FLAG_POSIX_SEMANTICS flag causes CreateFile to use a case-sensitive filename search when creating or opening a file. Use the FILE_FLAG_POSIX_SEMANTICS flag with extreme caution–if you use it when you create a file, that file might not be accessible to Windows applications.

FILE_FLAG_OPEN_REPARSE_POINT

In my opinion, this flag should have been called FILE_FLAG_IGNORE_REPARSE_POINT because it tells the system to ignore the file’s reparse attribute (if it exists). Reparse attributes allow a file system filter to modify the behavior of opening, reading, writing, and closing a file. Usually, the modified behavior is desired, so using the FILE_FLAG_OPEN_REPARSE_POINT flag is not recommended.

FILE_FLAG_OPEN_NO_RECALL

This flag tells the system not to restore a file’s contents from offline storage (such as tape) back to online storage (such as a hard disk). When files are not accessed for long periods of time, the system can transfer the file’s contents to offline storage, freeing up hard disk space. When the system does this, the file on the hard disk is not destroyed; only the data in the file is destroyed. When the file is opened, the system automatically restores the data from offline storage. The FILE_FLAG_OPEN_NO_RECALL flag instructs the system not to restore the data and causes I/O operations to be performed against the offline storage medium.

FILE_FLAG_OVERLAPPED

This flag tells the system that you want to access a device asynchronously. You’ll notice that the default way of opening a device is synchronous I/O (not specifying FILE_FLAG_OVERLAPPED). Synchronous I/O is what most developers are used to. When you read data from a file, your thread is suspended, waiting for the information to be read. Once the information has been read, the thread regains control and continues executing.

Because device I/O is slow when compared with most other operations, you might want to consider communicating with some devices asynchronously. Here’s how it works: Basically, you call a function to tell the operating system to read or write data, but instead of waiting for the I/O to complete, your call returns immediately, and the operating system completes the I/O on your behalf using its own threads. When the operating system has finished performing your requested I/O, you can be notified. Asynchronous I/O is the key to creating high-performance, scalable, responsive, and robust applications. Windows offers several methods of asynchronous I/O, all of which are discussed in this chapter.

File Attribute Flags

Now it’s time to examine the attribute flags for CreateFile’s dwFlagsAndAttributes parameter, described in Table 10-7. These flags are completely ignored by the system unless you are creating a brand new file and you pass NULL for CreateFile’s hFileTemplate parameter. Most of the attributes should already be familiar to you.

Table 10-7 File Attribute Flags That Can Be Passed for CreateFile’s dwFlagsAndAttributes Parameter

Flag	Meaning
FILE_ATTRIBUTE_ARCHIVE	The file is an archive file. Applications use this flag to mark files for backup or removal. When CreateFile creates a new file, this flag is automatically set.
FILE_ATTRIBUTE_ENCRYPTED	The file is encrypted.
FILE_ATTRIBUTE_HIDDEN	The file is hidden. It won’t be included in an ordinary directory listing.
FILE_ATTRIBUTE_NORMAL	The file has no other attributes set. This attribute is valid only when it’s used alone.
FILE_ATTRIBUTE_NOT_CONTENT_INDEXED	The file will not be indexed by the content indexing service.
FILE_ATTRIBUTE_OFFLINE	The file exists, but its data has been moved to offline storage. This flag is useful for hierarchical storage systems.
FILE_ATTRIBUTE_READONLY	The file is read-only. Applications can read the file but can’t write to it or delete it.
FILE_ATTRIBUTE_SYSTEM	The file is part of the operating system or is used exclusively by the operating system.
FILE_ATTRIBUTE_TEMPORARY	The file’s data will be used only for a short time. The file system tries to keep the file’s data in RAM rather than on disk to keep the access time to a minimum.

Use FILE_ATTRIBUTE_TEMPORARY if you are creating a temporary file. When CreateFile creates a file with the temporary attribute, CreateFile tries to keep the file’s data in memory instead of on the disk. This makes accessing the file’s contents much faster. If you keep writing to the file and the system can no longer keep the data in RAM, the operating system will be forced to start writing the data to the hard disk. You can improve the system’s performance by combining the FILE_ATTRIBUTE_TEMPORARY flag with the FILE_FLAG_DELETE_ON_CLOSE flag (discussed earlier). Normally, the system flushes a file’s cached data when the file is closed. However, if the system sees that the file is to be deleted when it is closed, the system doesn’t need to flush the file’s cached data.

In addition to all these communication and attribute flags, a number of flags allow you to control the security quality of service when opening a named-pipe device. Because these flags are specific to named pipes only, I will not discuss them here. To learn about them, please read about the CreateFile function in the Platform SDK documentation.

CreateFile’s last parameter, hFileTemplate, identifies the handle of an open file or is NULL. If hFileTemplate identifies a file handle, CreateFile ignores the attribute flags in the dwFlagsAndAttributes parameter completely and uses the attributes associated with the file identified by hFileTemplate. The file identified by hFileTemplate must have been opened with the GENERIC_READ flag for this to work. If CreateFile is opening an existing file (as opposed to creating a new file), the hFileTemplate parameter is ignored.

If CreateFile succeeds in creating or opening a file or device, the handle of the file or device is returned. If CreateFile fails, INVALID_HANDLE_VALUE is returned.