Advanced IO and JBoss Web

Introduction

With usage of the APR API as the basis of its connectors, JBoss Web is able to provide a number of extensions over the regular blocking IO as provided with support for the Servlet API.

IMPORTANT NOTE: Usage of these features requires using the APR HTTP connector. The classic java.io HTTP connector and the AJP connectors do not support them.

This API allows writing Servlets which can process IO asynchronously, receiving events when data is available for reading on the connection (rather than always using a blocking read), and writing data back on connections asynchronously without blocking (most likely responding to some event raised from some other source).

The number of concurrent connections that can be handled by JBoss Web is constrained by the poller size that has been configured on the connector (see the pollerSize attribute in the APR documentation). JBoss Web uses two separate pollers for keep alive and event processing, so by default it can handle a maximum of 8192 kept alive connections and 8192 event driven connections (for event connections, failure to put a socket in the poller will result in the end of the processing of the connection).

HttpEvent

Servlets which implement the org.jboss.servlet.http.HttpEventServlet interface will have their event method invoked rather than the usual service method. The event type field indicates the event which occurred. The event object gives access to the usual request and response objects. The main difference is that those objects remain valid and fully functional at any time between processing of the BEGIN event until processing an END or ERROR event, so that asynchronous operations are possible. The following event types exist:

EventType.BEGIN - will be called at the beginning of the processing of the connection. It can be used to initialize any relevant fields using the request and response objects. Between the end of the processing of this event, and the beginning of the processing of the end or error events, it is possible to use the response object to write data on the open connection. Note that the response object and dependent OutputStream and Writer are not synchronized, so when they are accessed by multiple threads adequate synchronization is needed. After processing the initial event, the request is considered to be committed.

EventType.READ - This indicates that input data is available, and that at least one read call can be made without blocking. The available and ready methods of the InputStream and Reader may be used to determine if there is a risk of blocking: the servlet must continue reading while data is reported available. When encountering a read error, the servlet should report it by propagating the exception properly. Throwing an exception will cause the error event to be invoked, and the connection will be closed. Alternately, it is also possible to catch any exception, perform clean up on any data structure the servlet may be using, and using the close method of the event. It is not allowed to attempt reading data from the request outside of the event method scope.

EventType.END - End may be called to end the processing of the request. Fields that have been initialized in the begin method should be reset. After this event has been processed, the request and response objects, as well as all their dependent objects will be recycled and used to process other requests. In particular, this event will be called if the HTTP session associated with the connection times out, if the web application is reloaded, if the server is shutdown, or if the connection was closed asynchronously.

EventType.EOF - The end of file of the input has been reached, and no further data is available. This event is sent because it can be difficult to detect otherwise, for example when chunked transfer encoding is being used. Following the processing of this event and the processing of any subsequent event, the event will be automatically suspended, as and no additional read events will be sent.

EventType.ERROR - Error will be called by the container in the case where an IO exception or a similar unrecoverable error occurs, or if the Servlet has thrown an exception during the invocation of the event method. Fields that have been initialized in the begin method should be reset (similar to what should be done when the END event is received). After this event has been processed, the request and response objects, as well as all their dependent objects will be recycled and used to process other requests.

EventType.TIMEOUT - The connection timed out according to the timeout value which has been set (the default value is the timeout value of the connector), but the connection will not be closed unless the servlet uses the close method of the event. The timeout is calculated from the last event sent to the Servlet, any asynchronous writes are not monitored. Depending on the situation, the handling of the event could change.

EventType.EVENT - Event will be called by the container after the resume() method is called, during which any operation can be performed, including closing the connection using the close() method.

EventType.WRITE - Write is sent if the servlet is using the isWriteReady method. This means that the connection is ready to receive data to be written out without blocking. This event will never be received if the servlet is not using the isWriteReady method, or if the isWriteReady method always returns true.

As described above, the typical lifecycle of a request will consist in a series of events such as: BEGIN -> READ -> READ -> READ -> TIMEOUT -> END. At any time, the servlet may end processing of the request by using the close method of the event object.

The close() method ends the request, which marks the end of the processing. This will send back to the client a notice that the server has no more data to send as part of this request. An END event will be sent to the servlet.

The setTimeout() method sets the timeout in milliseconds of idle time on the connection. A timeout occurs if the amount of time since the last event processed by the Servlet is greater than the timeout value. If a timeout occurs, the Servlet will receive an TIMEOUT event which will not result in automatically closing the event (the event may be closed using the close() method as usual).

The isReadReady() method returns true when data may be read from the connection (the flag becomes false if no data is available to read). When the flag becomes false, the Servlet can attempt to read additional data, but it will block until data is available. This method is equivalent to Reader.ready() and (InputStream.available() > 0).

The isWriteReady() method returns true when data may be written to the connection without blocking (the flag becomes false when the client is unable to accept data fast enough). When the flag becomes false, the servlet must stop writing data. If there's an attempt to flush additional data to the client and data still cannot be written immediately, an IOException will be thrown. If calling this method returns false, it will also request notification when the connection becomes available for writing again, and the servlet will receive a write event. Note: If the servlet is not using ready, and is writing its output inside the container threads, using this method is not mandatory, but any incomplete writes will be performed again in blocking mode. Note 2: When using the flag to perform non blocking writes, the Servlet should not be using large buffers, and should rather rely more on the buffering done inside the OutputStream and Writer objects provided through the Servlet API. In particular, a converted char array written through the Writer should be able to fit inside the byte buffer internally used for the Servlet output (which is 8 KB), and can be configured on the ServletResponse object if needed.

The suspend() method suspends processing of the connection until the configured timeout occurs, or resume() is called. In practice, this means the servlet will no longer receive read events. Reading should always be performed synchronously in the container threads unless the connection has been suspended.

The resume() method will cause the servlet container to send a generic event to the servlet, where the request can be processed synchronously (for example, it is possible to use this to complete the request after some asynchronous processing is done). This also resumes read events if they have been disabled using suspend. It is then possible to call suspend again later. It is also possible to call resume without calling suspend before.

The Servlet container will never, under any circumstance, invoke the event method of a Servlet with an event pertaining to a particular connection concurrently. For example, if a READ event is being processed, and some asynchronous thread closes the event using event.close(), an END event will only be called after the Servlet has completed processing of the READ event. However, the application must still synchronize any concurrent access it makes to an event or any objects provided by the Servlet API. The event object remains allocated for the entire duration of the connection, and can safely be used for synchronization purposes. For example, two possibly concurrent writes done on the same connection should likely synchronize on the event object.

When the processing of the END or ERROR event is done, the connection may be closed, and no additional data should be written on the connection. Unsynchronized asynchronous writes may cause stability issues when this happens. It is valid however to add the proper synchronization in the END or ERROR event processing (so that they wait for the asynchronous write loop to get done rather than synchronizing all writes).

HttpEventFilter

Similar to regular filters, a filter chain is invoked when events are processed. These filters should implement the HttpEventFilter interface (which works in the same way as the regular Filter interface), and should be declared and mapped in the deployment descriptor in the same way as a regular filter. The filter chain when processing an event will only include filters which match all the usual mapping rules, and also implement the HttpEventFiler interface.

Servlet 3.0 Async

Servlet 3.0 introduces async support to regular Servlets, roughly equivalent to this event API excluding the events generated from actual IO (READ and WRITE). Internally, async support is implemented using the event functionality, and presented to the user through the Servlet API.

Example code

The following pseudo code servlet implements asynchronous chat functionality using the API described above:

public class ChatServlet
    extends HttpServlet implements HttpEventServlet {

    protected ArrayList<HttpServletResponse> connections = 
        new ArrayList<HttpServletResponse>();
    protected MessageSender messageSender = null;
    
    public void init() throws ServletException {
        messageSender = new MessageSender();
        Thread messageSenderThread = 
            new Thread(messageSender, "Sender[" + getServletContext().getContextPath() + "]");
        messageSenderThread.setDaemon(true);
        messageSenderThread.start();
    }

    public void destroy() {
        connections.clear();
        messageSender.stop();
        messageSender = null;
    }

    /**
     * Process the given event.
     * 
     * @param event The event that will be processed
     * @throws IOException
     * @throws ServletException
     */
    public void event(HttpEvent event)
        throws IOException, ServletException {
        HttpServletRequest request = event.getHttpServletRequest();
        HttpServletResponse response = event.getHttpServletResponse();
        switch (event.getType()) {
        case BEGIN:
            log("Begin for session: " + request.getSession(true).getId());
            PrintWriter writer = response.getWriter();
            writer.println("<!doctype html public \"-//w3c//dtd html 4.0 transitional//en\">");
            writer.println("<head><title>JSP Chat</title></head><body bgcolor=\"#FFFFFF\">");
            writer.flush();
            synchronized(connections) {
                connections.add(response);
            }
            break;
        case ERROR:
            log("Error for session: " + request.getSession(true).getId());
            synchronized(connections) {
                connections.remove(response);
            }
            event.close();
            break;
        case END:
            log("End for session: " + request.getSession(true).getId());
            synchronized(connections) {
                connections.remove(response);
            }
            PrintWriter writer = response.getWriter();
            writer.println("</body></html>");
            event.close();
            break;
        case READ:
            InputStream is = request.getInputStream();
            byte[] buf = new byte[512];
            while (is.available() > 0) {
                int n = is.read(buf); //can throw an IOException
                if (n > 0) {
                    log("Read " + n + " bytes: " + new String(buf, 0, n) 
                            + " for session: " + request.getSession(true).getId());
                } else {
                    error(event, request, response);
                    return;
                }
            }
        }
    }

    public class MessageSender implements Runnable {

        protected boolean running = true;
        protected ArrayList<String> messages = new ArrayList<String>();
        
        public MessageSender() {
        }
        
        public void stop() {
            running = false;
        }

        /**
         * Add message for sending.
         */
        public void send(String user, String message) {
            synchronized (messages) {
                messages.add("[" + user + "]: " + message);
                messages.notify();
            }
        }

        public void run() {

            while (running) {

                if (messages.size() == 0) {
                    try {
                        synchronized (messages) {
                            messages.wait();
                        }
                    } catch (InterruptedException e) {
                        // Ignore
                    }
                }

                synchronized (connections) {
                    String[] pendingMessages = null;
                    synchronized (messages) {
                        pendingMessages = messages.toArray(new String[0]);
                        messages.clear();
                    }
                    // Send any pending message on all the open connections
                    // If a connection backlogs, there will be an exception
                    for (int i = 0; i < connections.size(); i++) {
                        try {
                            PrintWriter writer = connections.get(i).getWriter();
                            for (int j = 0; j < pendingMessages.length; j++) {
                                writer.println(pendingMessages[j] + "<br>");
                            }
                            writer.flush();
                        } catch (IOException e) {
                            log("IOExeption sending message", e);
                        }
                    }
                }

            }

        }

    }

}

Tomcat Bayeux API

JBoss Web supports the Apache Tomcat Bayeux API which provides a ready to use toolkit for building simple Comet enabled web applications.

Asynchronous writes

When APR is enabled, JBoss Web supports using sendfile to send large static files. These writes, as soon as the system load increases, will be performed asynchronously in the most efficient way. Instead of sending a large response using blocking writes, it is possible to write content to a static file, and write it using a sendfile code. A caching valve could take advantage of this to cache the response data in a file rather than store it in memory. Sendfile support is available if the request attribute org.apache.tomcat.sendfile.support is set to Boolean.TRUE.

Any servlet can instruct JBoss Web to perform a sendfile call by setting the appropriate response attributes. It is also necessary to correctly set the content length for the response. When using sendfile, it is best to ensure that neither the request or response have been wrapped, since as the response body will be sent later by the connector itself, it cannot be filtered. Other than setting the 3 needed request attributes, the servlet should not send any response data, but it may use any method which will result in modifying the response header (like setting cookies).

org.apache.tomcat.sendfile.filename: Canonical filename of the file which will be sent as a String
org.apache.tomcat.sendfile.start: Start offset as a Long
org.apache.tomcat.sendfile.end: End offset as a Long