This page provides answers to several frequently asked questions about the Spark and FreeMarker frameworks:

Do's and Don'ts of web development

Here are a few tips that you should keep in mind when developing web applications in any framework not just Spark and FreeMarker; these are general principles.

Read this article first then read the following three articles and then with that context re-read this article again and it should make more sense. This FAQ is a compression of knowledge and wisdom that could fill a couple of courses in web development and concurrent programming.

This is just scratching the surface, if you have more general questions about web development use the myCourses Discussion forum to post your questions.

How do I know where to put data?

There are roughly three scopes in web development:

Knowing exactly where to place data (information or objects) is the hardest job of system development. Follow these heuristics: (A) Put short-term (request scope) data in the local variables of methods starting at the Route.handler method and all methods it calls. (B) Put long-term data in Model (or Application) tier components. Lastly, you usually put one essential object in the Spark Session that is the root of interaction with the rest of the system; that object is usually some form of User. With that object as a key you can find other application objects within other tiers.

How does an HTTP request really work?

Note: this information will not help you with your coding effort but it is useful to understand the Layers involved in a particular Tier; in this case the UI tier. Your UI components interact with the layer below: Spark. And Spark interacts with the layer below it: Jetty. The layer below Jetty is the Java EE Web specification and core Java and the operating system (OS), of course.

As we talked about in the Web Architecture and Development lesson, HTTP is a network protocol that makes a TCP/IP connection to the server and sends a request. The server then generates a response that is sent back along the TCP connection and the browser renders the HTML in the response body.

The simplified view of an HTTP request.

As you can guess there is quite a lot going on behind the scenes. The following diagram shows a few of the more critical aspects of making and processing an HTTP request.

A detailed look at an HTTP request.

Let's look at each step in the process.

  1. Some user action triggers an HTTP request.
    For example, clicking on a link will trigger a GET HTTP request for the resource at the URL in the href attribute of the link tag.
  2. Request the web page
    The browser has a block of code that initiates the HTTP request by creating a TCP/IP Socket that establishes a network connection to the server.
  3. The HTTP request is sent to the server
    The HTTP request is sent on the output stream of the TCP socket connection. The browser waits for the HTTP response to arrive on the client's input stream.
  4. Jetty's Server awaits HTTP connections
    When the webapp starts up Jetty creates a ServerSocket listening to port 4567. It is this action that turns a regular Java application into a web server.
    When a client attempts to establish a TCP connection the Jetty server will "accept" the connection by creating a server-side Socket. This socket automatically associates the client's output stream to the server's input stream; and the server's output stream to the client's input stream; thus creating a path way from the request to the response.
  5. The Jetty Socket invokes the Spark Servlet
    According to the Java EE Web Specification, a Servlet is a software component that handles an HTTP request. Thus Jetty parses the HTTP input stream to an HttpServletRequest object and connects the output stream to an HttpServletResponse object. These two objects are passed to the Spark Servlet.
    Also at this point Jetty will assign a unique Java thread to process this HTTP request. Thus every invocation of the Spark Servlet will be in its own thread.
  6. The Spark Servlet dispatches to an application Route object
    Based upon the HTTP verb and URL in the HttpServletRequest object Spark will select an appropriate Application layer Route object. It will also wrap the servlet request and response objects in Spark's own Request and Response classes. These two objects are passed to the handle method of the selected Route object.
  7. The Route object uses FreeMaker to render an HTML response
    The UI Controller Route object handles the HTTP request and determines which UI View to render. It delegates rendering to the FreeMaker template engine by passing it the file name of the template and a View-Model map of view attributes.
  8. The HTML is sent back to the browser
    The HTML text that the template engine renders is passed as the return value of the Application layer's Route handle method. The Spark servlet knows how to push that text content onto the HttpServletResponse object that ultimately pushes the text as the body of the HTTP response output stream.
  9. The browser renders the new HTML content
    The HTTP response stream is then parsed by the client's socket on its input stream. This HTML content is then rendered by the browser.
  10. The user enjoys the new HTML view
    Finally, the browser will close the TCP socket connection and that closes the server socket connection. Closing the server-side socket will also free up the thread used by that HTTP request; the thread goes back to a pool for the next HTTP request to be processed.

How does the Jetty web server get started?

As mentioned in the previous FAQ "How does an HTTP request really work?" Jetty is a Java library that provides a server that listens to a TCP port (default: 4567) and handles HTTP requests. Spark builds on top of this library by providing an API for configuring pairs of HTTP verbs (GET, POST, and so on) with URL patterns. For example, in the Guessing Game web application you use the Spark get and post methods to build the routes of the webapp. 

import static spark.Spark.*;
import spark.TemplateEngine;
import spark.template.freemarker.FreeMarkerEngine;

public class HelloApp {
  public static void main(String[] args) {
    final TemplateEngine templateEngine = new FreeMarkerEngine();
    get("/", new GetHomeRoute(templateEngine));
    post("/hello", new PostHelloRoute(templateEngine));
  }
}

The Spark get method call on line 8 adds the first route of the application that "ignites" the Spark framework. The SparkServer uses dependency injection to have the appropriate Jetty Handler that knows how to dispatch HTTP requests to Spark router handlers. That is the connection between the HTTP/TCP processing by Jetty and the Spark framework. Ultimately, the Jetty server passes up each HTTP request that Spark then "routes" to Router objects based upon the HTTP verb and URL combination.

How to share a View between multiple UI Controllers?

There are times when the result of a given UI Controller is to render a View that is also generated by another existing UI Controller. As the developer you have a few choices. First you could copy the code that builds the Model-View data map from the original Controller into the new Controller. This leads to duplicated code and thus is not very DRY.

Alternatively, you can use an HTTP trick to tell the browser to render the view by redirecting the browser to perform an additional HTTP request that invokes the other Controller that already knows how to render this View. How to perform a redirect is the next FAQ.

There is a third possibility: You can put this common code into a parent class abstraction or in a helper class.

How do you do a redirect?

First you have to ask yourself: What is an HTTP redirect?. It is when the Route handler for one URL pattern responds by telling the browser to navigate to a different URL; thus a re-direction.

To demonstrate Spark code for doing an HTTP redirect we will illustrate the use case described in the previous FAQ: delegating View rendering to another Controller. Imagine you are building an eCommerce webapp what happens if the user clicks the Home link while filling-out an order? Wouldn't you want the user to be sent back to the order form? Let's assume the answer is yes. 

public class GetHomeRoute implements Route {

  public String handle(Request request, Response response) {
    // retrieve the HTTP session
    final Session httpSession = request.session();
    // retrieve an Order, if in-progress
    final Order order = shoppingCart.getActiveOrder();

    // if there is no active order
    if (order == null) {
      // render the Home page with plenty of product placement
      return templateEngine.render(new ModelAndView(vm, VIEW_NAME));
    }
    else {
      // there is an active order so go back to the Order form
      // so redirect the user to the Game view
      response.redirect(WebServer.ORDER_FORM_URL);
      halt();
      return null;
    }
  }
}
In reality this scenario has dubious value and could really frustrate the customer, but this is just a mock example.

The redirect magic occurs on lines 17-19. The redirect method on the Response object add an HTTP header to give the browser a new Location to navigate to. Recall that Spark is expecting that the return value of a Route is the body of the response (an HTML page or a JSON object) but in this situation there is no body. You might think that the null returned by this Route's handle method tells Spark that there is no HTTP response body but that is not exactly true. You see the Spark halt method actually throws an exception that the Spark framework catches and handles; thus circumventing the body generation step of the handling the HTTP request. There, now aren't you glad you asked?

How to share information across an HTTP redirect?

If you do choose to use an HTTP redirect to have a different UI Controller generate the UI View, there is one major gotcha: Frequently you might need to send the user a message. But the other UI Controller won't know about this message.

First let's look at a sketch of the Controller doing the redirect: 

public class PostAddItemToCartRoute implements Route {

  public String handle(Request request, Response response) {
    // retrieve the HTTP session
    final Session httpSession = request.session();

    // process adding a line item to the user's shopping cart
    // NOT SHOWN
    
    // create the user message and put into Session scope
    final String myMessage = ""; // MESSAGE NOT SHOWN
    httpSession.attribute("myMessage", myMessage);

    // use the GetShoppingCartRoute controller to display the View
    response.redirect("/shoppingCart");
    halt();
    return null;
  }
}

Now let's look at a sketch of the Controller handling the redirect: 

public class GetShoppingCartRoute implements Route {

  public String handle(Request request, Response response) {
    // retrieve the HTTP session
    final Session httpSession = request.session();

    // start building the View-Model
    final Map<String, Object> vm = new HashMap<>();
    vm.put(TITLE_ATTR, TITLE);
    
    // is there a session-scoped message to display?
    final String myMessage = httpSession.attribute("myMessage");
    if (myMessage != null) {
      vm.put("message", myMessage);
      httpSession.removeAttribute("myMessage");
    }
    
    // render the View
    return templateEngine.render(new ModelAndView(vm, VIEW_NAME));
  }
}

The solution is to use the Session scope to store (temporarily) a single message object. The primary Controller generates the message and stores it on the Session. Then that Controller performs a redirect that another Controller will handle (in a different HTTP request thread). This second Controller looks into the Session to see if an outstanding message has been put on the Session scope. If such a message exists it is added to the View-Model attribute map and then removes the message from the Session. The second action is necessary so the same message isn't displayed too many times.

How does a View refresh it's data?

Imagine you have a View that displays a list of users. Once the view has been generated by the server and rendered by the browser, that list of users can become stale as new users sign-in and other sign-out. What you want is for the View to update itself periodically so that the current user sees a fresh set of data every few seconds.

The solution has nothing to do with any server-side code but the refresh must be requested by the browser. This can be accomplished with a meta tag in the HTML of the View; like this: 

<!DOCTYPE html>
<head>
    <meta http-equiv="refresh" content="10">
    <title>${title} | My App</title>
</head>
<body>
  <!-- BODY NOT SHOWN -->
</body>
</html>

Line 3 is the HTML metadata tag that tells the browser that this View would be refreshed after 10 seconds.

There is one gotcha with this technique. The HTTP request that generated this response becomes subsequent HTTP requests to perform the refresh. Thus if this View was generated from a POST /someAction request then the browser will attempt to re-POST the same URL (and form data if applicable) and this is usually a bad idea because a POST request usually makes some change to the backend data and typically you don't want to repeat these changes. Therefore, if you need to generate a View that refreshes from a POST UI Controller then that Controller should perform a redirect to an HTTP GET request that renders the View that can then be refreshed from that same GET HTTP Route component.

There are other techniques for making sure a page has up-to-date information such as Ajax and Web Sockets. However, we urge teams to use the easiest technique to accomplish this goal; the HTTP refresh meta tag is by far the easiest, no coding required.

How can I connect multiple users to my application?

To thoroughly test your application, you will need to have multiple users working with the application. Each user will have to connect to the server in a different HTTP session. The different sessions are needed so that the Spark framework will use a different Session object for each connection and your application can distinguish between the different users when your routes process HTTP requests.

Here are several ways that you can get multiple connections to use different sessions:

How do you handle when a client connection breaks?

There are two scenarios relevant to the concept of a "broken HTTP connection". First, the user initiates an HTTP request and then immediately kills the browser's tab or exits the browser application altogether. In either situation, the server only knows that the TCP connection has been severed. The web server then closes the TCP socket and terminates the Java thread that is processing that HTTP request.

At no time will the Application layer code be aware of this situation. So there is nothing that you need to code to handle a broken connection. However, continue reading the next FAQ question.

How to recognize (handle) session timeout?

In the "How do you handle when a client connection breaks?" FAQ above we talked about what happens when an HTTP connection is lost. What is more interesting is the other side of the question: How do you know when a user has left the application running on the browser but has walked away?

The Java EE Servlet specification provides a wide variety of events that a web application can listen to. Unfortunately, Spark doesn't provide any hooks to register such event listeners. However, there is one type of event listener that doesn't require an explicit event registration hook: HttpSessionBindingListener. This is a Java interface that you implement on your own class and whenever an object of this type is added to the session Jetty will fire the valueBound method of this interface. When the object is removed from the session the valueUnbound method is called.

The next piece of information is that every session has a built-in timeout. The Jetty web server will track the timeout count-down and when the inactivity has lasted past the timeout threshold then Jetty will invalidate that user's session. Part of invalidating a session Jetty will remove (un-bind) each session attribute. This is how the binding listener API can be used to discover when a session times-out.

So what you can do is create a UI tier helper class that contains some Application tier component that can handle the session timeout. Here is an example session timeout watchdog helper class that uses the PlayerServices object to handle the end of the player's session and do any cleanup. 

public class SessionTimeoutWatchdog implements HttpSessionBindingListener {
  private static final Logger LOG = Logger.getLogger(SessionTimeoutWatchdog.class.getName());

  private final PlayerServices playerServices;

  public SessionTimeoutWatchdog(final PlayerServices playerServices) {
    LOG.fine("Watch dog created.");
    this.playerServices = Objects.requireNonNull(playerServices);
  }

  @Override
  public void valueBound(HttpSessionBindingEvent event) {
    // ignore this event
    LOG.fine("Player session started.");
  }

  @Override
  public void valueUnbound(HttpSessionBindingEvent event) {
    // the session is being terminated do some cleanup
    playerServices.endSession();
    //
    LOG.fine("Player session ended.");
  }
}

So to use this helper class you just need to add a single instance of this watchdog component to the session when the session is first created (see line 12). 

public class GetHomeRoute implements Route {

  public String handle(Request request, Response response) {
    // retrieve the HTTP session
    final Session httpSession = request.session();

    // if this is a brand new browser session
    if (httpSession.isNew()) {
      // get the object that will provide client-specific services for this player
      final PlayerServices playerService = gameCenter.newPlayerServices();
      httpSession.attribute(PLAYERSERVICES_KEY, playerService);
      httpSession.attribute(TIMEOUT_WATCHDOG_KEY, new SessionTimeoutWatchdog(playerService));
      // FIXME remove this line after testing the timeout hook with a short session lifetime
      httpSession.maxInactiveInterval(60);  // session terminates after 1 minute of inactivity
      
      // render the Home view
      // CODE NOT SHOWN
    }
    else {
      // redirect to Game View
      // CODE NOT SHOWN
    }
  }
}

Notice also line 14; here the code is explicitly declaring that the timeout threshold is 60 seconds. I did this just to test this watchdog component without having to wait a long time. BTW: Jetty does not have a default timeout; meaning sessions never timeout by default. So you can use code like line 14 to set a desired timeout for each newly created session. 30 minutes is a common timeout threshold; don't make the timeout too short.

How do I keep data across server restarts?

All of your data will disappear when you restart the application. How do I keep data across server restarts? The short answer is: You don't. At least for this project, you are not required to do that. In a full-blown Enterprise-scale web application you would use some form of database, but for this project we decided to eliminate the need for persistent storage requiring integration with a database, which is a big undertaking.

Do's and Dont's of Ajax Development

As mentioned in the Web Architecture and Development lecture slides, Ajax is the process sending an HTTP request that does not render and HTML page but rather provides an action that returns a JSON object representation.