Introduction

For many years, we have been using Java Threads for developing numerous Client-Server based applications. While it's always desirable to have a highly responsive Client application, handling a high volume of client requests has always been a prime goal of any Server application. However, designing a highly-scalable server requires lots of analysis. Without careful design and adherence to a well thought out underlying policy, a thread based system can fail to produce the desired outcome.

In this article, we'll discuss certain design principles that should be followed when the objective is to build an efficient, thread-safe application.

Thread Safety lies in Domain

As a design principle of thread safety, the vital point to look into is "Domain First." After all, the whole structure of an application lies on the Domain itself. If you get your domain model right, most of the problems are automatically taken care of. A model represents certain aspects of business logic consisting of various constraints and invariants. Understanding the pre and post conditions of a model are much needed in order to make the realized application thread-safe.

Let's imagine an travel agency that arranges budgeted cars for their traveler guests. They need an interactive car rental application that allows them to book multiple cars at different tourist locations on different dates within a specified price range. As they keep making multiple rental requests, at the same time if the system fails to find a car at particular location within price range, they want to be notified with suggested locations on the same screen.

We can think of this as a booking Requester taking the renal requests and putting them in a thread safe collection CarBookings that would be read concurrently by an actual CarFinder process to provide suggested locations. If CarFinder succeeds, it forwards this via the booking engine to the booking rental location. If CarFinder does not succeed, it tries to find the cars based on the price range and nearest location. Once found, it adds the info to another thread safe collection CarBookingAdvices.

There are many thread safe data structures available, but if you're sure about the Thread Safety policy that you want to adhere to, it's best to create a domain model on your own. For example:

 class CarBookings{     private final Set bookings = new HashSet();

    public synchronized void putCar(CarBooking booking){
        bookings.add(booking);
    }

    public synchronized Set getBookings(){
        return Collections.unmodifiableSet(bookings);
    }
}

Since the methods putCar() and getBookings() provide a secure way of accessing the non-thread-safe instance variable bookings, the CarBookings can be used safely by both the BookingRequester and CarFinder process running in different threads. This meets our simple thread safety requirement.

 //Defining BookingAdvice class BookingAdvice{     private final String bookingId="";     private final BigDecimal rate=new BigDecimal("");     private final String location=""; //Getter and Setter ommited }  class BookingAdvices{     private final Set<BookingAdvice> advices = new HashSet<BookingAdvice>();      public synchronized void putAdvice(BookingAdvice advice){         advices.add(advice);     }      public synchronized Set<BookingAdvice> getAdvices(){         return Collections.unmodifiableSet(advices);     } }  //Defining Requester class BookingRequester{     private CarBookings bookings;     private BookingAdvices advices;      public void requestBooking(CarBooking booking){         bookings.putCar(booking);                     //Line 1         displayAdvice(advices.getAdvices());      //Line 2     }      private void displayAdvices(Set<BookingAdvice> advices){         //Displays the advices on the same screen     } }  //Defining Finder class CarFinder{     private BookingAdvices advices;     private CarBookings bookings;      public void processAdvice(){         List<BookingAdvice> adviceList = getGeneratedAdvices();           for(BookingAdvice advice: adviceList)             advices.putAdvice(advice);                //Line 3          findAndProcessBookings(bookings.getBookings()); //Line 4     }      private List<BookingAdvice> getGeneratedAdvices(){         //Generate advice based on requests     }      private void findAndProcessBookings(Set<CarBooking> bookings){         //Finds advice for the booking if required     } }  

Hiding Model State

But, there is a caveat: if the CarBooking class is mutable, then there is a possibility of modifying each CarBooking instance in the set returned by getBookings(). Does it make our model brittle? Well, as long as we can make sure that it won't be modified later on, we'll be fine with that. But, sometimes it's easier get thread safety by hiding the model state and blocking the access to it thereafter. Here, we make CarBooking immutable and add a copy constructor:

 class CarBooking{     private final String bookingId;     private final String location;     private final BigDecimal rate;      //Copy constructor     public CarBooking(CarBooking booking){         this.bookingId= booking.getBookingId();         this.location= booking.getLocation();         this.rate= booking.getRate();     }     //public Getter methods below }  class CarBookings{     //Modified version     public synchronized Set getBookings(){
        Set newBookings = new HashSet();
        for(CarBooking b : bookings){
            CarBooking newBooking = new CarBooking(b);
            newBookings.add(newBooking);
        }
        return Collections.unmodifiableSet(newBookings);
    }
}

So, when getBookings() returns, it returns a copy of the CarBooking, thus not allowing modification of the original state--which makes it purely thread safe.

Do Thread Delegation if you can

Sometimes you might find yourself being overburdened with handling the thread safety in your domain model; in that case you can simply delegate it to a range of Java built-in data structures that are already proven and tested to be thread safe. Like in our example, if we had to use bookings as HashMap<String, CarBooking>, where String could represent bookingId, we could have easily replaced that with ConcurrentHashMap--which is a thread safe HashMap. Likewise there are many other data structures available (under the java.util.concurrent package) like CopyOnWriteArrayList and ConcurrentLinkedQueue, to name a few.

Figuring Single Thread Confinement

To coordinate multiple threads in a lock-based application, the OS and JVM have to burn fair amount of CPU cycles for context switching and thread scheduling--making the system less responsive sometimes.

This can be avoided by allowing only one thread to gain access to work on a unit model (either CarBooking or BookingAdvice) at a time, and also making sure that same unit model should not be re-worked by the same thread. To make this happen, we can employ bounded BlockingQueues (BookingQ and AdviceQ) that would act as a mediator between Requester and CarFinder. So, for a rental request, the Requester would just create a CarBooking and put it in the BookingQ queue, and on the other hand, CarFinder would take that off of the queue. The same thing will be done by the CarFinder to return a BookingAdvice in the AdviceQ queue. This way, multiple Requesters and CarFinders can work asynchronously, making the system highly responsive.

Use synchronized lock wisely

While synchronized lock provides an easy way to work with multiple threads, improper usage of the lock may cause unpredictable results. This becomes obvious when multiple locks are acquired to perform an atomic operation where the first lock is held until last lock operation is performed. Let's consider our BookingRequester and CarFinder that are using simple java Sets instead of CarBookings and BookingAdvices. Here two operations: putting request and getting advices for Requester (vice versa for CarFinder, lines 5,6,7 and 8) become implicitly atomic under the same lock:

 class BookingRequester{     private final Set bookings = new HashSet();
    private final CarFinder finder;

    public BookingRequester(CarFinder cf){
        finder = cf;
    }
    public synchronized void requestBooking(CarBooking booking){
          bookings.add(booking);                    //Line 5

        displayAdvice(finder.getAdvices());    //Line 6
    }

    public synchronized Set getBookings(){
        return bookings;
    }
}
class CarFinder{
    private final Set advices = new HashSet();
    private BookingRequester bookings;

    public CarFinder(BookingRequester br){
        bookings = br;
    }
    public synchronized void processAdvice(){
        List adviceList = getGeneratedAdvices();
          for(Advice advice: adviceList
              advices.add(advice);            //Line 7

        findAndProcessBookings(bookings.getBookings());    //Line 8
    }

    public synchronized Set getAdvices(){
        return advices;
    }

}

Now, a problem starts to creep in when two threads T1 and T2 try to call Requester requestBooking() and CarFinder processAdvice() respectively at the same time. During requestBooking, before T1 calls getAdvices() on CarFinder (Line 6), it has to wait for T2 to complete processAdvice. But to complete processAdvice, T2 needs to call getBookings() on Requester (Line 8), which won't happen until T1 completes requestBooking()--leading to a deadlock situation.

In our previous implementation of BookingRequester and CarFinder, we have already delegated the two operations to be handled independently under different locks (one for CarBookings and another for BookingAdvices, lines 1, 2, 3 and 4) to avoid any deadlock.

But, here we can simply resolve it by moving synchronized from the entire method to only the time required to put the booking (Line 5) and advice (line 7).

Another important point regarding deadlocks would be the order in which the resources are accessed in an atomic operation. If two threads work on an operation with two resource locks being held in a different order, it can lead to a deadlock situation too. So, make sure all threads calling an atomic operation get all resource locks in the same order.

Conclusion

Designing and improvising a thread based application is a challenge. But by following certain design principles and guidance, this can be easily overcome. At the same time, clear understanding of thread safety policy is also essential as it helps you simply the design. There are many other techniques available that we couldn't cover here. However, the principles presented here will always assist you in overcoming some of the thread safety related hurdles you might be facing as you develop thread-safe applications intended for operation on modern multicore and multiprocessor computers.