Create a subclass of DefaultTableCellRenderer. Override the method
getTableCellRendererComponent() to call the superclass version. Cast
its return value to JLabel and call setBorder() on it before returning
it yourself, to set the desired border. (See the API docs for the
Border interface to learn how to set a border).

Then you must set the table to use your custom renderer to render
table header cells. Assuming that your renderer subclass is called
MyHeaderCellRenderer, and your JTable instance is in a variable called
myTable, do this:

   myTable.getTableHeader().setDefaultRenderer(new
MyHeaderCellRenderer());