Mother of a Refactor

Thread view:

VisionSet - 31 Jul 2006 15:56 GMT

I/we have inherited a 'Java' web application that is written in a style
that has to be seen to believed. We need to refactor/rewrite it!

The application is a content management system dealing with orders,
users, questionaires, products etc. Our clients write plain html with
some minimal custom tags that are actually html comments. Our JSP's
parse these in and render new html with relevent forms.

Here is a quick tour of its 'architecture'
JSPs seem to be exclusively written in raw scriptlet and parse the
client html:

// replace tags on the fly
while (data.indexOf("", data.indexOf(contentTag)) + 4);

you get the idea!

Helper classes have no encapuslation and are just a series of heavily
nested hashTables keyed by whatever parameter was convenient at the
time.

Data access is raw and dynamically built sql without even a prepared
statement in sight. MySQL is the DBMS, we don't envisage a change, but
obviously a good architecture won't preclude this.

Decoupling is non-existent, encapsulation is non-existent, the OO
concept has completely escaped the author.

Where the heck do we start?

Obviously we want JSPs written as view only using JSTL.
We should probably have our clients essentially write a minimalist JSP
that we use as @includes?
We don't want any more of the ludicrous custom parsing.
I don't want to write raw SQL, I think hibernate is an option, does it
handle dynamic querying these days, what other option is there.
I don't think we will go down J2EE route, but it is possible.
We may not use Struts, JSF, Tapestry but would like to consider the
most suitable framework before dismissing it.

The team is small, the author has left and half the team is brand new.
I think we should have a period of discovery and analysis and write
some UML analysis to eventually objectify this mess.

The product is on-line, making money but further developement that is
being demanded is obviously painfully slow and hack upon hack.

Any advice on a suitable approach would be appreciated.

TIA
Mike W

Reply to this Message

Andy Dingley - 31 Jul 2006 17:23 GMT

> I/we have inherited a 'Java' web application that is written in a style
> that has to be seen to believed. We need to refactor/rewrite it!

Where's your desk? I think we're on the same project. 8-)

> Where the heck do we start?

Start? Before _ANYTHING_ else, you get automated unit tests running,
together with a good bug track / SVN / ant / automated build /
cruisecontrol / fitnesse setup. Obviously the test coverage will be
negligible at first, and it will be some long time before you have your
heads around Fitnesse. But you do this _before_ you start the
refactoring work, and you make sure it keeps working (dedicate someone
to it full time if you need to).

Don't refactor stuff that isn't already under good control via unit
test. That's the key to making slow progress, but at least guaranteeing
that it _is_ forward progress.

I'm currently working on a very nasty JSP / JDBC project that's showing
the benefits of 2 years of this approach. I can't take credit for it
(the hard bit was done before I arrived) but the results are
impressive. This beast is vile, but we have a dozen coders hacking
away all over it and every build we pack and ship works straight out of
the box. We just don't _ever_ have that problem of, "Sorry, but I
changed the foobar formatting a little bit and now it's killing users".

(and you've read Fowler, I assume)

Reply to this Message

VisionSet - 31 Jul 2006 22:26 GMT

> > I/we have inherited a 'Java' web application that is written in a style
> > that has to be seen to believed. We need to refactor/rewrite it!
[quoted text clipped - 4 lines]
>
> Start? Before _ANYTHING_ else, you get automated unit tests running...

Well by start I didn't mean jumping straight in!

Don't we do some analysis first? How'd we know what to test?

> (and you've read Fowler, I assume)

No, have you a title please?

Thanks Andy.

--
Mike W

Reply to this Message

Patricia Shanahan - 31 Jul 2006 22:50 GMT

>>> I/we have inherited a 'Java' web application that is written in a style
>>> that has to be seen to believed. We need to refactor/rewrite it!
[quoted text clipped - 10 lines]
>
> No, have you a title please?

Martin Fowler, "Refactoring: Improving the Design of Existing Code"
http://www.amazon.com/gp/product/0201485672

One of my all time favorite programming books, and one that I would
reread immediately if I were faced with your problem.

Many of the step-by-step approaches to keeping everything consistent can
be automated by using an IDE with refactoring support, such as Eclipse.

Also, Martin Fowler has a Refactoring page with other resources:
http://www.refactoring.com/

Patricia

Reply to this Message

jmcgill - 01 Aug 2006 01:23 GMT

> Martin Fowler, "Refactoring: Improving the Design of Existing Code"

I suspect Martin himself would, at this point, rewrite the whole thing
in Rails, given the situation described and the kind of application it is.

Just because the project is online and is a revenue stream does *NOT*
mean it's good and is no guarantee that it will *continue* to be a
revenue stream.

Then again, there's not much information in the OP's original complaint.
Maybe it's really not as bad as described. Java does tend to make it
difficult to write *really* bad code.

Reply to this Message

Eric Sosman - 01 Aug 2006 02:29 GMT

>> Martin Fowler, "Refactoring: Improving the Design of Existing Code"
>
> I suspect Martin himself would, at this point, rewrite the whole thing
> in Rails, given the situation described and the kind of application it is.

Isn't this exactly the fate from which component-based
architectures and reusable software were supposed to rescue us?

(In other words: Sooner or later, it all comes down to
understanding and execution, insight and skill. Myself, I've
seen entire fusillades of silver bullets whizz by and hit only
the prearranged demo targets; out in the shadows the vampires
still flit and the werewolves prowl, and only the trusty wooden
stake and well-wielded mallet stand between you and oblivion.)

Signature

Eric Sosman
esosman@acm-dot-org.invalid

Reply to this Message

Patricia Shanahan - 01 Aug 2006 02:56 GMT

>> Martin Fowler, "Refactoring: Improving the Design of Existing Code"
>
[quoted text clipped - 8 lines]
> Maybe it's really not as bad as described. Java does tend to make it
> difficult to write *really* bad code.

It is possible that the program is messed up to the point where only a
rewrite can rescue it.

However, that approach has serious business risks. Either they go on
adding features to the old program without improving its design, or they
freeze the features until the new program is ready.

Adding features without improving its design may work out, or may be
catastrophic. At a minimum, a small team is going to have to divide its
efforts between upgrading the old program and writing the new one.

Freezing also has risks, depending both on the urgency of the new
features compared to the time it will take to get the rewrite up to the
current functionality.

If they choose a refactoring strategy, they can pick out aspects of the
old design that are getting in the way, or need to be changed to support
new features, and selectively refactor those areas.

It's impossible to say which is the better strategy without knowing much
more about both the program and the business situation.

Patricia

Reply to this Message

AndrewMcDonagh - 31 Jul 2006 18:13 GMT

> I/we have inherited a 'Java' web application that is written in a style
> that has to be seen to believed. We need to refactor/rewrite it!
>
> The applicat....

snipped.

But does it work?

If it does, I wouldn't change it. If I need to extend it, I'd try to
isolate those changes from the old code as much as possible - even so
far as to duplicate state.

> Any advice on a suitable approach would be appreciated.
>
> TIA
> Mike W

Reply to this Message

Patricia Shanahan - 31 Jul 2006 19:20 GMT

>> I/we have inherited a 'Java' web application that is written in a
>> style that has to be seen to believed. We need to refactor/rewrite
[quoted text clipped - 9 lines]
> isolate those changes from the old code as much as possible - even
> so far as to duplicate state.

The OP did say "The product is on-line, making money but further
developement that is being demanded is obviously painfully slow and hack
upon hack."

That means freeze-and-forget is not an option.

Also, adding features as barnacles on the side, with duplicated state,
is a sure recipe for exponentially increasing maintenance problems.
Every piece of code that has its own copy of some state is another piece
of code that may need to be modified, and may get forgotten, when that
aspect of the state is involved in a change.

In particular, with scattered SQL construction, I would be worried about
SQL spoofing. It would be hard to be certain that all user data inserted
in SQL is being handled in spoof-proof ways.

Patricia

Reply to this Message

Wibble - 01 Aug 2006 12:03 GMT

> I/we have inherited a 'Java' web application that is written in a style
> that has to be seen to believed. We need to refactor/rewrite it!
[quoted text clipped - 50 lines]
> TIA
> Mike W

The answer here is just too obvious.

RAISE YOUR RATE!!!

Reply to this Message

Chris Uppal - 01 Aug 2006 12:48 GMT

> The product is on-line, making money but further developement that is
> being demanded is obviously painfully slow and hack upon hack.

Unlikely, IMO, that a complete rewrite would be justified then -- too much risk
that the real, underlying, logic (as opposed to its ghastly expression) is more
complex than you yet realise.

Still, I did once solve a similar (but /much/ smaller) problem by working on
the old code and a new implementation in parallel (the new implementation was a
sort of skunkworks project). I could use the new implementation to prototype
stuff to be added to the old. And I could work on untangling the old code and
by doing so gain confidence that my new implementation did everything that the
old one did (or more). In the end (after a couple of months -- wasn't a large
project) I found it quite easy to persuade the powers that be that we could
ditch the old stuff, because my new version was better, faster, and already had
a lot more functionality than the old was ever likely to gain.

Whatever you do, the important point is never to be in a position where after N
days/weeks/months, you find yourself worse off than if you'd spent that time
playing minesweeper.

> Decoupling is non-existent, encapsulation is non-existent, the OO
> concept has completely escaped the author.
>
> Where the heck do we start?

[...]

> The team is small, the author has left and half the team is brand new.
> I think we should have a period of discovery and analysis and write
> some UML analysis to eventually objectify this mess.

Before trying to objectify it, you'll have to understand it. In detail --
sometimes seemingly simple and obscure requirements can force a large change to
the object design. I hate to say it, but it sounds to me as if you'd be
better off attempting to clean up the existing code /before/ trying to make it
fit into good OO shape.

One technique that might be useful is recursive splitting and hiding. Choose a
large chunk of ill-understood functionality/code. Put a nice, minimal, and
clean, interface around it -- whilst leaving the mess inside almost untouched.
Don't change the rest of the system yet, because this attempt to split out a
sub-system is only a first attempt. Now try (in a separate development branch)
to make the rest of the system use only that interface. If that attempt fails,
then you didn't fully understand the relationship between the proposed
sub-system and the rest of the system -- so repeat with your newly gained
knowledge. Once you have managed to do this, fold the changes back into the
main development branch. Repeat. Eventually you'll (with luck) have the
system split into several pieces with well-understood interfaces between them.

Now, you can start again from the top. For each subsystem, you can decide
whether to clean the code up incrementally, do a complete re-write (possibly in
parallel), or to recurse the procedure to sub-divide that chunk. Since the
chunks are smaller, you have a better chance of making the right decision, and
it should be easier to implement whatever decision you make.

Note that this procedure is not guaranteed to produce a "good" OO design -- in
fact it has more in common with functional decomposition than the roll-based
decomposition of good OO.

-- chris

Reply to this Message

Luke Webber - 01 Aug 2006 23:25 GMT

> I/we have inherited a 'Java' web application that is written in a style
> that has to be seen to believed. We need to refactor/rewrite it!
[quoted text clipped - 15 lines]
>
> you get the idea!

Whew! There has been lots of good advice already posted, but I just want
to add one rather obvious suggestion. If the JSP is really all scriptlet
code, and the HTML is separate, then it should be rewritten as a
servlet. Moreover, you could probably get away with a single servlet and
a bunch of parser classes.

Better yet, you could actually rework the HTML into XML/XSL and use XSLT
in place of code. But that might be a bridge too far. <g>

Luke

Reply to this Message

Mother of a Refactor

Free Magazines