Java Forum / General / October 2006
Has anyone here ditched ant in favor of scripting for builds?
Danno - 03 Oct 2006 19:12 GMT I just want to know you're experience. Do you use both ant and scripting? Pure script builds? Do you use Groovy or Ruby?
Andy Dingley - 03 Oct 2006 20:14 GMT > Has anyone here ditched ant in favor of scripting for builds? > Do you use both ant and scripting? Hmmm... Two quite different questions there, but they're damn good ones.
I have build problems like you wouldn't believe. My full-time job title is "Build Manager", so you can imagine how big a problem it has to be to grow and keep a full-time role involved in it.
Currently we're using both Ant and Python, with a bunch of XSLT and still some legacy shell. Bugzilla and Subversion are tightly integrated into this. The Ant build.xml is over 1000 lines and there's another couple of thousand in Python.
My attempt to impose sanity on this is focussed round the ideas of "Internal" and "External" builds. The Internal build is Ant, and it begins by assuming an accurate source tree that's all in place locally and ends with a couple of well-behaved JARs and ZIPs sitting locally. Nothing changes outside this local directory tree, and especially nothing changes in the repositories.
Everything else, both before and after this, is handled in the "External" build, which is in Python. This starts with a build request in Bugzilla and ends with big single ZIPs on an ftp server, including generated docs, and will also have updated Subversion, Bugzilla and sent a few emails.
The idea really is that the Internal part is idempotent, whilst the External bit generates paperwork. Developers use the same Internal part over and over, the External bit only gets fired up for real Builds that are going out to QA or customers.
In fact my developers _don't_ use the Internal build. They tend to use Eclipse and let Eclipse handle things its own way. This sucks when they then hand over a completed project with its unused and ignored buildfile in tatters. If another one does it at 5pm after telling the customer "it's ready now", then there'll be blood on the carpet 8-(
We're also heavily into CurseControl and Fitnesse. Those have a dependency on Ant we could work round, but haven't felt any urge to.
Ant sucks for its lack of string handling. This is so poor that I don't even like using Ant with a command line, as it's impractical to validate command line parameters. Ant is good in that it understands dependencies, which is slightly more use than a chocolate teapot. In all this mess though, I only have one place where I actually use dependencies.
Ant is probably more use for deployment, where a simple initial choice (particularly which application server is in use) implies a whole chain of other tasks. I'm also using it as a SQL client tool during installation, because I know it's available, I know JDBC is already set up, it doesn't care whether it's connected to Oracle or SQL Server and it even does dependencies between successive patch scripts.
I'd probably use Ant again, if I started from scratch. -- But I'm having a hard time to really think _why_.
I'm impressed with Python though. Six months ago I wrote bad Perl, then I inherited this bundle mostly as shell scripts. I've gone from nowhere on Python to devotee in just that time.
Scott Harper - 04 Oct 2006 00:19 GMT >I have build problems like you wouldn't believe. My full-time job title >is "Build Manager", so you can imagine how big a problem it has to be >to grow and keep a full-time role involved in it. [...]
>In fact my developers _don't_ use the Internal build. They tend to use >Eclipse and let Eclipse handle things its own way. This sucks when they >then hand over a completed project with its unused and ignored >buildfile in tatters. If another one does it at 5pm after telling the >customer "it's ready now", then there'll be blood on the carpet 8-( Andy,
This is probably not the exact best forum for this discussion, but since I didn't start the thread, I'll tag on anyway... Perhaps there are others out there who can add to it as well.
We've been trying to get our build system in place. The majority of our developers use Eclipse, but we want to be able to build from the command line (automated) as well. I *think* I can handle this by exporting the Eclipse build files and by some judicious use of environment variables/properties.
But what we are having problems with is how to best organize our "projects" in Eclipse and CVS.
We have a bunch of Java packages that we want to mix and match in various application projects -- think libraries of utilities. How do people generally orgainize these in CVS and Eclipse? I'd like to keep the library package projects separate from the application projects, so someone doesn't accidentally change some "library" code -- not completely realizing that it's really global to their app -- and breaking a bunch of other apps.
Is it a good idea to keep each package as it's own directory in the CVS tree? We've looked at trying to use CVS modules to rearrage and combine source code into more modular units, but it really isn't well suited for that.
Then, do people import these projects individually into the Eclipse workspace, then create inter-project dependencies? Or is there a better way to incorporate subordinate projects into a bigger project? Seems like if you "import" code from another project it makes a local copy, which kind of defeats the purpose of source code control...
It sounds complicated, but surely this isn't an uncommon way to approach software development...
scott
Danno - 04 Oct 2006 16:33 GMT > > Has anyone here ditched ant in favor of scripting for builds? > > Do you use both ant and scripting? [quoted text clipped - 5 lines] > is "Build Manager", so you can imagine how big a problem it has to be > to grow and keep a full-time role involved in it. Wow, you can have a full time job out of it?
> Currently we're using both Ant and Python, with a bunch of XSLT and > still some legacy shell. Bugzilla and Subversion are tightly integrated [quoted text clipped - 13 lines] > generated docs, and will also have updated Subversion, Bugzilla and > sent a few emails. So you have people request a product build in Bugzilla, and you use that to initiate all the work to get the product build started?
> The idea really is that the Internal part is idempotent, whilst the > External bit generates paperwork. Developers use the same Internal part [quoted text clipped - 6 lines] > buildfile in tatters. If another one does it at 5pm after telling the > customer "it's ready now", then there'll be blood on the carpet 8-( Since CruiseControl requires that the external build works, wouldn't you know then who is suspect when someone commits a tattered project?
> We're also heavily into CurseControl and Fitnesse. Those have a > dependency on Ant we could work round, but haven't felt any urge to. [quoted text clipped - 5 lines] > all this mess though, I only have one place where I actually use > dependencies. Exactly, this is my problem which led me to put up this post.
> Ant is probably more use for deployment, where a simple initial choice > (particularly which application server is in use) implies a whole chain [quoted text clipped - 9 lines] > I inherited this bundle mostly as shell scripts. I've gone from nowhere > on Python to devotee in just that time. I am going to move towards Groovy instead of Python. Groovy has some documentation (http://groovy.codehaus.org/Groovy+Ant+Task) that can integrate with ant as far as building. Since I know Ruby, I think Groovy would be a nice fit. I appreciate you taking time to tell me about your background and your building techniques.
Andy Dingley - 04 Oct 2006 23:36 GMT > Wow, you can have a full time job out of it? 1/3rd getting builds out
1/3rd improving the build process, writing scripts to reduce #1, telling developers where to stick their code.
1/3rd chasing the developers who didn't follow #2 with my clueiron http://codesmiths.com/shed/things/clueiron/
1/3rd generally being sworn at over the phone
> So you have people request a product build in Bugzilla, and you use > that to initiate all the work to get the product build started? Yes. You also use Bugzilla to request your PC to be fixed, or to recruit a new developer.
> Since CruiseControl requires that the external build works, wouldn't > you know then who is suspect when someone commits a tattered project? Usually, but not always. The usual culprit is brand new work, where it hasn't yet been added to our Fitnesse tests. Even when it has been "added", it needs an active inspection by me to ensure that everything is as it should be. The process isn't watertight, particularly when timescales are pressured.
The "Agile" malarkey is great, but it's not very agile. Works a _lot_ better when you're doing incremental work on an established project, rather than creating new projects.
> I am going to move towards Groovy instead of Python. I considered that too, but it seemed too much like my previous experience of embedding Java snippets inside my Ant, when what I really needed was a wrapper around it.
Thomas Weidenfeller - 05 Oct 2006 09:53 GMT >> Wow, you can have a full time job out of it? > [quoted text clipped - 7 lines] > > 1/3rd generally being sworn at over the phone And another 1/3rd to finish your math degree along the way, right? :-)
/Thomas
 Signature The comp.lang.java.gui FAQ: http://gd.tuwien.ac.at/faqs/faqs-hierarchy/comp/comp.lang.java.gui/ ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
Andy Dingley - 05 Oct 2006 13:38 GMT > And another 1/3rd to finish your math degree along the way, right? :-) If three thirds add up to 1, that's arithmetic.
If four thirds add up to 1, that's maths.
If an uncertain number of thirds definitely add up to 1, but you can't tell how many, that's crypto.
David Zimmerman - 05 Oct 2006 18:01 GMT Sounds like the 90/10 rule of software
The first 90% of the job takes 90% of the time. The remaining 10% of the job takes the other 90% of the time.
>> And another 1/3rd to finish your math degree along the way, right? :-) > [quoted text clipped - 4 lines] > If an uncertain number of thirds definitely add up to 1, but you can't > tell how many, that's crypto. Chris Uppal - 06 Oct 2006 10:52 GMT > The first 90% of the job takes 90% of the time. > The remaining 10% of the job takes the other 90% of the time. Nice...
Up there with "n+1 trivial tasks will take the same time as n trivial tasks".
-- chris
Simon Brooke - 03 Oct 2006 22:36 GMT > I just want to know you're experience. Do you use both ant and > scripting? Pure script builds? Do you use Groovy or Ruby? Most of us have ditched scripting (i.e. makefiles) for ant; ant is a very, very good solution. Ant is, of course, a scripting approach with a XML-syntax scripting language, but its flexibility and extensibility make it extremely effective.
 Signature simon@jasmine.org.uk (Simon Brooke) http://www.jasmine.org.uk/~simon/
<p>Schroedinger's cat is <blink><strong>NOT</strong></blink> dead.</p>
Gordon Beaton - 04 Oct 2006 07:20 GMT > scripting (i.e. makefiles) When did these become equivalent?
/gordon
 Signature [ don't email me support questions or followups ] g o r d o n + n e w s @ b a l d e r 1 3 . s e
Simon Brooke - 04 Oct 2006 10:37 GMT >> scripting (i.e. makefiles) > > When did these become equivalent? Make is a scripting language interpreter in itself, and uses conventional shell scripting languages as the basis both for its macros and for its actions.
 Signature simon@jasmine.org.uk (Simon Brooke) http://www.jasmine.org.uk/~simon/
'graveyards are full of indispensable people'
Thomas Weidenfeller - 04 Oct 2006 11:20 GMT > Make is a scripting language interpreter in itself, Make is a rule-based engine - of course interpreting its own language. However, that language is fare from being a scripting language.
> and uses conventional > shell scripting languages as the basis both for its macros No. Make interprets its macros, and does not enlist the help of a shell or similar to do so.
> and for its > actions. It forwards the actions to an interpreter of your choice. That one is typically a shell, but can be anything you like it to be.
/Thomas
 Signature The comp.lang.java.gui FAQ: http://gd.tuwien.ac.at/faqs/faqs-hierarchy/comp/comp.lang.java.gui/ ftp://ftp.cs.uu.nl/pub/NEWS.ANSWERS/computer-lang/java/gui/faq
Andy Dingley - 04 Oct 2006 11:37 GMT > >> scripting (i.e. makefiles) > > > > When did these become equivalent? > > Make is a scripting language interpreter in itself, The fact (debatable) that Make is a scripting language certainly doesn't mean that scripting languages are make! "Equivalence" would require both of these to be true.
While it's arguable that "make does some scripting" the key capability of make is the ability to take a graph of dependencies and linearise it to a procedural sequence. Scripting languages (for many and various values of "scripting language") don't have this ability.
Andy Dingley - 04 Oct 2006 11:07 GMT > > scripting (i.e. makefiles) > > When did these become equivalent? They clearly didn't!
There are (at least) three levels here: Make, Ant and procedural scripts.
Make is a good solution to a problem of 30 years ago, that of reducing compilation to the minimum number of affected compilation units. These days compilation is so fast that we just don't care about this - we can re-compile from scratch in negligible time (certainly compared to running the unit tests) and so the "blunderbuss" approach to compilation is adequate. I don't need to spend time (and lots of it) managing the file-level dependencies I used to do with Make.
Ant is powerful, but I'm increasingly coming to the opinion that it's power I don't need. It does "task level" dependencies such that I can re-run the compile target before the JAR-maker, but this is of little _real_ benefit. I have a big Ant file with lots of separate targets, but the practicality is that I only ever use two execution paths through this: "Changed some source, so re-compile it and do the rest" or "This one is going out to a punter, so run the whole lot from scratch in clean directories". So there's the great virtue of Ant, and I'm not even using it. I don't need target dependencies here, just a few lines of rigid procedural script would be adequate.
I also hate Ant's lack of string-slicing. IMHO (to paraphrase emacs and email), every scripting tool should expand until it has the ability to use regexes. It's not much to ask, but I feed my buildfile a build tag parameter and need to make deliverable zipfiles with names based on it. This is _painful_ in Ant. One of my Ants has about 800 lines of 1500 total devoted just to parameter-handling logic. I could probably replace this with 20 lines of Python / Visual Basic.
When I started this project, I went through an aberrant phase of writing custom Ant tasks. These were supposedly great - I could do anything, my Ant was simple and elegant afterwards, and we're a Java shop so the language was familiar. I no longer do this anywhere, and would be most reluctant to ever do it again. Custom tasks are a terrible way to work - just don't go there!
The problem is that custom tasks split your build in two, quite probably even in two different editors. If you have "business logic" in the build, this is an obvious maintenance problem.
_Extending_ Ant by writing new tasks is OK, provided that the requirements for the task are sufficiently generic. A task that understands regexes is one thing, because it's stable within itself. A task that understands your local build tag format and maps it in and out of SVN paths is quite another problem -- day-to-day maintenance and process development means working across two platforms, with an unstable interface.
I'm also reluctant to use Ant tasks that are outside the basic Ant package (so JUnit and FOP are just about OK) because I have to assume an Ant environment in many locations, even customer sites. It's no use using Ant for an easy product deployment if I first have to work through a non-standard Ant deployment.
Ant might still have something to offer for multi-platform deployment. Problem of the week is deploying what began as a pure Tomcat development onto Weblogic / Websphere and JBoss as well. This is messy and Ant (or at least the Ant community and collected best practices) has things to offer for cleaning it up.
So if I don't (very often) need task-level dependencies and I don't need file level dependencies in compilation, then what's left for Ant to do ? And of this remainder, is Any any good at doing it?
As I mentioned before, I'm moving to more and more scripting (Python) but keeping this outside a somewhat arbiitrary boundary of the Internal/External build. With hindsight though (and some process dev time before the build-request-batphone next rings) I'd certainly be thinking about a pure procedural script approach.
pkriens@gmail.com - 05 Oct 2006 10:55 GMT > > scripting (i.e. makefiles) > When did these become equivalent? Agree with this question! Only bad makefiles looked like scripts. Good makefiles used the dependency facility which is declarative. However, most makefiles I have seen became highly procedureal because the maintainers did not understand the declarative approach of make.
Kind regards,
Peter Kriens
Danno - 04 Oct 2006 16:41 GMT > > I just want to know you're experience. Do you use both ant and > > scripting? Pure script builds? Do you use Groovy or Ruby? [quoted text clipped - 8 lines] > > <p>Schroedinger's cat is <blink><strong>NOT</strong></blink> dead.</p> I've used ant for years, I just have more complicated scenarios that I need to use a scripting language (e.g. Groovy, Ruby, JavaScript, Perl). That being said, I have to heartily disagree that make, ant, and XML are scripting languages. Make and ant are tools, that use scripts, but that doesn't mean those scripts are scripting languages. XML is a markup language (hence the ML in XML) , so XML is not a scripting language. I do agree that make is much better than ant though.
Chris Stiles - 04 Oct 2006 17:11 GMT > are scripting languages. Make and ant are tools, that use scripts, but > that doesn't mean those scripts are scripting languages. Ant is a batch file interpreter that understands XML and has flow control via implicit gotos (task dependancies).
 Signature regards, chris
Andy Dingley - 04 Oct 2006 17:53 GMT > Make and ant are tools, that use scripts, but > that doesn't mean those scripts are scripting languages. Ant is a scripting language. Make one target, stick a sequence of tasks in it. Use <antcall> as a subroutine mechanism. You've got yourself a scripting language.
It's not a _good_ scripting language, but it's still better than Windows offers.
The question is whether Ant's advantage of _also_ (and separately) being a dependency-flattening langauge is sufficient to excuse the poor scripting features, when our application of it is generally more dependent on scripting than anything else.
Danno - 04 Oct 2006 23:46 GMT > > Make and ant are tools, that use scripts, but > > that doesn't mean those scripts are scripting languages. > > Ant is a scripting language. Make one target, stick a sequence of tasks > in it. Use <antcall> as a subroutine mechanism. You've got yourself a > scripting language. Whew, I'd have to strain to see that.
> It's not a _good_ scripting language, but it's still better than > Windows offers. [quoted text clipped - 3 lines] > scripting features, when our application of it is generally more > dependent on scripting than anything else.
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|