Daniel wrote On 04/23/07 13:53,:

   You could certainly inspect the compiled byte code for
references to "forbidden" classes.  The java.lang.instrument
package might be helpful here.

   However, I doubt such an approach will be all that useful
as a detector of cheating.  To begin with, you need to make
a useful list of banned classes -- For example, if I were
told to sort an int[], I would use Arrays.sort() and never
touch the Collections class at all.  Also, the sources for
Arrays (and Collections) are readily available; a cheater
could simply copy the source and change the package names,
and thus evade your detection.

   Finally, JUnit is probably not a good framework for
this sort of thing.  If I try to write an array-sorter but
make an error and introduce an infinite loop, you're going
to want some kind of timeout mechanism that prevents my
blunder (or my sabotage!) from monopolizing your page until
somebody notices the problem and restarts it.  As far as
I'm aware, JUnit has no such facility; it just runs tests
until they finish or fail.  If they do neither ...

   Various organizations have public web pages that run
externally-submitted programs under controlled conditions,
with provision for catching runaways, guarding against hacks,
and so on.  Instead of reinventing the wheel, I'd suggest
you look up a few of those sites and find out what they're
using for software; perhaps you can adapt something that
already exists as opposed to building it all from scratch.

   Although this shares some aspects with the programming-
assignment problem, I think it's fundamentally a different
issue.  There's no "antagonist" to worry about, and the
body of code to be inspected is more or less "known," so I
think static analysis of byte code (or source code) may be
a more appropriate tool than black-box run-time testing.

I didn't read everything, but I wonder if some form of dependancy
injection would work here.  You supply the object (let's call it
SortTest) to sort, rather than allow them to declare one internally.

Also, make each method call for SortTest, for a read or a write, record
what is being done.  So if they are supposed to be implementing a shell
sort, you'll know if the reads and writes for the object are being done
in the correct order.  This could be useful feedback for the student, to
point out common errors.  If on the other hand they read each element
once, in order, then write them back once, in the sorted order, they are
probably cheating.

Rather than calling main, I think you'd have to define your own
Interface, which the user would then implement.  Do the compile and run
both on your end, that way you can control which libraries are accessed.
 You could also define your own classloader, to deny access to certain
classes by the user.

You need to get the classloader used to tell you.

Here are a starting point:

import java.net.URL;
import java.net.URLClassLoader;

public class TraceClassLoader extends URLClassLoader {
    public TraceClassLoader(String cp) throws Exception {
        super(new URL[] { new URL(cp) });
    }
    protected Class<?> loadClass(String name, boolean resolve) throws
ClassNotFoundException {
        System.out.println(name);
        return super.loadClass(name, resolve);
    }
    public static void main(String[] args) throws Exception {
        Class.forName(args[0], true, new
TraceClassLoader(args[1])).getMethod("main", new Class[] {
String[].class }).invoke(null, new Object[] { new String[0] });
    }
}

import java.util.*;

public class Foobar {
   public static void main(String[] args) throws Exception {
       List lst = new ArrayList();
       lst.add("CCC");
       lst.add("BB");
       lst.add("A");
       for(int i = 0; i < lst.size(); i++) {
           System.out.println(lst.get(i));
       }
       Collections.sort(lst);
       for(int i = 0; i < lst.size(); i++) {
           System.out.println(lst.get(i));
       }
   }
}

java TraceClassLoader Foobar file:/C:/

Foobar
java.lang.Object
java.util.List
java.lang.String
java.lang.Exception
java.util.ArrayList
java.lang.System
java.io.PrintStream
CCC
BB
A
java.util.Collections
A
BB
CCC

Arne

PS: It is important that Foobar.class is not in the classpath where
    TraceClassLoader is running.