From: <HIDDEN>
Path: news.mathworks.com!newsfeed.mathworks.com!WebX
Newsgroups: comp.soft-sys.matlab
Subject: Re: Scoring Algorithm
Message-ID: <eeb4a65.64@WebX.raydaftYaTP>
Date: Tue, 12 Nov 2002 16:22:06 -0500
References: <eeb4a65.-1@WebX.raydaftYaTP> <eeb4a65.9@WebX.raydaftYaTP> <eeb4a65.10@WebX.raydaftYaTP> <eeb4a65.22@WebX.raydaftYaTP> <eeb4a65.59@WebX.raydaftYaTP>
Lines: 79
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
Xref: news.mathworks.com comp.soft-sys.matlab:142703


Stefan Stoll wrote:
> 
> 
> Kevin Spiteri wrote:
>>
>> score = result + exp(-1.95 + 0.05*runtime)
>>
> 
> The current top entry contains 1771 lines
> of poorly documented M-code, making it al-
> most impossible to grasp what's being changed
> or added from one entry to the next.
> 
> I admit that all the submitted "spaghetti
> codes" are very good at folding the proteins
> from the contest suite, but I can't escape
> the feeling that most GPRs (good programming
> rules) are completely neglected. To put it
> a bit sharper:
> 
> The contest discourages good programming.
> 
> Don't misunderstand me. I am enjoying the
> contest and facinated by the zoology of
> modules in the current code!
> 


This restates in part what I said very early on in this thread, which
was:


> 
> <http://www.mathworks.com/contest>
> 
> These contests are a great way to learn new programming tricks, as
> well as show off the ones you have.


While it is true that these contests are fun to watch, I doubt that
many "learn new programming tricks" from them. It is next to
impossible for outside observers (i.e., those who watch but don't
submit) to track changes in submitted code as the contest proceeds.
Since the submitted code does not use any or many comments and since
the code often uses cryptic single-letter variables, only those folks
who have the time and desire to win the contest take the time to
study what's been done by others and improve it.


I also doubt that the contests give folks the chance to "show off"
programming tricks either. Once again the code is so cryptic and
poorly documented that learning anything from it requires a
substantial investment in time and effort.


To me, the number one thing I learn from these contests is the
fundamental need to write good, readable, well-commented code.
Cryptic code may run a little faster, but overall productivity falls
to near zero as soon as someone says, "Can you modify this code
written by someone else to do _______?"


*************************


I concur with Stefan's comments. I also like another one I read
somewhere: That execution time should not be penalized at all until
it reaches some significant percentage of the maximum time allotted.
That way well written algorithms that do not consume an excess amount
of time do not suffer from variations in the timing accuracy while
running the test suite.


In a past contest I tested this timing variability by simply
resubmitting a midcontest winning solution with no changes other than
adding a comment line to the code. After several such "faked"
submissions, "my" solutions were sometimes faster, sometimes slower
than the midcontest winner that I chose to test! If I could have done
this at the last minute and was lucky enough, then I could have been
the contest winner, despite the fact that I did no real work--I just
exploited a scoring system weakness.


As long as execution time and solution validity w.r.t. a given test
suite are the only measures that determine winning, these contests
will always gravitate to unreadable code that is a mixture of code
from a variety of submitters. If code readability is important (and I
believe it is), then it must contribute somehow to the scoring system.