Path: news.mathworks.com!not-for-mail
From: "Yi Cao" <y.cao@cranfield.ac.uk>
Newsgroups: comp.soft-sys.matlab
Subject: Re: MATLAB Central Spring Contest
Date: Sun, 11 May 2008 07:22:04 +0000 (UTC)
Organization: Cranfield University
Lines: 51
Message-ID: <g066qs$drt$1@fred.mathworks.com>
References: <fv82i0$nql$1@fred.mathworks.com> <fvsk1d$m84$1@fred.mathworks.com> <fvsmhv$se6$1@fred.mathworks.com> <fvsot0$gr9$1@fred.mathworks.com> <fvsqp9$8gg$1@fred.mathworks.com>
Reply-To: "Yi Cao" <y.cao@cranfield.ac.uk>
NNTP-Posting-Host: webapp-05-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1210490524 14205 172.30.248.35 (11 May 2008 07:22:04 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sun, 11 May 2008 07:22:04 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 69713
Xref: news.mathworks.com comp.soft-sys.matlab:467774


"Steve Hoelzer" <shoelzer@gmail.com> wrote in message 
<fvsqp9$8gg$1@fred.mathworks.com>...
> "Yi Cao" <y.cao@cranfield.ac.uk> wrote in message
> <fvsot0$gr9$1@fred.mathworks.com>...
> > However, I would like to suggest to use a random 
function 
> > in runcontest function to make tuning random parameters 
> > imposible. For example, in runcontest function, a line 
of 
> > rand('state',sum(100*clock)) can make any intention to 
tune 
> > a random parameter impossible because even with the 
same 
> > parameters, the results could be totally different if a 
> > random function is used in a code. 
> > 
> > Yi
> 
> No. The same code must give the same score every time it
> runs. Otherwise, a new type of random variable "tuning" is
> possible: submitting the same code multiple times. I too
> wish that magic number tuning was not a part of the 
contest,
> but I don't think it can be programmatically enforced.
> 
> Steve

If most of us agree to discourage using random functions, 
then including a random seeds in runcontest is a solution. 
It wont affect entries which do not use any random function 
but will make results unpredictable if a random function is 
used. It is against "The same code must give the same score 
every time it runs". However, this is what the 
user "deserved" because he/she makes some decision 
randomly. OK, they can resubmit the same code several 
times, but seldom one of them can get to the top because 
they cannot tune parameters systematically. 

I support the idea to let the final stage of contests a 
twilight phase. It is also logical, a contest starting from 
darkness, then twilight, then full daylight, finally 
twilight again because it is going to finish. It does not 
need to be a full day, but 4 hours will be enough. If 
possible, I would like only those submitted during the last 
4 hours to be protected from download but not old entries. 
This will help us to search the complete list to find a 
good code to work with during the final stage.

Yi