Path: news.mathworks.com!not-for-mail
From: "Yi Cao" <y.cao@cranfield.ac.uk>
Newsgroups: comp.soft-sys.matlab
Subject: Re: MATLAB Central Spring Contest
Date: Sat, 10 May 2008 06:35:04 +0000 (UTC)
Organization: Cranfield University
Lines: 41
Message-ID: <g03fmo$jnt$1@fred.mathworks.com>
References: <fv82i0$nql$1@fred.mathworks.com> <fvthcn$82i$1@fred.mathworks.com> <fvu2lr$j89$1@fred.mathworks.com> <fvu4c8$jkj$1@fred.mathworks.com> <fvucs2$d4v$1@fred.mathworks.com> <fvuln8$m57$1@fred.mathworks.com> <fvv9u4$eo9$1@fred.mathworks.com> <fvvmj3$5rt$1@fred.mathworks.com> <g000v3$gg9$1@fred.mathworks.com> <g00t34$kj5$1@fred.mathworks.com> <g00unh$dnk$1@fred.mathworks.com> <g02jgr$a2d$1@fred.mathworks.com> <g036ef$i42$1@fred.mathworks.com>
Reply-To: "Yi Cao" <y.cao@cranfield.ac.uk>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1210401304 20221 172.30.248.37 (10 May 2008 06:35:04 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sat, 10 May 2008 06:35:04 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 69713
Xref: news.mathworks.com comp.soft-sys.matlab:467669


"Ned Gulley" <gulley@mathworks.com> wrote in message > 
> Hi Sergey:
> 
> You are right. Introducing a new idea to compete with
> hyper-tweaked code is very difficult. We'd like to make it
> easier. Can you think of things we might do to the contest
> machinery to make it easier to introduce new algorithmic
> ideas? Based on all the feedback so far, we will probably
> introduce some kind of rate limiter (whether by CAPTCHA or
> otherwise). What are some other ideas? I've often wondered
> if people might flag new ideas explicitly with a message
> like "please help me tweak a new concept into first 
place."
> We tried a late stage twilight last time, but it didn't 
seem
> to help, and it wasn't much fun.
> 
> -Ned.
> Contest Team

I think to encourage new idea, we have to introduce some 
generality index to be included in the overall score. When 
a new algorithm is submitted, even it may not be able to 
compete with current best code, but if it shows some 
advantage in generality, it might be pick up by someone 
else to tweak it into top. The generality index can come 
from another hidden testsuit or simply an average results 
by dropping different portion of cases in the testsuite 
(something like cross validation).

Another suggestion, can we make the last 30 minutes or so 
as the twilight phase, or disable downloading an entry from 
the waiting queue? This will encourage people to submit 
their final code earlier.

The node index and 1000 node contest were good. But, it 
seems that the weight of the node index was too small, no 
one cared about it. 

Yi