Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: MATLAB Central Spring Contest
Date: Sat, 10 May 2008 16:20:20 +0000 (UTC)
Organization: Harvard Medical School
Lines: 22
Message-ID: <g04i04$86v$1@fred.mathworks.com>
References: <fv82i0$nql$1@fred.mathworks.com> <fvthcn$82i$1@fred.mathworks.com> <fvu2lr$j89$1@fred.mathworks.com> <fvu4c8$jkj$1@fred.mathworks.com> <fvucs2$d4v$1@fred.mathworks.com> <fvuln8$m57$1@fred.mathworks.com> <fvv9u4$eo9$1@fred.mathworks.com> <fvvmj3$5rt$1@fred.mathworks.com> <g000v3$gg9$1@fred.mathworks.com> <g00t34$kj5$1@fred.mathworks.com> <g00unh$dnk$1@fred.mathworks.com> <g02jgr$a2d$1@fred.mathworks.com> <g036ef$i42$1@fred.mathworks.com>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1210436420 8415 172.30.248.37 (10 May 2008 16:20:20 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Sat, 10 May 2008 16:20:20 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 51934
Xref: news.mathworks.com comp.soft-sys.matlab:467715


Seem to me, we are coming to some consensus. I would like 
to see last one hour before all deadlines to be twilight 
too. It will give some time for new code to be adjusted and 
make sure that the real author of good modification will 
get credit. However it does not solve completely problem of 
automatic submissions because they still may clog queue and 
deny everybody feedback from test, making it essentially 
dark.

There were a lot of discussions about test set overfitting. 
I do not think that any suggested method of using subsets 
will really work; it will just change test set (or weighted 
combination of sets) to fit. 
Another idea is to make &#8220;generality&#8221; prize the main prize 
with limited number of submission from each participant. 

All this may be (?) less important with different kind of 
contest problem. I would say that Blackbox contest having 
its own problem (probing) was not under the danger of 
overfitting and unnecessary parameter tweaking.