Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

To resolve issues starting MATLAB on Mac OS X 10.10 (Yosemite) visit: http://www.mathworks.com/matlabcentral/answers/159016

What do you think of Cody, new service for MATLAB Central

Asked by bym on 28 Jan 2012
Latest activity Edited by Laura Pop on 21 Oct 2013

What do you think of Cody? I'm interested in your comments

4 Comments

Image Analyst on 29 Jan 2012

I haven't, unless he means the kid in my Boy Scout troop, which I doubt. Maybe someone can post a link, if it's worth knowing about.

Walter Roberson on 29 Jan 2012

http://www.mathworks.com/matlabcentral/cody

Matt Tearle on 22 Feb 2012

Well you guys are just behind the curve, then!
http://www.mathworks.com/matlabcentral/trendy/plots/677
(Haha. "Behind the curve". See what I did there?)

bym

Tags

Products

14 Answers

Answer by Jan Simon on 29 Jan 2012

As Oleg has mentioned already, the measurement of the code leads to bad programming practize. Therefore the Cody-users learn to use Matlab inefficiently.

The "code-size" system for "measuring" the submitted solution do not reward:

  • check number and type inputs
  • consider NaN's
  • handle arrays of all size, even empty arrays
  • pre-allocation
  • local constructions instead of calling external functions, e.g. "y = x(end:-1:1)" is worse than "y = fliplr(x)"
  • optional parenthesis added to improve the readibility.

While some hard-working contributors in Answers try to provide as much good programming practice as possible to improve the total quality of the user-land code, Cody encourages to be accustomed to a compact but for larger projects horrible programming style. In consequence I think that Answers and Cody are antagonists.

Imagine I try to employ some programmers for a serious Matlab project. Will I ask the top ten of Answers, FileExchange or Cody?

It is not really clear to me, in which order I should solve the problems. Sometimes the solution of one problem let me look on the solutions of other problems I've solved some days before. This has a limited pedagogical quality only. I can see that there is a very compact solution for a certain problem. To learn how it was implemented, I have to submit any valid solution for another problem. This encourages to submit even bad solutions get the the wanted information.

Currently the method to measure the code-size can easily be confused. What happens, if the actual calculations are embedded as string in the comment section and EVALed? Can we create M-files dynamically, which shadow the functions required for testing the success? How long will it take until somebody breaks out of the sandbox and injects some SQL code?

I see a growing number of droll cheating already. But I see an invitation to (for?) fraud also.

[EDITED] See also: Blogs: Scoring in Cody

5 Comments

Sean de Wolski on 1 Feb 2012

Jan, you can write problems for CODY that meet those requirements :)

There are definitely some problems on there already that do use empty sets/nans etc and expect the solution to be correct.

Sean de Wolski on 1 Feb 2012

Perhaps you could make one that is strictly - Does this data get to move forward?

Jan Simon on 2 Feb 2012

Sean, I see a fast evolution in Cody. That's fine.
I still do not think, that the magic code-size measurement it a good method. Are there better methods to measure the complexitiy of the code?

Jan Simon
Answer by Oleg Komarov on 28 Jan 2012

I think it's:

  • fun
  • it teaches you may things apart matlab

But:

  • Encourages bad programming practice with it's rating system
  • Some problems are buggy, badly stated or the checks allow cheating

EDIT Now it's getting too tempting to brute force the solution.

EDIT2

The voting system is bugged. Once you get it right with any high score, you can fail with a low score and see the solutions of those with lower score than you.

1 Comment

Jan Simon on 29 Jan 2012

+1 for mentioning, that the "short code size" measurement provokes bad programmint practize.

Oleg Komarov
Answer by samil on 29 Jan 2012

It's good for people who want to improve themselves in a casual manner (e.g. me), but might be bad for people who are obsessed with collecting points.

0 Comments

samil
Answer by Albert Yam on 22 Feb 2012

I found that a lot of the Cody (official) problems after a while turned into a lot of 'regexp' stuff and was a bit too esoteric for me.

At first, looking at other people's solutions was good. Seeing other ways to solve to problems was interesting, but after a while, the 'best' solution just gets saturated with the same thing. An explanation of what is going on in people's code would be helpful.

Have the 'best' solution be voted by players. Perhaps only allow a player's first successful submission be shown (or branch further optimizations), so that people don't just copy the 'best' and resubmit.

0 Comments

Albert Yam
Answer by Kevin Holst on 22 Feb 2012

I like Cody as a set of brain teasers that may force you to think about fairly unique problem that you may not have encountered before. I would echo what has been said about the rating system. On the surface, it seems very good to be able to write just a few lines of code to do a task, but like Jan has said, it produces code that may be less efficient and that is usually not very readable.

Cody also allows the user to see what tests their codes will encounter if they fail the first time. I feel like this does not make the programmer write general enough functions, and allows them to just write a function that solves that SPECIFIC problem. My solution to this would be to have many different tests and only use 2 or 3 for any given check.

3 Comments

Sean de Wolski on 22 Feb 2012

That's an interesting idea for tests. What if the author could check the tests (s)he wants visible and hide the others? Hidden tests would require a verbose error message but you wouldn't be able to see the test.

E.g:

A test that uses nans and is hidden could say, "failed with nans".

Kevin Holst on 23 Feb 2012

That would be good, too. Still allows all tests to be conducted, but some more generic ones are hidden.

Jan Simon on 23 Feb 2012

Some problems cannot be solved without seeing the tests, because the description does not contain enough information. Therefore I think that hidden tests are not useful in general, but the opposite: Show the tests initially also, to define unequivocally what must be done to solve the tests.
Of course this allows to create poor solutions which solve only the defined tests by using a lookup table. This can be controlled either by letting an editor delete the nonsense-solutions, or by adding further tests and re-evaluating all submitted solutions.

Kevin Holst
Answer by Matt Tearle on 22 Feb 2012

OK, so you know that I'm biased, but let me be clear that I'm stating my personal opinion here, not that of my corporate overlords.

I don't see Cody as antithetical or antagonistic to Answers. I see them as complementary, serving different purposes. I think it's important to see Cody for what it is: a game. A fun way to flex your MATLAB muscles.

For me in my professional life, I see Cody as a great resource for those learning MATLAB. A few minutes a day on Cody would, I think, help a new user develop and solidify their skills. Once they are writing serious code for a specific purpose, I'd expect them to use Answers as a resource for getting help with specific difficulties.

I understand, and share, the concern about teaching bad practices. However, I'm not worried about things like checking input types and dimensions, dealing with NaNs, and so on. Again, I don't see Cody as a way to develop serious MATLAB programming or application development skills. I see it as a way to develop MATLAB language skills. It's a game, and part of the rules of the game is defining what the input to this function might be. What does concern me with the scoring is that inefficient or inelegant approaches may score the best. My canonical example for that is the "pyramid number" problem, where sum(1:n) is "better" than n*(n+1)/2. FWIW, the Cody team are aware of these issues.

One specific quibble: as others have noted, there's a lot of regexp, which, to me, is not really why MATLAB is so awesome.

Occasionally I've seen some neat tricks that I may or may not tuck away in some spare neurons for future reference.

Bottom line(s):

  • I like Cody as a fun way to get you to play around in MATLAB.
  • I like Answers as an efficient way to get help on particular MATLAB issues.
  • I like the File Exchange as a way to get code that achieves a common task, but isn't part of MATLAB.
  • I like CSSM as a way to have pointless arguments about 0- or 1-based indexing ;)

0 Comments

Matt Tearle
Answer by James Tursa on 26 Feb 2012

My first experience with CODY:

Tried the "sum integers from 1 to 2^n" problem. Example used only a scalar input, but I coded up a vectorized version anyway and submitted it. It wound up in 139th place according to size. Hmmm...

So I looked at the solutions ... WHOOPS! It wouldn't let me. I had to solve another problem just to gain the right to look at the other solutions to the first problem I already solved. OK, so a bit of wasted time but I finally got permission to look at the solutions. Here is what I found, of the 184 correct solutions all ranked on size:

Solutions #1-#93 all use slight variations of sum(1:2^n). Nice, compact, short, ... and horrible programming. Forming the explicit integer array up to 2^n is a bad use of time and resources to solve this problem.

Solution #94 on the list is the first one that goes directly after the solution based on a closed form expression instead of forming the integer array explicitly. It is vectorized, but it suffers from calling the exponentation twice. Well, at least we are getting significantly better than the 1:2^n approach, but we had to wade through several pages of the 1:2^n stuff to get here ... not good.

Solution #106 on the list is the first one to use the pow2 function, but it still calls it twice.

Solution #121 on the list is the first one that calls the exponentiation ^ only once, but it is not vectorized.

My solution, #139 on the list, is the first one that gets the answer via the closed form expression, does the exponentiation only once, and is vectorized. In fact, after scanning all of the solutions mine was the only one that had all three of these features. (One would have gotten to it quicker by scanning the list of 184 correct solutions in reverse order.) I'm pretty sure that it would have wound up in dead last place had I included any argument checking.

Well, so what? CODY is being touted as just a game by TMW, not a serious programming aid. On that level, fine, I suppose. And maybe as a teaching aid in MATLAB syntax or as a general introduction to functions you didn't know about, it is OK as well. But as an aid to good programming practice I think it fails, unless there are comments to go along with the solutions.

Q: Does TMW want comments on bad programming practice etc to appear on the leading solutions? I.e., is this really just a game or does TMW want CODY to be used as a legitimate programming aid as well?

1 Comment

Jan Simon on 26 Feb 2012

As far as I understand, Cody is just a game: http://blogs.mathworks.com/desktop/2012/02/06/scoring-in-cody
The sum of 1:2^n is an important example: The smart method Gauss has found in the school is a basement of the education of efficient algorithms. It hurts to see that "sum(1:2^n)" is called the "leading solution".

James Tursa
Answer by Daniel on 11 Mar 2012

I tried Cody when it first came out. At that time it appeared to be about two things: solving problems and writing "good" code. After I submit, I can see if I solved a problem and if not, where I failed. I like that, so the solving problems part is fine. For the "good" code part Cody gives me a seemingly random number. I don't like playing games in which I do not know what I am being scored on.

Then I saw http://blogs.mathworks.com/desktop/2012/02/06/scoring-in-cody/#comment-8624 and read in the about cody section: http://www.mathworks.co.uk/matlabcentral/about/cody/ that I could see the scoring code. Okay, this sound fun, I may disagree with there definition of "good", but at least I can read the rules.

The rules are basically:

x = length(mtree(answer))

sounds good, except that most of the work in mtree appears to depend on the compiled closed source built-in mtreemex. So the rules apparently are written down, but are not allowed to be read. That to me is the definition of a stupid game.

3 Comments

Jan Simon on 11 Mar 2012

Matlab 2009a and 2011b contain the source code of mtree, see "edit mtree" and "depfun mtree". It has 2600 lines of code and a lean documentation only. It reveals some interesting information, but it is not easy to get behind the meaning of the internal comments.

Daniel on 11 Mar 2012

Sorry about that, my comment/answer was confusing. I edit it to try and clarify. In 2011a the source of mtree is also available (3200 lines of code). The issue is that it appears that most of the 3200 lines of code for mtree depend on what is returned by mtreemex (a built-in function).

Jan Simon on 11 Mar 2012

Thanks, now I've found the mtreemex call also.
'a+b' has a shorter mtree than 'a+(b)'. At least this detail is a stupid rule for a coding contest.

Daniel
Answer by bym on 1 Feb 2012

I find myself in the past few days exploring Cody, rather than providing answers on "answers". Probably not a great drawback to answers, but I do feel a little guilty for not providing help where I can.

I like seeing Cody solutions to problems that are elegant, that I would not have thought of. Kind of a mini contest primer.

Now if I can just hunker down and improve my regexp() skills :(

1 Comment

Jan Simon on 1 Feb 2012

I prefer to get back Matt Fig and his humpday puzzlers.
Matt, where are you?

bym
Answer by Bjorn Gustavsson on 1 Feb 2012

Once uppon a time there were irregularly golfing contest problems on CSSM. I found those fun, inspirational and occasionally educational. There nothing was hidden behind any fancy locks and organized scoring, but worked just the same or more efficiently if learning is concerned.

Cody seems a total waste.

0 Comments

Bjorn Gustavsson
Answer by David Young on 1 Feb 2012

I just wrote a longish answer but lost it when the site went down briefly. The gist was that I think Cody is entertaining, a bit educational, but not to be taken too seriously.

However, I've noticed that solutions are now appearing that use evalin to hack the answer. This is so boring! I think it's vital that eval, evalin and assignin are banned. 'Solutions' that use these spoil the fun.

10 Comments

Ned Gulley on 3 Feb 2012

We have blocked some of the obvious ways to evaluate code, but there are others. Cody is just a game, and at some point people will have to decide whether they'd rather spend their time cheating or playing. We'll try to remove obvious cases of cheating, and over time we'll add features that help mitigate cheating. All online games have to deal with this.

Jan Simon on 4 Feb 2012

Thanks, Ned. Now it is getting clearer to me. I'm interested in blocking EVAL securely after I tried to create a bullet-proof P-file for an access limitation and copy protection.
It is very hard to define accurately, what a "game" is. Without doubt it needs well defined rules. Why is EVAL() cheating, while TRUE+'0' is not?
I think the reason is, that we tell the users in Matlab Answers every day, the EVAL is cheating in real programs also.

David Young on 4 Feb 2012

Jan, I think also that if someone uses eval() to reduce the apparent size of their program, or evalin() to poof the answer in rather than compute it, that's boring for everyone else and a waste of time for them. They will not learn new techniques or solve new problems, but they will add noise to the set of solutions.

Other slightly strange methods, like '011'-'0' to reduce the node count, may only make sense in the context of the game, but to exploit them you do have to think about the problem and how to solve it.

Those are the sort of reasons that make me feel it's sensible to ban eval etc.: they reduce the need to solve the problems, while tricks with different classes and so on don't have that effect.

Oh, and I do agree with you about avoiding eval() in real programs, in general.

David Young
Answer by David Young on 5 Feb 2012

It would be helpful to know what toolboxes are available to solution code. I'm sure I saw a solution that used a function from the stats toolbox (which I don't have) but functions from the Image Processing Toolbox aren't found. Should be all or none, or perhaps a definite list with some rationale behind it.

2 Comments

David Young on 7 Feb 2012

... and it seems that imfilter (in the IPT) used to be available, but isn't any more.

Jan Simon on 7 Feb 2012

There was a nice solution using IMDILATE to create a special pattern in a matrix. But as David found out already, the IPT is not available.

David Young
Answer by Aurelien Queffurust on 17 Feb 2012

Cody is my new drug

3 Comments

Jan Simon on 19 Feb 2012

Is this a positive or negative opinion about Cody?

Aurelien Queffurust on 21 Feb 2012

positive , I have learnt a lot of tricks from others solutions
drawback : sometimes the problem is unclear

Jan Simon on 21 Feb 2012

What kind of tricks did you learn? Are they useful only inside Cody, or also inside real-world Matlab programs?

Aurelien Queffurust
Answer by K E on 17 Feb 2012

Would love it, but keep getting errors that 'License checkout failed' on some of the testing of my solutions which requires resubmitting. Why?

3 Comments

Sean de Wolski on 17 Feb 2012

Please contact files@mathworks.com

Lindsay Coutinho on 22 Feb 2012

Hi K E, was just looking over your problem. Sorry we didn't get back to you sooner about this but the problem seems like you are using a flavor of 'diff' function from symbolic toolbox. We only allow MATLAB now. Make sure you are not using any toolboxes other than MATLAB. I hope this helps.

K E on 22 Feb 2012

Thanks!

K E

Contact us