Code covered by the BSD License  

Highlights from
JSONlab: a toolbox to encode/decode JSON files in MATLAB/Octave

4.36364
4.4 | 24 ratings Rate this file 545 Downloads (last 30 days) File Size: 38.9 KB File ID: #33381
image thumbnail

JSONlab: a toolbox to encode/decode JSON files in MATLAB/Octave

by

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

 

19 Oct 2011 (Updated )

JSONlab is an open-source JSON/UBJSON encoder and decoder (parser) for MATLAB and Octave.

Editor's Notes:

This file was selected as MATLAB Central Pick of the Week

| Watch this File

File Information
Description

** JSONLAB v1.0 (Optimus - Final) is released on 01/02/2015.**
JSONlab is a component of the "iso2mesh" toolbox (http://iso2mesh.sf.net). For the latest information regarding JSONlab, please visit its homepage at http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab

JSON (JavaScript Object Notation) is a highly portable, human-readable and "fat-free" text format to represent complex and hierarchical data. It is as powerful as XML, but less verbose. JSON format is widely used for data-exchange in applications, and is essential for the wild success of Ajax and Web2.0.

UBJSON (Universal Binary JSON) is a binary JSON format, specifically optimized for compact file size and better performance while keeping the semantics as simple as the text-based JSON format. Using the UBJSON format allows to wrap complex binary data in a flexible and extensible structure, making it possible to process complex and large dataset without accuracy loss due to text conversions.

We envision that both JSON and its binary version will serve as part of the mainstream data-exchange formats for scientific research in the future. It will provide the flexibility and generality achieved by other popular general-purpose file specifications, such as HDF5, with significantly reduced complexity and enhanced performance.

JSONlab is a free and open-source implementation of a JSON/UBJSON encoder and a decoder in the native MATLAB language. It can be used to convert a MATLAB data structure (array, struct, cell, struct array and cell array) into JSON/UBJSON formatted strings, or to decode a JSON/UBJSON file into MATLAB data structure. JSONlab supports both MATLAB and GNU Octave (a free MATLAB clone).

JSONlab provides two functions, loadjson.m -- a MATLAB->JSON decoder,
and savejson.m -- a MATLAB->JSON encoder, for the text-based JSON, and
two equivallent functions -- loadubjson and saveubjson for the binary
JSON. The savejson, loadubjson and saveubjson functions were written by
Qianqian Fang, while the loadjson.m script was derived from the previous works by the following people:

- Nedialko Krouchev: http://www.mathworks.com/matlabcentral/fileexchange/25713
            date: 2009/11/02
- François Glineur: http://www.mathworks.com/matlabcentral/fileexchange/23393
            date: 2009/03/22
- Joel Feenstra: http://www.mathworks.com/matlabcentral/fileexchange/20565
            date: 2008/07/03

Please find detailed online help at http://iso2mesh.sourceforge.net/cgi-bin/index.cgi?jsonlab/Doc

Acknowledgements

Json Parser, (Another) Json Parser, and Highly Portable Json Input Parser inspired this file.

This file inspired Jsonrpc2: A Set Of Classes To Encode/Decode Json Rpc Messages, Deprecated Light Field Toolbox V0.2 V0.3 Now Available, Deprecated Light Field Toolbox V0.3 V0.4 Now Available, Jc Dstatus Plot, Slack Matlab, and Light Field Toolbox V0.4.

Required Products Communications System Toolbox
Simulink Verification and Validation
MATLAB
MATLAB release MATLAB 7.4 (R2007a)
MATLAB Search Path
/
/jsonlab
/jsonlab/examples
Other requirements JSONlab is platform independent.
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (85)
16 Apr 2015 Samaksh Behl

Can one create markups/annotations in a PDF using JASONlab?

Thanks!

Comment only
14 Apr 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Graham, your input, json, is not exactly an object. it is a string. based on JSON grammar, double quotes inside a string needs to be escaped, thus the backslashes, see http://www.json.org/

to convert json to an actually matlab object, you need to do

jsonobj=loadjson(json);

then you can save jsonobj to a JSON string:

savejson('',jsonobj)

if all you want to do is to save an already converted JSON string to a file, you can just use the standard file IO functions - fopen/fprintf/fclose, no need to use savejson.

Comment only
14 Apr 2015 Graham Warner

Why does savejson add backslashes to the string? I have an object "json" which reads:
>> json

json =

{"Duration":16420.479075,"Num_Samples":40000,"Frequency":13449312,"Num_Channels":17}

and then when I try to save it, I get backslashes:
>> savejson('',json,'/space/jazz/1/users/gwarner/jason.mat')

ans =

"{\"Duration\":16420.479075,\"Num_Samples\":40000,\"Frequency\":13449312,\"Num_Channels\":17}"

Comment only
30 Mar 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Pinghung, first of all, the loadjson function was derived from parse_json.m that you used as the reference. I am sure there is a simple tweak to restore the old behavior. Have you profiled both codes (profile on; loadjson(...); profile viewer)? it can tell you right away where was the difference.

Secondly, it is fairly easy to optimize for a special input to get good speed (again, use the matlab profiler is a good start), IMO, what is harder to achieve is to optimize the code so that it can get good speed for most types of inputs. This often needs a balanced solution. The changes in loadjson from the prior works were focused on accelerating the parsing of large arrays, which are most commonly encountered by matlab users. Try running the built-in speedtest benchmark and you will see the difference.

https://github.com/fangq/jsonlab/blob/master/examples/jsonlab_speedtest.m

thirdly, try a different versions of matlab, I found the later versions of matlab are ~10x faster when running loadjson.

PS: feel free to email me (fangqq at gmail.com) your input JSON file so I can find out exactly what happened with loadjson.

Comment only
30 Mar 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Laurent, you can disable the "FastArrayParser" by
setting it to 0, for example:

a = { 8 , 9 };
loadjson(savejson('',a),'FastArrayParser',0)

this will force treating all containers (array/object) to cells.

Comment only
30 Mar 2015 Laurent

this is great!
but one thing: cell arrays turned into matrices; how can I avoid this behavior ?
ex.:
a = { 8 , 9 };
savejson('',a,'f.mat');
b = loadjson('f.mat');
-> b is now the 1-x-2 matrix [ 8 9 ]
big thanks for the help

18 Mar 2015 Mark Mikofski

Mark Mikofski (view profile)

Pinghung, have you tried the Java implementation org.json or the .NET implementations json.NET and fastJSON. I explain how to use them in this Undocumented MATLAB post: http://undocumentedmatlab.com/blog/json-matlab-integration and in this Gist: https://gist.github.com/mikofski/2492355

Comment only
17 Mar 2015 Pinghung

Much slower than other implementations like parse_json.m. We need to quickly deserialize datasets fetched from a JSON API. Will need to find a faster way to deserialize in Matlab.

Benchmark 1:
deserialize 500 JSON datapoints (each with nested data)

parse_json.m 0.785s
loadjson.m 3.914s

Benchmark 2:
deserialize 5000 JSON datapoints (each with nested data)

parse_json.m 7.837s (scales linearly)
loadjson.m stopped test after 3 minutes and 100% CPU consumption.

03 Mar 2015 Iain Robinson  
07 Feb 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Alexander, you need to use the SimplifyCell flag, like this:

loadjson(savejson('',structarray),'SimplifyCell',1)

Comment only
06 Feb 2015 Alexander Mont

I have noticed that when I save a struct array in Matlab and then load the json file back again, it turns into a cell array. Is there a way to prevent it from doing this?

Comment only
15 Jan 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Kyle, you need to use the following command to reproduce the input:

y=loadjson(myjson,'FastArrayParser',2)

if a branch can be collapsed (by cell2mat), the FastArrayParser controls the depth of the output cell array. 0 disables this feature. Try setting it from 0 to 3 to see the difference.

In the future, I agree that loadjson should be more intelligent on which level can be simplified.

Comment only
14 Jan 2015 Kyle

Kyle (view profile)

Hi Qianqian Fang,

I cannot recover a cell containing numbers from your software. For example:

x = {[1],[1,2],[1,2,3]};
myjson = savejson('data',x);
y = loadjson(myjson);

In the example above, y does not equal x. I would like for y to be a cell just like x, but instead the numbers within x are collapsed into an array. Would you please change your code to make the input and output equivalent?

Comment only
03 Jan 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Calos, I think what you need is a vector transpose, see my example:

a=rand(100,1);
savejson ('myvar',a')

does this work for you?

Comment only
03 Jan 2015 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi everyone, JSONlab 1.0 is here! enjoy

Comment only
29 Dec 2014 Carlos

Carlos (view profile)

Hi Qianqian Fang. I have a var which is a 100x1 double and I would like convert it to JSON in the following form:

{
"myvar": [num1, num2, num3...],
}

I tried converting "myvar" to char and to cell but i don't achieve this form.
Thanks for your efforts.

Comment only
24 Dec 2014 Paul Macey

This just works - I had started writing my own decoder when I came across JSONlab, saving me a lot of effort. Thanks for the contribution.

22 Nov 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

a progress bar is now added to loadjson, see the latest svn commit:

http://sourceforge.net/p/iso2mesh/code/452/

use loadjson(...,'ShowProgress',1) to activate this feature.

Comment only
21 Nov 2014 b

b (view profile)

Is it possible to output the record number on the prompt as it loads each record in the datafile? If the number of records are too large, can the cursor give any indication or percentage of the amount of file read or number of records read? Thanks.

Comment only
19 Nov 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

Unfortunately, JSON comment using // or % is not part of the specification. A common workaround is to put comments into an element with special names, for example "_help" or "::comment::" etc, load the entire object, but not to use those values. JSONlab can handle special characters in the name fields, so you can name a comment quite distictively.

Comment only
19 Nov 2014 b

b (view profile)

Hello Qianqian,

How to use % or // or anything else to comment a particular field in a json file so that loadjson will not give an error:
Error using loadjson>error_pos (line 431)
JSONparser:invalidFormat: String starting with " expected at position 3267:
30:02.091Z",

Alternately, if we want skip a field while loading (say, "_id" : ), is there a way to do this?

Thanks.

Comment only
17 Nov 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@tmartin

Compact JSON output is now supported, see my latest commit:

http://sourceforge.net/p/iso2mesh/code/449/

add ('Compact',1) to the savejson parameter list to enable this feature.

the only formatting white space left now is the space after the colon. I will remove them in my future updates.

Comment only
17 Nov 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Nicolas

I added the requested help info in the latest commit, please see

http://sourceforge.net/p/iso2mesh/code/448/

Comment only
17 Nov 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Alessandro

thanks for pointing out this issue. It is indeed a bug related to the fast array parser that I added. I don't really have a good fix for this as it is related to how matlab eval() command handles an input string. However, I just added an option to loadjson that allows you to skip the fast parser for some array dimensions.

See my changes at:

http://sourceforge.net/p/iso2mesh/code/448/

your example can be properly handled using

B = loadjson('temp.json','FastArrayParser',2)

let me know if you see any addition issues related to this change.

Comment only
07 Nov 2014 Nicolas

Thanks a lot for your work.
Would it be possible to get some information on the parsing options that can be passed as input arguments to the loadjson function?
I've had a look on the toolbox home page but couldn't find anything.
Thanks for your help.

Comment only
06 Nov 2014 Timo Dörsam  
01 Nov 2014 tmartin

Is there a convenient way to produce a compact form of the JSON file?

Comment only
26 Oct 2014 T Lankhorst

I am amazed by the speed and usability of this software.

26 Oct 2014 T Lankhorst  
21 Oct 2014 Alessandro

Hi Qianqian,

thank you for making available this very useful library.

I want to report what it looks like to be a bug. I noticed that if a cell-array contains matrices having same number of columns, and save/load it, the library concatenates vertically the matrices, outputting a single matrix. Please look at the code here below:
A = {rand(2,4), rand(3,4)};
savejson('', A, 'temp.json');
B = loadjson('temp.json');
whos A B

which outputs:
Name Size Bytes Class Attributes
A 1x2 384 cell
B 5x4 160 double

Comment only
23 Sep 2014 Joerg Buchholz

Joerg Buchholz (view profile)

 
31 Jul 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Richard

feel free to share your patch/update by sending me an email: fangqq <at> gmail.com. I will review it and commit it to the code repository (and add acknowledge to your contribution in the documentation).

just to understand your request, by "serialization", did you mean adding data as additional parameters? In the past, I would often do it by using a cell array "{obj1, obj2, obj3 ...}"

Comment only
30 Jul 2014 Richard

Excellent library, really useful. I have a feature request though - adding support for serialisation of Matlab objects. I have modified the code so that struct2json is called with an object, which works correctly. Is there anywhere where I can contribute these changes back?

23 Jul 2014 Evgeny

Evgeny (view profile)

Was really useful for me for a simple task. One command and you're done

23 Jul 2014 Evgeny

Evgeny (view profile)

 
11 Jul 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

that's not true. high dimensional arrays are supported. what's not supported is the high-dimensional cell arrays.

Comment only
11 Jul 2014 Dmitry

Dmitry (view profile)

You can't save multidimensional arrays with dimensionality larger than 2, is this true?

Comment only
30 Mar 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Kai

can you try converting your input "review" (a char array) to unicode first by using native2unicode()?

not sure if it helps, in the last test in jsonlab/sample/demo_jsonlab_basic.m, I have a utf-8 encoded Chinese string. loadjson/savejson seem to work well with this example (my locale is en_US.UTF-8).

Comment only
30 Mar 2014 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

sorry, not in jsonlab_selftest.m, but demo_jsonlab_basic.m

Comment only
29 Mar 2014 Kai

Kai (view profile)

Works great with Romanized characters. However, I ran into some character encoding issues. I am trying to read in script-type languages (Arabic, Urdu, Dari, Farsi, etc.) and the imported text is

{"text":"حمایت اتحادیه میوه فروشان و سبزی فروشان افغانستان از تیم اصلاحات و همگرایی؛\nحوالی ساعت 8 امشب 27 ماه حوت سال روان... http://t.co/AAcvfHxYAN","isoLanguageCode":"fa"}

Resulting text stored in the array:

حمایت اتحادیه میوه فروشان و سبزی فروشان افغانستان از تیم اصلاحات و همگرایی؛ حوالی ساعت 8 امشب 27 ماه حوت سال روان... http://t.co/AAcvfHxYAN

I can get the characters to show up properly from a text file if:
a. I set my local Language Settings for
Persian/Arabic and

b. I specify UTF-9 encoding during fopen

>> file = fopen('tweet.txt','r','n','UTF-8');
>> review = fread(file,'*char')';
>> fclose(file);

Any thoughts?

18 Mar 2014 Augustin

A typical example is a large music library. That being said it does not seem to be the use this toolbox was intended for.

Comment only
18 Mar 2014 Augustin

Pros : it's easy to use

Cons : 1) poor documentation (what are the options of the parser ?)

2) This toolbox appears to be optimized for large arrays, but it is not efficient when fields are small (e.g. strings, numeric values) but the json file is still big.

29 Jan 2014 Albert

Albert (view profile)

Very useful, thank you. Have been using it for months. Quite fast.

29 Jan 2014 E. L. Rayle

Works great. Performs the conversions between string and struct like a champ.

17 Oct 2013 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

based on the error, it sounds like you had an invalid json file. have you tested your json file by one of the json validators? for example:

http://jsonlint.com/

Comment only
17 Oct 2013 J

J (view profile)

(someone please delete my previous question I have found the solution)

Problem with input:

data = loadjson('data.json')

I get error:

Error using loadjson>error_pos (line 415)
JSONparser:invalidFormat: Outer level structure must be an object or an array: /<error>

Error in loadjson (line 70)
error_pos('Outer level structure must be an object or an array');

Comment only
16 Oct 2013 J

J (view profile)

I've attempted to use loadjson.m but I keep getting the error at line 47 saying "input file does not exist". All I am doing is "data = loadjson('data(5).json')" and that data file is within the same directory. Please help (or redirect me to where I should type my question if this is not the appropriate place). Many thanks!

Comment only
19 Feb 2013 Mark Mikofski

Mark Mikofski (view profile)

@Jan, I've been using a JSON.jar [1] I made from the java-json source [2] and it works perfectly. I don't know how native java speed compares with native MATLAB though.

[1] http://dl.dropbox.com/u/19049582/JSON.jar
[2] http://json.org/java/

Comment only
19 Feb 2013 Leah

Leah (view profile)

I would have liked if there was an option to return as a dataset. Added these lines

data=data.data;
data=struct2cell(data);
data=cell2mat(squeeze(data)');
data= dataset({data(:,1), 'TimeEPOC'}, {data(:,2), 'Value'});

28 Jan 2013 Anssi Klapuri

Correction to my previous post: The mentioned line should read "currstr=inStr(pos:min(pos+100,end));" of course.

Comment only
28 Jan 2013 Anssi Klapuri

Excellent contribution. For large json files (mine was >100MB), you can make the code >100 times faster by replacing the line "currstr=inStr(pos:end);" with "currstr=inStr(pos:pos+100);" in parse_number().

08 Oct 2012 John Reinert Nash

Useful, fast parser. Much appreciated!

06 Sep 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Chuck37

the issue you had reported is related to the cell2mat call on line#248 of loadjson. you can compare the following:

a=struct('one',1,'two',2);
b=[a a];
c={a,a};
class(b)
class(c)
class(cell2mat(c))

in many other cases, this kind of conversion is beneficial because it simplifies the data by looking into the internal regularity. However, it does come with the side-effect you have described.

To solve this issue, I just committed SVN#384, accepting a flag to control cell array->struct array conversion, see

http://iso2mesh.svn.sourceforge.net/viewvc/iso2mesh?view=revision&revision=384

you may check out this version and run the following to compare:

d=loadjson(savejson('',c))
class(d)
d=loadjson(savejson('',c),'SimplifyCell',1)
class(d)

by default, loadjson no longer converts a cell array to a struct array.

Comment only
06 Sep 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

@Arnout: 2D cell is currently not supported by jsonlab. I hope this can be added in the future versions. Contribution on this is welcome!

Comment only
29 Aug 2012 Arnout Standaert

Thanks a bunch for JSONLab, it is saving me a lot of time writing my own serializer...

One question: I notice that 2-D cell arrays of strings get squashed to a 1-D JSON array of strings. Is it possible to keep the dimensionality of the cell array?

22 Aug 2012 Chuck37

Thanks for making this. One issue I have is that sometimes the reader converts JSON files to cells and sometimes to structures. If I understand right, this is a function of whether the fields are exactly the same. For my application, this is a real pain because sometimes the fields are out of order (cell) and sometimes they aren't (array) and my processing code can't work with both without a lot of work. Is there a way I can force it to always make cells? I know this may not be as fast, but I need to know what I'm going to get.

Comment only
22 Aug 2012 Chuck37  
06 Jul 2012 Jonathan Lister

Jonathan Lister (view profile)

Works great. Seems really fast for my use. I wanted to save structs in human readable format. At first I went for XML tools, but found them lacking. JSON is less bloated than XML. I like it.

20 Jun 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Mark

I just uploaded a new JSONLAB release, v0.9.0. Aside from various bug fixes, one of the main features of this release is the better round-trip consistency you suggested earlier. A leading underscore is now converted to x0x5F_; all hex codes generated by loadjson are automatically converted back to string forms in savejson.

A remaining issue is the handling of multi-byte characters in matlab. If this is the case, you may have to run feature('DefaultCharacterSet','latin1') in matlab to get consistent results.

Let me know if you are happy with this new release. (download from http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab/Download )

Comment only
06 Jun 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Mark

taking out the try-catch block can give you very verbose output. For a simple array input, the output becomes a cell array with each element a cell. It becomes very difficult to read and process.

I think the difficulty we are facing comes from the ambiguity when mapping a matlab data structures to JSON. For example, when you say "foo", you can map it to "foo", {"foo"} or ["foo"], or even {["foo"]}. I chose the simplest one for now, because I don't know what we would gain from having the brackets (differentiate from an array origin or a cell origin?)

For the underscores, a unique prefix/suffix seems to be the safe way to go. If I were to implement this feature, I would add something like JSONLAB_LEADING_UNDERSCORE_ to both loadjson/savejson, and use opt.LeadingUnderscore as an input option.

Comment only
30 May 2012 Mark

Mark (view profile)

I should have been clearer with my first question: What I would like is for savejson to keep cell arrays of strings as cell arrays (i.e., to keep the single brackets around them). That is, I would like
O.test = {'x'};
s = savejson('',O)
to output:
s = '{"test":["x"]}'

You said, "both loadjson and savejson were designed to eliminate a JSON array that has a single element by default." - what was your reason for making this choice? And is there some way I can hack around in the code to get the behavior I want out of it without having 289 other things go wrong?

Also: I found that if I comment out the try-catch-end block you mentioned in your response to Nancy (loadjson.m lines 247-253), I can ensure that bracketed json objects (even with only one element: '["x"]') are interpreted as cell arrays ({'x'}) (which I want). Is making that change going to get me in trouble elsewhere in the code?

As to the underscores - well, I was thinking about a unique prefix you could use (or post-fix? two trailing underscores is legal (and uncommon) in matlab...?) But I can work around this myself if you'd rather not change both functions. Thanks!

Comment only
30 May 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Mark

regarding your first question, no, the NoRowBracket only applies to "numerical elements". I don't see any problems for processing array of strings though.

For your second question, it is not difficult for me to add a non-standard option to do the conversion. However, the current approach to deal with a leading underscore is to prepend an "x", similar to the behavior of MATLAB function genvarname(). However, once converted, a name in the form of "x_..." becomes difficult to separate from the genuine ones. A conversion from "x_..." to "_..." may have false positives. I am not sure if this is the best way to do, unless I define a unique prefix in the loadjson for this scenario.

Comment only
30 May 2012 Mark

Mark (view profile)

Hi Qianqian,

Thanks for the excellent code!

Can you please clarify your answer to Nancy's question above? When you wrote,

For savejson, there is a flag "NoRowBracket" controls if you want to force a [] around a single numerical element. By default, [] is not printed. Although this was not implemented for string arrays, a similar behavior is probably preferred to be consistent.

...did you mean that you intend to implement NoRowBracket functionality for arrays of strings? (And if so, might I add some encouragement for you to do so?)

Also, though I understand the complexity of achieving full round-trip invariance with loadjson and savejson, I wonder if you could be convinced to make one special case of round-trip invariance:

Leading "_" characters are commonly used in python variables, and are necessary for "_id" and "_rev" field names if you are working with json objects in couchdb. Is there any chance you could ensure that savejson un-did the string substitutions made to leading underscores in loadjson?

Comment only
25 May 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Cyril

thanks for pointing out this bug. I just committed a fix in the SVN:

http://iso2mesh.svn.sourceforge.net/viewvc/iso2mesh?view=revision&revision=369

Please check it out. Your example will have a output of

{
"test": {
"foo": null,
"bar": 0
}
}

a struct element with either {} or [] will be mapped to null.

Let me know if this works out for you.

Qianqian

Comment only
23 May 2012 Cyril Davin

Hi Qianqian,

Thanks for your nice work.
Here is a small bug :
s.foo={};s.bar=0; savejson('test',s)
The foo field and empty cell array disappear.

regards,
Cyril.

Comment only
25 Apr 2012 Mark Mikofski

Mark Mikofski (view profile)

FYI: for those interested in using Java org.json in MATLAB

Demo:
https://gist.github.com/2492355
git clone git://gist.github.com/2492355.git orgJSONdemo

JSON in Java (org.json):
http://www.json.org/java/index.html

Using Java Collections in Matlab:
http://undocumentedmatlab.com/blog/using-java-collections-in-matlab/

Storing MATLAB structs in Java Objects

Java SE 6 (1.6) API:
http://docs.oracle.com/javase/6/docs/api/

Java HashMap
http://docs.oracle.com/javase/6/docs/api/java/util/HashMap.html

Java ArrayList
http://docs.oracle.com/javase/6/docs/api/java/util/ArrayList.html

Passing Data to a Java Method:
http://www.mathworks.com/help/techdoc/matlab_external/f6425.html

Handling Data Returned from a Java Method:
http://www.mathworks.com/help/techdoc/matlab_external/f6671.html

Bringing Java Classes and Methods into MATLAB Workspace: Simplifying Java Class Names:
http://www.mathworks.com/help/techdoc/matlab_external/f4863.html#f46341

Working with Java Arrays:
http://www.mathworks.com/help/techdoc/matlab_external/f15351.html

Concatenating Java Objects:
http://www.mathworks.com/help/techdoc/matlab_external/f4873.html#f48488

javaArray:
http://www.mathworks.com/help/techdoc/ref/javaarray.html

Comment only
24 Apr 2012 Mark Mikofski

Mark Mikofski (view profile)

Thank you Q. Fang for this excellent program. As an alternative anyone could just use the org.json files directly from java. they are here:
https://github.com/douglascrockford/JSON-java
and directions are here:
http://www.json.org/java/index.html
You need to compile them or just export them to a jar file which requires jdk. I have compiled it already for you here:
http://dl.dropbox.com/u/19049582/JSON.jar
Before you can use the org.json library you have to add to your java class path (either dynamic or static) by following the directions here:
http://www.mathworks.com/help/techdoc/matlab_external/f4863.html
For example: javaaddpath('full-path\JSON.jar') will add the jar file to your dynamic path.
Then you can use java which is native to MATLAB to use the org.json library.
For example:
jsonObj = org.json.JSONObject('{myKey:myVal}')

Comment only
20 Apr 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Nancy

both loadjson and savejson were designed to eliminate a JSON array that has a single element by default. In loadjson, line#248, the call cell2mat() attempts downgrading a 1x1 cell array to an array of the element type (unless there are mixed types). Test class(loadjson('["x"]')) with and without that try/catch block, you can see the difference.

For savejson, there is a flag "NoRowBracket" controls if you want to force a [] around a single numerical element. By default, [] is not printed. Although this was not implemented for string arrays, a similar behavior is probably preferred to be consistent.

To keep round-trip invariance is not likely, partly because JSON syntax has redundancies. What it might be achievable is to keep loadjson/loadjson->savejson->loadjson, or similarly, savejson/savejson->loadjson->savejson, giving similar answers. But it is still a quite challenging task.

Qianqian

Comment only
20 Apr 2012 Nancy

Nancy (view profile)

Hi Qianqian,

I noticed that when I have only 1 value in a json array, a roundtrip load/save doesn't give me the original json string (square brackets are left out).
Example:
>> jsonStr1 = '{"keys" : ["value1"]}';
>> savejson('',loadjson(jsonStr1))

ans =

{
"keys": "value1"
}

When there is more than 1 value in the json array, there is no problem.
Example:
>> jsonStr2 = '{"keys" : ["value1", "value2"]}';
>> savejson('',loadjson(jsonStr2))

ans =

{
"keys": [
"value1",
"value2"
]
}

Also, will there be future updates to convert the illegal character substitutions created by loadjson, when using savejson? For example, '_0x2F_' will be converted back to '/' when using savejson.

This is great though, thanks!

Comment only
21 Mar 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Francois

both loadjson and savejson are expected to handle the input items in a sequential order.

For savejson, I am sure an numeric array/cell/cell array/string array are saved by element indices.

For struct inputs, savejson encodes each sub-field based on its "defined" orders. That means
savejson('root',struct('f1',1,'f2',2))
and
savejson('root',struct('f2',2,'f1',1))

give you different outputs. If you want to order it in a deterministic way, you can use orderfields() function to treat the struct first.

I do want to mention one thing, currently jsonlab can not handle high-dimensional cell/struct arrays. It converts it to 1D array based on matlab's column-major ordering. For example:
a=struct('f1',1,'f2',2);
a=repmat(a,2,2);
savejson('',a)

I expect to improve this in the later versions. Any contribution on this is welcome!

Qianqian

Comment only
21 Mar 2012 Francois Rongère

Promising work !!
I will try to implement it in my researchs where we have a lot of heterogenous data as parameter of our simulations.

@Fang : do we have the insurance to always have the same order of nodes in the json file so I can use diffs and version control over json parameters files ?

Regards

15 Mar 2012 Sally

Sally (view profile)

Release 0.8.1 solved the problem I had on 27 Feb. The bracket depth on my updated json remains identical to the depth in the input file.

We can now use jsonlib to record steps in our image processing pipeline. Thank you for adjusting this and doing it so quickly.

13 Mar 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

I don't see problems for either case on my side:

>> works = struct('root', struct('root2', struct('root3', [1 2]')));
>> works_out = loadjson(savejson('' , works));
>> works_out.root.root2.root3

ans =

1
2

>> doesnt_work = struct('root', struct('root2', struct('root3', [1 2])));
>> doesnt_work_out = loadjson(savejson('' , doesnt_work));
>> doesnt_work_out.root.root2.root3

ans =

1 2

I even tested on octave, no problem at all. Can you double check and make sure you have downloaded the latest version (version 0.8.1)?

Comment only
13 Mar 2012 Oscar

Oscar (view profile)

I'm sorry again(!). Here is how the example should be.

works = struct('root', struct('root2', struct('root3', [1 2]')));
works_out = loadjson(savejson('' , works));

doesnt_work = struct('root', struct('root2', struct('root3', [1 2])));
doesnt_work_out = loadjson(savejson('' , doesnt_work));

Comment only
13 Mar 2012 Oscar

Oscar (view profile)

I'm sorry, my previous post didn't appear for some reason. Here is it:
works = struct('root', struct('root2', struct('root3', [1 2]')));
works_out = loadjson(savejson('' , meta2));

donesnt_work = struct('root', struct('root2', struct('root3', [1 2])));
donesnt_work_out = loadjson(savejson('' , meta2));

Comment only
13 Mar 2012 Oscar

Oscar (view profile)

also, in the case that works. The ouptut 'works_out' is not exactly the same as the input. In fact, it has turned into 'doenesnt_work'.

Comment only
13 Mar 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Oscar

I tested the following commands, it works fine with the current release of jsonlab:

a=struct('f1',[1 2 3 4])
savejson('',a)
a2=loadjson(savejson('',a))

I am happy to consider your patch, but it would be really helpful if you can give me your test script that failed earlier.

Comment only
13 Mar 2012 Oscar

Oscar (view profile)

hi Qianqian,

I found the problem. If the struct contains row-vectors, they don't get properly encoded. I added these simple lines to the mat2json subfunction, that convert the row-vectors to column-vectors, and now it works like a charm.

if ((size(item, 1) == 1) && size(item, 2) > 1)
item = item';
end

thanks

Comment only
13 Mar 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Oscar

please post your sample script and data so we can debug.

Qianqian

Comment only
13 Mar 2012 Oscar

Oscar (view profile)

In my experience, this doesn't work for all kinds of object, and remarkably, I couldn't load the output json struct with 'loadjson' that was created by 'savejson'. I wonder if there might be fundamental problems / ambiguities with a json matlab - struct interface?

06 Mar 2012 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Sally, I just updated jsonlab to 0.8.1. A change in this release is to avoid forcing a "root" object in savejson.m. To do this, you will need to use savejson('',obj) to save your structure.

So, download the new version and give it a try. Let me know if this is what you want.

Comment only
27 Feb 2012 Sally

Sally (view profile)

My json gets an extra set of curly brackets on each loadjson/savejson round trip. This effect is shown in each example on your nice page: http://iso2mesh.sourceforge.net/cgi-bin/index.cgi?jsonlab/Doc/Examples.

Is there a way to avoid this? I would like to use matlab to add an element to an existing json file, without nesting the content deeper in the result file.

thanks, sally

Comment only
24 Oct 2011 Jan Simon

Jan Simon (view profile)

@Fang: (I hope this is your "1st" name) Thanks for your answer. I will try to test the speed improvement of my suggestions soon.
The current version number starting with 0 implies, that further development is planned. I expect, this is going to be a 5 star submission finally.

Comment only
22 Oct 2011 Qianqian Fang

Qianqian Fang (view profile)

  • 1 file
  • 545 downloads
  • 4.36364

hi Jan

thanks for your comments. The issues you pointed out in loadjson.m are largely inherited from the previous authors (except for the OCTAVE_VERSION one). I will consider your comments and improve the loading speed in the future. You are also welcome to submit patches and profiling results to help me out.

Also, please go to http://iso2mesh.sf.net/cgi-bin/index.cgi?jsonlab/Download and download version 0.5.1, released yesterday. I realized I made a mistake in 0.5.0 and my optimized code on loading large data was not enabled. With the new code, loading a large matrix from JSON file can be 30x faster (by passing the parse_number calls in parse_array).

Hopefully version 0.5.1 will be updated on Matlab File Exchange soon.

Comment only
22 Oct 2011 Jan Simon

Jan Simon (view profile)

The excessive usage of global variables reduces the speed remarkably and might interfer with other programs.
exist('OCTAVE_VERSION', 'builtin') is a better test, because it does not search the file "OCTAVE_VERSION.m" in the complete Matlab path.
With pre-allocations the code could be accelerated very much. Dynamic fieldnames are faster than GETFIELD. STRMATCH is extremly slow and deprecated.
"eval(sprintf('object.%s = val;', valid_field(str)))" is cruel. Better: "object.(valid_field(str)) = val".
MLint suggests to omit some FINDs.
This function is very useful, because JSON files are highly efficient for a certain kind of data. If the speed is improved furtherly, this is a very good submission.

Comment only
Updates
21 Oct 2011

ChangeLog:

 2011/10/21 fix a bug in loadjson, previous code does not use any of the acceleration
 2011/10/20 loadjson supports JSON collections: appendable JSON objects

19 Jan 2012

ChangeLog:
-2012/01/13 speed up loadjson by 20 fold when parsing large data arrays in matlab
-2012/01/11 remove row bracket if an array has 1 element
-2011/12/22 accept sequence of 'param',value input
-2011/11/18 fix struct array bug

05 Mar 2012

ChangeLog:

 2012/02/28 loadjson quotation mark escape bug, see http://bit.ly/yyk1nS
 2012/01/25 patch to handle root-less objects, contributed by Blake Johnson

20 Jun 2012

ChangeLog

- 2012/06/17 *new format for an invalid leading char, unpacking hex code in savejson
- 2012/06/01 support JSONP in savejson
- 2012/05/25 fix the empty cell bug
- 2012/04/05 savejson can save to a file

26 Aug 2013

update to 1.0 alpha release, included functions (saveubjson and loadubjson) for writing and reading Universal Binary JSON files

03 Feb 2014

- update to jsonlab 1.0 beta release
- fix file read and write bugs in loadubjson and saveubjson

24 Nov 2014

ChangeLog:
- fix UBSJON data endianness for all integer types
- fix several compatibility issues for GNU octave
- support 2D cell and struct arrays
- escape special characters in a JSON string
- fix a bug when saving UBSJON files

03 Jan 2015

ChangeLog:
- update documentation, finalize 1.0
- fix a bug to strictly respect NoRowBracket in savejson

Contact us