@@ -28,14 +28,14 @@ VIII.Acknowledgement
28280. What's New
2929
3030JSONLab v2.0 - code named "Magnus Prime" - is a stable release of JSONLab and
31- a new milestone towards a stable, complete reference implementation for the
31+ a new milestone towards a stable, complete reference implementation of the
3232JData Specification (http://openjdata.org) for portable scientific data storage.
3333
3434There have been many major updates added to this release since the previous
3535release v1.9.8 in Oct. 2019. A list of the major changes are summarized below
3636(with key features marked by *), including the support to `_ArrayShape_` to
3737efficiently encode special matrices and the addition of `jsave/jload` to save
38- and restore variables in MATLAB/Octave like `save/load` commands (experimental):
38+ and restore variables in MATLAB/Octave like the `save/load` commands (experimental):
3939
4040* 2020-06-09*[ ] created `jdata` and `bjdata` python modules to share data with MATLAB
4141* 2020-06-08*[cbde607] add savebj and loadbj to dedicate to loading and saving bjdata
@@ -71,27 +71,28 @@ and restore variables in MATLAB/Octave like `save/load` commands (experimental):
7171* 2019-10-22*[650b5ec] enable jdataencode in savejson and saveubjson
7272
7373
74- Please note that JSONLab v2.0 is now compliant with JData Spec Draft 3, in
74+ Please note that JSONLab v2.0 is now compliant with JData Spec Draft 3; in
7575comparison, v1.9.8 is compatible with Draft 2; v1.9 and previous releases are
7676compatible with Draft 1. JSONLab v2.0 can read all data files generated by
7777v1.9.8, but v1.9.8 can not read the new UBJSON markers introduced in v2.0.
7878
79- The `jsave/jload` functions are experimental. The generated `.jamm` files are
80- renamed binary-JData/UBJSON files; they can be 50% smaller than `.mat` files
81- (if using `jsave(...,'compression','lzma')`) and can be readily loaded among
82- a long list of programming environments such as Python, JavaScript and Go.
79+ The newly introduced `jsave/jload` functions are in the experimental stage.
80+ They generate `.jamm` files which are renamed binary-JData/UBJSON files;
81+ they can be 50% smaller than `.mat` files if using `jsave(...,'compression','lzma')`
82+ and can be readily opened among a long list of programming environments
83+ such as Python, JavaScript and Go.
8384
84- The `saveubjson/loadubjson` functions now supports Binary JData specification (BJData)
85+ The `saveubjson/loadubjson` functions added support to the Binary JData specification (BJData)
8586Draft 1 (https://github.com/fangq/bjdata) and are now renamed as `savebj/loadbj`
86- (we kept `saveubjson/loadubjson` for compatibility purposes as aliases to the new
87+ (`saveubjson/loadubjson` are kept for compatibility purposes as aliases to the new
8788functions). The BJData spec is largely compatible with UBJSON spec Draft 12, with the
8889following differences (we are working with the UBJSON maintainer to merge
8990these two specifications):
9091
9192* BJData adds 4 new numeric data types: `uint16 [u]`, `uint32 [m]`, `uint64 [M]` \
9293 and `float16 [h]` ('''new in JSONLab v2.0''')
9394* BJData supports an optimized ND array container (supported in JSONLab since 2013)
94- * BJData does not convert NaN/Inf/-Inf to `null` (supported in JSONLab since 2013)
95+ * BJData does not convert ` NaN/Inf/-Inf` to `null` (supported in JSONLab since 2013)
9596
9697To avoid using the new type markers, one should attach `'UBJSON',1` in the `savebj`
9798command as
@@ -106,27 +107,27 @@ To convert an older file (JSON/UBJSON) to the new format, you should run
106107 data=loadjson('my_old_data_file.json','FormatVersion',1.9)
107108 savejson('',data,'FileName','new_file.json')
108109
109- You are strongly encouraged to convert all previously generated data files using the new
110+ You are strongly encouraged to convert all pre-v1.9.8 generated data files using the new
110111format.
111112
112113
113114-------------------------------------------------------------------------------
114115
115116I. Introduction
116117
117- JSONLab is a free and open-source JSON/UBJSON/MessagePack encoder and a
118- decoder in the native MATLAB language. It can be used to convert a MATLAB
118+ JSONLab is a free and open-source JSON/UBJSON/MessagePack encoder and
119+ decoder written in the native MATLAB language. It can be used to convert a MATLAB
119120data structure (array, struct, cell, struct array, cell array, and objects) into
120- JSON/UBJSON/MessagePack formatted strings, or to decode a
121+ JSON/UBJSON/MessagePack formatted strings and files , or to parse a
121122JSON/UBJSON/MessagePack file into MATLAB data structure. JSONLab supports both
122123MATLAB and [http://www.gnu.org/software/octave GNU Octave] (a free MATLAB clone).
123124
124125JSON ([http://www.json.org/ JavaScript Object Notation]) is a highly portable,
125126human-readable and [http://en.wikipedia.org/wiki/JSON "fat-free"] text format
126127to represent complex and hierarchical data, widely used for data-exchange in applications.
127128UBJSON ([http://ubjson.org/ Universal Binary JSON]) is a binary JSON format,
128- specifically designed to complement the limitations of JSON, permitting the
129- storage of binary data with strongly typed data records, resulting in smaller
129+ specifically designed to specifically address the limitations of JSON, permitting
130+ efficient storage of binary data with strongly typed data records, resulting in smaller
130131file sizes and fast encoding and decoding. MessagePack is another binary
131132JSON-like data format widely used in data exchange in web/native applications.
132133It is slightly more compact than UBJSON, but is not directly readable compared
@@ -143,7 +144,10 @@ to standardize serializations of complex scientific data structures, such as
143144N-D arrays, sparse/complex-valued arrays, trees, maps, tables and graphs using
144145JSON/binary JSON constructs. The text and binary formatted JData files are
145146syntactically compatible with JSON/UBJSON formats, and can be readily parsed
146- using existing JSON and UBJSON parsers.
147+ using existing JSON and UBJSON parsers. JSONLab is not just a parser and writer
148+ of JSON/UBJSON data files, but one that systematically converts complex scientific
149+ data structures into human-readable and universally supported JSON forms using the
150+ standardized JData data annotations.
147151
148152-------------------------------------------------------------------------------
149153
@@ -200,20 +204,20 @@ JSONLab is also available on Arch Linux. You may install it using the below comm
200204
201205III.Using JSONLab
202206
203- JSONLab provides a pair of functions, `loadjson` -- a JSON-to-MATLAB parser,
204- and `savejson` -- a MATLAB-to-JSON encoder, for the text-based JSON, and
205- two equivallent function pairs -- `loadubjson` and ` saveubjson` for binary
206- JSON and `loadmsgpack` and ` savemsgpack` for MessagePack. The `load*` functions
207- for the 3 supported data formats share almost the same input parameters ;
207+ JSONLab provides a pair of functions, `loadjson` -- a JSON parser, and
208+ `savejson` -- a MATLAB-to-JSON encoder, to read/write the text-based JSON; and
209+ two equivallent pairs -- `loadubjson/ saveubjson` for binary
210+ JSON and `loadmsgpack/ savemsgpack` for MessagePack. The `load*` functions
211+ for the 3 supported data formats share almost the same input parameter format ;
208212similarly for the 3 `save*` functions (`savejson/saveubjson/savemsgpack`).
209- These encoders and decoders are capable of converting/storing many different
213+ These encoders and decoders are capable of processing/sharing almost all
210214data structures supported by MATLAB, thanks to `jdataencode/jdatadecode` -
211- a pair of in-memory data converters that translate complex data structures
215+ a pair of in-memory data converters translating complex data structures
212216to the easy-to-serialized forms according to the JData specifications.
213217The detailed help information can be found in the `Contents.m` file.
214218
215- In the below section, we simply provide a few examples on how to use
216- each of the core functions for encoding/decoding JSON/UBJSON/MessagePack data
219+ In the below section, we provide a few examples on how to us each of the
220+ core functions for encoding/decoding JSON/UBJSON/MessagePack data.
217221
218222=== savejson.m ===
219223
@@ -277,7 +281,8 @@ will see the conversions from MATLAB data structure to JSON text and backward.
277281In `jsonlab_selftest.m`, we load complex JSON files downloaded from the Internet
278282and validate the loadjson/savejson functions for regression testing purposes.
279283Similarly, a `demo_ubjson_basic.m` script is provided to test the `saveubjson`
280- and `loadubjson` functions for various matlab data structures.
284+ and `loadubjson` functions for various matlab data structures, and
285+ ``demo_msgpack_basic.m`` is for testing ``savemsgpack`` and ``loadmsgpack``.
281286
282287Please run these examples and understand how JSONLab works before you use
283288it to process your data.
@@ -288,7 +293,7 @@ it to process your data.
288293Under the `test` folder, you can find a script to test individual data types and
289294inputs using various encoders and decoders. This unit testing script also serves as
290295a '''specification validator''' to the JSONLab functions and ensure that the outputs
291- are compliant to the specifications.
296+ are compliant to the underlying specifications.
292297
293298
294299-------------------------------------------------------------------------------
@@ -297,38 +302,38 @@ IV.Using `jsave/jload` to share workspace
297302
298303Starting from JSONLab v2.0, we provide a pair of functions, `jsave/jload` to store
299304and retrieve variables from the current workspace, similar to the `save/load`
300- functions in MATLAB and Octave. The files `jsave/jload` use is by default
301- a binary JData file with a suffix `.jamm`. The file size is comparable
305+ functions in MATLAB and Octave. The files that `jsave/jload` reads/writes is by
306+ default a binary JData file with a suffix `.jamm`. The file size is comparable
302307(can be smaller if use `lzma` compression) to `.mat` files. This feature
303308is currently experimental.
304309
305310The main benefits of using .jamm file to share matlab variables include
306311
307312* a `.jamm` file can be 50% smaller than a `.mat` file when using \
308313 `jsave(..., "compression","lzma")`; the only drawback is longer saving time.
309- * a `.jamm` file can be readily read/used among many programming environments, including \
310- Python, JavaScript, Go, Java etc, where .mat file support is not available. \
314+ * a `.jamm` file can be readily read/opened among many programming environments, including \
315+ Python, JavaScript, Go, Java etc, where ` .mat` file support is not generally available. \
311316 Parsers of `.jamm` is largely compatible with UBJSON's parsers available at \
312317 http://ubjson.org/?page_id=48
313318* a `.jamm` file is quasi-human-readable, one can see the internal data fields \
314319 even in a command line, for example using `strings -n 2 file.jamm | astyle`, \
315- making the binary data easy to be understood, shared and reused.
320+ making the binary data easy to be understood, shared and reused.
316321* `jsave/jload` can also use MessagePack and JSON formats as the underlying \
317322 data storage format, addressing needs from diverse applications. \
318323 MessagePack parsers are readily available at https://msgpack.org/
319324
320325
321326=== jsave.m ===
322327
323- jsave % save workspace to jamdata.jamm
328+ jsave % save the current workspace to jamdata.jamm
324329 jsave mydata.jamm
325330 jsave('mydata.jamm','vars',{'var1','var2'})
326331 jsave('mydata.jamm','compression','lzma')
327332 jsave('mydata.json','compression','gzip')
328333
329334=== jload.m ===
330335
331- jload % load from jamdata.jamm
336+ jload % load variables from jamdata.jamm to the current workspace
332337 jload mydata.jamm
333338 jload('mydata.jamm','vars',{'var1','var2'})
334339 jload('mydata.jamm','simplifycell',0)
@@ -338,23 +343,23 @@ The main benefits of using .jamm file to share matlab variables include
338343
339344V. Sharing JSONLab created data files in Python
340345
341- Despite the use of data annotation schemes defined by the JData Specification,
346+ Despite the use of portable data annotation defined by the JData Specification,
342347the output JSON files created by JSONLab are 100% JSON compatible (with
343348the exception that long strings may be broken into multiple lines for better
344349readability). Therefore, JSONLab-created JSON files (`.json, .jnii, .jnirs` etc)
345350can be readily read and written by nearly all existing JSON parsers, including
346351the built-in `json` module parser in Python.
347352
348353However, we strongly recommend one to use a lightweight `jdata` module,
349- developed by the same author, to pefrorm the extra JData encoding and decoding
354+ developed by the same author, to perform the extra JData encoding and decoding
350355and convert JSON data directly to convenient Python/Numpy data structures.
351356The `jdata` module can also directly read/write UBJSON/Binary JData outputs
352357from JSONLab (`.bjd, .ubj, .bnii, .bnirs, .jamm` etc). Using binary JData
353358files are exptected to produce much smaller file sizes and faster parsing,
354359while maintainining excellent portability and generality.
355360
356- In short, to conveniently read/write data files created by JSONLab in Python,
357- whether they are JSON based or binary JData/UBJSON based, one should download
361+ In short, to conveniently read/write data files created by JSONLab into Python,
362+ whether they are JSON based or binary JData/UBJSON based, one just need to download
358363the below two light-weight python modules:
359364
360365* **jdata**: PyPi: https://pypi.org/project/jdata/ ; Github: https://github.com/fangq/pyjdata
@@ -365,18 +370,18 @@ To install these modules on Python 2.x, please first check if your system has
365370
366371 sudo apt-get install python-pip python3-pip python-numpy python3-numpy
367372
368- After installation is done, one can then install the `jdata` and `bjdata` modules by
373+ After the installation is done, one can then install the `` jdata`` and `` bjdata` ` modules by
369374
370375 pip install jdata --user
371376 pip install bjdata --user
372377
373- To install these modules on Python 3.x, please replace `pip` by `pip3`.
378+ To install these modules for Python 3.x, please replace `` pip`` by `` pip3` `.
374379If one prefers to install these modules globally for all users, simply
375380execute the above commands using `sudo` and remove the `--user` flag.
376381
377- The above modules requires built-in Python modules `json` and NumPy (`numpy`).
382+ The above modules require built-in Python modules `json` and NumPy (`numpy`).
378383
379- Once the necessary modules are installed, one can start `python` (or `python3`), and run
384+ Once the necessary modules are installed, one can type `python` (or `python3`), and run
380385
381386 import jdata as jd
382387 import numpy as np
@@ -386,13 +391,13 @@ Once the necessary modules are installed, one can start `python` (or `python3`),
386391 data2=jd.loadb('myfile.ubj',object_pairs_hook=OrderedDict);
387392 data3=jd.loadb('myfile.jamm',object_pairs_hook=OrderedDict);
388393
389- where `jd.loadt()` function loads a text-based JSON file and perform
390- JData decoding and convert the enclosed data into Python `dict`, `list`
394+ where `jd.loadt()` function loads a text-based JSON file, performs
395+ JData decoding and converts the enclosed data into Python `dict`, `list`
391396and `numpy` objects. Similarly, `jd.loadb()` function loads a binary
392- JData/UBJSON file and perform similar conversion . One can directly call
397+ JData/UBJSON file and performs similar conversions . One can directly call
393398`jd.load()` to open JSONLab (and derived toolboxes such as jnifti:
394399https://github.com/fangq/jnifti or jsnirfy: https://github.com/fangq/jsnirfy)
395- generated files based on their respective default file suffix.
400+ generated files based on their respective file suffix.
396401
397402Similarly, the `jb.savet()`, `jb.saveb()` and `jb.save` functions
398403can revert the direction and convert a Python/Numpy object into JData encoded
0 commit comments