How to Use JSON in Your z/OS Program
First created in 2001, JavaScript Object Notation (JSON) has become the preferred way of creating structured data. Not limited to JavaScript applications, JSON formats can be found in web requests and responses and NoSQL databases. So, it makes sense that we will want our traditional z/OS programming languages (COBOL, PL/I, Assembler, C and REXX) to use and work with JSON data. And there’s good news: they already can.
Can JSON Work With Old School z/OS?
Before we get too excited, let’s take a look at an example of JSON data:
{"Client": { "id_":"00000001","GivenName":"John","Surname":"Smith"}}
Here’s our first problem: JSON doesn’t follow a fixed format. If we switched the Surname and GivenName fields, it’s still valid JSON. Similarly, if our JSON text doesn’t specify a surname, it’s still valid. This is actually one of the advantages of JSON: no predefined format. However, traditional languages are used to data in a fixed format: think C or COBOL structures.
JSON data is almost always encoded in UTF-8, which is our second problem: Traditional z/OS languages normally work in EBCDIC, with occasional ASCII. So, we’re also looking at code page conversions.
We could do things the hard way: manually using traditional language features to create, read and process JSON text. For example, the following REXX code could create our example JSON text:
json_text_ebc = '{"Client": { ' ||
'"id_:"' || ID || '”' ||
',"Surname":"' || SURNAME || '”' ||
',"GivenName":"' || GIVEN_NAME '”' ||
'}' /* Create JSON text in EBCDIC */
call bpxwunix 'iconv -f IBM-037 -t UTF8', json_text_ebc , json_text_utf8, err.
/* Convert JSON text to UTF-8 */
This manually creates our JSON in EBCDIC, then calls the z/OS UNIX command iconv to convert it to UTF-8.
This is a bit hard. It will get worse as our JSON text gets more complicated. Reading and processing incoming JSON text will be even harder. There has to be an easier way—and there is.
COBOL
From Enterprise COBOL 6.1, JSON documents can be parsed into COBOL structures. The trick is that we need to know the JSON structure before we can do this. So, to parse our string above, we need to define the COBOL structure that maps to the JSON string:
Working-Storage Section. 01 Client. 03 id_ Pic X(8). 03 Surname Pic X(20). 03 GivenName Pic X(20).
Variable names must be the same as the JSON names. We also need an area for our JSON text:
01 JSON-STRING Pic X(100).
Once our JSON string is in the variable JSON-STRING, we can parse it into our Client structure:
JSON PARSE JSON-STRING INTO Client END-JSON
A few things to note:
- The incoming JSON text is assumed to be in UTF-8. The Client structure output is in the codepage specified in the CODEPAGE COBOL compiler option: This must be a single-byte EBCDIC codepage.
- Escaped JSON strings like n for newline and ” for double quote are supported.
- The JSON PARSE statement has other parameters we haven’t shown: These can help match JSON fields with the COBOL fields (if the COBOL field names are different to the JSON ones), and decide what should, or should not, be converted from UTF-8 to EBCDIC.
- The JSON names can be in any order. If names are missing, this is fine as well.
From Enterprise COBOL 6.2, the opposite can be done—creating a JSON document from a COBOL structure. Using our previous JSON-DOC structure, the following command will create a UTF-8 JSON string from JSON-DOC:
JSON GENERATE JSON-STRING FROM Client END-JSON
PL/I
IBM Enterprise PL/I has offered native JSON features since version 4.5. But before we dive in, let’s look at PL/I variables used for JSON data. We’re going to need a character area to hold the JSON text, a pointer to this area, and a variable with the length of this area:
dcl JSONtext char(1000) var; /* Buffer holding JSON text */ dcl pJSON pointer; /* pointer to buffer */ dcl nJSON fixed bin(31); /* length of buffer */ pJSON = addrdata(JSONtext); /* set buffer pointer */ nJSON = length(JSONtext); /* set buffer length */
Let’s work with some incoming JSON text. The PL/I statement JSONVALID can be used to check if text is a valid JSON statement. So, we could code something like:
dcl posBad fixed bin(31); /* Position of first bad character */
posBad = JSONVALID(pJSON,nJSON); /* get location of first bad byte */
if posBad = 0 Then Display ("JSON OK"); /* no bad bytes */
else Display ("JSON not OK");
The JSONGETVALUE function can be used in a similar way to the COBOL JSON PARSE statement. Again, we first need a structure that maps to the format of our JSON text:
dcl /* Structure to hold our JSON record */ 1 Client, 2 id_ char(8), 2 Surname char(20), 3 GivenName char(20) varying;
Again, the variable names in the structure must match the JSON names.
To parse our JSON text into this structure, we could use:
dcl bytesJSON fixed bin(31); /* Number of bytes read from JSON text */ bytesJSON= JSONGETVALUE(pJSON,nJSON, Client);
The incoming JSON data is assumed to be in UTF-8, and is converted to the code page of the program (usually EBCDIC). Like COBOL, the JSON values can be in any order.
The JSONPUTVALUE function does the opposite—create JSON text from a structure:
bytesJSON= JSONPUTVALUE (pJSON,nJSON, Client);
There are other PL/I functions that can be used to work through a JSON text piece by piece. For example, JSONGETARRAYSTART checks to see if the next character starts a JSON record (i.e., starts with ‘[‘), JSONGETCOLON checks if the next character is a colon, and JSONGETOBJECTSTART checks to see if the next character is an opening brace ({). So, you could use this code to read in our JSON text:
locJSON = 0 /* Point to start*/ locJSON+= JSONGETOBJECTSTART(pJSON+locJSON, nJSON-locJSON); /* Jump to start */ locJSON+= JSONGETMEMBER(pJSON+locJSON, nJSON-locJSON,id_); /* Get ID */ locJSON+= JSONGETCOMMA(pJSON+locJSON, nJSON-locJSON); /* Jump to next */ locJSON+= JSONGETMEMBER(pJSON+locJSON, nJSON-locJSON,Surname); /* Get surname */ locJSON+= JSONGETCOMMA(pJSON+locJSON, nJSON-locJSON); /* Jump to next */ locJSON+= JSONGETMEMBER(pJSON+locJSON, nJSON-locJSON,GivenName);/* Get give name */ locJSON+= JSONGETOBJECTEND(pJSON+locJSON, nJSON-locJSON); /* Jump to end */
Other Languages
This is where we come to a stop. None of the other traditional programming languages have native features for JSON.
C and C++ programmers will find some libraries around that can be used to create and use JSON text. However, these are usually designed to work on other platforms (that don’t use EBCDIC). To port these to z/OS, we’ll probably need to do some code page translations to and from UTF-8.
At this point, REXX and Assembler programmers may be getting ready to throw this article away. But all is not lost. Enter the z/OS JSON parser.
z/OS JSON Parser
Introduced as a small programming enhancement to z/OS 2.1, the z/OS Client Web Enablement Toolkit is a set of callable programs that traditional z/OS programs can use to work with web services. There are two components of this toolkit: the HTTP protocol enabler to allow these programs to use HTTP/HTTPS; and the z/OS JSON parser—no prizes for guessing what this does.
So, how does it work? Supported programming languages (C, C++, COBOL, PL/I, REXX, Assembler) call a JSON parser program to do something with JSON text. User programs can be in almost any environment: batch, started task or z/OS UNIX process.
Suppose we want to process an incoming JSON text. Our program would:
- Initialize the environment (call JSON Parser program HWTJINIT)
- Parse a JSON text into a buffer for later processing (HWTJPARS)
- Work with this parsed JSON: for example, get values (HWTJGVAL), search for names (HWTJSRCH), get number of entries (HWTJGNUE)
- Terminate the environment (HWTJTERM)
Let’s take an example C program that reads our JSON text. As with COBOL and PL/I, we want some variables, including a variable to hold our JSON text, and a structure that maps to this JSON:
#include <hwtjic.h> /* JSON Parse includes needed for C */
typedef struct { /* Structure for incoming JSON values */
char[8] id;
char[20] surname;
char[20] given_name;
} JSON_record;
incoming_JSON JSON_record; /* Area to receive incoming JSON data */
HWTJ_PARSERHANDLE_TYPE parser_instance; /* Every call needs a parser instance */
HWTJ_DIAGAREA_TYPE diag_area; /* Area to use for incoming JSON data */
HWTJ_HANDLE_TYPE value_handle = 0; /* Needed when working with parsed JSON */
char *jsonText; /* Our incoming JSON text */
int retcode; /* Return Code */
So, now we need to initialize our environment, and parse the JSON code:
hwtjinit(&retcode, MAX_WORKAREA_SIZE,
parser_instance, &diag_area); /* initialize */
hwtjpars(&retcode, parser_instance,
(char *)& jsonText, /* JSON text string address(input) */
strlen(jsonText), /* JSON text string length (input) */
&diag_area); /* Put incoming JSON into a work area */
hwtjgval(&retcode, parser_instance,
value_handle, /* handle to a value (input) */
&string_value_addr, /* value address (output) */
&value_length, /* returned value length (output) */
&diag_area); /* area used for HWTJPARS output */
If we want to build JSON text, it’s not much different:
- Initialize the environment (HWTJINIT)
- Add entries to an outgoing JSON text stream (HWTJCREN)
- Terminate the environment (HWTJTERM)
The z/OS JSON parser is pretty smart, and will try to determine the code page of the incoming JSON text. For outgoing, the HWTJSENC program can be called to specify the outgoing code page.
More JSON Functionality
This isn’t all the JSON functionality you can find on z/OS. For example, there are a lot of ways to work between traditional mainframe applications and resources and JSON, including z/OS Connect and CICS web services.
Other tools such as Zowe can be used with more “modern” languages like JavaScript with Node.js to publish and consume web services. Finally, NoSQL database systems can be implemented on z/OS using Db2, or just plain old VSAM.
However, if you want to work with the “raw” JSON from your traditional mainframe programming language, there are some excellent options to get you going.