YAJL Meets DATA-INTO

There’s now another even easier way in which you can apply YAJL to many tasks where you need to consume JSON

 We first introduced readers to Scott Klement’s port of the YAJL JSON toolkit in “An RPGer’s First Steps With JSON.” That article discussed using YAJL to generate JSON. We covered the use of YAJL to consume JSON in the follow-up article “Consuming JSON data with YAJL.”  However, since that article was published, things have changed and there’s now another even easier way in which you can apply YAJL to many tasks where you need to consume JSON. How? Well the ever-prolific Scott Klement has now added a DATA-INTO parser to the YAJL toolkit and it works beautifully.

Before we start the discussion of how to use the YAJLINTO parser, a few words on why we are so pleased to see this new addition to the toolkit. Those of you who have some familiarity with DATA-INTO may know that IBM actually ships a JSON parser along with the support materials for DATA-INTO that are delivered in library QOAR. So why not just use that one? The simple answer is that IBM didn’t supply this parser as a “production ready” piece of code with the same level of resilience as the XML parser that is supplied with XML-INTO.

Rather, the supplied JSON parser is intended as a teaching tool to demonstrate how to write a DATA-INTO parser. If you want to use it in production you can do so, but you will soon begin to encounter some limitations in the types of JSON document that it can process.

Enter the YAJLINTO parser, which, as best we can tell so far, can handle any JSON document that you can throw at it. Let’s take a look at how to use it.

The document we are going to process in this example looks like this:

 [ { "ID": 12345, "Active": true, "Name": "Paris", "Address": { "Street": "Main Street", "City": "Jasontown", "State": "CA", "Zip": "12345" } }, { "ID": 23456, "Active": false, "Name": "Rich", "Address": { "Street": "South Road", "City": "Hailsham", "State": "UK", "Zip": null ... 

The file consists of an array of objects that represent customer information. There are two main items of interest in this particular document. The first is that the element “Active” is a boolean true/false. The second is that the “Zip” entry in the second array element has a value of null. Both of these are common in JSON documents and YAJLINTO provides specific options for handling them which we will discuss later. However, for our first example, we’ll just stick with using the parser’s default handling.

As we mentioned in our first article on DATA-INTO, “XML-INTO meets Open Access With RPG’s New DATA-INTO” the way in which DATA-INTO works is very similar to XML-INTO. For that reason, we’re not going to delve too deeply into the base mechanics of DATA-INTO. Rather, we’ll draw comparisons and note the differences between the two. If you’re not familiar with the way that XML-INTO works and want more details than we’re providing here, we suggest that you read “A Traditional Approach to a Modern Technology.” 

The first thing we need to do is to define the target data structure to hold the document. It’s shown below starting at (A). Since there are multiple customers represented in the document we have coded it as a DS array of 99 elements. Within each element we have the fields ID, Active, and Name, followed by the nested DS Address (B), which maps to the “Address” object in the JSON document. Note that since Active is a Boolean, we’ve represented it as an indicator here since YAJLINTO’s default behavior is to return the value “0” for false and “1” for true.

Since the target for our DATA-INTO operation is an array, we can take advantage of the fact that RPG will supply a count of the active elements in the PSDS. You can see how we have defined that at (C).

 ((A) dcl-ds Customers Dim(99) Qualified; ID packed(5); Active ind; Name char(30); (B) dcl-ds Address; Street char(40); City char(30); State char(2); Zip char(10); end-ds; end-ds; (C) dcl-ds *N psds; count Int(20) Pos(372); end-ds; dcl-s jsonFile varchar(200) inz('/home/Paris/Extra/customerData1.json'); dcl-s i int(10); dcl-s wait char(1); (D) Data-into Customers %Data( jsonFile : 'doc=file case=any') %Parser( 'YAJL/YAJLINTO' ); (E) For i = 1 to count; // Loop through results Dsply ( 'Customer ' + %EditC( Customers(i).ID: 'X') + ' Active = ' + Customers(i).Active + ' Zip = ' + Customers(i).Address.Zip ); endfor; Dsply ('Processed ' + %Char(count) + ' Customers' ) ' ' wait; *InLr = *On; 

Where DATA_INTO really differs from its sibling XML-INTO is in the third parameter %PARSER. XML-INTO has no need of this parameter since it uses IBM’s built-in XML parser. But with DATA-INTO we’re responsible for providing the parser and it is %PARSER’s job to identify the program or subprocedure to be used. It can also optionally provide additional parameters to control the parsing process. This option isn’t used in this first example as we are taking all of the defaults. More on this in a moment.

Once the DATA-INTO operation completes, we then use the PSDS element count (count) to loop through the results (E). If you were to study the output, you’d see that the field Active is set to ‘1’ for values of true and ‘0’ for false. These are the defaults used by YAJLINTO. Similarly the value of the second customer’s zip code has been set to the default value of ‘*NULL’.

Changing the Defaults

As we noted, the YAJLINTO parser provides the capability for changing a number of default behaviors via the second parameter to the %PARSER BIF. This parameter data must be in the form of a valid JSON document and can contain any of the following options:

• “value_true” – this supplies the value to be used for boolean true values. By default, it’s set to “1” but can be any character value you choose.
• “value_false” – you can probably guess what this one does! The default is “0”. As an example, suppose our RPG application requires the values “Y” and “N” for the true/false values. We can achieve this by using the parameter string: { “value_true” : “Y”, “value_false” : “N” }
• “value_null” – the default value is ‘*NULL’ but in some cases you might want to set it to blanks, or to some high or low value.
• “skip_document_node” – Defaults to false. This is a difficult one to explain so we’ll do so in a separate section below.
• “document_name” – This allows you to provide a name for an outer element so that inner elements can be referenced by the path= %DATA option. There is no default value.

Notes on “skip_document_node”

Some JSON documents are designed to mimic an existing XML document. The problem in designing such documents is that unlike XML, JSON does not allow a name for the root element. In order to provide a name at this level, therefore, they are coded with an additional outer layer. For example:

 { "Customers" : [ { "ID": 12345, .... 

To process this with YAJL-INTO would normally require that the DS be modified to reflect this additional level like this:

 dcl-ds results Qualified; count_Customers Int(10); dcl-ds Customers Dim(99); ID packed(5); .... 

It also means that we cannot use the PSDS count as we are no longer loading directly into an array. This in turn would force us to add the countprefix processing option and the associated count_Customers variable. If you’re not familiar with the use of countprefix see “XML-INTO Revisited.” 

In order to simplify the required Data Structures etc. YAJLINTO allows us to specify { “skip_document_node” : true }. This causes the parser to completely ignore the outer document node with the result that the same Data Structure and options used in our original example are valid. In fact this is the default behavior of the IBM supplied JSON parser, but it results in it not being able to process a number of conventional JSON documents.

Sample Programs and JSON Documents

A number of sample programs are included in the downloadable code package that you can find on our website.

DIYAJL and CustomerData1.json are the files used in the basic example shown in this article.

DIYAJL2 and CustomerData2.json are used to demonstrate how to code without using the skip_document_node option. Program DIYAJL2A demonstrates its use. These programs also demonstrate the use of other YAJLINTO parsing options.

Wrapping Up

As you can see the combination of YAJLINTO and RPG’s DATA-INTO provide a powerful and simple method for handling the processing of JSON documents. They may not be able to handle every JSON processing requirement that you encounter, but they will certainly be able to handle the majority of cases.

If there are any questions you have on either DATA-INTO in general or YAJLINTO specifically, please let us know via the comments section and we’ll do our best to address them.

---