jonathanstowe/json-infer

Infer the structure of a JSON object and make some Raku class to represent it

JSON::Infer

Create Raku classes to represent JSON data by some dodgy inference.

Synopsis

Use the script to do it simply:

# Create the module in the directory "foo"

raku-json-infer --uri=http://api.mixcloud.com/spartacus/party-time/ --out-dir=foo --class-name=Mixcloud::Show

Or do it in your own code:

use JSON::Infer;

my $obj = JSON::Infer.new();
my $ret = $obj.infer(uri => 'http://api.mixcloud.com/spartacus/party-time/', class-name => 'Mixcloud::Show');

say $ret.make-class; # Print the class definition

Description

JSON is nearly ubiquitous on the internet, developers love it for making APIs. However the webservices that use it for transfer of data rarely have a machine readable specification that can be turned into code so developers who want to consume these services usually have to make the client definition themselves.

This module aims to provide a way to generate Raku classes that can represent the data from a JSON source. The structure and the types of the data is inferred from a single data item so the accuracy may depend on the consistency of the data.

Installation

Assuming you have a working Rakudo installation you should be able to install this with zef :

# From the source directory

zef install .

# Remote installation

zef install JSON::Infer

Other install mechanisms may be become available in the future.

Support

Suggestions/patches are welcomed via github at https://github.com/jonathanstowe/JSON-Infer/issues

Licence

This is free software.

Please see the LICENCE file in the distribution for details.

© Jonathan Stowe 2015, 2016, 2017, 2019, 2020, 2021

json-infer's Issues

Infer JSON::Class

Might be nice to have the inferred classes extend your JSON::Class so that they can be round tripped to/from the server?

Duplicate class definitions in nested JSON structures

Given a multi-tiered document with object structures at each layer like:

{
   "links" : {
      "self" : "https://petition.parliament.uk/petitions/241584.json"
   },
   "data" : {
      "id" : 241584,
      "attributes" : {
         "government_response" : null,
         "created_at" : "2019-02-14T12:14:59.326Z",
         "closed_at" : null,
         "rejected_at" : null,
         "scheduled_debate_date" : null,
         "creator_name" : "Margaret Anne Georgiadou",
         "rejection" : null,
         "debate_outcome_at" : null,
         "response_threshold_reached_at" : "2019-03-18T13:26:30.257Z",
         "signature_count" : 2833656,
         "updated_at" : "2019-03-22T10:18:42.117Z",
         "signatures_by_constituency" : [
            {
               "ons_code" : "S14000022",
               "name" : "Edinburgh East",
               "mp" : "Tommy Sheppard MP",
               "signature_count" : 4906
            },
            {
               "mp" : "Deidre Brock MP",
               "signature_count" : 7339,
               "ons_code" : "S14000023",
               "name" : "Edinburgh North and Leith"
            }
         ],
         "additional_details" : "",
         "background" : "The government repeatedly claims exiting the EU is 'the will of the people'. We need to put a stop to this claim by proving the strength of public support now, for remaining in the EU. A People's Vote may not happen - so vote now.",
         "government_response_at" : null,
         "action" : "Revoke Article 50 and remain in the EU.",
         "signatures_by_country" : [
            {
               "signature_count" : 4,
               "code" : "AF",
               "name" : "Afghanistan"
            },
            {
               "name" : "Albania",
               "signature_count" : 3,
               "code" : "AL"
            }
         ],
         "moderation_threshold_reached_at" : "2019-02-14T14:57:53.747Z",
         "debate_threshold_reached_at" : "2019-03-20T20:33:35.184Z",
         "state" : "open",
         "debate" : null,
         "opened_at" : "2019-02-20T10:25:02.393Z"
      },
      "type" : "petition"
   }
}

The generated classes are defined multiple times (once for every parent level) like:

use JSON::Name;
use JSON::Class;
class Foo does JSON::Class {
    class SignaturesByConstituency does JSON::Class {
        has Str $.mp;
        has Int $.signature-count is json-name('signature_count');
        has Str $.name;
        has Str $.ons-code is json-name('ons_code');
    }
    class SignaturesByCountry does JSON::Class {
        has Str $.code;
        has Int $.signature-count is json-name('signature_count');
        has Str $.name;
    }
    class Attributes does JSON::Class {
        class SignaturesByConstituency does JSON::Class {
            has Str $.mp;
            has Int $.signature-count is json-name('signature_count');
            has Str $.name;
            has Str $.ons-code is json-name('ons_code');
        }
        class SignaturesByCountry does JSON::Class {
            has Str $.code;
            has Int $.signature-count is json-name('signature_count');
            has Str $.name;
        }
        has Any $.government-response is json-name('government_response');
        has Any $.rejected-at is json-name('rejected_at');
        has Any $.scheduled-debate-date is json-name('scheduled_debate_date');
        has Str $.background;
        has Str $.action;
        has Str $.additional-details is json-name('additional_details');
        has Str $.opened-at is json-name('opened_at');
        has Any $.rejection;
        has SignaturesByCountry @.signatures-by-country is json-name('signatures_by_country');
        has Str $.moderation-threshold-reached-at is json-name('moderation_threshold_reached_at');
        has Str $.creator-name is json-name('creator_name');
        has Str $.updated-at is json-name('updated_at');
        has SignaturesByConstituency @.signatures-by-constituency is json-name('signatures_by_constituency');
        has Any $.government-response-at is json-name('government_response_at');
        has Str $.debate-threshold-reached-at is json-name('debate_threshold_reached_at');
        has Any $.debate-outcome-at is json-name('debate_outcome_at');
        has Str $.response-threshold-reached-at is json-name('response_threshold_reached_at');
        has Int $.signature-count is json-name('signature_count');
        has Any $.debate;
        has Str $.state;
        has Any $.closed-at is json-name('closed_at');
        has Str $.created-at is json-name('created_at');
    }
    class Data does JSON::Class {
        class SignaturesByConstituency does JSON::Class {
            has Str $.mp;
            has Int $.signature-count is json-name('signature_count');
            has Str $.name;
            has Str $.ons-code is json-name('ons_code');
        }
        class SignaturesByCountry does JSON::Class {
            has Str $.code;
            has Int $.signature-count is json-name('signature_count');
            has Str $.name;
        }
        class Attributes does JSON::Class {
            class SignaturesByConstituency does JSON::Class {
                has Str $.mp;
                has Int $.signature-count is json-name('signature_count');
                has Str $.name;
                has Str $.ons-code is json-name('ons_code');
            }
            class SignaturesByCountry does JSON::Class {
                has Str $.code;
                has Int $.signature-count is json-name('signature_count');
                has Str $.name;
            }
            has Any $.government-response is json-name('government_response');
            has Any $.rejected-at is json-name('rejected_at');
            has Any $.scheduled-debate-date is json-name('scheduled_debate_date');
            has Str $.background;
            has Str $.action;
            has Str $.additional-details is json-name('additional_details');
            has Str $.opened-at is json-name('opened_at');
            has Any $.rejection;
            has SignaturesByCountry @.signatures-by-country is json-name('signatures_by_country');
            has Str $.moderation-threshold-reached-at is json-name('moderation_threshold_reached_at');
            has Str $.creator-name is json-name('creator_name');
            has Str $.updated-at is json-name('updated_at');
            has SignaturesByConstituency @.signatures-by-constituency is json-name('signatures_by_constituency');
            has Any $.government-response-at is json-name('government_response_at');
            has Str $.debate-threshold-reached-at is json-name('debate_threshold_reached_at');
            has Any $.debate-outcome-at is json-name('debate_outcome_at');
            has Str $.response-threshold-reached-at is json-name('response_threshold_reached_at');
            has Int $.signature-count is json-name('signature_count');
            has Any $.debate;
            has Str $.state;
            has Any $.closed-at is json-name('closed_at');
            has Str $.created-at is json-name('created_at');
        }
        has Int $.id;
        has Str $.type;
        has Attributes $.attributes;
    }
    class Links does JSON::Class {
        has Str $.self;
    }
    has Links $.links;
    has Data $.data;
}

Which is way LTA and means that the generated code can't be usefully used without editing.

There should be an option to determine the placement of the child classes (either raised to the top level or left local to the object in which they are used.

Use template for generated classes.

Currently the generated classes are "hardcoded", devolving this to an external template would make things a lot better:

• Easier maintenance
• User defined style and/or special requirements
• "best practice" (whatever that meands.

Fairly agnostic about the choice of templating engine, Template6 is probably most familiar.

`kebab` makes an absolute pigs of repeated uppercase

e.g.

class YNAP::Hesabu::AWS::SNS::Message does JSON::Class {
    has Str $.token is json-name('Token');
    has Str $.message-id is json-name('MessageId');
    has Str $.timestamp is json-name('Timestamp');
    has Str $.signature is json-name('Signature');
    has Str $.signature-version is json-name('SignatureVersion');
    has Str $.topic-arn is json-name('TopicArn');
    has Str $.type is json-name('Type');
    has Str $.signing-cert-u r l is json-name('SigningCertURL');
    has Str $.subscribe-u r l is json-name('SubscribeURL');
    has Str $.message is json-name('Message');
}