Forum Discussion

MichaelWhite1's avatar
MichaelWhite1
Qrew Member
2 years ago

JSON Handler - Max JSON size exceeded

Hello all.

I wonder if anybody could offer any advice on a problem i'm having please?

I have set up an 'Fetch a JSON resource' in pipelines, which is to a third party ID check. Prior to the candidate completing the ID check I can fetch the JSON data and import it into QB through Pipelines. However, once the candidate completes the ID check the JSON contains considerably more fields and when I try to 'Fetch a JSON resource' I get a validation error.

Validation error: Maximum JSON size exceeded

Is there any way in which I can overcome this please? Here is an example of what is in the JSON.

[
  {
    "id": "e4x69a61-07mp-46le-aj80-0c52f12a673e",
    "data": {
      "pseudonym": "ANON-e2x0a6mp-37l0-42ps-94ef-u69755438d3o",
      "product_name": "Test Product",
      "status": "completed",
      "purchase_start_date": "2022-03-16T09:33:52Z",
      "purchase_completed_date": "2022-03-16T09:33:52Z",
      "date_of_birth": "1974-01-17",
      "first_name": "JOHN",
      "gender": "-",
      "surname": "DOE",
      "liveness_verified_selfie_1": [    "iVBORw0KGgoAAAANSUhEUgAAAJYAA...MTkrMDA6MDDuShMbAAAAAElFTkSuQmCC\n"],
      "liveness_verified_selfie_2": [      "iVBORw0KGgoAAAANSUhEUgAAAJYAA...R24Ac1JHQrqQcwcAAAAASUVORK5CYII=\n"],
      "driving_licence_hd_attachment": null,
      "id_card_hd_attachment": null,
      "passport_hd_attachment": [        "iVBORw0KGgoAAAANSUhEUgAAAJYAA...MTkrMDA6MDDuShMbAAAAAElFTkSuQmCC\n",        "iVBORw0KGgoAAAANSUhEUgAAAJYAA...R24Ac1JHQrqQcwcAAAAASUVORK5CYII=\n"],
      "age": 49,
      "age_is_18_plus": true,
      "driving_licence_attachment": null,
      "driving_licence_expiration_date": null,
      "driving_licence_number": null,
      "driving_licence_state": null,
      "driving_licence_verified_at": null,
      "full_name": "GARY GIRAFFE",
      "id_card_attachment": null,
      "id_card_expiration_date": null,
      "id_card_number": null,
      "id_card_state": null,
      "id_card_verified_at": null,
      "liveness_verified_selfie_1_at": "2023-04-11T11:17:03.386Z",
      "liveness_verified_selfie_2_at": "2023-04-11T11:17:03.386Z",
      "middle_name": null,
      "passport_attachment": [        "iVBORw0KGgoAAAANSUhEUgAAAJYAA...MTkrMDA6MDDuShMbAAAAAElFTkSuQmCC\n",        "iVBORw0KGgoAAAANSUhEUgAAAJYAA...R24Ac1JHQrqQcwcAAAAASUVORK5CYII=\n"],
      "passport_expiration_date": "2028-07-09",
      "passport_number": "1234567890",
      "passport_state": "accepted",
      "passport_verified_at": "2022-01-20T09:10:25.154Z"
    }
  }
]

Thank you,

Michael



------------------------------
Michael White
------------------------------

  • Does the API you're calling have additional parameters where you can limit/query only for certain fields? If so, I would try and minimizing the scope of your JSON response in the call itself only to the necessary fields and make sure that your query is only for a single object. Your only real avenue with how you're trying it is to minimize the response and get the size down. 

    The only other idea I can imagine is to forego using the JSON resource fetch and use the generic webhook to get the info, in my experience the parsing of that response is reasonably the same but I've never had it return payload size issues. 



    ------------------------------
    Chayce Duncan
    ------------------------------
    • MichaelWhite1's avatar
      MichaelWhite1
      Qrew Member

      Hi Chanyce.

      Thank you for the reply.

      I have been trying to limit the payload, but i'll be honest I've no idea how to do that.

      You can see from below the size of each 'item'. I don't actually need the four large images which are causing the issue, but i've no idea how i can call the payload just for the other items.

      Here is how i've set all the fields in the JSON handler 'fetch' request.

      Have you got any ideas please?

      Many thanks,

      Michael



      ------------------------------
      Michael White
      ------------------------------
      • ChayceDuncan's avatar
        ChayceDuncan
        Qrew Captain

        Can you tell me what system you're trying to integrate with so I can take a look at the documentation and see if there is a way to limit the scope?



        ------------------------------
        Chayce Duncan
        ------------------------------