Forum Discussion

BradleyPayne's avatar
BradleyPayne
Qrew Trainee
3 years ago

Efficiently use pipelines for record change logs

Hi,

I was going through the quickbase university courses and came across this use case of using pipelines' Quickbase Records Updated feature to log changes to a table. 

Building Powerful Pipelines Using included Features

I have a similar use case, but I have nearly 40 fields in the table and I only want the log a single changed field in the log table. 

In coding, I would do something like: 


for field in table_fields:
    if field.prev.value != field.current.value:
        create_logs_record(field.name, field.prev.value, field.current.value, ...)​

Is there a way I can efficiently do this in pipelines using jinja or some other method?
Currently, I have a trigger on update and then a long series of `if changed, create record else ...` which is tedious to create and would be difficult to modify.

------------------------------
Bradley
------------------------------
  • One solution is to log every field in 40 Steps. That will take two pipelines. 


    Then each night run a step to purge records where old value and new value are the same. 


    Filter off from any reports where the old and new values are the same and your users will never know that a day's worth of record where there was, for the most part, no change  



    ------------------------------
    Mark Shnier (YQC)
    mark.shnier@gmail.com
    ------------------------------

  • One solution is to log every field in 40 Steps. That will take two pipelines. 


    Then each night run a step to purge records where old value and new value are the same. 


    Filter off from any reports where the old and new values are the same and your users will never know that a day's worth of record where there was, for the most part, no change  



    ------------------------------
    Mark Shnier (YQC)
    mark.shnier@gmail.com
    ------------------------------

    • BradleyPayne's avatar
      BradleyPayne
      Qrew Trainee
      So if I'm understanding the docs correctly, It is not possible to loop through the record fields? Basically, I could solve this problem in 2 steps if I could do a for each field in the record, but that doesn't appear possible.

      ------------------------------
      Bradley
      ------------------------------
      • MarkShnier__You's avatar
        MarkShnier__You
        Icon for Qrew Legend rankQrew Legend
        I'm not aware how to loop through a field checking for changes.  I don't think it's possible.  The best practice would be to load the 40 records (I guess you will need two pipelines so maybe 20 + 20) via an upsert, that will have the least impact on app performance.

        ------------------------------
        Mark Shnier (YQC)
        mark.shnier@gmail.com
        ------------------------------