Sample Header Ad - 728x90

Query optimisation - how to reference lateral join table to avoid duplicate lookup

1 vote
1 answer
761 views
First time posting here but long time reader. I'm fairly new in my role working on optimising sql queries. Most are generated from KNEX.js leading to some peculiar artifacts that might make sense in javascript but don't play well with SQL. I'm trying to improve the query below. The goal of the query is to produce a table listing a merchant's vending machines and the total amounts and counts taken through different gateways (eg AMEX, VISA, EBT, COUPON, NULL[Cash]) for accounting periods. I've reproduced it fully although I've altered the gateway names to ABC / XYZ for example purposes. Clearly there is plenty of potential for improvement but where I would specifically would like to ask for help is this: The query has two lateral joins that are essentially identical. vend_total and vend_partial. The only distinction is in the timestamp- and "vend"."timestamp" <= cash_accounting_period_ids.closed_at AS vend_total - or between cash_accounting_period_ids.created_at and cash_accounting_period_ids.closed_at AS vend_partial To my mind that makes vend_partial just a subset of vend_total. I would like to rewrite the query so that it does not have the obvious duplication of filters and does not fetch data twice, but I cannot wrap my head around how. It seems to me I should be able to reference vend_total as the basis of vend_partial instead of duplicating it, or instead rewriting the whole filter aggregate "totals" as a cte and then reference that, but tbh the complexity of this query is beating me. I'd really appreciate any suggestions people might have as to how to streamline this query, and more than that I'd love some recommendations for resources or strategies for query analysis and optimisation. I'm new to this and I enjoy the challenge but I work for a small startup and at this point I'm apparently the SQL expert so will take any advice I can get! George.
with "cash_accounting_period_ids" as (select "cash_accounting_period".*
                                      from "cash_accounting_period"
                                      where "cash_accounting_period"."merchant_id" = ?1
                                        and "cash_accounting_period"."closed_at" is not null
                                      order by closed_at DESC NULLS FIRST, cash_accounting_period.created_at DESC
                                      limit ?2)
select "cash_accounting_period_ids".*,
       (select row_to_json(vm)
        from (select vending_machine.*, ST_AsGeoJSON(location) as location) as vm) AS "vendingMachine",
       row_to_json("verified_by".*)                                                AS "verifiedBy",
       row_to_json("closed_by".*)                                                  AS "closedBy",
       row_to_json("company".*)                                                    AS "client",
       row_to_json(vend_total.*)                                                   AS "vendTotal",
       row_to_json(vend_partial.*)                                                 AS "vendPartial",
       row_to_json(route.*)                                                        AS route
from "cash_accounting_period_ids"
         left join "vending_machine" on "vending_machine"."id" = "cash_accounting_period_ids"."vending_machine_id"
         left join "user" AS "verified_by" on "verified_by"."id" = "cash_accounting_period_ids"."verified_by"
         left join "user" AS "closed_by" on "closed_by"."id" = "cash_accounting_period_ids"."closed_by"
         left join "company" on "company"."id" = "cash_accounting_period_ids"."client_id"
         left join lateral (select "totals".*
                            from (select "vend"."vending_machine_id",
                                         COUNT(*) FILTER (WHERE vend.gateway_name = 'ABC')           AS abc_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name = 'ABC'),
                                                  0)                                                   AS abc_amount,
                                         COUNT(*) FILTER (WHERE vend.gateway_name = 'XYZ')           AS xyz_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name = 'XYZ'),
                                                  0)                                                   AS xyz_amount,
                                         COUNT(*) FILTER (WHERE vend.gateway_name IS NULL)             AS money_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name IS NULL),
                                                  0)                                                   AS money_amount,
                                         COUNT(*)                                                      AS total_count,
                                         COALESCE(SUM(vend.amount), 0)                                 AS total_amount
                                  from "vend"
                                           left join lateral (select coalesce(array_agg(route_vending_machine.route_id), '{}') as id
                                                              from "route_vending_machine"
                                                              where route_vending_machine.vending_machine_id = vend.vending_machine_id) subquery
                                                     ON TRUE
                                  where "vend"."merchant_id" = ?1
                                    and "vend"."display" = true
                                    and "vend"."currency_id" = 'EUR'
                                    and ("succeeded" = true or "succeeded" is null)
                                    and cash_accounting_period_ids.vending_machine_id = vend.vending_machine_id
                                    and (((TRUE or TRUE or TRUE)) and not (("subquery"."id" && '{}' or 1 = 0 or 1 = 0)))
                                    and "vend"."timestamp" <= cash_accounting_period_ids.closed_at
                                  group by "vend"."vending_machine_id", "vending_machine"."name") as "totals"
                            limit 1) vend_total ON TRUE
         left join lateral (select "totals".*
                            from (select "vend"."vending_machine_id",
                                         COUNT(*) FILTER (WHERE vend.gateway_name = 'ABC')           AS abc_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name = 'ABC'),
                                                  0)                                                   AS abc_amount,
                                         COUNT(*) FILTER (WHERE vend.gateway_name = 'XYZ')           AS xyz_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name = 'XYZ'),
                                                  0)                                                   AS xyz_amount,
                                         COUNT(*) FILTER (WHERE vend.gateway_name IS NULL)             AS money_count,
                                         COALESCE(SUM(vend.amount) FILTER (WHERE vend.gateway_name IS NULL),
                                                  0)                                                   AS money_amount,
                                         COUNT(*)                                                      AS total_count,
                                         COALESCE(SUM(vend.amount), 0)                                 AS total_amount
                                  from "vend"
                                           left join lateral (select coalesce(array_agg(route_vending_machine.route_id), '{}') as id
                                                              from "route_vending_machine"
                                                              where route_vending_machine.vending_machine_id = vend.vending_machine_id) subquery
                                                     ON TRUE
                                  where "vend"."merchant_id" = ?1
                                    and "vend"."display" = true
                                    and "vend"."currency_id" = 'EUR'
                                    and ("succeeded" = true or "succeeded" is null)
                                    and cash_accounting_period_ids.vending_machine_id = vend.vending_machine_id
                                    and (((TRUE or TRUE or TRUE)) and not (("subquery"."id" && '{}' or 1 = 0 or 1 = 0)))
                                    and "vend"."timestamp" between cash_accounting_period_ids.created_at and cash_accounting_period_ids.closed_at
                                  group by "vend"."vending_machine_id", "vending_machine"."name") as "totals"
                            limit 1) vend_partial ON TRUE
         left join "route_vending_machine"
                   on "route_vending_machine"."vending_machine_id" = "cash_accounting_period_ids"."vending_machine_id"
         left join "route" on "route"."id" = "route_vending_machine"."route_id";
EXPLAIN (ANALYSE, BUFFERS) is too long to share here in post or comment so I dumped it here: https://justpaste.it/bolah
Asked by George B (13 rep)
Mar 10, 2023, 11:20 AM
Last activity: Mar 10, 2023, 10:42 PM