tradon/modules/purchase_trade/docs/business rules/shipment_fee_allocation.md

BR-PT-005 - Shipment Fee Allocation

Intent

Allocate shipment costs by combining shipment-level fees and contract-level budgeted fees, while avoiding double counting and keeping fee detail rows available for BI analysis.

Scope

• Domain: purchase_trade
• Target: SQL view/query for shipment fee allocation
• Source tables:
  • lot_lot
  • lot_qt
  • lot_qt_hist
  • fee_fee
  • product_product
  • party_party
  • currency_currency
  • product_uom
• Fee sources:
  • shipment fees linked with fee_fee.shipment_in
  • purchase contract fees linked with fee_fee.line
  • sale contract fees linked with fee_fee.sale_line

Business Context

Every purchase contract line has a virtual lot by default. The virtual lot represents the remaining quantity of the contract line.

When physical lots are created, their quantity reduces the virtual lot quantity. Physical lots represent confirmed physical quantities, typically once the Bill of Lading quantity is known.

Before the vessel is sailing or before the exact Bill of Lading quantity is known, costs can already be ordered or scheduled at shipment level. In that case, virtual shipment quantities must still be included as forecast quantities.

Quantity Source Rules

Shipment quantity is built directly from lot and shipment tables.

Quantity priority:

1. Use physical lots linked to the shipment when they exist.
2. Use virtual lot shipment quantities only when no physical lot exists yet for the same shipment and contract pair.

This prevents double counting:

• physical lots represent confirmed shipment quantities
• virtual lots represent forecast/open quantities until physical lots exist

Fee Selection Rules

For each shipment_id + product_id + supplier_id pair:

1. Use shipment-level fees of type ordered first.
2. If shipment quantity remains uncovered, use shipment-level fees of type scheduled.
3. If shipment quantity still remains uncovered, use contract-level fees of type budgeted.

Contract-level fallback fees:

• purchase contract fees are eligible
• sale contract fees are eligible
• only type = 'budgeted' is eligible
• budgeted contract fees are applied to the remaining uncovered shipment quantity

Shipment-level fees:

• only type IN ('ordered', 'scheduled') are eligible
• null fee quantity is treated as 0 and does not allocate quantity
• allocated shipment fee quantity is capped to the shipment quantity

If multiple fees exist for the same product/supplier pair at the selected level, keep multiple rows. The view must preserve details for analysis.

Sign Rules

Use the same sign logic as the existing fee utility views.

Shipment fees:

• rec = +1
• any other p_r value = -1

Purchase contract budgeted fees:

• rec = -1
• any other p_r value = +1

Sale contract budgeted fees:

• rec = +1
• any other p_r value = -1

Expected Behavior

Example:

• Shipment quantity: 1000 Mt
• Shipment ordered freight: 800 Mt at 60 USD/Mt
• Contract budgeted freight: 50 USD/Mt

Expected output:

• 800 Mt at 60 USD/Mt from shipment ordered fee
• 200 Mt at 50 USD/Mt from contract budgeted fee

If a scheduled fee also exists:

• Shipment quantity: 1000 Mt
• Ordered freight: 800 Mt at 60 USD/Mt
• Scheduled freight: 150 Mt at 55 USD/Mt
• Contract budgeted freight: 50 USD/Mt

Expected output:

• 800 Mt at 60 USD/Mt from shipment ordered fee
• 150 Mt at 55 USD/Mt from shipment scheduled fee
• 50 Mt at 50 USD/Mt from contract budgeted fee

Query Draft

WITH physical_shipment_lots AS (
    SELECT
        l.lot_shipment_in AS shipment_id,
        l.id AS lot_id,
        l.lot_type,
        l.line AS purchase_line_id,
        l.sale_line AS sale_line_id,
        qh.quantity AS shipment_quantity,
        l.lot_unit_line AS shipment_unit_id
    FROM lot_lot l
    LEFT JOIN lot_qt_hist qh
        ON qh.lot = l.id
       AND qh.quantity_type = l.lot_state
    WHERE l.lot_type = 'physic'
      AND l.lot_shipment_in IS NOT NULL
),
virtual_shipment_lots AS (
    SELECT
        q.lot_shipment_in AS shipment_id,
        vp.id AS lot_id,
        vp.lot_type,
        vp.line AS purchase_line_id,
        COALESCE(vp.sale_line, ps.sale_line) AS sale_line_id,
        q.lot_quantity AS shipment_quantity,
        q.lot_unit AS shipment_unit_id
    FROM lot_qt q
    JOIN lot_lot vp
        ON vp.id = q.lot_p
       AND vp.lot_type = 'virtual'
    LEFT JOIN lot_lot ps
        ON ps.id = q.lot_s
    WHERE q.lot_shipment_in IS NOT NULL
      AND NOT EXISTS (
          SELECT 1
          FROM physical_shipment_lots pl
          WHERE pl.shipment_id = q.lot_shipment_in
            AND pl.purchase_line_id = vp.line
            AND (
                pl.sale_line_id = COALESCE(vp.sale_line, ps.sale_line)
                OR COALESCE(vp.sale_line, ps.sale_line) IS NULL
            )
      )
),
shipment_lots AS (
    SELECT * FROM physical_shipment_lots
    UNION ALL
    SELECT * FROM virtual_shipment_lots
),
shipment_context AS (
    SELECT
        shipment_id,
        purchase_line_id,
        sale_line_id,
        shipment_unit_id,
        SUM(COALESCE(shipment_quantity, 0)) AS shipment_quantity
    FROM shipment_lots
    WHERE shipment_id IS NOT NULL
    GROUP BY
        shipment_id,
        purchase_line_id,
        sale_line_id,
        shipment_unit_id
),
shipment_base AS (
    SELECT
        shipment_id,
        SUM(shipment_quantity) AS shipment_quantity
    FROM shipment_context
    GROUP BY shipment_id
),
shipment_fee_candidates AS (
    SELECT
        f.id AS fee_id,
        'Shipment' AS fee_source,
        f.type AS fee_type,
        CASE
            WHEN f.type = 'ordered' THEN 1
            WHEN f.type = 'scheduled' THEN 2
        END AS priority,
        sb.shipment_id,
        NULL::integer AS purchase_line_id,
        NULL::integer AS sale_line_id,
        f.product AS product_id,
        f.supplier AS supplier_id,
        f.mode AS packaging,
        f.p_r AS pay_or_rec,
        f.state,
        f.weight_type,
        COALESCE(f.quantity, 0) AS requested_quantity,
        sb.shipment_quantity,
        f.price AS fee_price,
        f.currency AS currency_id,
        f.unit AS unit_id,
        CASE
            WHEN upper(f.p_r::text) = 'REC' THEN 1
            ELSE -1
        END AS sign_multiplier
    FROM shipment_base sb
    JOIN fee_fee f
        ON f.shipment_in = sb.shipment_id
    WHERE f.type IN ('ordered', 'scheduled')
),
shipment_fee_allocated AS (
    SELECT
        c.*,
        LEAST(
            c.requested_quantity,
            GREATEST(
                c.shipment_quantity
                - COALESCE(
                    SUM(c.requested_quantity) OVER (
                        PARTITION BY c.shipment_id, c.product_id, c.supplier_id
                        ORDER BY c.priority, c.fee_id
                        ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING
                    ),
                    0
                ),
                0
            )
        ) AS allocated_quantity
    FROM shipment_fee_candidates c
),
shipment_fee_coverage AS (
    SELECT
        shipment_id,
        product_id,
        supplier_id,
        MAX(shipment_quantity) AS shipment_quantity,
        SUM(allocated_quantity) AS allocated_quantity
    FROM shipment_fee_allocated
    GROUP BY shipment_id, product_id, supplier_id
),
contract_budgeted_fee_candidates AS (
    SELECT
        f.id AS fee_id,
        'Purchase Contract' AS fee_source,
        f.type AS fee_type,
        3 AS priority,
        sc.shipment_id,
        sc.purchase_line_id,
        NULL::integer AS sale_line_id,
        f.product AS product_id,
        f.supplier AS supplier_id,
        f.mode AS packaging,
        f.p_r AS pay_or_rec,
        f.state,
        f.weight_type,
        sc.shipment_quantity,
        f.price AS fee_price,
        f.currency AS currency_id,
        COALESCE(f.unit, sc.shipment_unit_id) AS unit_id,
        CASE
            WHEN upper(f.p_r::text) = 'REC' THEN -1
            ELSE 1
        END AS sign_multiplier
    FROM shipment_context sc
    JOIN fee_fee f
        ON f.line = sc.purchase_line_id
    WHERE f.type = 'budgeted'
      AND sc.purchase_line_id IS NOT NULL

    UNION ALL

    SELECT
        f.id AS fee_id,
        'Sale Contract' AS fee_source,
        f.type AS fee_type,
        3 AS priority,
        sc.shipment_id,
        NULL::integer AS purchase_line_id,
        sc.sale_line_id,
        f.product AS product_id,
        f.supplier AS supplier_id,
        f.mode AS packaging,
        f.p_r AS pay_or_rec,
        f.state,
        f.weight_type,
        sc.shipment_quantity,
        f.price AS fee_price,
        f.currency AS currency_id,
        COALESCE(f.unit, sc.shipment_unit_id) AS unit_id,
        CASE
            WHEN upper(f.p_r::text) = 'REC' THEN 1
            ELSE -1
        END AS sign_multiplier
    FROM shipment_context sc
    JOIN fee_fee f
        ON f.sale_line = sc.sale_line_id
    WHERE f.type = 'budgeted'
      AND sc.sale_line_id IS NOT NULL
),
contract_budgeted_allocated AS (
    SELECT
        c.*,
        GREATEST(
            c.shipment_quantity - COALESCE(fc.allocated_quantity, 0),
            0
        ) AS allocated_quantity
    FROM contract_budgeted_fee_candidates c
    LEFT JOIN shipment_fee_coverage fc
        ON fc.shipment_id = c.shipment_id
       AND fc.product_id = c.product_id
       AND fc.supplier_id = c.supplier_id
),
final_fees AS (
    SELECT
        fee_id,
        fee_source,
        fee_type,
        priority,
        shipment_id,
        purchase_line_id,
        sale_line_id,
        product_id,
        supplier_id,
        packaging,
        pay_or_rec,
        state,
        weight_type,
        allocated_quantity AS fee_quantity,
        fee_price,
        currency_id,
        unit_id,
        sign_multiplier
    FROM shipment_fee_allocated
    WHERE allocated_quantity > 0

    UNION ALL

    SELECT
        fee_id,
        fee_source,
        fee_type,
        priority,
        shipment_id,
        purchase_line_id,
        sale_line_id,
        product_id,
        supplier_id,
        packaging,
        pay_or_rec,
        state,
        weight_type,
        allocated_quantity AS fee_quantity,
        fee_price,
        currency_id,
        unit_id,
        sign_multiplier
    FROM contract_budgeted_allocated
    WHERE allocated_quantity > 0
)
SELECT
    f.shipment_id AS "intShipmentId",
    f.fee_id AS "intFeeId",
    f.fee_source AS "Fee Source",
    f.fee_type AS "Fee Type",
    f.priority AS "Priority",
    f.purchase_line_id AS "intPurchaseLineId",
    f.sale_line_id AS "intSaleLineId",
    f.product_id AS "intProductId",
    p.code AS "Fee",
    f.supplier_id AS "intSupplierId",
    sup.name AS "Supplier",
    f.packaging AS "Packaging",
    f.pay_or_rec AS "Pay or Rec",
    f.state AS "State",
    CASE
        WHEN upper(f.weight_type::text) = 'BRUT' THEN 'Gross'
        ELSE 'Net'
    END AS "Weighing Type",
    f.fee_quantity AS "Quantity",
    f.fee_price * f.sign_multiplier AS "Price",
    cur.name AS "Currency",
    COALESCE(uom.name, 'Mt') AS "Unit",
    f.fee_quantity * f.fee_price * f.sign_multiplier AS "Amount",
    CASE
        WHEN upper(p.code::text) LIKE '%FREIGHT%' THEN 'Freight'
        WHEN upper(p.code::text) LIKE '%PROFIT SHARING%' THEN 'Profit Sharing'
        ELSE 'Other Costs'
    END AS "Cost Group"
FROM final_fees f
JOIN product_product p
    ON p.id = f.product_id
JOIN party_party sup
    ON sup.id = f.supplier_id
LEFT JOIN currency_currency cur
    ON cur.id = f.currency_id
LEFT JOIN product_uom uom
    ON uom.id = f.unit_id
ORDER BY
    f.shipment_id,
    f.product_id,
    f.supplier_id,
    f.priority,
    f.fee_id;

Impacted Files

Expected SQL/view impact:

• BI SQL view or migration file that will materialize the shipment fee allocation query
• Existing related views for comparison:
  • vw_utility_contract_fees
  • vw_utility_shipment_fees
  • vw_bi_itsa_fct_contract_fees
  • vw_bi_itsa_fct_shipment_fees

No Python code impact is expected unless the view is generated by module migration code.

Tests

Recommended tests:

• shipment with no ordered/scheduled fee uses full budgeted contract fee quantity
• shipment with ordered fee covering full quantity does not use budgeted fallback
• shipment with ordered fee partially covering quantity uses budgeted fallback for remaining quantity
• shipment with ordered and scheduled fees uses ordered first, scheduled second, budgeted third
• shipment fee quantity above shipment quantity is capped
• null shipment fee quantity does not allocate quantity
• multiple fees at the same priority are kept as multiple detail rows
• physical lot quantity replaces virtual forecast quantity for the same shipment/contract pair
• virtual lot quantity is used when no physical lot exists yet

Open Questions

• Q: Should this query become a permanent PostgreSQL view?

• Q: What should the final view name be?

• Q: Should the output include both allocated quantity and original fee quantity for auditability?