tradon/modules/purchase_trade/docs/rules/market-price-import.md

BR-PT-004 - Market Price Import

Intent

Import dated market prices from an .xlsx Excel file into price.price_value, using price.price as the market price index.

Scope

• Wizard: purchase_trade.import_prices
• Input model: purchase_trade.import_prices.start
• Result model: purchase_trade.import_prices.result
• Target models:
  • price.price
  • price.price_value
• Menu entry: under price.menu_price

Inputs

The wizard reads only the first worksheet of an .xlsx file.

The first row is treated as the header row. Header names are normalized by lowercasing and removing non-alphanumeric characters, so labels such as price_index, price index, and Price Index map to the same field.

Required columns:

• price_index
• price_date
• high_price
• low_price
• open_price
• price_value

The wizard options are:

• Create price index if missing
• Overwrite existing price

Date and Numeric Parsing

Accepted price_date values:

• Excel serial date numbers
• YYYY-MM-DD
• DD/MM/YYYY
• MM/DD/YYYY

Numeric price fields accept decimal commas or decimal points. Empty numeric cells are imported as empty values. Invalid numeric values are reported as row errors.

Expected Behavior

For each non-empty data row, starting from Excel row 2:

1. Trim and validate price_index.
2. Parse price_date.
3. Search price.price by exact price_index.
4. If the price index is missing:
   • create it when Create price index if missing is checked
   • otherwise skip the row with price_index missing
5. Search price.price_value by (price, price_date).
6. If an existing price value is found:
   • update it when Overwrite existing price is checked
   • otherwise skip the row with price_date already exists
7. If no existing price value is found, create a new price.price_value.

Created Price Index Defaults

When the wizard creates a missing price.price, it sets:

• price_index = imported price_index
• price_desc = imported price_index
• price_curve_type = future

It also tries to default these references when matching records exist:

• price_type: price.fixtype where name = Market price
• price_currency: currency.currency where code = USD
• price_calendar: price.calendar where name = Argus EU
• price_unit: product.uom where name = Mt

If the price_index contains a YYYY-MM style period, the wizard derives a product.month:

• pattern accepted in the name: YYYY-MM, YYYY/MM, YYYY_MM, YYYY.MM, or YYYY MM
• month name format: MONYY, for example JUL26
• if no matching product.month exists, it is created with is_cotation = True

Result Reporting

The result screen always shows counts and detail sections for:

• created price indexes
• imported prices
• updated existing prices
• skipped records
• errors

Row-level errors do not stop the whole import; the wizard records the error and continues with the next row.

Edge Cases

• Missing price_index: skipped.
• Missing price_date: skipped.
• Invalid .xlsx file: blocking UserError.
• Missing required columns: blocking UserError.
• Invalid date: row error.
• Invalid numeric value: row error.
• Existing (price, price_date) without overwrite option: skipped.
• Existing (price, price_date) with overwrite option: updated.
• Empty rows are ignored.

Impacted Files

Direct purchase_trade files:

• modules/purchase_trade/pricing.py
• modules/purchase_trade/pricing.xml
• modules/purchase_trade/view/import_prices_start_form.xml
• modules/purchase_trade/view/import_prices_result_form.xml
• modules/purchase_trade/__init__.py
• modules/purchase_trade/tryton.cfg
• modules/purchase_trade/tests/test_module.py

External model dependencies:

• modules/price/price.py
• modules/price/price_value.py
• modules/price/view/price_value_form.xml

Tests

Existing focused tests cover:

• missing price index skipped when creation is disabled
• missing price index created when creation is enabled
• default fields on newly created price indexes
• period reuse/creation from price_index
• existing price_date skipped when overwrite is disabled
• existing price_date updated when overwrite is enabled
• invalid row values collected as errors
• result screen formatting

Recommended additional tests:

• .xlsx header normalization
• missing required columns
• Excel serial date parsing
• empty row ignored
• invalid workbook raises UserError

Open Questions

• Q: Should (price, price_date) be enforced unique at model/database level?

• A: Enforce uniqueness at model level for now. Do not add a database constraint yet.

• Q: Should created price index defaults remain hardcoded to Market price, USD, Argus EU, and Mt?

• A: Yes. Keep these defaults hardcoded for this import.

• Q: Should ambiguous slash dates prefer DD/MM/YYYY over MM/DD/YYYY, as currently implemented?

• A: Prefer the user's/default locale date format when possible. Fall back to the current order only if no locale preference is available.

• Comment: current code tries DD/MM/YYYY before MM/DD/YYYY and does not inspect locale. Implementing this answer requires a code change.

• Q: Should missing price_value be allowed, or should it skip/error while high/low/open remain optional?

• A: Missing price_value is not allowed. Report the row as an error. high_price, low_price, and open_price remain optional.

• Comment: current code allows empty price_value and imports it as an empty value. Implementing this answer requires a code change.

• Q: Should duplicate rows for the same (price_index, price_date) inside the same Excel file be treated as an error, skipped after the first row, or resolved by the overwrite option?

• A: Report duplicate rows as row errors and do not import or update the duplicate row.

• Q: When Create price index if missing is enabled, should missing default reference records (Market price, USD, Argus EU, Mt) block index creation or remain optional as currently implemented?

• A: Remain optional. Create the price index with the reference records that can be found.

• Q: Should the import result distinguish business validation errors from technical parsing errors?

• A: Yes. Distinguish business validation errors from file, parsing, and technical errors in the import result.