Product Distribution Internals

Product distribution is the transfer of a product record from one Fieldpine system to another where:

• The sender and receiver are different businesses. This might be a franchise Head Office sending known products to franchisee stores (if they are not directly company controlled), or it might be a supplier maintaining product records for retailers that stock their products
• This method is NOT used where Head Office and the stores are one retail operation. It is only for cross retailer scenarios.
• It is also used where sales reporting is sent out of one retailer to another (unrelated) business

User Level Configuration

To create this product replication, the sender creates a "membership". This generates a token/key that subscribers add to their Fieldpine systems to subscribe to that feed. The token is long and intended to be cut/paste not hand typed. The token includes details such as API keys, host names and other technical aspects.

Once the membership is created and the end retailer subscribes then any product changes in the sender will replicate to end stores. The change is available immediately for stores, but often it takes a couple of minutes to be flow. Longer if the change is after hours.

On the "membership" management page, retailers can click "download all data". This queues an internal operation to reload all products. Which is primarily a support "get out of jail" option. You do not need to click this often, it is only when you want to force download/application.

Technical Considerations

So replicating products is easy right? Send a CSV like file to the store and insert/replace into their database. While thats the general approach there are some considerations:

• Retail Price. If head office change the price, that does not mean the price should change in the store immediately. The store may need to change signage etc.
• Retail Price. Depending on the relationship and country, it might be illegal for the sender to force the retail price, they can perhaps set RRP, but not mandate retail price
• Cost Price. Head office might send that the cost price is $5. The store however may have different cost pricing
• Operation Flags. Product definitions can include operation controls. An example is "selling messages" which display a message to staff when sold. Are these messages defined by the sender, or do stores maintain their own.
  Another example is that the sender might send their complete catalog (say kitchen ware and furniture) but a receiving retailer may only sell furniture from that supplier, so they want to block some products - otherwise they would all appear instore to staff, which is confusing and error prone.

Sender Operation

• Each product definition contains a hidden internal field called "RVE". This is a UTC timestamp stored in a double that maintains the last change date of the product. Change is used loosely here, changing related data can alter a product too - it does not mean that product was directly changed.

  Tip. The quickcode "mark all products changed" can be entered into the web pages (not PosGreen) and the RVE on all products will be updated. This will force a complete retransmit to all stores. This is not required for normal daily operation.

• There is an API endpoint that has the highest known RVE for products. This updates to the highest RVE as products are editted. Client stores poll this endpoint to detect changes. The endpoint is optimised and cached using various rules to reduce load on the sender server. The endpoint is protected and requires an API key to read, but the endpoint is just an opaque bunch of numbers, not data, and could be public with little security risk.

Receiver Operation - Detection

TBS

Receiver Operation - Applying Changes

Preparation Phase

1. The routine (ApplyInboundDataDistribution) is called with an array of all changes that potentially should be applied
2. The setting DistributeApply.Products.MEMBERSHIP-ID.f103.AutoAccept is loaded
3. The setting DistributeApply.Products.MEMBERSHIP-ID.f103 is loaded
4. The setting DistributeApply.Products.MEMBERSHIP-ID.f108 is loaded
5. The setting DebugPTrace.ProductsDataDistribute is loaded. This controls if tracing is enabled. Tracing is enabled by default.
   If tracing is enabled, the file DebugTrace_ProductsDataDistribute.out is created in the retailers directory. This is a text file readable with notepad.
6. An inbound gateway is created. All products are sent through the gateway. A gateway is a generic function that can restrict and change the inbound data feed to suit the local store.

Per Product Loop

1. The inbound product.physkey is located and the database is checked to see if this product is new or an edit.
2. If the product is new, we attempt to use the inbound product.pid if free, otherwise a new Pid is allocated
3. All inbound fields are sent via the gateway, allowing the gateway to block/change/allow each field change depending on per store configuration. Some fields are handled specifically:
   • Unit Price
     • If the Unit Price policy is "always block" (from setting DistributeApply.Products.MEMBERSHIP-ID.f103) the field is ignored
     • If this is a new product, the price is written regardless of other settings. It is pointless to create a new product and set the price to zero.
     • If the policy is "create only", further processing is skipped.
     • If the policy is "if not set local" AND the store has product overlay functionality (rare) then a check is made to see if it has an overlay price record (ie local store price)
     • The new price is checked to any current price (using 2 decimal places, so 9.87 and 9.8692 are considered equal) the new price is ignored
     • If the cost price is to change, decisions are made as to immediate change, or to place this on the "pending price change" report. See trace file section below to understand where the price change is written
   • Cost Price
     • If the Cost Price policy is "always block" (from setting DistributeApply.Products.MEMBERSHIP-ID.f108) the field is ignored
     • If the policy is "if not set local", then if the product has not been set locally OR this is a new product, the change is saved.
     • If the policy is "create only" and the product is being created, the field is saved
     • If the policy is "always accept", the field is saved
   • Pid.
     • Blocked as already processed or revectored.
   • Spid, r_depid, r_depidN
     • These are all blocked as direct values, they are sent as sub records (see below)
   • RmSystem.
     • The membership RmSystem is applied if available, otherwise the inbound rmsystem. This field essentially tracks who defined/owns this product record.
4. The gateway is queried, "can we store this, now that you've seen everything?". If the answer is yes, operation continues, otherwise it is discarded, "Gateway blocked write" is written to the trace file and we move to the next inbound product
5. TBC...(line 25398)

Finalisation Phase

1. This retailers product RVE are updated. Potentially this retailer is also distributing products out to other retailers as well.
2. All internal caches relating to products are flushed.

Trace File Notes

The trace file contains technical dumps as the engine executes. Here is a typical example.

a. pid=3893,7459 Vc=188.900000000 newPrice=188.900000000
b. Pid=3893,7459 bHaveCurUnitPrice=1 CurUnitPrice=188.9000 FullyDone=0 LogicDone=0 WritePriceBuffer=0 has=0
Cost NOT applied.  Is set locally. pid=3893,7459
Subpacket Supplier
DistributeApply.Supplier.KEPZMTFQN35QWLWELECEYNAAACDAAAAB=0
Subpacket Department
DistributeApply.Department.KEPZMTFQN35QWLWELECEYNAAACDAAAAB=00000000 (1)
2nd pass Subpacket Supplier
2nd pass Subpacket Department
a. pid=3894,7458 Vc=188.900000000 newPrice=188.900000000
b. Pid=3894,7458 bHaveCurUnitPrice=1 CurUnitPrice=188.9000 FullyDone=0 LogicDone=0 WritePriceBuffer=0 has=0
Subpacket Supplier
Subpacket Department
2nd pass Subpacket Supplier
2nd pass Subpacket Department

If you have an issue, then the trace file contains the last processed records. Depending on timing that might not be the product you are interested in. A trick is to use the membership page to "download all" and wait few minutes, then grab the trace file. This will lilely have trace records of all products.

Trace lines explained

The order of trace lines depends largely on the field order sent from the sender.

a. pid=3893,7459 Vc=188.900000000 newPrice=188.900000000

This line is added when processing the "unitprice" field. The first pid (3893) is the local database Pid. This may be zero if this is a new product. The second pid (7459) is the senders Pid. The remainder of the line is the price that is being loaded.

Cost NOT applied.  Is set locally. pid=3893,7459

Added when the cost price field is processed. Cost price tracing was added in P254. There are several messages that can appear:

• Ignoring costprice. Policy is always block
• Cost applied. Not set locally
• Cost NOT applied. Is set locally
• Cost saved. This is a create
• Cost saved. Not otherwise handled and always accept set

b. Pid=3893,7459 bHaveCurUnitPrice=1 CurUnitPrice=188.9000 FullyDone=0 LogicDone=0 WritePriceBuffer=0 has=0

This is written during unitprice processing. The contents are:

• bHaveCurUnitPrice - flag indicating we found a unit price
• CurUnitPrice - unit price we found
• FullyDone - flag, if set indicates we have already fully completed handling this costprice change.
• LogicDone - flag which indicates if we have already matched a logic rule, or if we should continue to process logic rules
• WritePriceBuffer - set if we should be writing this change to a price change pending report.
• has - flag to indicate if this retailer has enabled pending price changes.

Retailers can chose to enable "pending price changes" themselves, hence why the logic above does not always simply write to that field.
When writing to pending price change, various checks are made to see if pricing is block (temp or perm), or if the price was already pending and this is a secondary revision.