Where and how is my data stored?
Internally within Gds, it is stored in binary format files. These files are controlled by application, but we make it easy for you to get data in and out. A binary format is used so that we can provide guarantees of operation. Essentially this binary format is used for transport and intenal use, but it is exposed to more "standard" formats for you to use.
Some worked examples should make this clearer.
- A head office user creates a new product using the web edit screen. ("Drink, $2")
- The local machine (or web server) which first receives this request:
- Validates the input
- Verifies the user is authorised to perform this action
- Adds the record to the local databases (if present) as well as adding to internal storage.
- Sends the record details as a "business transaction" to other systems, so they can add it too.
The code that performs the above steps is called the implementor ie it implements the users command.
- Remote machines that receive the business transaction, will typically skip steps (a) and (b), and just store and forward
At step 2(c), the implementor "adds to local databases". This means that individual machine will update whatever underlying storage it is using. Typically this will be Oracle or Sql/Server in head office, but might be Sql/Server Express in larger stores, or Access.
The implementor can write to several types of storage, ODBC, Acces, CSV and flat files etc. They can be "current view", or log of all changes - depending on what you want.
How can I edit reference data?
You can use any or all of the following (varies by what you are editing exactly)
- Fieldpine Web Pages
- Mobile Apps
- HTTP API directly (eg your own applications)
- Bulk loading Excel Spreadsheets (or CSV etc)
- XML transfers
- Email commands to the Retail System.
- Database import tables
Each of the above will eventually call the API "product.edit". This API is the implementor.
If you are authorised to edit records, you can edit them anywhere in the system, you do not need to login to head office (this can be blocked in some cases)
What is missing, is that you cannot (by default) simply edit the SQL database we explained in the first question above. There could be many databases in your network, with various levels of security, so all edits must go through the API to ensure system consistency.
In reality, if you wish, you can mark your "master database" in a special way, and the system will read through this looking for changes to send through the API. Your changes to applied in this way must still be valid, you cannot set a date field to "Call Alice"
A powerful side effect of this system is that every change gets a log of changes, including where and when is was made.
Can I edit other data eg sales?
Yes and no. There aren't many inherent restrictions on you changing data, if an API exists. There is an API to edit sales and you can change many attributes, or even delete them. However the system maintains a change log of everything.
Obvously, editing a sale 3 weeks after it was completed may not cause it to reprocess. If an email address was wrongly entered to email a receipt to a customer, and then editted correctly later, the system will not retrigger the emailing of a receipt just because you editted a field
What data is where?
Each table or business transaction, decides for itself the scope of data. Some examples are:
| Customers | Generally everywhere, although possibly hidden from view |
| Products | Generally everywhere, although possibly hidden from view |
| Valid Tender Types | Everywhere |
| Sales Records | Typically up to higher ranked systems (Lanes --> Store Servers --> Head Office). Although temporary backups are sent "sideways" |
| Audit Records | Up to higher ranked systems |
Can I read data that isn't stored in SQL etc?
Yes. There is a lot of data stored that doesnt make sense to store in SQL. You can ask Gds/2 (via an API) to extract any data, returning it either in a HTTP response, or direct to a file (best for large/full extracts). A good example is full logging (if enabled), you may wish to extract a single date/store to investigate
There are also support utilities to directly read the mesh data files and dump them to readable formats, but these require the encryption keys. (Note, these utilities are restricted distribution)
What if the network is down?
Edits and other records will buffer and send when the network comes back up. You can be offline for an extended period (days) without too much effect, although oviously you cannot get updates.
The system is also capable of using other techniques to send and receive records if you allow it. You can copy the files manually, use Dropbox or cloud storage, IoT routing nodes, email etc. By default the system will not (knowingly) allow data out of your network
Side note, the system is capable of communicating sometimes, at reduced speeds, even if your TCP network is misconfigured and "not working"
What about security?
We can be so aggresive with your data updates and routing, because all information is encrypted. (more. See "Network Packets" section). The default encryption keys are at least 16Mbits in size, from which sub keys and session keys are derived. When sending over insecure channels, data is encrypted and then split into multiple packets, with the reassembly information encrypted as well.