“Shop data” is used to link database pages (currently those for publications and albums) to product pages in online shops offering the corresponding items for purchase. This is a service that the Strathspey SCD Database offers equitably and at no charge to all online purveyors of SCD media, in the interest and for the convenience of the SCDDB user community. If you sell SCD publications and/or albums online to the public, by all means do get in touch> so we can set you up. The rest of this document is meant for shop operators, whom we will call “you” for simplicity.
In order to be able to upload shop data, you will need to have some basic configuration done on the site.
We need an official name for the shop (this is what will be displayed on the product pages) as well as the my.strathspey username of the account that will be uploading the data (for authentication).
We need a base URL for the products that will be linked. This should be the longest common prefix of all product page URLs in the shop. For example, if product pages in the online shop look like
https://scd-shop.example.com/products/foo/bar/baz.html
https://scd-shop.example.com/products/quux/glorp/corge.html
etc., the base URL could be https://scd-shop.example.com/products.
We use this to ensure that shop data can only contain URLs that point to
pages within the shop (for security).
We assume that you will periodically upload data that informs the database about the availability of products in your shop and how to get to the product page to order them there. We don’t worry about actual availability as in “this item is currently on back-order”; that is something that you can do on your own site. What we’re interested in is whether a shop is willing and able to sell a product in principle — this means that we would expect you to eventually remove links to materials which are out of print and no longer available for sale at all, but temporary shortages don’t imply something ought to be removed. In other words, we don’t need (or for that matter even want) hourly updates from you.
To upload shop data to the database, you must prepare a CSV (“comma-separated values”) file where the first three columns on each line have the following meaning:
The database ID of the item in question. This is a number which
you can find to the right of the title on the item’s detail page,
or at the end of the URL of the item’s detail page. For example,
the detail page for Book 50 by the RSCDS is at
https://my.strathspey.org/dd/publication/1325/, and the 1325
is also prominently displayed on that page. — Unfortunately there
is currently no convenient way of obtaining the database IDs for
a large set of items other than by looking them up one by
one. Fortunately this is something you do once for every item. A
reasonably quick method is to use the publication and album list
pages (they have a convenient “filter” field near the top right
corner of the page that will let you narrow down the list to a
particular set of items very quickly) and to point one’s mouse at
the title of an item, without clicking on it, so the URL of the
detail page, which contains the database ID, is displayed by the
browser.
A product type code, which is “P” for a publication on paper, “E” for an electronic publication (e.g., PDF file), or “C” for a CD. (We’re happy to add more product type codes if necessary). Items marked “P” or “E” will be looked for in the publications table, and items marked “C” in the albums table. Undefined media codes will cause error messages.
A URL path, which is essentially the part of the item’s product
page in your online shop that is not covered by the “base URL”
mentioned above. For example, in the first specimen mentioned
above, the base URL would be
https://scd-shop.example.com/products and the URL path would be
/foo/bar/baz.html. (You should ensure that the result of
concatenating the base URL and URL path contains the correct
number of slashes. In our example, if the base URL was
https://scd-shop.example.com/products and the URL path was
foo/bar/baz.html, the resulting concatenation
https://scd-shop.example.com/productsfoo/bar/baz.html might not
work. Spare slashes are usually not a problem but that would
depend on the web server that is hosting your online shop.) (We
don’t want to add our own slashes because it could be the case
that the base URL is something like
https://scd-shop.example.com/?product= and the URL path
something like 1234, and adding a slash between the two would
be Wrong.)
Any further columns in the CSV file will be ignored. You should feel free to add information that can be useful when debugging issues, such as the title and author of the item in question.
The basic idea here is that ideally you should be able to get the database system that is running your shop to produce such a list for you “on demand”. You may have to add database columns to hold the database ID or media code, but this should not be an insurmountable obstacle. Alternatively, you could manually maintain an Excel spreadsheet with this information but that is not going to be as much fun.
There are two basic methods of uploading shop data to the
database. The manual method involves navigating to the shop upload
page, which is https://my.strathspey.org/dd/shop/SLUG/upload/
(SLUG stands for a short identifier for your shop that we will
assign when the shop is first configured in the database). This page
offers an HTML form where you can specify the name, on your own
computer, of the CSV file containing your shop data. If you submit
that form, the shop data file will be uploaded and processed.
The automated method, on the other hand, makes it possible to update
the database without human intervention if you manage to create a shop
data file automatically. You can then upload this using a program like
curl to perform an HTTP POST request that includes the shop data
file. To ensure that the upload comes from you, the POST request must
also contain the “upload key”, a string of random characters which you
can obtain from the shop upload page. (Note that the shop upload page
is only available to the my.strathspey account which is designated as
the official uploader for the shop in question, so there shouldn’t be
a big problem with keeping this confidential.) If your CSV file is
called data.csv, and the upload key is wa6AXLjgRd0VwzDB, a sample
curl call to upload the shop data to the shop named scdshop could
look like
$ curl -F key='wa6AXLjgRd0VwzDB' -F data=@data.csv https://my.strathspey.org/dd/shop/scdshop/upload/
It would of course also be perfectly possible to use a tool like the Python “requests” library to do this from within a program. Make sure that the POST data is uploaded using the “multipart/form-data” content type which allows file data to be included.
When you upload shop data to the database (using either of the two approaches), the system tries to match all the database IDs (from the first column of the file) to entries in the database, using the media codes (from the second column) to identify the table to use. Only if there are no errors in the uploaded data — missing fields, object IDs that aren’t integers, invalid media codes, or empty URL paths — will the shop data actually be written to the database.
The database returns the number of records read, the number of errors
detected, and a list of the entries read including error messages for
any erroneous lines. This should help you figure out and correct any
problems in the shop data before you try uploading the data again.
These results are returned as an HTML page if the data was submitted
manually via the HTML form, and as a JSON document if the data was
submitted via curl or a similar tool using the upload key for
authentication.
Sign in to see recent visitors!