More and more people are using the resources of the Strathspey SCD
Database for their own purposes, which is great – that is what it is
supposed to be there for. One area that seems to be receiving quite
some interest lately is dance lists, but so far, external users have
been forced to scrape the screen or use the /list/embed interface
(which returns HTML) for information about dance lists, which is less
than optimal.
Our plan is to make more of the SCDDB accessible via APIs that are based on HTTP and JSON, and we’re making a tentative start here in order to make it easier for programmers to get at dance lists. The API support right now is fairly rudimentary but more features will be added in the future.
The Lists API comprises a set of HTTP endpoints which are all located below the
https://my.strathspey.org/dd/api/lists/v1/
URL path. The /v1/ component specifies that this is version 1 of the
API; if at some point in the future the API changes in a way that is
incompatible with what it was before, the version number will be
incremented (and presumably there will be several versions of the API
available at least for a while). In the interest of expediency,
though, we will feel free to add stuff to the v1 API as long as the
existing endpoints are not changed incompatibly; we can do this
because my.strathspey.org is the only server which implements the
Lists API, and as long as clients keep working everything should be
fine.
For the time being, the Lists API has the following restrictions:
Dance lists can only be read, not modified or deleted, and no new lists can be added via the API. For this to change, we will need to sort out authentication (i.e., we want you to be able to prove to the API’s satisfaction that you are who you say you are before we’re going to let you diddle any dance lists), and we’re not entirely sure what the most convenient method would be.
Only dance lists that are visible to the general public (as opposed to private dance lists visible only to their owners) are accessible via the API. Again, this is not going to change until we introduce authentication, and then of course you will only be able to get at your private dance lists, not those of other users.
For the most part, the HTTP endpoints accept parameters either as part
of the URL path or as part of the URL query string. Eventually once we
accept PUT or POST requests we will of course look at request
bodies. Results are generally returned as JSON data. Strings are
encoded in UTF-8 format but note that some characters may be escaped
as HTML entities.
You can query the URL experimentally using, e.g., curl or a
specialised HTTP API application such as Postman. You can also use the
OpenAPI browser to
look around and try things. If you’re programming, your programming
language will presumably have some sort of convenient support for HTTP
requests and responses that saves you from having to stuff bytes down
raw TCP connections. For example, if you’re a Python programmer, check
out the requests library.
In the easiest case,
https://my.strathspey.org/dd/api/lists/v1/list
will give you a JSON dictionary that looks roughly like
{
"items": [
{
"id": 1,
"name": "Nov Dance Part 1",
"owner": "katemnic",
"type": "unknown",
"date": null,
"is_private": false,
"is_template": false,
"item_count": 10
},
{
"id": 2,
"name": "41st Frankfurt Spring Ball 2009",
"owner": "anselm",
"type": "function",
"date": "2009-04-18",
"is_private": false,
"is_template": false,
"item_count": 18
},
… more lists omitted for brevity …
]
}
(you get the idea). That is, the dictionary contains a single entry,
items, whose value is an array of dictionaries, each describing one
dance list, ordered ascending by database ID.
Theoretically we could return the array of dictionaries directly, but doing that sort of thing is not secure; best practices suggest that JSON replies should always be dictionaries. It is arguable whether this is still actually an issue in 2023, but who knows what browsers people are using on their Windows XP machines, and in any case it is at best a minor inconvenience for us programmers.
The items in each list dictionary mostly speak for themselves. The
owner is the my.strathspey username of the account whose list we’re
talking about, and date is either null (for undated lists) or a
date in ISO format (YYYY-MM-DD). The type can be function,
class, informational, other, or unknown. The item_count is
the number of items on the list; this includes dances and extras as
well as headings, step and formation practice entries, and “Other”.
If you look closely you will notice that there are gaps in the sequence of database IDs for the lists. This is because this endpoint, by default, returns only lists that are readable by the public; private lists don’t show up.
As you’ve seen, the /list API endpoint will give you a list of all
public dance lists on the server in ascending order of database
ID. Well, not actually all of them; by default only the first 100 will
be returned. This can be controlled by adding a limit parameter to
the URL, as in
https://my.strathspey.org/dd/api/lists/v1/list?limit=1000
(You could ask the server for all lists at once by setting
limit=9999999 or something but please don’t do that. At least, not
often.) Or else, once you have retrieved the first hundred you could
ask for the second hundred by adding an offset, as in
https://my.strathspey.org/dd/api/lists/v1/list?offset=100
Of course, offset and limit can be combined if you want to process
lists in chunks that are larger or smaller than 100 specimens.
Chances are that you’re not going to be interested in all public lists on the server, and downloading a list of all of them to go through them on your own machine is both tedious and a waste of resources (we don’t really care what you do on your own computer but we’d much rather not tie up our server with your requests of all public dance lists if all you’re really looking for is the list for last week’s class, thank you). So for your convenience (and ours) there are various methods to narrow down which dance lists you can retrieve.
For example, if you’re only interested in lists by a specific owner, such as yours truly, you could say something like
https://my.strathspey.org/dd/api/lists/v1/list?owner=anselm
(all you need to know to make this work is that yours truly is
anselm on my.strathspey). This restricts the output to my dance
lists.
You can also search for lists whose name contains a particular string. For example,
https://my.strathspey.org/dd/api/lists/v1/list?name=FSCDC%20Class
will return only lists with FSCDC Class in their name (note how the
space between FSCDC and Class had to be URL-encoded in the query).
These two (or indeed, any) “filters” can of course be combined,
which will result in their being applied one after the other
(effectively an AND operation):
https://my.strathspey.org/dd/api/lists/v1/list?owner=anselm&name=Ball
will return only those of my dance lists with Ball in their names.
You can use type=function, type=class, etc. to restrict the output
to particular types of dance list, and date=YYYY-MM-DD to restrict
the output to lists associated with that particular calendar date. You
can also use date_from to look for lists whose date lies on or
after a given date, and date_to to locate lists whose date is on
a given date or earlier. For example,
https://my.strathspey.org/dd/api/lists/v1/list?type=function&date_from=2023-01-01&date_to=2023-03-31
gives you all lists describing functions that took place in the first quarter of 2023.
Use order=date to sort the result by date rather than database
ID. Other valid values for order include name, owner, type,
and item_count. Put a minus sign (-) at the start of the value to
return the results in reverse order. For example, if you’re interested
in the most recent class at the Frankfurt SCD Club, use
https://my.strathspey.org/dd/api/lists/v1/list?name=FSCDC%20Class&order=-date&limit=1
You can also combine order keys; for example, to sort the result in
descending order by date and entries on the same date in ascending
order by name, use order=-date,name.
Any public dance list can be retrieved by passing its database ID to
the /list API endpoint, as in
https://my.strathspey.org/dd/api/lists/v1/list/40054
This will produce output like
{
"id": 40054,
"name": "FSCDC Class 28 March 2023",
"owner": "anselm",
"type": "class",
"date": "2023-03-28",
"is_private": false,
"is_template": false,
"notes": "Dances from *RSCDS Tokai 25th Anniversary Book*",
"time_base": "19:15",
…
The first seven entries in the dictionary correspond to those that the
/list endpoint (without a dance list ID) would return when this dance
list appears in a list of dance lists. The notes item contains the
explanatory note for the list as a whole (it can extend across several
lines), and the time_base gives the time of day based on which
timings in the dance list are calculated (we will get back to that
later).
Next come the items on the list, in an array of dictionaries:
"items": [
{
"id": 693537,
"number": 1,
"type": "O",
"description": "Warmup",
"dance": null,
"notes": "",
"scratch": false,
"time": 10,
"recordings": []
},
…
{
"id": 693538,
"number": 3,
"type": "D",
"description": "",
"dance": {
"id": 20877,
"name": "Happy Dancing",
"displayname": "Happy Dancing",
"type": "J32",
"set": "3/4L",
"source": "Kajino: Tokai 25th Anniversary"
},
"notes": "",
"scratch": false,
"time": 15,
"recordings": [
{
"id": 10689,
"name": "Happy Dancing",
"artist": "Bluebell Scottish Country Dance Trio",
"album": "RSCDS Tokai 25th Anniversary",
"type": "J32 8"
}
]
},
Again, these data structures are fairly self-explanatory. The id
items refer to the database IDs of the dance list items and may become
more important later when we add functionality that allows them to be
modified via the Lists API. The type
specifies the type of dance list item:
| Code | Type of dance list item |
|---|---|
| D | Dance |
| X | Extra |
| S | Step Practice |
| F | Formation Practice |
| P | Programme Item |
| H | Heading |
| O | Other |
For S, F, P, H, and O dance list
items, the description contains the
text that was entered in the non-dance item input field on the “Edit
List” page, such as the type of step or formation taught, the nature
of a programme item such as “Highland demonstration”, or the text for
a heading.
For D and X dance list items, the dance item contains another
dictionary giving details about the dance in question, including the
database ID of the dance, the name (with articles like “The” at the
end, as in “Happy Meeting, The”) and displayname (with articles at
the front, as in “The Happy Meeting”).
The notes item contains the note for that item (which on
the “Edit List” page can be entered after clicking on the “pencil and
pad” icon at the right edge of the item’s box), and time contains
the duration of the item in integer minutes (0 if no duration was
specified). The time values of the list items can be added to the
list’s time_base to figure out when each item is supposed to start,
like (in Python)
hh, mm = map(int, dance_list.time_base.split(':'))
for k, item in enumerate(dance_list.items):
print(f"Item {k} starts at {hh:2d}:{mm:02d}"
mm += item.time
if mm >= 60:
hh += mm // 60
mm %= 60
The Boolean value scratch specifies whether this item belongs to the
actual dance list or the “scratch space” which can be used to store
items temporarily. The number gives the position of the dance list
item in the dance list or scratch space (each time starting from 1).
Every dance list item can be associated with one or more recordings in
the database. These are listed in recordings and are largely
obvious. For dance list items without recordings, recordings
contains an empty list.
The database ID of a dance list can be obtained, e.g., from a listing
returned by the /list endpoint of the Lists API. Sometimes, though, it
is more convenient to refer to a dance list by its name without having
to retrieve a list first just to get at its database ID. For
convenience, the /list endpoint supports queries like
https://my.strathspey.org/dd/api/lists/v1/list/OWNER:NAME
where OWNER is the my.strathspey username of the list owner and
NAME the name of the dance list. (Remember that dance list names are
guaranteed to be unique only among the dance lists of a single owner.)
For example,
https://my.strathspey.org/dd/api/lists/v1/list/anselm:FSCDC%20Class%2028%20March%202023
(note, again, the URL encoding of the spaces) would retrieve the same list we looked at before (with ID 40054).
Sign in to see recent visitors!