Commit 42dc14ec authored by Mattias Päivärinta's avatar Mattias Päivärinta Committed by mats Dufberg
Browse files

API documentation (#233)

* Started integrating API and notes_api

* Comments from Mats

* Further work on API documentation

* Comments from Mats

* Document parameter defaults

* Replace grammatical error with placeholder about missing documentation

* Add documentation for translation language

* Document current state of domain name validation

* Document restriction of single concurrent batch per api user

* Typo

* Parallel Workers support currently lacks documentation

* Fix #170: Deprecate parameter "advanced"
parent 9401ca8c
......@@ -30,19 +30,35 @@ Follow the detailed [installation instructions](docs/installation.md).
### Configuration
Text for configuring the backend are found in the [installation
instructions](docs/installation.md).
Zonemaster *Backend* is configured as a whole from `/etc/zonemaster/backend_config.ini`.
>
> At this time there is no documentation for `backend_config.ini`.
>
### Documentation
There is a fully documented [API](docs/API.md), which is the primay way
to use the backend. The [docs](docs/) directory also contains the SQL commands
for manipulating the database.
The Zonemaster Backend documentation is split up into several documents:
* A number of [Typographic Conventions](docs/TypographicConventions.md) are used
throughout this documentation.
* The [Architecture](docs/Architecture.md) document describes each of the
Zonemaster Backend components and how they operate. It also discusses all
central concepts needed to understand the Zonemaster backend, and contains a
glossary over domain specific technical terms.
* The [Getting Started](docs/GettingStarted.md) guide walks you through creating
a *test* and following it through its life cycle, all using JSON-RPC calls to
the *Web backend*.
* The [API](docs/API.md) documentation describes the *Web backend* inteface in
detail.
The [docs](docs/) directory also contains the SQL commands for manipulating the
database.
License
=======
The software is released under the 2-clause BSD license. See separate
[LICENSE](LICENSE) file.
This diff is collapsed.
# Architecture
The Zonemaster *Backend* is a system for performing domain health checks and
keeping records of performed domain health checks.
A Zonemaster *Backend* system consists of at least three components: a
single *Database*, a single *Worker* and one or more *Web backends*.
## Components
### Database
The *Database* stores health check requests and results. The *Backend*
architecture is oriented around a single central *Database*.
### Worker
A Zonemaster *Worker* is a daemon that picks up *test* requests from the
*Database*, runs them using the *Engine* library, and records the results back
to the *Database*. A single *Worker* may handle several requests concurrently.
The *Backend* architecture supports a single *Worker* interacting with a single *Database*.
>
> TODO: List all files these processes read and write.
>
> TODO: List everything these processes open network connections to.
>
> TODO: Describe in which order *test* are processed.
>
> TODO: Describe how concurrency, parallelism and synchronization works within a single *Worker*.
>
> TODO: Describe how synchronization works among parallel *Workers*.
>
### Web backend
A Zonemaster *Web backend* is a daemon providing a JSON-RPC interface for
recording *test* requests in the *Database* and fetching *test* results from the
*Database*. The *Backend* architecture supports multiple *Web backends*
interacting with the same *Database*.
>
> TODO: List all ports these processes listen to.
>
> TODO: List all files these processes read and write.
>
> TODO: List everything these processes open network connections to.
>
## Glossary
### Test
### Batch
### Test result
### Test module
### Message
### Policy
### Profile
>
> TODO: Come up with a better name to distinguish it from *config profiles*.
>
### Config profile
*Config profiles* are configured under the the `ZONEMASTER` section of `zonemaster_backend.ini`.
>
> TODO: Describe this in greater detail.
>
### Engine
The Zonemaster *Engine* is a library for performing *tests*. It's hosted in [its
own repository](https://github.com/dotse/zonemaster-engine/).
# Getting started
This is a guide for getting started with the Zonemaster Web Backend JSON-RPC API.
>
> Note: This guide makes a number of assumptions about you setup:
>
> * that it's a Unix-like environment
> * that you have Zonemaster *backend* installed according to the [installation guide](installation.md)
> * that you have the tools `curl` and `jq` installed
>
First, check that the *Web backend* is running and answering properly.
```sh
curl -sS -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "version_info"}' http://localhost:5000/ | jq .
```
Enqueue a *test* of the domain `zonemaster.net`.
```sh
curl -sS -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 2, "method": "start_domain_test", "params": {"domain": "zonemaster.net", "ipv4": true}}' http://localhost:5000/ | jq .
```
However, we need the *test id* of the *test* we just enqueued.
Let's query the same method with the same params again, let `jq` filter out the *test id* for us, and then store it in an environment variable.
```sh
TESTID=`curl -sS -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 3, "method": "start_domain_test", "params": {"domain": "zonemaster.net", "ipv4": true}}' http://localhost:5000/ | jq .result`
echo "$TESTID"
```
Watch the *test* progress (`"result"`) up to `100` (percent) by repeatedly running this command.
```sh
curl -sS -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 4, "method": "test_progress", "params": '"$TESTID"'}' http://localhost:5000/ | jq .
```
Once the progress value has reached `100`, you can query for the *test result*.
```sh
curl -sS -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 5, "method": "get_test_results", "params": {"id": '"$TESTID"'}}' http://localhost:5000/ | jq .
```
If you're moderatly quick and repeatedly re-run the last command you should be able to see the progress value increase in uneven steps from `0` to `100`.
Never mind updating the JSON-RPC `"id"` - the server doesn't care.
Once the progress has reached 100, lots of test results should also be showing up.
# Typographic conventions
The Zonemaster Backend documentation uses the following typographic conventions:
*Italic* text is used for:
* technical terms defined in the [Architecture](Architecture.md) document
* data types defined in the [API](API.md) document
`Monospace` text is used for:
* snippets of JSON or sh
* JSON-RPC method names
* JSON values
* single or strings of characters
* internet addresses (e.g. domain names and IP addresses)
* file names with or without paths (e.g. configuration files and command line
tools)
* config section names
>
> Block quotes are used for:
>
> * notes and commentary
>
# Common
All requests consist of JSON objects with four keys.
1) "jsonrpc"
The value of this key is the fixed string `"2.0"`.
2) "method"
The name of the method to be called.
3) "id"
An id-value meant to connect a request with a response. The value
has no meaning, and will simply be copied to the response.
4) "params"
This key holds the parameters to the method being called.
All responses consist of JSON objects with three keys.
1) "jsonrpc"
The value of this key is the fixed string `"2.0"`.
2) "id"
An id-value meant to connect a request with a response. The value
has no meaning, and is simply copied from the request.
3) "result"
This key holds the results returned from the method that was called.
In the descriptions below, only the contents of the "params" (input)
and "result" (output) keys are discussed, since they are the only ones
where the content differs between different methods.
# API Methods
## version_info
### Input
None.
### Output
Version information string.
## get_ns_ips
### Input
A domain name.
### Output
A list of objects. The objects each have a single key and value. The
key is the domain name given as input. The value is an IP address for
the name, or the value `0.0.0.0` if the lookup returned no A or AAAA
records.
## get_data_from_parent_zone
### Input
A domain name.
### Output
An object with two keys, `ns_list` and `ds_list`.
The value of the `ns_list` key is a list of objects. Each object in
the list has two keys, `ns` and `ip`. The values of the `ns` key is
the domain name of a nameserver for the input name, and the value of
the `ip` is an IP address for that nameserver name or the value
`0.0.0.0`.
As of this writing, the value of the `ds_list` key is always an empty
list.
## validate_syntax
### Input
The input for this method is a JSON object. The object may have the
keys `domain`, `ipv4`, `ipv6`, `ds_info`, `nameservers`,
`profile`, `advanced`, `client_id` and `client_version`. If any other
key is present, an error will be returned.
If the key `nameservers` exists, its value must be a list of objects
each of which has exactly the two keys `ip` and `ns`. The value of the
`ns` key must meet the criteria described below for the `domain`
value, and the value of the `ip` key must be a syntactically valid
IPv4 or IPv6 address.
If the key `ds_info` exists, its value must be a list of
objects each of which has exactly the two keys `algorithm` and
`digest`. The value of the `algorithm` key must be either the string
`"sha1"`, in which case the value of the `digest` key must be 40
hexadecimal characters, or the value `"sha256"`, in which case the
value of the `digest` key must be 64 hexadecimal characters.
At least one of the keys `ipv4` and `ipv6` must exist and have one of
the values `1`, `0`, `true` or `false`.
If the key `advanced` exists, it must have one of the values `true`
and `false`.
If the key `profile` exists, it must have a value that is one of the
three strings `"default_profile"`, `"test_profile_1"` and
`"test_profile_2"`.
The key `domain` must exist and have a value that meets the following
criteria.
1) If the value contains characters outside the ASCII character set,
the entire value must be possible to convert to IDNA format.
2) If the value is a single character, that character must be `.`.
3) The length of the value must not be greater than 254 characters.
4) When the value is split at `.` characters (after IDNA conversion,
if needed), each component part must be at most 63 characters long.
5) Each such component part must also consist only of the characters
`0` to `9`, `a` to `z`, `A` to `Z` and `-`.
If the `nameservers` key is _not_ set, a recursive query made by the
server to its locally configured resolver for `NS` records for the
value of the `domain` key must return a reply with at least one
resource record in the `answer` section.
### Output
An object with the two keys `message` and `status`. If all criteria
above were met, the `status` key will have as its value the string
`"ok"`. If not, it will have the value `"nok"`. The value of the
`message` key is a human-readable string with more details in the
status.
## start_domain_test
### Input
The same as for `validate_syntax`.
### Output
The numeric ID of a newly started test, or a test with the same
parameters started within the recent configurable short time.
## test_progress
### Input
A numeric test ID.
### Output
An integer value (possible encoded in a string) between 0 and 100
inclusive, describing the progress of the test in question as a
percentage.
## get_test_params
### Input
A numeric test ID.
### Output
A JSON object with the parameters used to start the test (that is, the
input parameters to `start_domain_test`).
## get_test_results
### Input
A JSON object with the two keys `id` and `language`. `id` is a numeric
test ID. `language` is a string where the two first characters are a
language code to be used by the translator. As of this writing, the
language codes that are expected to work are `"en"`, `"sv"` and
`"fr"`. If the code given does not work, the translator will use
English.
### Output
A JSON object with a the following keys and values:
* `batch_id`
The ID number of the batch of tests this one belongs to. `null` if
it does not belong to a batch.
* `creation_time`
The time at which the test request was created.
* `domain`
The name of the tested domain.
* `id`
The numeric ID of the test.
* `params_deterministic_hash`
An MD5 hash of the canonical JSON representation of the test
parameters, used internally to identify repeated test request.
* `params`
The parameters used to start the test (that is, the values used as
input to `start_domain_test`).
* `priority`
The priority of the test. Used by the backend execution daemon to
determine the order in which tests are run, if there are more
requests than available test slots.
* `progress`
An integer in the interval 0 to 100 inclusive, showing the
percentage of the test process that has been completed.
* `results`
A list of objects representing the results of the test. Each
object has three keys, `module`, `message` and `level`. The values
of them are strings. The `module` is the test module that produced
the result, the `level` is the severity of the message as set in
the policy used (that is, one of the strings `DEBUG`, `INFO`,
`NOTICE`, `WARNING`, `ERROR` and `CRITICAL`) and `message` is a
human-readable text describing that particular result.
* `test_end_time`
The time at which the test was completed.
* `test_start_time`
The time at which the test was started.
## get_test_history
### Input
A JSON object with three keys, `frontend_params`, `offset` and
`limit`. The value of `frontend_params` is an object in turn, with the
keys `domain` and `nameservers`.
The values of `limit` and `offset` will be used as-is as the
corresponding values in SQL expressions. `domain` and `nameservers`
will be used to look up all tests for the given domain, separated
according to if they were started with a `nameservers` parameter or
not.
### Output
A JSON object with four keys. `id` is the numeric ID of the test.
`creation_time` is the time when the test request was created.
`advanced_options` is true if the corresponding flag was set in the
request. `overall_results` is the most severe problem level logged in
the test results.
## add_api_user
### Input
A JSON object with two keys, `username` and `api_key`. Both should be
strings, both are simply inserted into the users table in the
database.
### Output
The numeric ID of the just created user.
## add_batch_job
### Input
A JSON object with three keys, `username`, `api_key` and
`batch_params`. The first two are strings, and must match a pair
previously created with `add_api_user`.
`batch_params` is a JSON object. It should be exactly the same as the
input object described for `validate_syntax`, except that instead of
the `domain` key there should be a key `domains`. The value of this
key should be a list of strings with the names of domains to be
tested. All the domains will be tested using identical parameters. The
domain names should probably obey the rules for domain names, but in
this case no attempt is made to enforce those prior to starting the
tests.
### Output
## api1
### Input
None.
### Output
A string with the version of the Perl interpreter running the backend
(more specifically, the value of the `$]` variable in the Perl
interpreter).
# Examples
## Minimal request to start a test
```
{
"domain": "example.org",
"ipv4": true
}
```
## Non-minimal request to start a test
```
{
"domain": "example.org",
"ipv4": 1,
"ipv6": 1,
"ds_info": [],
"profile": "default_profile",
"advanced": false,
"client_id": "Documentation Example",
"client_version": "1.0",
"nameservers": [
{
"ns": "ns1.example.org",
"ip": "192.168.0.1"
},
{
"ns": "ns2.example.org",
"ip": "2607:f0d0:1002:51::4"
}
]
}
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment