From 3c6605196bf7b2ef7f4b74463fb9f7c8c45580b9 Mon Sep 17 00:00:00 2001 From: hiletmis <31757771+hiletmis@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:16:29 +0300 Subject: [PATCH] Switch proseWrap to never Co-Authored-By: Claude Opus 5.5 --- .prettierrc | 2 +- README.md | 25 +++----- packages/airnode-feed/README.md | 36 ++++------- packages/airnode-feed/config/configuration.md | 59 ++++++------------- packages/e2e/README.md | 11 +--- packages/performance-test/README.md | 57 +++++------------- packages/signed-api/README.md | 32 ++++------ packages/signed-api/config/configuration.md | 54 +++++------------ 8 files changed, 81 insertions(+), 195 deletions(-) diff --git a/.prettierrc b/.prettierrc index 5c91792b..649b8991 100644 --- a/.prettierrc +++ b/.prettierrc @@ -10,7 +10,7 @@ "files": "*.md", "options": { "parser": "markdown", - "proseWrap": "always" + "proseWrap": "never" } } ] diff --git a/README.md b/README.md index 407cf479..836e4dea 100644 --- a/README.md +++ b/README.md @@ -2,16 +2,12 @@ A monorepo for managing signed data. Consists of: -- [signed-api](./packages/signed-api/README.md) - A service for storing and accessing signed data. It provides endpoints - to handle signed data for a specific airnode. +- [signed-api](./packages/signed-api/README.md) - A service for storing and accessing signed data. It provides endpoints to handle signed data for a specific airnode. - [airnode-feed](./packages/airnode-feed/README.md) - A service for pushing data provider signed data. - [e2e](./packages/e2e/README.md) - End to end test utilizing Mock API, Airnode feed and signed API. -- [performance-test](./packages/performance-test/README.md) - Configurations and scripts to allow running performance - tests and benchmarks on AWS. +- [performance-test](./packages/performance-test/README.md) - Configurations and scripts to allow running performance tests and benchmarks on AWS. -Read the -[specification](https://docs.google.com/document/d/1-kUPIXSD4ZW1SGs_P8HsejC9k9aHB-NXs9_6-OclnmE/edit#heading=h.i307237rdfda) -for more details on the architecture and the components. +Read the [specification](https://docs.google.com/document/d/1-kUPIXSD4ZW1SGs_P8HsejC9k9aHB-NXs9_6-OclnmE/edit#heading=h.i307237rdfda) for more details on the architecture and the components. ## Getting started @@ -27,22 +23,15 @@ and to build the packages: pnpm run build ``` -Note, that every time you make a change to a workspace that is used as a dependency of another, you need to rebuild the -changed package (otherwise you might get weird JS/TS errors). +Note, that every time you make a change to a workspace that is used as a dependency of another, you need to rebuild the changed package (otherwise you might get weird JS/TS errors). ## Versioning and release -Signed API and Airnode feed use [semantic versioning](https://semver.org/). Packages are published to NPM to export the -configuration schemas and various utilities. The services are dockerized and published to Docker Hub. +Signed API and Airnode feed use [semantic versioning](https://semver.org/). Packages are published to NPM to export the configuration schemas and various utilities. The services are dockerized and published to Docker Hub. There is a script that automates the process of creating new NPM packages and Docker images. Full release procedure: -1. `pnpm run create-release:npm [major|minor|patch]` - The script ensures publishing happens from up-to-date `main` - branch. It updates the package versions for `airnode-feed` and `signed-api`, updates fixtures and example files, does - basic checks to ensure the changes are valid and creates a version commit. The command intentionally does not do the - publishing so that the changes can be reviewed before pushing. +1. `pnpm run create-release:npm [major|minor|patch]` - The script ensures publishing happens from up-to-date `main` branch. It updates the package versions for `airnode-feed` and `signed-api`, updates fixtures and example files, does basic checks to ensure the changes are valid and creates a version commit. The command intentionally does not do the publishing so that the changes can be reviewed before pushing. 2. `git show` - To inspect the changes of the version commit. 3. Run the e2e tests locally. This is not automated due to implementation complexity. -4. `git push` - Push the version commit upstream. This will trigger the `tag-and-release` GitHub Actions job and result - in 1) the commit being tagged with the new version, 2) the release being created on GitHub, 3) both packages being - released on npm, and 4) both Docker images being built and pushed to Docker Hub. +4. `git push` - Push the version commit upstream. This will trigger the `tag-and-release` GitHub Actions job and result in 1) the commit being tagged with the new version, 2) the release being created on GitHub, 3) both packages being released on npm, and 4) both Docker images being built and pushed to Docker Hub. diff --git a/packages/airnode-feed/README.md b/packages/airnode-feed/README.md index 438ddb5b..53d8bc56 100644 --- a/packages/airnode-feed/README.md +++ b/packages/airnode-feed/README.md @@ -2,14 +2,10 @@ > A service for storing and accessing signed data. -Airnode feed is a Node.js service, dockerized and deployable on any cloud provider or hostable on premise. It is -continuously running two core loops: +Airnode feed is a Node.js service, dockerized and deployable on any cloud provider or hostable on premise. It is continuously running two core loops: -1. `Fetch beacon data` - Each `triggers.signedApiUpdates` entry defines a group of templates. Airnode feed makes a - template request to the API specified in the OIS to get the template data. Airnode feed's wallet is used to sign the - responses and these are then saved to in-memory storage. -2. `Push signed beacon data to signed API` - For each `triggers.signedApiUpdates`, periodically checks the in-memory - storage and pushes the signed data to the configured API. +1. `Fetch beacon data` - Each `triggers.signedApiUpdates` entry defines a group of templates. Airnode feed makes a template request to the API specified in the OIS to get the template data. Airnode feed's wallet is used to sign the responses and these are then saved to in-memory storage. +2. `Push signed beacon data to signed API` - For each `triggers.signedApiUpdates`, periodically checks the in-memory storage and pushes the signed data to the configured API. ## Configuration @@ -24,20 +20,16 @@ The Airnode feed needs a configuration in order to run. The `config` folder cont To start the the Airnode feed in dev mode run the following: -1. `cp config/airnode-feed.example.json config/airnode-feed.json` - To copy the Airnode feed configuration from the - example. Note, the `airnode-feed.json` file is ignored by git. If you are using Docker Desktop, you need to change - the URL from localhost to `host.docker.internal`. For example: +1. `cp config/airnode-feed.example.json config/airnode-feed.json` - To copy the Airnode feed configuration from the example. Note, the `airnode-feed.json` file is ignored by git. If you are using Docker Desktop, you need to change the URL from localhost to `host.docker.internal`. For example: ```jsonc "url": "http://host.docker.internal:8090" ``` -2. `cp config/secrets.example.env config/secrets.env` - To copy the secrets.env needed for the configuration. This file - is also ignored by git. +2. `cp config/secrets.example.env config/secrets.env` - To copy the secrets.env needed for the configuration. This file is also ignored by git. 3. Set the `NODARY_API_KEY` inside the secrets file. Ask someone from development team for the key. 4. `cp .env.example .env` - To copy the example environment variables. Optionally change the defaults. -5. `pnpm run dev` - To run the Airnode feed. This step assumes already running signed API as specified in the - `airnode-feed.json` configuration. +5. `pnpm run dev` - To run the Airnode feed. This step assumes already running signed API as specified in the `airnode-feed.json` configuration. ### Testing @@ -53,8 +45,7 @@ LOGGER_ENABLED=true pnpm run test ### Docker instructions -You can use shorthands from package.json. To understand how the docker image is built, read the -[Dockerfile](../../Dockerfile). +You can use shorthands from package.json. To understand how the docker image is built, read the [Dockerfile](../../Dockerfile). ```sh pnpm run docker:build @@ -65,9 +56,7 @@ pnpm run docker:run -To deploy Airnode feed on AWS you can use the Cloud Formation template created by the API integrations team. The -template can be found in the private api-integrations repository -[here](https://github.com/api3dao/api-integrations/blob/main/data/cloudformation-template.json). +To deploy Airnode feed on AWS you can use the Cloud Formation template created by the API integrations team. The template can be found in the private api-integrations repository [here](https://github.com/api3dao/api-integrations/blob/main/data/cloudformation-template.json). @@ -80,14 +69,11 @@ To run the Airnode feed docker image you need to: 1. Mount config folder to `/app/config`. The folder should contain the `airnode-feed.json` and `secrets.env` files. -2. Pass the `-it --init` flags to the docker run command. This is needed to ensure the docker is stopped gracefully. See - [this](https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals) for details. +2. Pass the `-it --init` flags to the docker run command. This is needed to ensure the docker is stopped gracefully. See [this](https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals) for details. 3. Specify the `--env-file` with the path to the `.env` file containing the [ENV configuration](#environment-variables). 4. Optionally, pass the `--rm` flag to remove the container after it is stopped. -5. If running on Linux, you need to use the `--network host` to access the host network. This has no effect for Docker - Desktop. -6. Lastly, if you are using Docker Desktop and you want to access the host machine, you need to change the host URL from - `localhost` to `host.docker.internal` in the configuration files. +5. If running on Linux, you need to use the `--network host` to access the host network. This has no effect for Docker Desktop. +6. Lastly, if you are using Docker Desktop and you want to access the host machine, you need to change the host URL from `localhost` to `host.docker.internal` in the configuration files. diff --git a/packages/airnode-feed/config/configuration.md b/packages/airnode-feed/config/configuration.md index a1983d70..c8475a4a 100644 --- a/packages/airnode-feed/config/configuration.md +++ b/packages/airnode-feed/config/configuration.md @@ -1,12 +1,10 @@ # Configuration -Airnode feed can be configured via a combination of [environment variables](#environment-variables) and -[configuration files](#configuration-files). +Airnode feed can be configured via a combination of [environment variables](#environment-variables) and [configuration files](#configuration-files). ## Environment variables -Logging needs to be initialized prior the configuration files are loaded. This is done via environment variables. All of -the environment variables are optional and or set with default values for convenience. +Logging needs to be initialized prior the configuration files are loaded. This is done via environment variables. All of the environment variables are optional and or set with default values for convenience. Example: @@ -46,16 +44,13 @@ Default: `true`. The format of the log output. Options: - `json` - Specifies JSON log format. This is suitable when running in production and streaming logs to other services. -- `pretty` - Logs are formatted in a human-friendly "pretty" way. Ideal, when running the service locally and in - development. +- `pretty` - Logs are formatted in a human-friendly "pretty" way. Ideal, when running the service locally and in development. Default: `json`. ### `LOG_HEARTBEAT` _(optional)_ -Enables or disables the heartbeat log. The heartbeat log is a cryptographically secure log that is emitted every 60 -seconds to indicate that the service is running. The log includes useful information such as the deployment timestamp -and configuration hash. Options: +Enables or disables the heartbeat log. The heartbeat log is a cryptographically secure log that is emitted every 60 seconds to indicate that the service is running. The log includes useful information such as the deployment timestamp and configuration hash. Options: - `true` - Enables the heartbeat log. - `false` - Disables the heartbeat log. @@ -66,8 +61,7 @@ Default: `true`. Enables or disables colors in the log output. Options: -- `true` - Enables colors in the log output. The output has special color setting characters that are parseable by CLI. - Recommended when running locally and in development. +- `true` - Enables colors in the log output. The output has special color setting characters that are parseable by CLI. Recommended when running locally and in development. - `false` - Disables colors in the log output. Recommended for production. Default: `false`. @@ -85,16 +79,13 @@ Default: `info`. ## Configuration files -Airnode feed needs two configuration files, `airnode-feed.json` and `secrets.env`. All expressions of a form -`${SECRET_NAME}` are referring to values from secrets and are interpolated inside the `airnode-feed.json` at runtime. -You are advised to put sensitive information inside secrets file. +Airnode feed needs two configuration files, `airnode-feed.json` and `secrets.env`. All expressions of a form `${SECRET_NAME}` are referring to values from secrets and are interpolated inside the `airnode-feed.json` at runtime. You are advised to put sensitive information inside secrets file. You can also refer to the example configuration files. ### `templates` -Configuration for the template requests. Each template request is defined by a `templateId` and a `template` object. For -example: +Configuration for the template requests. Each template request is defined by a `templateId` and a `template` object. For example: ```jsonc // Defines a single template. @@ -122,9 +113,7 @@ The ID of the endpoint to which the template request is made. ##### `parameters` -The parameters of the template request. Refer to -[Airnode ABI](https://airnode-docs.api3.org/reference/airnode/latest/specifications/airnode-abi.html) specification for -details. +The parameters of the template request. Refer to [Airnode ABI](https://airnode-docs.api3.org/reference/airnode/latest/specifications/airnode-abi.html) specification for details. ##### `parameters[n]` @@ -132,8 +121,7 @@ Defines one of the parameters of the template request. `type` -Refer to -[Airnode ABI available types](https://airnode-docs.api3.org/reference/airnode/latest/specifications/airnode-abi.html#details). +Refer to [Airnode ABI available types](https://airnode-docs.api3.org/reference/airnode/latest/specifications/airnode-abi.html#details). `name` @@ -177,8 +165,7 @@ The title of the OIS to which the endpoint belongs. ### `triggers.signedApiUpdates` -Configuration for the signed API update triggers. There can be multiple triggers, each specifying a different update -configuration. +Configuration for the signed API update triggers. There can be multiple triggers, each specifying a different update configuration. For example: @@ -202,15 +189,9 @@ For example: #### `triggers.signedApiUpdates[n]` -Configuration for one of the signed API update triggers. Airnode feed periodically fetches data from data provider and -pushes it to all Signed APIs defined in the configuration file. +Configuration for one of the signed API update triggers. Airnode feed periodically fetches data from data provider and pushes it to all Signed APIs defined in the configuration file. -Airnode feed only makes a single template request independently of the number of template IDs specified. This is to -reduce the number of data provider calls. This implies that all of the templates in the trigger must use the same -endpoint and parameters. You can use -[OIS processing](https://airnode-docs.api3.org/reference/ois/latest/processing.html) to remove the parameters before -making the request (using pre-processing) and later get the corresponding template value based on the endpoint -parameters (using-processing). +Airnode feed only makes a single template request independently of the number of template IDs specified. This is to reduce the number of data provider calls. This implies that all of the templates in the trigger must use the same endpoint and parameters. You can use [OIS processing](https://airnode-docs.api3.org/reference/ois/latest/processing.html) to remove the parameters before making the request (using pre-processing) and later get the corresponding template value based on the endpoint parameters (using-processing). ##### `templateIds` @@ -249,8 +230,7 @@ The URL of the signed API. #### `authToken` -The authentication token used to authenticate with the signed API. It is recommended to interpolate this value from -secrets. +The authentication token used to authenticate with the signed API. It is recommended to interpolate this value from secrets. If the signed API does not require authentication, set this value to `null`. @@ -266,8 +246,7 @@ Refer to the [OIS documentation](https://airnode-docs.api3.org/reference/ois/lat ### `apiCredentials` -Refer to Airnode's -[API credentials](https://airnode-docs.api3.org/reference/airnode/latest/deployment-files/config-json.html#apicredentials). +Refer to Airnode's [API credentials](https://airnode-docs.api3.org/reference/airnode/latest/deployment-files/config-json.html#apicredentials). ### `nodeSettings` @@ -275,13 +254,11 @@ Contains general deployment parameters of the Airnode feed. #### `nodeVersion` -The version of the Airnode feed. The version specified in the config must match the version of the Airnode feed at -deployment time. +The version of the Airnode feed. The version specified in the config must match the version of the Airnode feed at deployment time. #### `airnodeWalletMnemonic` -Mnemonic for the airnode wallet used to sign the template responses. It is recommended to interpolate this value from -secrets. For example: +Mnemonic for the airnode wallet used to sign the template responses. It is recommended to interpolate this value from secrets. For example: ```jsonc // The mnemonic is interpolated from the "WALLET_MNEMONIC" secret. @@ -290,6 +267,4 @@ secrets. For example: #### `stage` -An identifier of the deployment stage. This is used to distinguish between different deployments of Airnode feed, for -example `dev`, `staging` or `production`. The stage value can have 256 characters at maximum and can only include -lowercase alphanumeric characters and hyphens. +An identifier of the deployment stage. This is used to distinguish between different deployments of Airnode feed, for example `dev`, `staging` or `production`. The stage value can have 256 characters at maximum and can only include lowercase alphanumeric characters and hyphens. diff --git a/packages/e2e/README.md b/packages/e2e/README.md index 7b0e9ddc..6af11dd2 100644 --- a/packages/e2e/README.md +++ b/packages/e2e/README.md @@ -4,13 +4,9 @@ ## Getting started -1. Copy the Airnode feed secrets. Whilst inside `src` run - `cp airnode-feed/secrets.example.env airnode-feed/secrets.env`. If you are using Docker Desktop, you need to change - the URL in `airnode-feed/secrets.env` from `localhost` to `host.docker.internal`, because Airnode feed is running - inside a Docker container. +1. Copy the Airnode feed secrets. Whilst inside `src` run `cp airnode-feed/secrets.example.env airnode-feed/secrets.env`. If you are using Docker Desktop, you need to change the URL in `airnode-feed/secrets.env` from `localhost` to `host.docker.internal`, because Airnode feed is running inside a Docker container. 2. Copy the Signed API secrets. Whilst inside `src` run `cp signed-api/secrets.example.env signed-api/secrets.env` -3. Build the latest Docker images. Run `pnpm run docker:build` from the monorepo root. The e2e flow uses the docker - images. +3. Build the latest Docker images. Run `pnpm run docker:build` from the monorepo root. The e2e flow uses the docker images. 4. This module contains services (or configurations) that are integrated together. Specifically: - `airnode-feed` - Contains the configuration for the Airnode feed service. - `signed-api` - Contains the configuration for the signed API service. @@ -19,8 +15,7 @@ You are free to modify the configurations to test different scenarios. -5. There are `start:` scripts to start the services. It is recommended to start each service in a separate - terminal and in this order: +5. There are `start:` scripts to start the services. It is recommended to start each service in a separate terminal and in this order: 1. `pnpm run start:data-provider-api` 2. `pnpm run start:signed-api` 3. `pnpm run start:airnode-feed` diff --git a/packages/performance-test/README.md b/packages/performance-test/README.md index a71866e6..b2d7e916 100644 --- a/packages/performance-test/README.md +++ b/packages/performance-test/README.md @@ -2,26 +2,20 @@ > Basic configurations, scripts and instructions to run a performance test on AWS. -Similar performance tests can be run on the host machine, but you may be bottlenecked by the host machine performance or -network bandwidth. Also the AWS side is not tested this way. For these reasons the instructions here are tailored to -AWS. If you want to run the services in a Docker locally, modify the commands accordingly. +Similar performance tests can be run on the host machine, but you may be bottlenecked by the host machine performance or network bandwidth. Also the AWS side is not tested this way. For these reasons the instructions here are tailored to AWS. If you want to run the services in a Docker locally, modify the commands accordingly. ## Overview -The `airnode-feed` and `signed-api` directory contain the configurations for the Airnode feed and Signed API -respectively. The basic instructions to run the performance test: +The `airnode-feed` and `signed-api` directory contain the configurations for the Airnode feed and Signed API respectively. The basic instructions to run the performance test: -1. Deploy a "source" Signed API that is going to be used by Airnode feeds as a data source. This is better than spamming - some existing API with requests. +1. Deploy a "source" Signed API that is going to be used by Airnode feeds as a data source. This is better than spamming some existing API with requests. 2. Initialize the Signed API with sample data. -3. Deploy a "target" Signed API that Airnode feeds are going to push the signed data to. Optionally, use the "source" - also as a "target". +3. Deploy a "target" Signed API that Airnode feeds are going to push the signed data to. Optionally, use the "source" also as a "target". 4. Deploy Airnode feed(s) that fetch data from "source" and push it to "target". ## Deploying/removing CloudFormation template -You can either do so manually via AWS UI or use the CLI. The CLI is more convenient because you have to click less. To -use the CLI: +You can either do so manually via AWS UI or use the CLI. The CLI is more convenient because you have to click less. To use the CLI: 1. Install the AWS CLI. 2. Log into AWS CLI (or just paste the short-term credential ENV variables to the console). @@ -33,27 +27,17 @@ The rest of the instructions assume you are using the CLI and are logged in. To deploy the Signed API, use the `cloudformation.json` file in the `signed-api` directory. Specifically: -1. Replace `` with the URL of the configuration file in the CF template. You can use the GitHub - remote URL of the `signed-api.json` file. When developing you use the file from a different branch and each time you - edit the file push the changes upstream. +1. Replace `` with the URL of the configuration file in the CF template. You can use the GitHub remote URL of the `signed-api.json` file. When developing you use the file from a different branch and each time you edit the file push the changes upstream. 2. Replace the `` with the actual docker image to be used in the CF template. 3. Make sure the template is correct. -4. Consider changing `-perf-test` ID into something more meaningful. This step is necessary if deploying multiple Signed - APIs in the same region. +4. Consider changing `-perf-test` ID into something more meaningful. This step is necessary if deploying multiple Signed APIs in the same region. 5. Optionally change the `CPU` and `Memory`. -6. Run - `aws cloudformation create-stack --stack-name signed-api- --template-body file://signed-api/cloudformation.json --capabilities CAPABILITY_NAMED_IAM --region `. - Be sure to replace `` (e.g. "source") and `` (e.g. "eu-west-1") with the appropriate values. -7. Run `aws cloudformation delete-stack --stack-name signed-api- --region ` to remove the Signed API. - Replace `` and `` with the same values used in the deployment. +6. Run `aws cloudformation create-stack --stack-name signed-api- --template-body file://signed-api/cloudformation.json --capabilities CAPABILITY_NAMED_IAM --region `. Be sure to replace `` (e.g. "source") and `` (e.g. "eu-west-1") with the appropriate values. +7. Run `aws cloudformation delete-stack --stack-name signed-api- --region ` to remove the Signed API. Replace `` and `` with the same values used in the deployment. ## Populating the Signed API -To initialize the Signed API, we run Airnode feed(s) on the host machine. There are multiple ways to do this, one of -them is to use some existing signed API as a data source and run multiple Airnode feeds (with different mnemonic) with a -single trigger and push the signed data to the Signed API. The number of Airnode feeds (X) and the number of templates -(Y) in the trigger can be configured as necessary. The end result is a Signed API with X endpoints, each serving Y -signed data. +To initialize the Signed API, we run Airnode feed(s) on the host machine. There are multiple ways to do this, one of them is to use some existing signed API as a data source and run multiple Airnode feeds (with different mnemonic) with a single trigger and push the signed data to the Signed API. The number of Airnode feeds (X) and the number of templates (Y) in the trigger can be configured as necessary. The end result is a Signed API with X endpoints, each serving Y signed data. ### Create Airnode feed configuration @@ -62,9 +46,7 @@ You can use a script to create an Airnode feed configuration. You need to: 1. Use some existing signed API and set `SOURCE_SIGNED_API_URL` and `SOURCE_SIGNED_API_ENDPOINT_PATH`. 2. Set `SIGNED_DATAS_PER_API_RESPONSE` (e.g. `1` when there is no batching, `100` to simulate API batch of 100). 3. Set `TARGET_SIGNED_API_URL` to the URL of the target Signed API. -4. Leave `TRIGGERS_COUNT` to `1` and `FETCH_INTERVAL` to `86400`. The point is to create just a single trigger and push - the signed data only once. Set `AIRNODE_FEED_CONFIG_PATH` to - `./airnode-feed/initialize-source-api/config/airnode-feed.json`. +4. Leave `TRIGGERS_COUNT` to `1` and `FETCH_INTERVAL` to `86400`. The point is to create just a single trigger and push the signed data only once. Set `AIRNODE_FEED_CONFIG_PATH` to `./airnode-feed/initialize-source-api/config/airnode-feed.json`. For example, to prepare Airnode feed config with the Nodary pool you could use the following: @@ -94,8 +76,7 @@ pnpm run run-script ./airnode-feed/create-config.ts ### Run Airnode feed(s) -To run the Airnode feed(s) you can use the `start-airnode-feeds.ts`. You can specify `TOTAL_AIRNODE_FEEDS` for how many -Airnode feeds should be run. Each Airnode feed will use a different mnemonic. For example: +To run the Airnode feed(s) you can use the `start-airnode-feeds.ts`. You can specify `TOTAL_AIRNODE_FEEDS` for how many Airnode feeds should be run. Each Airnode feed will use a different mnemonic. For example: To run 300 Airnode feeds (simulating API that does not support batching): @@ -120,20 +101,14 @@ To deploy the Airnode feed, use the `cloudformation.json` file in the `airnode-f 3. Replace `` with the configuration URL in the `cloudformation.json` file in the CF template. 4. Replace the `` with the actual docker image to be used in the CF template. 5. Make sure the template is correct. -6. Consider changing `-perf-test` ID into something more meaningful. This step is necessary if deploying multiple - Airnode feeds in the same region. +6. Consider changing `-perf-test` ID into something more meaningful. This step is necessary if deploying multiple Airnode feeds in the same region. 7. Optionally change the `CPU` and `Memory`. -8. Run - `aws cloudformation create-stack --stack-name airnode-feed- --template-body file://airnode-feed/cloudformation.json --capabilities CAPABILITY_NAMED_IAM --region `. - Be sure to replace `` (e.g. "perf-test") and `` (e.g. "eu-west-1") with the appropriate values. -9. Run `aws cloudformation delete-stack --stack-name airnode-feed- --region ` to remove the Signed API. - Replace `` and `` with the same values used in the deployment. +8. Run `aws cloudformation create-stack --stack-name airnode-feed- --template-body file://airnode-feed/cloudformation.json --capabilities CAPABILITY_NAMED_IAM --region `. Be sure to replace `` (e.g. "perf-test") and `` (e.g. "eu-west-1") with the appropriate values. +9. Run `aws cloudformation delete-stack --stack-name airnode-feed- --region ` to remove the Signed API. Replace `` and `` with the same values used in the deployment. ### Create Airnode feed configuration -You can use a script to create an Airnode feed configuration, similarly to the "source" Signed API initialization. The -`SIGNED_DATAS_PER_API_RESPONSE` should remain the same, `TRIGGERS_COUNT` should equal to `TOTAL_AIRNODE_FEEDS`, and -`FETCH_INTERVAL` can be set arbitrarily. +You can use a script to create an Airnode feed configuration, similarly to the "source" Signed API initialization. The `SIGNED_DATAS_PER_API_RESPONSE` should remain the same, `TRIGGERS_COUNT` should equal to `TOTAL_AIRNODE_FEEDS`, and `FETCH_INTERVAL` can be set arbitrarily. For example, to create an Airnode feed configuration for no batching case: diff --git a/packages/signed-api/README.md b/packages/signed-api/README.md index d5bfeb85..15ceb27f 100644 --- a/packages/signed-api/README.md +++ b/packages/signed-api/README.md @@ -2,8 +2,7 @@ > A service for storing and accessing signed data. It provides endpoints to handle signed data for a specific airnode. -Signed API is a Node.js API server, dockerized and deployable on any cloud provider or hostable on premise. It stores -the data in memory and provides endpoints to push and retrieve beacon data. +Signed API is a Node.js API server, dockerized and deployable on any cloud provider or hostable on premise. It stores the data in memory and provides endpoints to push and retrieve beacon data. ## Configuration @@ -11,15 +10,13 @@ See [configuration](./config/configuration.md) for details. ## Local development -1. `cp config/signed-api.example.json config/signed-api.json` - To create a config file from the example one. Optionally - change the defaults. +1. `cp config/signed-api.example.json config/signed-api.json` - To create a config file from the example one. Optionally change the defaults. 2. `cp .env.example .env` - To copy the example environment variables. Optionally change the defaults. 3. `pnpm run dev` - To start the API server. The port number can be configured by `SERVER_PORT` environment variable. ### Docker instructions -You can use shorthands from package.json. To understand how the docker image is built, read the -[Dockerfile](../../Dockerfile). +You can use shorthands from package.json. To understand how the docker image is built, read the [Dockerfile](../../Dockerfile). ```sh pnpm run docker:build @@ -43,26 +40,19 @@ LOGGER_ENABLED=true pnpm run test The API provides the following endpoints: - `POST /{airnode}`: Insert a batch of signed data. - - The batch is validated for consistency and data integrity errors. If there is any issue during this step, the whole - batch is rejected. Otherwise the batch is accepted. Also, all data that is no longer needed is removed during this - step. + - The batch is validated for consistency and data integrity errors. If there is any issue during this step, the whole batch is rejected. Otherwise the batch is accepted. Also, all data that is no longer needed is removed during this step. - `GET /{endpoint-name}/{airnode}`: Retrieve signed data for the Airnode respecting the endpoint configuration. - Returns the freshest signed data available for the given Airnode, respecting the configured endpoint delay. - `GET /airnodes`: Retrieve list of all available Airnode address. - - Returns all Airnode addresses for which there is signed data. It is possible that this data cannot be shown by the - delayed endpoints (in case the data is too fresh and there is not an older alternative). + - Returns all Airnode addresses for which there is signed data. It is possible that this data cannot be shown by the delayed endpoints (in case the data is too fresh and there is not an older alternative). - `GET /`: Retrieve system status information. - - Returns current system configuration details including deployment stage, version, current timestamp, deployment - timestamp, configuration hash, and certified Airnode addresses. + - Returns current system configuration details including deployment stage, version, current timestamp, deployment timestamp, configuration hash, and certified Airnode addresses. ## Deployment -To deploy signed API on AWS you can use a CloudFormation template in the `deployment` folder. You need to specify the -docker image of the signed API and the URL of the signed API configuration which will be download when the service is -started. +To deploy signed API on AWS you can use a CloudFormation template in the `deployment` folder. You need to specify the docker image of the signed API and the URL of the signed API configuration which will be download when the service is started. -The template will create all necessary AWS resources and assign a domain name to access the the API. You can get the URL -from the output parameters of the CloudFormation stack or by checking the DNS record of the load balancer. +The template will create all necessary AWS resources and assign a domain name to access the the API. You can get the URL from the output parameters of the CloudFormation stack or by checking the DNS record of the load balancer. To deploy on premise you can use the Docker instructions below. @@ -72,11 +62,9 @@ The API is also dockerized. To run the dockerized APi, you need to: -1. Publish the port of the API to the host machine. The port number of signed API in the container is set to `80`. So - the command should look like `--publish :80`. +1. Publish the port of the API to the host machine. The port number of signed API in the container is set to `80`. So the command should look like `--publish :80`. 2. Mount config folder to `/app/config`. The folder should contain the `signed-api.json` file. -3. Pass the `-it --init` flags to the docker run command. This is needed to ensure the docker is stopped gracefully. See - [this](https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals) for details. +3. Pass the `-it --init` flags to the docker run command. This is needed to ensure the docker is stopped gracefully. See [this](https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals) for details. 4. Specify the `--env-file` with the path to the `.env` file containing the [ENV configuration](#environment-variables). 5. Optionally, pass the `--rm` flag to remove the container after it is stopped. diff --git a/packages/signed-api/config/configuration.md b/packages/signed-api/config/configuration.md index ee7db9f5..c1791806 100644 --- a/packages/signed-api/config/configuration.md +++ b/packages/signed-api/config/configuration.md @@ -1,12 +1,10 @@ # Configuration -The API is configured via combination of [environment variables](#environment-variables) and -[configuration files](#configuration-files). +The API is configured via combination of [environment variables](#environment-variables) and [configuration files](#configuration-files). ## Environment variables -Parts of the API needs to be initialized prior the configuration files are loaded. This is done via environment -variables. All of the environment variables are optional and or set with default values for convenience. +Parts of the API needs to be initialized prior the configuration files are loaded. This is done via environment variables. All of the environment variables are optional and or set with default values for convenience. Example: @@ -43,16 +41,13 @@ Default: `true`. The format of the log output. Options: - `json` - Specifies JSON log format. This is suitable when running in production and streaming logs to other services. -- `pretty` - Logs are formatted in a human-friendly "pretty" way. Ideal, when running the service locally and in - development. +- `pretty` - Logs are formatted in a human-friendly "pretty" way. Ideal, when running the service locally and in development. Default: `json`. ### `LOG_API_DATA` _(optional)_ -Enables or disables logging of the API data at the `info` level. When set to `true`, received valid signed data will be -logged with the fields `airnode`, `encodedValue`, `templateId`, and `timestamp`. The `signature` field is intentionally -excluded for security reasons. Options: +Enables or disables logging of the API data at the `info` level. When set to `true`, received valid signed data will be logged with the fields `airnode`, `encodedValue`, `templateId`, and `timestamp`. The `signature` field is intentionally excluded for security reasons. Options: - `true` - Enables logging of the API data. - `false` - Disables logging of the API data. @@ -63,8 +58,7 @@ Default: `false`. Enables or disables colors in the log output. Options: -- `true` - Enables colors in the log output. The output has special color setting characters that are parseable by CLI. - Recommended when running locally and in development. +- `true` - Enables colors in the log output. The output has special color setting characters that are parseable by CLI. Recommended when running locally and in development. - `false` - Disables colors in the log output. Recommended for production. Default: `false`. @@ -111,9 +105,7 @@ The path to the configuration file in the AWS S3 bucket. Required when `CONFIG_S ## Configuration files -The API is configured via combination of `signed-api.json` and `secrets.env` configuration files. All expressions of a -form `${SECRET_NAME}` are referring to values from secrets and are interpolated inside the `signed-api.json` at runtime. -You are advised to put sensitive information inside secrets file. +The API is configured via combination of `signed-api.json` and `secrets.env` configuration files. All expressions of a form `${SECRET_NAME}` are referring to values from secrets and are interpolated inside the `signed-api.json` at runtime. You are advised to put sensitive information inside secrets file. ### `endpoints` @@ -160,8 +152,7 @@ Configuration for one of the endpoints. ##### `urlPath` -The URL path on which the endpoint is served. Must start with a slash and contain only alphanumeric characters and -dashes. +The URL path on which the endpoint is served. Must start with a slash and contain only alphanumeric characters and dashes. ##### `delaySeconds` @@ -169,21 +160,17 @@ The delay in seconds for the endpoint. The endpoint will only serve data that is ##### `authTokens` -The nonempty list of -[Bearer authentication tokens](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#bearer) allowed to query -the data. +The nonempty list of [Bearer authentication tokens](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#bearer) allowed to query the data. In case the endpoint should be publicly available, set the value to `null`. ##### `hideSignatures` _(optional)_ -The boolean flag to hide the signatures in the response. The flag is optional for backwards compatibility with older -Signed API versions. +The boolean flag to hide the signatures in the response. The flag is optional for backwards compatibility with older Signed API versions. ##### `isOev` -The boolean flag to indicate if the endpoint serves the data for OEV beacons. This data can be used to update the API3 -proxies with the latest OEV data, but cannot be used to update the main API3 base feeds. +The boolean flag to indicate if the endpoint serves the data for OEV beacons. This data can be used to update the API3 proxies with the latest OEV data, but cannot be used to update the main API3 base feeds. ### `cache` _(optional)_ @@ -196,8 +183,7 @@ Defaults to no cache (no headers). The type of the cache. Options: - `browser` - Sets the standard `cache-control: max-age=XYZ` header. -- `cdn` - Similar to `browser` cache, but also sets the `cdn-cache-control` to the same value as `cache-control`. - Setting both headers is necessary to support CDNs which do not support `cdn-cache-control` header. +- `cdn` - Similar to `browser` cache, but also sets the `cdn-cache-control` to the same value as `cache-control`. Setting both headers is necessary to support CDNs which do not support `cdn-cache-control` header. #### `maxAgeSeconds` @@ -205,8 +191,7 @@ The maximum age of the cache in seconds. The cache is cleared after this time. ### `allowedAirnodes` -The list of allowed Airnodes with authorization details. If the list is empty, no Airnode is allowed. To whitelist all -Airnodes, set the value to `"*"` instead of an array. +The list of allowed Airnodes with authorization details. If the list is empty, no Airnode is allowed. To whitelist all Airnodes, set the value to `"*"` instead of an array. Example: @@ -239,24 +224,17 @@ The address of the Airnode. The address must be a valid Ethereum address. ##### `authTokens` -The nonempty list of -[Bearer authentication tokens](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#bearer). +The nonempty list of [Bearer authentication tokens](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#bearer). -To allow pushing data without any authorization, set the value to `null`. The API validates the data, but this is not -recommended. +To allow pushing data without any authorization, set the value to `null`. The API validates the data, but this is not recommended. ##### `isCertified` -A boolean flag indicating whether the Airnode is first-party verified. When set to true, the Airnode address will be -included in the `/` endpoint response under `certifiedAirnodes`. This flag is set by the deployer of the Signed API -instance and serves as a declaration of ownership or control over the listed Airnodes. However, as the deployer has -complete control over setting this flag, it does not constitute a cryptographic proof of ownership. +A boolean flag indicating whether the Airnode is first-party verified. When set to true, the Airnode address will be included in the `/` endpoint response under `certifiedAirnodes`. This flag is set by the deployer of the Signed API instance and serves as a declaration of ownership or control over the listed Airnodes. However, as the deployer has complete control over setting this flag, it does not constitute a cryptographic proof of ownership. #### `stage` -An identifier of the deployment stage. This is used to distinguish between different deployments of Signed API, for -example `dev`, `staging` or `production`. The stage value can have 256 characters at maximum and can only include -lowercase alphanumeric characters and hyphens. +An identifier of the deployment stage. This is used to distinguish between different deployments of Signed API, for example `dev`, `staging` or `production`. The stage value can have 256 characters at maximum and can only include lowercase alphanumeric characters and hyphens. #### `version`