Log shipping via fluentd to external processors

Architecture

Currently fluentd logging is realised via separate container that shares filesystem with app (vue-storefront).

That means that fluentd won't collect logs from stdout/stderr because they are not shared. So in order to make possible to fluentd to process application logs app needs to be configured to log to file.

Prerequisites

Application logging

Application (both the Alokai Frontend and the Middleware) needs to log to file(s) in order for fluentd to pick up its logs. Files should be located at /var/www/logs/*.log Example of how app logger (e.g. Winston) should be configured can be found at NewRelic configuration docs

Enabling via API

fluentd logging is not directly controlled. Instead you should enable one of log processors like Azure Log Analytics, DataDog or Loki and fluentd container will be set up automatically for them.

Azure Log Analytics

echo '
{
  "azure_log_analytics": {
    "enabled": true,
    "log_type": "ApacheAccessLog",
    "shared_key": "REDACTED",
    "customer_id": "REDACTED",
    "add_time_field": true,
    "add_tag_field": true,
    "time_field_name": "time",
    "local_time": true,
    "tag_field_name": "vsf"
  }
}' | curl -s https://farmer.vuestorefront.cloud/instance/my-instance -X PATCH -H 'X-User-Id: xxxx' -H 'X-Api-Key: yyyy' --json @-

Loki

Example request to enable Loki logging

echo '
{
  "loki": {
    "enabled": true,
    "username": "",
    "password": "",
    "extra_labels": {"some": "label"},
    "url": "https://logs-prod-us-central1.grafana.net",
  }
}' | curl -s https://farmer.vuestorefront.cloud/instance/my-instance -X PATCH -H 'X-User-Id: xxxx' -H 'X-Api-Key: yyyy' --json @-

Datadog

Example request to enable Datadog logging

echo '
{
  "datadog": {
    "logs": {
      "enabled": true
    },
    "api_key": "",
    "env_var": [],
  }
}' | curl -s https://farmer.vuestorefront.cloud/instance/my-instance -X PATCH -H 'X-User-Id: xxxx' -H 'X-Api-Key: yyyy' --json @-

For more details go to Datadog integration page.