axiomhq/axiom-lambda-extension

Ingest logs and platform events from your lambda functions

Axiom unlocks observability at any scale.

• Ingest with ease, store without limits: Axiom’s next-generation datastore enables ingesting petabytes of data with ultimate efficiency. Ship logs from Kubernetes, AWS, Azure, Google Cloud, DigitalOcean, Nomad, and others.
• Query everything, all the time: Whether DevOps, SecOps, or EverythingOps, query all your data no matter its age. No provisioning, no moving data from cold/archive to “hot”, and no worrying about slow queries. All your data, all. the. time.
• Powerful dashboards, for continuous observability: Build dashboards to collect related queries and present information that’s quick and easy to digest for you and your team. Dashboards can be kept private or shared with others, and are the perfect way to bring together data from different sources.

For more information, check out the official documentation and our community Discord.

---

The axiom-lambda-extension can send logs and platform events of your Lambda function to Axiom. Axiom will detect the extension and provide you with quick filters and a dashboard.

With the Axiom Lambda extension, you can forget about the extra configuration of CloudWatch and subscription filters.

Note: After the Axiom Lambda extension is installed, the Lambda service will still send logs to CloudWatch Logs.

To disable the CloudWatch logging, follow the steps below:

• Install the Axiom Lambda extension;
• Make sure everything is working properly in Axiom;
• Disable the permissions for CloudWatch.

For more detail on how to disable the CloudWatch logging, see the Axiom documentation.

Quickstart

1. Set these environment variables on your function:

   • AXIOM_DATASET: The dataset name to send logs to. Learn more about creating a dataset here
   • AXIOM_TOKEN: The Axiom API token (needs ingest permission into the dataset above). Learn more about creating tokens here

note the extensions will not work without correct credentials, but it will not crash your function. If you want it to crash for testing purposes check the Troubleshooting section below.

2. Add the extension as a layer with the AWS CLI:

$ aws lambda update-function-configuration --function-name my-function \
    --layers arn:aws:lambda:<AWS_REGION>:694952825951:layer:axiom-extension-<ARCH>:<VERSION>

• Use the latest version number specified on the Releases page for the VERSION parameter. For example, 4.
• For more detail on AWS_REGION and ARCH parameters, expand the table below:

All Lambda Layers

Region	arm64	x86_64
us-west-1	arn:aws:lambda:us-west-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:us-west-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
us-west-2	arn:aws:lambda:us-west-2:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:us-west-2:694952825951:layer:axiom-extension-x86_64:<VERSION>
us-east-1	arn:aws:lambda:us-east-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:us-east-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
us-east-2	arn:aws:lambda:us-east-2:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:us-east-2:694952825951:layer:axiom-extension-x86_64:<VERSION>
eu-west-1	arn:aws:lambda:eu-west-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:eu-west-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
eu-west-2	arn:aws:lambda:eu-west-2:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:eu-west-2:694952825951:layer:axiom-extension-x86_64:<VERSION>
eu-west-3	arn:aws:lambda:eu-west-3:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:eu-west-3:694952825951:layer:axiom-extension-x86_64:<VERSION>
eu-north-1	arn:aws:lambda:eu-north-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:eu-north-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
eu-central-1	arn:aws:lambda:eu-central-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:eu-central-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
ca-central-1	arn:aws:lambda:ca-central-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ca-central-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
sa-east-1	arn:aws:lambda:sa-east-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:sa-east-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-south-1	arn:aws:lambda:ap-south-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-south-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-southeast-1	arn:aws:lambda:ap-southeast-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-southeast-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-southeast-2	arn:aws:lambda:ap-southeast-2:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-southeast-2:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-northeast-1	arn:aws:lambda:ap-northeast-1:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-northeast-1:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-northeast-2	arn:aws:lambda:ap-northeast-2:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-northeast-2:694952825951:layer:axiom-extension-x86_64:<VERSION>
ap-northeast-3	arn:aws:lambda:ap-northeast-3:694952825951:layer:axiom-extension-arm64:<VERSION>	arn:aws:lambda:ap-northeast-3:694952825951:layer:axiom-extension-x86_64:<VERSION>

Terraform Example

You can also use Terraform to hook up your Lambda with Axiom Lambda layer in two different ways:

1. Using plain Terraform code:

resource "aws_lambda_function" "test_lambda" {
  filename      = "lambda_function_payload.zip"
  function_name = "lambda_function_name"
  role          = aws_iam_role.iam_for_lambda.arn
  handler       = "index.test"
  runtime       = "nodejs14.x"

  ephemeral_storage {
    size = 10240 # Min 512 MB and the Max 10240 MB
  }

  environment {
    variables = {
      AXIOM_TOKEN   = "axiom-token"
      AXIOM_DATASET = "axiom-dataset"
    }
  }

  layers = [
    "arn:aws:lambda:<AWS_REGION>:694952825951:layer:axiom-extension-<ARCH>:<VERSION>"
  ]
}

2. Using AWS lambda module:

module "lambda_function" {
  source = "terraform-aws-modules/lambda/aws"

  function_name = "my-lambda1"
  description   = "My awesome lambda function"
  handler       = "index.lambda_handler"
  runtime       = "python3.8"

  source_path = "../src/lambda-function1"

  layers = [
    "arn:aws:lambda:<AWS_REGION>:694952825951:layer:axiom-extension-<ARCH>:<VERSION>"
  ]

  environment_variables = {
    AXIOM_TOKEN   = "axiom-token"
    AXIOM_DATASET = "axiom-dataset"
  }
}

Troubleshooting

Double check that the API token has permission to ingest data into the dataset. If that is not the issue, please check the function logs on the AWS console, the extension will log any errors with setup or ingest.

For testing purposes you can also set the PANIC_ON_API_ERR environment variable to true to tell the extension to crash if couldn't connect to Axiom.

License

© Axiom, Inc., 2023

Distributed under MIT License (The MIT License).

axiom-lambda-extension's Issues

use a config struct instead of a raw global in main.go

          also, this is separate but how do we feel about using a struct instead of this global variable?

like, instead of this:

var (
	runtimeAPI        = os.Getenv("AWS_LAMBDA_RUNTIME_API")
	extensionName     = filepath.Base(os.Args[0])
	isFirstInvocation = true
	runtimeDone       = make(chan struct{})

	// API Port
	logsPort = "8080"

	// Buffering Config
	defaultMaxItems  = 1000
	defaultMaxBytes  = 262144
	defaultTimeoutMS = 1000

	developmentMode = false
	logger          *zap.Logger
)

something more like:

type Config struct {
    RuntimeAPI        string
    ExtensionName     string
    IsFirstInvocation bool
    LogsPort          string
    MaxItems          int
    MaxBytes          int
    TimeoutMS         int
    DevelopmentMode   bool
    Logger            *zap.Logger
    RuntimeDone       chan struct{}
}

var cfg = Config{
    RuntimeAPI:        os.Getenv("AWS_LAMBDA_RUNTIME_API"),
    ExtensionName:     filepath.Base(os.Args[0]),
    IsFirstInvocation: true,
    LogsPort:          "8080",
    MaxItems:          1000,
    MaxBytes:          262144,
    TimeoutMS:         1000,
    DevelopmentMode:   false,
    RuntimeDone:       make(chan struct{}),
}

and then used like

cfg.Logger.Error('ex')

Originally posted by @flbn in #17 (comment)

Serverless and axiom-lambda-extension

Hi there,

I'm trying to use your plugin with the serverless framework.
Basically I'm doing something like:

service: app

frameworkVersion: '3'
useDotenv: true

plugins:
  - serverless-plugin-typescript
  - serverless-plugin-optimize
  - serverless-extra-parameters
  - serverless-offline

package:
  excludeDevDependencies: true
  individually: true
  patterns:
    - '!**/node_modules/@aws-sdk/**'

provider:
  name: aws
  runtime: nodejs18.x
  architecture: arm64
  region: ${opt:region, 'eu-central-1'}
  stage: ${opt:stage, 'dev'}
  stackName: dev-app
  layers:
    axiom:
      path: arn:aws:lambda:${self:provider.region}:694952825951:layer:axiom-extension-arm64:4
      name: axiom-layer
      description: Axiom unlocks observability at any scale.
  environment:
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: 1
    AWS_SDK_JS_SUPPRESS_MAINTENANCE_MODE_MESSAGE: 1
    LOG_LEVEL: debug
    NODE_ENV: production
    NODE_OPTIONS: '--enable-source-maps --stack-trace-limit=1000'
    AXIOM_TOKEN: ${ssm:/${self:provider.stage}/AXIOM_TOKEN}
    AXIOM_DATASET: 'app'

functions:
  api:
    handler: ./src/events/api.handler
    description: 'REST API / GraphQL handler'
    events:
      - httpApi:
          path: /{proxy+}
          method: ANY

But unfortunately when I'm trying to deploy my lambda (using sls deploy command) I'm getting error:

Error:
No file matches include / exclude patterns

Any idea what's wrong?