apache/apisix

The Cloud-Native API Gateway

Apache APISIX API Gateway

Apache APISIX is a dynamic, real-time, high-performance API Gateway.

APISIX API Gateway provides rich traffic management features such as load balancing, dynamic upstream, canary release, circuit breaking, authentication, observability, and more.

You can use APISIX API Gateway to handle traditional north-south traffic, as well as east-west traffic between services. It can also be used as a k8s ingress controller.

The technical architecture of Apache APISIX:

Community

• Mailing List: Mail to [email protected], follow the reply to subscribe to the mailing list.
• Slack Workspace - invitation link (Please open an issue if this link is expired), and then join the #apisix channel (Channels -> Browse channels -> search for "apisix").
• - follow and interact with us using hashtag #ApacheAPISIX
• Documentation
• Discussions
• Blog

Features

You can use APISIX API Gateway as a traffic entrance to process all business data, including dynamic routing, dynamic upstream, dynamic certificates, A/B testing, canary release, blue-green deployment, limit rate, defense against malicious attacks, metrics, monitoring alarms, service observability, service governance, etc.

• All platforms

  • Cloud-Native: Platform agnostic, No vendor lock-in, APISIX API Gateway can run from bare-metal to Kubernetes.
  • Supports ARM64: Don't worry about the lock-in of the infra technology.

• Multi protocols

  • TCP/UDP Proxy: Dynamic TCP/UDP proxy.
  • Dubbo Proxy: Dynamic HTTP to Dubbo proxy.
  • Dynamic MQTT Proxy: Supports to load balance MQTT by client_id, both support MQTT 3.1.*, 5.0.
  • gRPC proxy: Proxying gRPC traffic.
  • gRPC Web Proxy: Proxying gRPC Web traffic to gRPC Service.
  • gRPC transcoding: Supports protocol transcoding so that clients can access your gRPC API by using HTTP/JSON.
  • Proxy Websocket
  • Proxy Protocol
  • HTTP(S) Forward Proxy
  • SSL: Dynamically load an SSL certificate.

• Full Dynamic

  • Hot Updates And Hot Plugins: Continuously updates its configurations and plugins without restarts!
  • Proxy Rewrite: Support rewrite the host, uri, schema, method, headers of the request before send to upstream.
  • Response Rewrite: Set customized response status code, body and header to the client.
  • Dynamic Load Balancing: Round-robin load balancing with weight.
  • Hash-based Load Balancing: Load balance with consistent hashing sessions.
  • Health Checks: Enable health check on the upstream node and will automatically filter unhealthy nodes during load balancing to ensure system stability.
  • Circuit-Breaker: Intelligent tracking of unhealthy upstream services.
  • Proxy Mirror: Provides the ability to mirror client requests.
  • Traffic Split: Allows users to incrementally direct percentages of traffic between various upstreams.

• Fine-grained routing

  • Supports full path matching and prefix matching
  • Support all Nginx built-in variables as conditions for routing, so you can use cookie, args, etc. as routing conditions to implement canary release, A/B testing, etc.
  • Support various operators as judgment conditions for routing, for example {"arg_age", ">", 24}
  • Support custom route matching function
  • IPv6: Use IPv6 to match the route.
  • Support TTL
  • Support priority
  • Support Batch Http Requests
  • Support filtering route by GraphQL attributes

• Security

  • Rich authentication & authorization support:
    • key-auth
    • JWT
    • basic-auth
    • wolf-rbac
    • casbin
    • keycloak
    • casdoor
  • IP Whitelist/Blacklist
  • Referer Whitelist/Blacklist
  • IdP: Support external Identity platforms, such as Auth0, okta, etc..
  • Limit-req
  • Limit-count
  • Limit-concurrency
  • Anti-ReDoS(Regular expression Denial of Service): Built-in policies to Anti ReDoS without configuration.
  • CORS Enable CORS(Cross-origin resource sharing) for your API.
  • URI Blocker: Block client request by URI.
  • Request Validator
  • CSRF Based on the Double Submit Cookie way, protect your API from CSRF attacks.

• OPS friendly

  • Zipkin tracing: Zipkin
  • Open source APM: support Apache SkyWalking
  • Works with external service discovery: In addition to the built-in etcd, it also supports Consul, Consul_kv, Nacos, Eureka and Zookeeper (CP).
  • Monitoring And Metrics: Prometheus
  • Clustering: APISIX nodes are stateless, creates clustering of the configuration center, please refer to etcd Clustering Guide.
  • High availability: Support to configure multiple etcd addresses in the same cluster.
  • Dashboard
  • Version Control: Supports rollbacks of operations.
  • CLI: start\stop\reload APISIX through the command line.
  • Standalone: Supports to load route rules from local YAML file, it is more friendly such as under the kubernetes(k8s).
  • Global Rule: Allows to run any plugin for all request, eg: limit rate, IP filter etc.
  • High performance: The single-core QPS reaches 18k with an average delay of fewer than 0.2 milliseconds.
  • Fault Injection
  • REST Admin API: Using the REST Admin API to control Apache APISIX, which only allows 127.0.0.1 access by default, you can modify the allow_admin field in conf/config.yaml to specify a list of IPs that are allowed to call the Admin API. Also, note that the Admin API uses key auth to verify the identity of the caller. The admin_key field in conf/config.yaml needs to be modified before deployment to ensure security.
  • External Loggers: Export access logs to external log management tools. (HTTP Logger, TCP Logger, Kafka Logger, UDP Logger, RocketMQ Logger, SkyWalking Logger, Alibaba Cloud Logging(SLS), Google Cloud Logging, Splunk HEC Logging, File Logger, SolarWinds Loggly Logging, TencentCloud CLS).
  • ClickHouse: push logs to ClickHouse.
  • Elasticsearch: push logs to Elasticsearch.
  • Datadog: push custom metrics to the DogStatsD server, comes bundled with Datadog agent, over the UDP protocol. DogStatsD basically is an implementation of StatsD protocol which collects the custom metrics for Apache APISIX agent, aggregates it into a single data point and sends it to the configured Datadog server.
  • Helm charts
  • HashiCorp Vault: Support secret management solution for accessing secrets from Vault secure storage backed in a low trust environment. Currently, RS256 keys (public-private key pairs) or secret keys can be linked from vault in jwt-auth authentication plugin using APISIX Secret resource.

• Highly scalable

  • Custom plugins: Allows hooking of common phases, such as rewrite, access, header filter, body filter and log, also allows to hook the balancer stage.
  • Plugin can be written in Java/Go/Python
  • Plugin can be written with Proxy Wasm SDK
  • Custom load balancing algorithms: You can use custom load balancing algorithms during the balancer phase.
  • Custom routing: Support users to implement routing algorithms themselves.

• Multi-Language support

  • Apache APISIX is a multi-language gateway for plugin development and provides support via RPC and Wasm.
  • The RPC way, is the current way. Developers can choose the language according to their needs and after starting an independent process with the RPC, it exchanges data with APISIX through local RPC communication. Till this moment, APISIX has support for Java, Golang, Python and Node.js.
  • The Wasm or WebAssembly, is an experimental way. APISIX can load and run Wasm bytecode via APISIX wasm plugin written with the Proxy Wasm SDK. Developers only need to write the code according to the SDK and then compile it into a Wasm bytecode that runs on Wasm VM with APISIX.

• Serverless

  • Lua functions: Invoke functions in each phase in APISIX.
  • AWS Lambda: Integration with AWS Lambda function as a dynamic upstream to proxy all requests for a particular URI to the AWS API gateway endpoint. Supports authorization via api key and AWS IAM access secret.
  • Azure Functions: Seamless integration with Azure Serverless Function as a dynamic upstream to proxy all requests for a particular URI to the Microsoft Azure cloud.
  • Apache OpenWhisk: Seamless integration with Apache OpenWhisk as a dynamic upstream to proxy all requests for a particular URI to your own OpenWhisk cluster.

Get Started

1. Installation

   Please refer to install documentation.

2. Getting started

   The getting started guide is a great way to learn the basics of APISIX. Just follow the steps in Getting Started.

   Further, you can follow the documentation to try more plugins.

3. Admin API

   Apache APISIX provides REST Admin API to dynamically control the Apache APISIX cluster.

4. Plugin development

   You can refer to plugin development guide, and sample plugin example-plugin's code implementation. Reading plugin concept would help you learn more about the plugin.

For more documents, please refer to Apache APISIX Documentation site

Benchmark

Using AWS's eight-core server, APISIX's QPS reaches 140,000 with a latency of only 0.2 ms.

Benchmark script has been open sourced, welcome to try and contribute.

APISIX also works perfectly in AWS graviton3 C7g.

Contributor Over Time

visit here to generate Contributor Over Time.

User Stories

• European eFactory Platform: API Security Gateway – Using APISIX in the eFactory Platform
• Copernicus Reference System Software
• More Stories

Who Uses APISIX API Gateway?

A wide variety of companies and organizations use APISIX API Gateway for research, production and commercial product, below are some of them:

• Airwallex
• Bilibili
• CVTE
• European eFactory Platform
• European Copernicus Reference System
• Geely
• HONOR
• Horizon Robotics
• iQIYI
• Lenovo
• NASA JPL
• Nayuki
• OPPO
• QingCloud
• Swisscom
• Tencent Game
• Travelsky
• vivo
• Sina Weibo
• WeCity
• WPS
• XPENG
• Zoom

Landscape

  

APISIX enriches the CNCF API Gateway Landscape.

Logos

• Apache APISIX logo(PNG)
• Apache APISIX logo source

Acknowledgments

Inspired by Kong and Orange.

License

Apache 2.0 License

apisix's Issues

feature: support service mechanism

a service maybe contain one or more routes.

{
	"id": "1",
	"name": "service demo",
	"plugin_conf": {
		......
	}
}

feature: put all of the Lua code and libraries to apisix home folder.

$ tree -L 1  apisix
apisix
├── apisix-0.5-0.rockspec
├── bin
├── CODE_OF_CONDUCT.md
├── conf
├── COPYRIGHT
├── deps            # dependent Lua files and libraries
├── doc
├── go
├── LICENSE
├── logs
├── lua
├── Makefile
├── README_CN.md
├── README.md
├── t
└── utils

feature: need a easy way to fetch the current status.

it can be a json response.

feature: support talking to etcd via TLS.

add user `apisix` instead of nobody

we should add the user apisix in CLI, rpm, luarocks, docker etc.

not found stub_status

the rpm package enable stub_status https://github.com/openresty/openresty-packaging/blob/master/rpm/SPECS/openresty.spec#L158 by default, however, the configure of source disable it: https://github.com/openresty/openresty/blob/master/util/configure#L67

so we need to packaging openresty into APISIX to avoid this problem.

report by Stone from QQ group

plugin: rate limit

implemented:

limit-req
limit-count

waiting todo:

• limit-conn
• limit-traffic

change: all of the config should be stored under `/apisix/`

feature: support ARM platform

bug: failed to fetch data from etcd

there is no data under /services and /upstreams.

2019/05/31 04:03:54 [error] 5370#5370: *875 [lua] config_etcd.lua:194: failed to fetch data from etcd: /usr/share/lua/5.1/resty/etcd.lua:240: bad argument #1 to 'ipairs' (table expected, got nil), etcd key: /services, context: ngx.timer
2019/05/31 04:03:59 [error] 5370#5370: *879 [lua] config_etcd.lua:194: failed to fetch data from etcd: /usr/share/lua/5.1/resty/etcd.lua:240: bad argument #1 to 'ipairs' (table expected, got nil), etcd key: /upstreams, context: ngx.timer

doc: need Chinese version, support to load certificate for HTTPS request.

https://github.com/iresty/apisix/blob/master/doc/https.md

there is English doc, we need a Chinese version too.

add test cases for plugins.

• round-robin balance

• consistent balance

• prometheus

• key-auth

• limit-conn

• limit-req

• limit-count

feature: schema check for consumer configuration

add consumer, route, service ID as key for limit-rate, limit-count, limit-conn

now we only support IP as key, which is not enough

API schema validation

the goal:

• code simplicity, we may add new feature to it
• open source and good performance
• pure Lua if possible

how about this:

-- JSON schema validation
https://github.com/jdesgats/ljsonschema
https://github.com/epoberezkin/ajv

Combine openresty source code compilation steps？

feature: support to match route with wildcard host.

centos 7: install shell

sudo yum install yum-utils
sudo yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo

sudo yum install git automake autoconf gcc pcre-devel libtool gcc+ gcc-c++ \
    luarocks etcd openresty -y

sudo service etcd start

luarocks install apisix

curl http://127.0.0.1:2379/v2/keys/apisix/routes -X PUT -d dir=true
curl http://127.0.0.1:2379/v2/keys/apisix/upstreams -X PUT -d dir=true
curl http://127.0.0.1:2379/v2/keys/apisix/services -X PUT -d dir=true

curl http://127.0.0.1:2379/v2/keys/apisix/routes/1 -X PUT -d value='
{
	"methods": ["GET"],
	"uri": "/index.html",
	"id": 1,
	"plugin_config": {
		"limit-count": {
			"count": 2,
			"time_window": 60,
			"rejected_code": 503,
			"key": "remote_addr"
		}
	},
	"upstream": {
		"type": "roundrobin",
		"nodes": {
			"39.97.63.215:80": 1
		}
	}
}'

sudo apisix init
sudo apisix start

$ curl -i http://127.0.0.1:9080/index.html

plugin.lua crash

I think, should not be caused by fetching things from an empty table, whatevery the etcd config bad or right

bug: maybe missing `with-http_stub_status_module` module in openresty

should show an error message if the local openresty is not compiled with with-http_stub_status_module module.

support dubbo with tengine's module

Dubbo has its own transport protocol and needs to be done on the tcp protocol if it is to be supported.

Please consider using `lua-resty-master` framework

lua-resty-mater is a framework for OpenResty development. Please check out my presentation https://github.com/openresty-fan/2017-12-23-openresty-meetup-presentation, if the API gateway is complicated, which might light you up.

move docs of plugins from `/lua/apisix/plugins` to `/doc`

feature: dynamic load balancing

feature: fetch the config from etcd.

feature: schema check for upstream configuration

related issue: #82

implement plugin: key-auth

Roadmap of Version 0.6

We are collecting new features for apisix version 0.6. If you have any suggestions, please leave a message here.

• support HTTPS: #152
• support for custom certificates for HTTPS: #152
• add one more port for admin API: #194
• health check and circuit-breaker
• dashboard: #170
• opentracing, implemnt Zipkin first: #204

add schema validation for plugins

for example: https://github.com/iresty/apisix/blob/master/lua/apisix/plugins/example-plugin.lua#L7

能否支持zookeeper来做配置分发，流量分发

我也大量看过lua 和zk的资料，目前没有很好地方案让lua支持zk，目前我们的服务是通过zk注册，不太好维护2套发现服务

rename the project to `apisix`

plugin: support for consistent hash algorithm for dynamic load balancing

https://github.com/iresty/apisix/blob/master/lua/apisix/balancer.lua#L29

test: test in travis.

plugin: support consumer.

feature: schema check for plugin configuration

balancer: support domain.

feature: support WebAssembly in apisix.

在 apisix 的生态圈中，目前主要是通过 Lua 来完成的，但由于 Lua 语言自身比较弱以及生态比较匮乏，是否可以通过其他语言来弥补短板，就非常有意义了。

貌似 WebAssembly 比较符合目前需求，它已经支持目前比较流行的语言主要有 C/C++、JavaScript 和 Golang 等，为这些语言提供了新的运行机制。

另外对于 serverless 以及边缘计算场景，更需要多语言的支持。

In the apisix ecosystem, it is mainly done by Lua, but because the Lua language itself is weak and the ecology is scarce, it is very meaningful to make up for the short board through other languages.

It seems that WebAssembly is more in line with current needs. It already supports the more popular languages, such as C/C++, JavaScript and Golang, providing a new operating mechanism for these languages.

In addition, for serverless and edge computing case, more need for multi-language support.

Collecting test samples in complex situations

在当前的测试结果中，还缺少复杂情况的测试。这需要一份真实的测试样本，包含各种不同 uri 、method 组合，在 uri 中最好还包含一些有正则匹配的情况，等待您的建议。

In the current test results, there is still a lack of testing for complex situations. This requires a real test sample, including a variety of different uri, method combinations, and it is best to include some regular matching cases in the uri, waiting for your suggestion.

test `limit count`: It is unstable and sometimes fails

bug: Need to control the number of cached shema document

Maybe lru table is more suitable

https://github.com/iresty/apisix/blob/master/lua/apisix/core/schema.lua#L8

doc: why do users need apisix? what problem can apisix solve?

doc: maybe a mistake in architecture-design-cn.md

In this section https://github.com/iresty/apisix/blob/master/doc/architecture-design-cn.md#consumer

Consumer 是某类具体服务的消费者，主要用来表述不同用户的概念。比如不用的客户请求同一个 API， 经过用户认证体系，网关服务需知道当前请求用户身份信息，针对不同的消费用户，会有不同的限制处理逻辑。

不用 maybe 不同 ？

benchmark: v0.5

single worker

0.4 version benchmark: #36 (comment)

this version: dc12e8d

# 1route + 1 upstream + no plugin

$ curl http://127.0.0.1:2379/v2/keys/apisix/routes/1 -X PUT \
    -d value='{"methods":["GET"],"uri":"/hello","id":1,"plugin_config":{},"upstream":{"type":"roundrobin","nodes":{"127.0.0.1:80":1,"127.0.0.2:80":1}}}'

$ wrk -d 5 http://127.0.0.2:9080/hello
Running 5s test @ http://127.0.0.2:9080/hello
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   518.57us  180.42us   8.61ms   91.77%
    Req/Sec     9.52k   659.49    10.89k    66.67%
  96627 requests in 5.10s, 18.15MB read
Requests/sec:  18944.55
Transfer/sec:      3.56MB

$ wrk -d 5 http://127.0.0.2:9080/hello
Running 5s test @ http://127.0.0.2:9080/hello
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   520.49us  145.73us   9.11ms   86.06%
    Req/Sec     9.50k   825.08    10.93k    77.00%
  94465 requests in 5.00s, 17.74MB read
Requests/sec:  18888.80
Transfer/sec:      3.55MB

# 1route + 1 upstream + 2 plugins

$ curl http://127.0.0.1:2379/v2/keys/apisix/routes/1 -X PUT     -d \
    value='{"methods":["GET"],"uri":"/hello","id":1,"plugins":{"limit-count":{"count":200000000,"time_window":60,"rejected_code":503,"key":"remote_addr"},"prometheus":{}},"upstream":{"type":"roundrobin","nodes":{"127.0.0.1:80":1,"127.0.0.2:80":1}}}'

$ wrk -d 30 http://127.0.0.2:9080/hello
Running 30s test @ http://127.0.0.2:9080/hello
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   595.39us  178.51us  22.24ms   90.63%
    Req/Sec     8.33k   642.91     9.46k    59.80%
  499149 requests in 30.10s, 124.22MB read
Requests/sec:  16582.76
Transfer/sec:      4.13MB

$ wrk -d 30 http://127.0.0.2:9080/hello
Running 30s test @ http://127.0.0.2:9080/hello
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   598.23us  188.41us  15.63ms   91.48%
    Req/Sec     8.29k   672.68     9.52k    72.00%
  495022 requests in 30.00s, 123.19MB read
Requests/sec:  16500.29
Transfer/sec:      4.11MB

feature: need a powerful cache layer.

doc: set the configuration with admin API.

for example:

curl http://127.0.0.1:2379/v2/keys/apisix/routes/1

change to

curl http://127.0.0.1:9080/apisix/admin/routes/1

feature: use schema check for service configuration

feature: heartbeat plugin

bugfix: priority to use openresty's cjson library

Openresty has a built-in cjson library, but users may also install another version of cjson via luarocks, in which case we should give priority to the former.

benchmark: v0.4, made a benchmark for `apisix` to determine if there is a performance issue

1 route with same upstream
10 routes with same upstream
100 routes with same upstream
1000 routes with same upstream

support `PATCH` for admin APIs

• routes: #365

• service: #370

• upstream: #371

• consumers: waiting to do

• ssl: waiting to do