ory/keto

Open Source (Go) implementation of "Zanzibar: Google's Consistent, Global Authorization System". Ships gRPC, REST APIs, newSQL, and an easy and granular permission language. Supports ACL, RBAC, and other access models.

Chat | Discusssions | Newsletter

Guide | API Docs | Code Docs

Support this project!

Work in Open Source, Ory is hiring!

---

Ory Keto is the first and most popular open source implementation of "Zanzibar: Google's Consistent, Global Authorization System"!

Get Started

You can use Docker to run Ory Keto locally or use the Ory CLI to try out Ory Keto:

# This example works best in Bash
bash <(curl https://raw.githubusercontent.com/ory/meta/master/install.sh) -b . ory
sudo mv ./ory /usr/local/bin/

# Or with Homebrew installed
brew install ory/tap/cli

create a new project (you may also use Docker)

ory create project --name "Ory Keto Example"
export project_id="{set to the id from output}"

and follow the quick & easy steps below.

Create a namespace with the Ory Permission Language

# Write a simple configuration with one namespace
echo "class Document implements Namespace {}" > config.ts

# Apply that configuration
ory patch opl --project $project_id -f file://./config.ts

# Create a relationship that grants tom access to a document
echo "Document:secret#read@tom" \
  | ory parse relation-tuples --project=$project_id --format=json - \
  | ory create relation-tuples --project=$project_id -

# List all relationships
ory list relation-tuples --project=$project_id
# Output:
#   NAMESPACE	OBJECT	RELATION NAME	SUBJECT
#   Document	secret	read		tom
#
#   NEXT PAGE TOKEN
#   IS LAST PAGE	true

Now, check out your project on the Ory Network or continue with a more in-depth guide.

Ory Keto on the Ory Network

The Ory Network is the fastest, most secure and worry-free way to use Ory's Services. Ory Permissions is powered by the Ory Keto open source permission server, and it's fully API-compatible.

The Ory Network provides the infrastructure for modern end-to-end security:

• Identity & credential management scaling to billions of users and devices
• Registration, Login and Account management flows for passkey, biometric, social, SSO and multi-factor authentication
• Pre-built login, registration and account management pages and components
• OAuth2 and OpenID provider for single sign on, API access and machine-to-machine authorization
• Low-latency permission checks based on Google's Zanzibar model and with built-in support for the Ory Permission Language

It's fully managed, highly available, developer & compliance-friendly!

• GDPR-friendly secure storage with data locality
• Cloud-native APIs, compatible with Ory's Open Source servers
• Comprehensive admin tools with the web-based Ory Console and the Ory Command Line Interface (CLI)
• Extensive documentation, straightforward examples and easy-to-follow guides
• Fair, usage-based pricing

Sign up for a free developer account today!

Ory Network Hybrid Support Plan

Ory offers a support plan for Ory Network Hybrid, including Ory on private cloud deployments. If you have a self-hosted solution and would like help, consider a support plan! The team at Ory has years of experience in cloud computing. Ory's offering is the only official program for qualified support from the maintainers. For more information see the website or book a meeting!

Ory Permissions, Keto and the Google's Zanzibar model

Determining whether online users are authorized to access digital objects is central to preserving privacy. This paper presents the design, implementation, and deployment of Zanzibar, a global system for storing and evaluating access control lists. Zanzibar provides a uniform data model and configuration language for expressing a wide range of access control policies from hundreds of client services at Google, including Calendar, Cloud, Drive, Maps, Photos, and YouTube. Its authorization decisions respect causal ordering of user actions and thus provide external consistency amid changes to access control lists and object contents. Zanzibar scales to trillions of access control lists and millions of authorization requests per second to support services used by billions of people. It has maintained 95th-percentile latency of less than 10 milliseconds and availability of greater than 99.999% over 3 years of production use.

Source

If you need to know if a user (or robot, car, service) is allowed to do something - Ory Permissions and Ory Keto are the right fit for you.

Currently, Ory Permissions [on the Ory Network] and the open-source Ory Keto permission server implement the API contracts for managing and checking relations ("permissions") with HTTP and gRPC APIs, as well as global rules defined through the Ory Permission Language ("userset rewrites"). Future versions will include features such as Zookies, reverse permission lookups, and more.

---

• Ory Permissions, Keto and the Google's Zanzibar model
• Who's Using It?
  • Installation
• Ecosystem
  • Ory Kratos: Identity and User Infrastructure and Management
  • Ory Hydra: OAuth2 & OpenID Connect Server
  • Ory Oathkeeper: Identity & Access Proxy
  • Ory Keto: Access Control Policies as a Server
• Security
  • Disclosing Vulnerabilities
• Telemetry
  • Guide
  • HTTP API Documentation
  • Upgrading and Changelog
  • Command Line Documentation
  • Develop
    • Dependencies
    • Install From Source
    • Formatting Code
    • Running Tests
      • Short Tests
      • Regular Tests
      • End-to-End Tests
    • Build Docker

Who's Using It?

The Ory community stands on the shoulders of individuals, companies, and maintainers. The Ory team thanks everyone involved - from submitting bug reports and feature requests, to contributing patches and documentation. The Ory community counts more than 33.000 members and is growing rapidly. The Ory stack protects 60.000.000.000+ API requests every month with over 400.000+ active service nodes. None of this would have been possible without each and everyone of you!

The following list represents companies that have accompanied us along the way and that have made outstanding contributions to our ecosystem. If you think that your company deserves a spot here, reach out to [email protected] now!

Type	Name	Logo	Website
Adopter *	Raspberry PI Foundation		raspberrypi.org
Adopter *	Kyma Project		kyma-project.io
Adopter *	Tulip		tulip.com
Adopter *	Cashdeck / All My Funds		cashdeck.com.au
Adopter *	Hootsuite		hootsuite.com
Adopter *	Segment		segment.com
Adopter *	Arduino		arduino.cc
Adopter *	DataDetect		unifiedglobalarchiving.com/data-detect/
Adopter *	Sainsbury's		sainsburys.co.uk
Adopter *	Contraste		contraste.com
Adopter *	Reyah		reyah.eu
Adopter *	Zero		getzero.dev
Adopter *	Padis		padis.io
Adopter *	Cloudbear		cloudbear.eu
Adopter *	Security Onion Solutions		securityonionsolutions.com
Adopter *	Factly		factlylabs.com
Adopter *	Nortal		nortal.com
Adopter *	OrderMyGear		ordermygear.com
Adopter *	Spiri.bo		spiri.bo
Adopter *	Strivacity		strivacity.com
Adopter *	Hanko		hanko.io
Adopter *	Rabbit		rabbit.co.th
Adopter *	inMusic		inmusicbrands.com
Adopter *	Buhta		buhta.com
Adopter *	Connctd		connctd.com
Adopter *	Paralus		paralus.io
Adopter *	TIER IV		tier4.jp
Adopter *	R2Devops		r2devops.io
Adopter *	LunaSec		lunasec.io
Adopter *	Serlo		serlo.org
Adopter *	dyrector.io		dyrector.io
Adopter *	Stackspin		stackspin.net
Adopter *	Amplitude		amplitude.com
Adopter *	Pinniped		pinniped.dev
Adopter *	Pvotal		pvotal.tech

Many thanks to all individual contributors

* Uses one of Ory's major projects in production.

Installation

Head over to the documentation to learn about ways of installing Ory Keto.

Ecosystem

We build Ory on several guiding principles when it comes to our architecture design:

• Minimal dependencies
• Runs everywhere
• Scales without effort
• Minimize room for human and network errors

Ory's architecture is designed to run best on a Container Orchestration system such as Kubernetes, CloudFoundry, OpenShift, and similar projects. Binaries are small (5-15MB) and available for all popular processor types (ARM, AMD64, i386) and operating systems (FreeBSD, Linux, macOS, Windows) without system dependencies (Java, Node, Ruby, libxml, ...).

Ory Kratos: Identity and User Infrastructure and Management

Ory Kratos is an API-first Identity and User Management system that is built according to cloud architecture best practices. It implements core use cases that almost every software application needs to deal with: Self-service Login and Registration, Multi-Factor Authentication (MFA/2FA), Account Recovery and Verification, Profile, and Account Management.

Ory Hydra: OAuth2 & OpenID Connect Server

Ory Hydra is an OpenID Certified™ OAuth2 and OpenID Connect Provider which easily connects to any existing identity system by writing a tiny "bridge" application. It gives absolute control over the user interface and user experience flows.

Ory Oathkeeper: Identity & Access Proxy

Ory Oathkeeper is a BeyondCorp/Zero Trust Identity & Access Proxy (IAP) with configurable authentication, authorization, and request mutation rules for your web services: Authenticate JWT, Access Tokens, API Keys, mTLS; Check if the contained subject is allowed to perform the request; Encode resulting content into custom headers (X-User-ID), JSON Web Tokens and more!

Ory Keto: Access Control Policies as a Server

Ory Keto is a policy decision point. It uses a set of access control policies, similar to AWS IAM Policies, in order to determine whether a subject (user, application, service, car, ...) is authorized to perform a certain action on a resource.

Security

Disclosing Vulnerabilities

If you think you found a security vulnerability, please refrain from posting it publicly on the forums, the chat, or GitHub. You can find all info for responsible disclosure in our security.txt.

Telemetry

Our services collect summarized, anonymized data which can optionally be turned off. Click here to learn more.

Guide

The Guide is available here.

HTTP API Documentation

The HTTP API is documented here.

Upgrading and Changelog

New releases might introduce breaking changes. To help you identify and incorporate those changes, we document these changes in UPGRADE.md and CHANGELOG.md.

Command Line Documentation

Run keto -h or keto help.

Develop

We encourage all contributions and recommend you read our contribution guidelines.

Dependencies

You need Go 1.19+ and (for the test suites):

• Docker and Docker Compose
• GNU Make 4.3
• NodeJS / npm >= v7

It is possible to develop Ory Keto on Windows, but please be aware that all guides assume a Unix shell like bash or zsh.

Install From Source

make install

Formatting Code

You can format all code using make format. Our CI checks if your code is properly formatted.

Running Tests

There are two types of tests you can run:

• Short tests (do not require a SQL database like PostgreSQL)
• Regular tests (do require PostgreSQL, MySQL, CockroachDB)

Short Tests

Short tests run fairly quickly. You can either test all of the code at once:

go test -short -tags sqlite ./...

or test just a specific module:

go test -tags sqlite -short ./internal/check/...

Regular Tests

Regular tests require a database set up. Our test suite is able to work with docker directly (using ory/dockertest) but we encourage to use the script instead. Using dockertest can bloat the number of Docker Images on your system and starting them on each run is quite slow. Instead we recommend doing:

source ./scripts/test-resetdb.sh
go test -tags sqlite ./...

End-to-End Tests

The e2e tests are part of the normal go test. To only run the e2e test, use:

source ./scripts/test-resetdb.sh
go test -tags sqlite ./internal/e2e/...

or add the -short tag to only test against sqlite in-memory.

Build Docker

You can build a development Docker Image using:

make docker

keto's Issues

engine: Implement (H)RBAC

Press 👍 if you want to see this in ORY Keto and 👎 if not!

Move to new release pipeline

roles: add a PUT method to overwrite members

see ory/hydra#768

Build and upload binaries upon release

question: performance & benchmarks

Would you be able to provide some insight into performace implications of switching to OPA based implementation. I saw in the original PR that says we have 10x improvement for simple cases

Some questions I have are..

• how many queries / data loaded per evaluate request
• do we load all collections data for every query? if so what is the better way to do resouce level permissions / field level permissions
• Is there a way to add cache at some level

Policy filters

Hey,

first of all thank you for this service, it seems awesome! Is there a way (planned) to filter policies, i.e. give me all the policies that match some-scope:some-resource:<.*>?

Thanks and cheers,
Sebastian

The Warden API should not check for refresh tokens, only access tokens.

• Check if we can return the token type in the introspection response
• Check if refresh tokens can be introspected
• Document that upgrading from prior 0.8.0 requires to upgrade to 0.8.0 first and then upgrade to 1.0.x
• There was a security issue where introspection with a client id would not be checked against the database, actually

Show default host in "Listening" log message

Currently, the log message shows Listening on http://:4466 which is missing the host. That should be fixed.

Re-release v1.0.0-beta.x as v0.1.x-sandbox+oryos.x

• v0.1.1-sandbox+oryOS.1
• v0.1.2-sandbox+oryOS.2
• v0.1.3-sandbox+oryOS.3
• v0.1.4-sandbox+oryOS.4
• v0.1.5-sandbox+oryOS.5
• v0.1.6-sandbox+oryOS.6
• v0.1.7-sandbox+oryOS.7
• v0.1.8-sandbox+oryOS.8
• v0.1.9-sandbox+oryOS.9

  release-npm:
    docker:
      - image: circleci/node:8.9.3
    working_directory: ~/keto
    steps:
      - checkout
      - run: echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc
      - run: |
          npm version -f --no-git-tag-version $CIRCLE_TAG
      - run: npm publish --access public

{
  "name": "@oryd/keto",
  "private": false,
  "version": "0.0.0-test.0",
  "description": "The official JavaScript / NodeJS SDK for ORY Keto.",
  "license": "Apache 2.0",
  "main": "sdk/js/swagger/src/index.js",
  "scripts": {
    "prettier": "prettier --single-quote --parser flow --no-semi --write \"sdk/js/**/*.js\""
  },
  "browser": {
    "fs": false
  },
  "dependencies": {
    "superagent": "~3.7.0",
    "opencollective": "^1.0.3"
  },
  "devDependencies": {
    "prettier": "^1.7.4"
  },
  "collective": {
    "type": "opencollective",
    "url": "https://opencollective.com/ory",
    "logo": "https://opencollective.com/ory/logo.txt"
  }
}

      - run: docker build --build-arg git_tag=$(git describe --tags) --build-arg git_commit=$(git rev-parse HEAD) -f Dockerfile -t oryd/keto:$(echo $CIRCLE_TAG | tr '+' '_') .
      - run: docker login --username "$DOCKER_USERNAME" --password "$DOCKER_PASSWORD"
      - run: docker push oryd/keto:$(echo $CIRCLE_TAG | tr '+' '_')

roles without members does not shown in roles list

Hello! In our case we have some roles without members and its does not shown in roles list, but if request an empty role detail page by its id, all works fine

Keto latency with large number of policies

Describe the bug
Keto's latency increases with policies and resources

To Reproduce
Steps to reproduce the behavior:

1. (Optionally) Start with a fresh database
2. Create 30k regex policies with 1 or more resources
3. Perform allowed checks
4. Witness requests take 1-10 seconds to complete

Expected behavior
Access allowed checks under 100ms

Version:

• Environment: MacOS Mojave, Postgres 11
• Version: Master (0.3.1)

Additional context
Initially, I noticed Keto requests taking longer than 1 minute in pods running on Kubernetes when our policy count exceeded 30k. We also had memory issues but it may be unrelated. I was unable to run a memory profile using ppof on our images, the pod would crash OOM before the memory profile was written.

The policy causing the latency had 16 resources. They all had 2 wildcards.
Ex: tenant:<.*>:resource:foo<.*>

Observations about latency:

• The number of resources directly affects latency.
• The number of policies (unused or not) affects latency
• SERVE_MAX_AGE caused 500 errors with this many policies

A temporary solution to this would be to split my policy into 16 individual policies, but 1-2 second requests (running locally) still seems extreme.

Removing member from role fails with `unknown error (status 200)`

Describe the bug

Attempting to remove a member from a role fails with unknown error (status 200).

Reproducing the bug

Steps to reproduce the behavior:

1. Start Keto v0.3.3-sandbox using the ORY guide: https://www.ory.sh/docs/keto/configure-deploy
2. Create a role with a member via the SDK (creating a role with no members works too)
3. Try removing that member from the role
4. Request fails with response: unknown error (status 200): {resp:0xc0000ac1b0}

package main

import (
	"github.com/ory/keto/sdk/go/keto/client"
	"github.com/ory/keto/sdk/go/keto/client/engines"
	"github.com/ory/keto/sdk/go/keto/models"
)

func main() {
	ketoClient := client.NewHTTPClientWithConfig(nil, &client.TransportConfig{
		Host:     "localhost:4466",
		BasePath: "",
		Schemes:  []string{"http"},
	})

	createReq := engines.NewUpsertOryAccessControlPolicyRoleParams().
		WithFlavor("regex").
		WithBody(&models.OryAccessControlPolicyRole{
			ID:      "test-role",
			Members: []string{"[email protected]"},
		})
	_, err := ketoClient.Engines.UpsertOryAccessControlPolicyRole(createReq)
	if err != nil {
		panic(err)
	}

	removeReq := engines.NewRemoveOryAccessControlPolicyRoleMembersParams().
		WithFlavor("regex").
		WithID("test-role").
		WithMember("[email protected]")
	_, err = ketoClient.Engines.RemoveOryAccessControlPolicyRoleMembers(removeReq)
	if err != nil {
		panic(err)
	}
}

Expected behavior

The SDK should handle the valid request and return an empty response. Which status code is correct: 201 created or 200 ok? Probably an easy fix either way (I can submit a PR for it if needed).

Environment

• Version: v0.3.3-sandbox
• Environment: Docker

Keto can not be consumed as a dependency

Describe the bug
[email protected] can not be resolved as a dependency to another project with both dep or go mod due to Gopkg.lock containing outdated and unused dependencies.

To Reproduce

# with go mod
$ go get github.com/ory/[email protected]

go build github.com/ory/go-convenience/corsx: no Go files in
go build github.com/ory/ladon/manager/sql: no Go files in

Expected behavior
Dependency resolution should pass

Role setup not working

Hey,

I'm still experimenting with this service - I tried to setup roles.

My roles are:
[ { "id": "admin", "members": [ "bob" ] } ]

My policies are:
[ { "id": "f1d0ff27-5afd-49dd-9f58-9dffc0b99e35", "description": "", "subjects": [ "admin" ], "effect": "allow", "resources": [ "project:1:ticket:1" ], "actions": [ "read" ], "conditions": {} } ]

However, my request to localhost:4466/warden/subjects/authorize with the following content returns { "allowed": false }.

Body:
{ "action": "read", "subject": "bob", "resource": "project:1:ticket:1" }

Am I doing anything wrong?

Reintroduce swagger task to CircleCI

This has been disabled because go-swagger does not yet work with go-modules

There is no way to get all roles using the ListOryAccessControlPolicyRoles() API

Describe the bug

Not sure if you will consider this a bug or a feature request but I am using the API ListOryAccessControlPolicyRoles(flavor, limit, offset) and my use case involves getting all the roles. I tried passing limit and offset as 0 to get all of them but got empty results.

My current workaround is to call it like this
ListOryAccessControlPolicyRoles("regex", math.MaxInt64, 0)

Expected behavior

When I pass limit and offset as 0, I expect to get all the roles.

Environment

• Version: 1cb8a36
• Environment: Local

warden: Check against multiple policies

See ory/hydra#264

Can not remember consent because no user interaction was required - when resp['skip'] false

The response from fetching log in details using the login_challenge contains resp['skip'] = false yet still an error is thrown with code 400

After retrying a log in resp['skip'] comes back as true without the user logging in!

0.2.2-sandbox+oryOS.10: Unable to initialize compiler: lstat /go: no such file or directory

Describe the bug
Version [0.2.2-sandbox+oryOS.10]
keto serve
=> Unable to initialize compiler: lstat /go: no such file or directory

Version [v0.1.9-sandbox+oryOS.9]
keto serve
=> work well

To Reproduce
Steps to reproduce the behavior:

1. Run keto serve
2. See error: Unable to initialize compiler: lstat /go: no such file or directory

Version:

• Environment: [binaries]
• Version [0.2.2-sandbox+oryOS.10]

Moving forward with this project's versioning

Hi there,

so I kinda messed up. This project is currently released as version v1.0.0-beta.X. The goal of this was to have one coherent version across all services to make integration of them as easy as possible. While that was a good intention, releasing something as v1.0.0 gave the wrong impression of stability, and rightful so. My initial idea was to label projects that are not fully backed yet with a -sandbox or -preview tag (v1.0.0-sandbox) but that contradicts how semantic versioning works and is probably even more confusing.

For so many reasons this was a bad decision, this project specifically moved right away to v1.0.0 instead of going through a "sandbox" phase where you, the community, evaluates the usefulness of the service and gives feedback on missing parts which may require breaking changes.

To resolve this, we worked on a new concept which I would like to present to you here. To not make the same mistake again, I want to get your feedback on this before moving forward with this plan. Implementing this plan implies that versions v1.0.0-beta.* will be purged and that we will start tagging this as a v0.y.z release instead.

Please read (and give feedback) on the following paragraphs which will be (upon approval) immortalized in the ORY Documentation:

Software Versions

The ORY ecosystem consists of multiple services versioned using semantic versioning. This section
explains how we define service versions and what they mean.

There are three types of services:

• Graduated services change rarely in a backwards incompatible fashion. A service can be considered graduated if
  the major version is >= 1 - for example: v1.0.1, v2.2.2. If a serious backwards incompatible
  change is made, the major version jumps one up. Most, if not all, REST APIs will provide backwards compatible transformations that
  make it possible to interact with the server using older API concepts.
• Incubating services have well defined concepts but do not provide backwards compatible REST APIs yet. Incubating
  services are indicated by major version numbers of 0 - for example: v0.10.0
• Sandbox services may implement concepts, provide APIs and CLIs which are not fully baked yet. It is possible that
  these services change in unpredictable ways. These services are indicated by major version numbers of 0 and the
  sandbox label - for example: v0.10.0-sandbox.

To make deployment easier but stay compatible with semantic versioning, each service is equipped with a ecosystem version
number denoted by +oes.X where X represents the version ecosystem. This is specifically useful when using
incubating or sandboxed services which do not share the version numbers of graduated services. Let's look at some examples:

• ORY Hydra v1.0.1+oes.6 is best compatible with ORY Oathkeeper v0.12.1+oes.6 and ORY Keto v0.5.1-sandbox+oes.6.
• ORY Hydra v1.0.2+oes.7 is best compatible with ORY Oathkeeper v0.12.2+oes.7 and ORY Keto v0.6.0-sandbox+oes.7.
• ORY Hydra v1.0.3+oes.8 is probably not fully compatible with ORY Oathkeeper v0.12.1+oes.6 nor with ORY Keto
  v0.5.1-sandbox+oes.6.
• ORY Hydra v1.1.0+oes.9 is best compatible with ORY Oathkeeper v1.1.0+oes.9 and ORY Keto v1.1.0+oes.9.

Important: Each release - unless explicitly labeled as -unstable - is going through extensive quality assurance
and is considered secure and reliable enough to be run in production. If you choose to go with an incubating or sandbox
service, it is likely that you will spend some time addressing breaking changes.

We always provide ways to migrate breaking changes, and all breaking changes are meticolously described in each project's
UPGRADE.md as well as HISTORY.md.

Moving forward with ORY Keto

I recently (re-)discovered the OPA project. This issue is about deprecating the ORY Ladon engine and aligning ORY Keto with OPA. The decision is not yet made and we are looking for valuable input regarding this.

OPA allows you to write authorization logic using a language specifically designed for that, called rego. Syntax is very go-like. Due to this, OPA is capable of providing all sorts of authorization mechanisms, like RBAC, ABAC, ACL, AWS IAM Policies, and more. In fact, I believe that ORY Ladon's logic is implementable using rego. I'm not sure if that holds true for conditions which still needs verification from my side.

Let's take a look at the current downsides of each project.

I believe that policy documents as implemented by ORY Ladon are very powerful, but also very complicated. Many developers struggle with proper resource & action naming. I think that regular expressions have their place here, but many developers struggle with writing and testing regular expressions and variable substitution is very flaky (currently only used in the ORY Oathkeeper adapter for ORY Keto iirc) from a ux perspective. Also, regular expressions do not scale well, especially if read from the database. I think we can fix this with caching, but that is not fixing the problem itself, only the symptom.

On the same hand, OPA is limited. I think rego is great for developers that really want to jump into this (like me). But it's a new syntax, new language, and new tools. I think the language is not incredibly intuitive and not always readable:

I believe that policy documents are, in general, quite complicated. In my opinion, rego has a steep learning curve as well. Can you tell me immediately what this does?

sod_roles = [
    ["create-payment", "approve-payment"],
    ["create-vendor", "pay-vendor"]
]

sod_violation[user] {
    role1 = user_role[user][_]
    role2 = user_role[user][_]
    sod_roles[_] = [role1, role2]
}

At least I can not - the point I'm trying to make is, you have to learn this.

OPA comes with a REST API but it's really a parser and execution engine. It parses rego files and executes the logic based on data you provide. The result is always true or false, depending on the authorization result.

The server is limited. It stores everything in-memory so pushing logic to the server is not realistic with more than one server running. Instead, you'll probably have to write a CI pipeline which builds a docker image that has all your policy definitions. IMO that can be very nice, especially if you have rego test as part of that pipeline. But, from my experience, most people do not want or know how to do these things.

Coming back to policies for a moment: Most developers do not need AWS IAM Policies, simple role management is enough. By the way, the Google Cloud Platform migrated completely to RBAC/ACL as well recently (at least in the UI) - very few people want to deal with complicated JSON documents. And many make mistakes, which is evident due to the many S3 leaks we're seeing recently (caused by misconfigured buckets, well really misconfigured AWS IAM Policies). I think the same would have happened if AWS used rego, or it would have probably caused less people to use this feature at all.

My vision for ORY Keto is to have a "policy decision point" (someone that says if something is allowed or not) that just works. I also believe that it should be possible to use several well-known patterns out of the box, this includes RBAC, ACL, ABAC and ORY Ladon Policies (for BC compatibility) and well, maybe your own rego definitions? I will experiment this month with different concepts and try to migrate Ladon Policies on top of rego. My preliminary tests showed that we can get a 10x performance improvement for simple use cases. We'll see how well this does for advanced ones.

We write this software is for you, so please participate in the discussion and leave your ideas and comments below.

Add SAML Authenticator

Upserting to memory database only inserts

Describe the bug
Upserting policies, with the same IDs/data, to the memory database, causes duplicate records, instead of updating.

To Reproduce
Steps to reproduce the behavior:

1. Run DATABASE_URL=memory go run main.go serve
2. PUT the same policy more than once

PUT /engines/acp/ory/exact/policies HTTP/1.1
Host: localhost:4466
Content-Type: application/json
{
  "actions": [
    "delete"
  ],
  "description": "Delete first blog post",
  "effect": "allow",
  "id": "alice-deletes-blog-post",
  "resources": [
    "blog_posts:my-first-blog-post"
  ],
  "subjects": [
    "alice"
  ]
}

3. Get the list of policies and withness duplicate entries

GET /engines/acp/ory/exact/policies HTTP/1.1
Host: localhost:4466
Content-Type: application/json

Expected behavior
Upserting a policy will update a pre-existing policy based on the ID. If this is a limitation of the memory database, I would expect some sort of warning.

Version:

• Go version 1.11
• Keto origin/master @ ed7af3f

engine: Implement ACL

Press 👍 if you want to see this in ORY Keto and 👎 if not!

Error on loading resty package with go modules

Hello!

When trying to install Keto SDK with go modules on:

GO111MODULE=on go get github.com/ory/keto/sdk/go/keto

I run into the following issue:

go: finding github.com/ory/keto/sdk/go/keto latest
go: finding github.com/ory/keto/sdk/go latest
go: finding github.com/ory/keto/sdk latest
go: github.com/go-resty/[email protected]: parsing go.mod: unexpected module path "gopkg.in/resty.v1"
go: error loading module requirements

The problem seems very similar to this one in Hydra.

glob not working on members of groups

Wildcards in the members of groups are not interpreted as wildcards for flavor glob.

Applies to recent versions of keto, e.g., to commit db94481.

The reason seems to be at the end of the file engine/ladon/rego/glob/main.rego in the function match_subjects. The intended implementation is preceded by the implementation for the flavor exact.

Add something like

decide_allow([group_policy], [{"id": "groups:6", "members": ["group-*"]}]) with input as {"resource": "articles:6", "subject": "group-member", "action": "actions:6"}

to main_test.rego for a corresponding test.

cannot import policies with regex

Describe the bug
when keto engines acp ory policies import is used against a JSON document representing a policy it fails to retain the angle brackets that are needed when using the regex flavor.

Reproducing the bug
Just run the example in the example repository using a resource policy like the following:

[{
  "id": "some-id",
  "description": "some description",
  "subjects": ["some-subject"],
  "effect": "allow",
  "resources": [
      "resource:sample:<.*>",
  ],
  "actions": [
      "action:sample:<.*>:foo",
  ]
}]

setup an oathkeeper policy that asks for a resource and an action that leverage those regular expressions.

Expected behavior
I expect to be able to access the resource. If you check how the policy was saved in the database, you'll see that those angle brackets have been escaped preventing, I guess, Ladon to parse the rule correctly.

When keto is importing the JSON document, as per GO policy when marshaling a JSON string, it will escape any HTML character, thus causing the unwanted behavior shown above.

Thanks

engine: Implement AWS IAM Policies

Press 👍 if you want to see this in ORY Keto and 👎 if not!

Slash (/) in role or policy id causes 404 error for GET and DELETE

Describe the bug

When I try to GET or DELETE a role with a / in the string for the id, a 404 error response is returned. The same issue exists for policy ids.

Reproducing the bug

Steps to reproduce the behavior:

1. Run docker run ....
2. Create a role with / in the id for example: curl -X PUT \ http://localhost:32034/engines/acp/ory/exact/roles \ -H 'Accept: application/json' \ -H 'Accept-Encoding: gzip, deflate' \ -H 'Cache-Control: no-cache' \ -H 'Connection: keep-alive' \ -H 'Content-Length: 53' \ -H 'Content-Type: application/json' \ -H 'Host: localhost:32034' \ -H 'cache-control: no-cache' \ -d '{ "id": "some/group", "members": ["string"] }' and confirm 200 OK response
3. Request details for the role with the request curl -X GET \ http://localhost:32034/engines/acp/ory/exact/roles/some%2Fgroup
4. Delete role with the request curl -X DELETE \ http://localhost:32034/engines/acp/ory/exact/roles/some%2Fgroup

Result: 404 responses for the GET and DELETE responses, however, the role is included when listing all roles

Server logs

INFO[29052] started handling request                      method=PUT remote="192.168.65.3:55658" request=/engines/acp/ory/exact/roles
INFO[29052] completed handling request                    measure#keto.latency=5997225 method=PUT remote="192.168.65.3:55658" request=/engines/acp/ory/exact/roles status=200 text_status=OK took=5.997225ms
INFO[29107] started handling request                      method=GET remote="192.168.65.3:55658" request="/engines/acp/ory/exact/roles/some%2Fgroup"
INFO[29107] completed handling request                    measure#keto.latency=209943 method=GET remote="192.168.65.3:55658" request="/engines/acp/ory/exact/roles/some%2Fgroup" status=404 text_status="Not Found" took="209.943µs"
INFO[29209] started handling request                      method=DELETE remote="192.168.65.3:56136" request="/engines/acp/ory/exact/roles/some%2Fgroup"
INFO[29209] completed handling request                    measure#keto.latency=129919 method=DELETE remote="192.168.65.3:56136" request="/engines/acp/ory/exact/roles/some%2Fgroup" status=404 text_status="Not Found" took="129.919µs"

Server configuration

Expected behavior

GET and DELETE should handle url encoded identifiers, or an id with a slash should be prevented from being used to create a role.

Environment

• Version: v0.3.6-sandbox+oryOS.12
  Build Commit: ef5aec8
  Build Timestamp: 2019-10-16T16:49:13Z
• Environment: Docker

Wrong oauth introspect url

In help command instrospect url is “hydra-url/oauth2/introspection”, but introspect on hydra work at “hydra-url/oauth2/introspect”

“Example: AUTHENTICATOR_OAUTH2_INTROSPECTION_INTROSPECTION_URL=https://my-server/oauth2/introspection”

engine: Support glob matching in ory acp

Allow multiple filter param values in roles and policies list API

This is a continuation of #124. Currently, only one value can per filter param is supported in the list API. For example, current filter only allows queries like /policies/member=foo&subject=bar, but often times a user needs multiple value search like policies/member=foo&member=baz&subject=bar

This is easy to implement in the storage filter, but the question is what does it mean to pass multiple values? Does it mean AND or OR? I doubt it is possible to support both with the same url format. This REST spec does not provide any guideline for this. This page(https://stackoverflow.com/questions/1746507/authoritative-position-of-duplicate-http-get-query-keys) seems to suggest that the intended behavior is AND.

What are your thoughts?

Move to go-swagger client generation

DELETE member from role

Describe the bug
When I try to delete a member from a role, it returns Method Not Allowed.

Expected behavior
The members should be deleted succesfully.

Version:

• Environment: Docker
• Version: v0.2.2-sandbox_oryOS.10:

Additional context
In the configure & deploy section the docker image used is oryd/keto:v0.2.2_oryOS.10, I tried fetching this image, In case this one allows me to delete roles, but I get a manifest error:

$ docker run oryd/keto:v0.2.2_oryOS.10
Unable to find image 'oryd/keto:v0.2.2_oryOS.10' locally
docker: Error response from daemon: manifest for oryd/keto:v0.2.2_oryOS.10 not found````

 

API documentation seems not up to date

Describe the bug

The API documentation explains how to list ORY Access Control Policy Roles but when we called it, we got a 301 - moved permanently . (https://www.ory.sh/docs/keto/sdk/api#list-ory-access-control-policy-roles)
It worked on Friday May 17th 2019.

Reproducing the bug

Steps to reproduce the behavior:

1. Follow the steps described here: https://www.ory.sh/docs/next/keto/configure-deploy
2. curl http://localhost:4466/engines/acp/ory/glob/roles/
3. Response received: Moved Permanently.

Server logs

time="2019-05-20T14:21:56Z" level=info msg="started handling request" method=GET remote="192.168.192.1:35094" request=/engines/acp/ory/glob/roles/
time="2019-05-20T14:21:56Z" level=info msg="completed handling request" measure#keto.latency=60777 method=GET remote="192.168.192.1:35094" request=/engines/acp/ory/glob/roles/ status=301 text_status="Moved Permanently" took="60.777µs"

Expected behavior

The ansmer must contain a list of policy roles

Environment

• Docker image: 0.3.1
• Docker version: 18.09.2
• Ubuntu: 18

policy: evaluate wildcard matching strategy

One issue with access control policies is that, at the moment, we have the burden of regular expression overhead. This implies that scalability is limited.

Most matches however require simple wildcard logic, like resources:* or users:*:emails. It should be explored if wildcard matches perform better in memory and in databases and decided if regexp matching will be replaced with wildcard matching, or at least complemented.

See ory/hydra#580

Upgrade from < 1.0.0-beta Hydra migration commands fail

From: https://github.com/ory/hydra/blob/master/UPGRADE.md#access-control--warden-api

# Migrate the policies and warden groups to keto
$ keto migrate hydra $DATABASE_URL

# Create other Keto database schemas
$ keto migrate sql $DATABASE_URL

Error output:

keto_1            | INFO[0000] Successfully applied SQL migrations           migrations=0 table=role
keto_1            | FATA[0000] An error occurred while trying to apply SQL migrations  error="Could not migrate sql schema, applied 2 migrations: pq: relation "ladon_subject_compiled_idx" already exists handling 3" migrations=0 table=policies

Cannot connect to MySQL

Describe the bug

It is not possible to connect to MySQL using the DSN format.

Reproducing the bug

The issue is related to a change in Golang and was first reported here: golang-migrate/migrate#264

Expected behavior

Keto should be able to connect to MySQL.

Environment

• Keto versions: Higher than v0.3.3 (it works in v0.3.3)
• Environment: Tested in Docker

Additional context

The issue has already been fixed: golang-migrate/migrate#265
Most probably the solution is simply to update the Go version.

gRPC API

Is your feature request related to a problem? Please describe.
No

Describe the solution you'd like
I was thinking, how nice would be to see gRPC API, besides HTTP in order to improve performance, avoid serialization with protobuf and stay type-safe. I am concerned about performance issues and making a bottleneck in the system. There is a lot of benchmarks online which says that gRPC is 10x faster than HTTP JSON API doing the same thing. This gRPC API doesn't need to have everything that HTTP has, just 2-3 methods would be enough, even 1 method so I can at least verify access by checking policy, this is most used API endpoint.

We are considering ory/Keto usage as AuthZ server and ACL for Mainflux IIoT platform. Our main goals are performance, scalability, and security, AuthZ and any kind of access control will have an impact on our performance (We are reaching hundreds of thousands msg/sec), but much less if we use gRPC API rather then HTTP.

I was wondering if gRPC API is something that yoiu would consider, at least for check access endpoint and do you see any drawbacks, blocks, etc...?

Thanks!

GetOryAccessControlPolicyRoles by member id as well

Is your feature request related to a problem? Please describe.

More and more of our clients are asking for getting roles for a specific member or a group.

Describe the solution you'd like

Have an API GetOryAccessControlPolicyRoles(flavor, Member ID/GroupID) to get all the roles associated with a certain member

Describe alternatives you've considered

Our current workaround involves getting all the roles using ListOryAccessControlPolicyRoles and filter through specific subjects/members. This is not a secure way and also a memory intensive operation. Another potential way I thought was to read the rego_data directly which basically means having another keto API to support this.

Additional context

I have already started working on this so if you would like I can submit a PR for this. I am pretty much only waiting to see if you already have something in the pipeline or have any strong objection against having this API.

Doing a go get will throw an error

Describe the bug

Attempting to go get the project will get you an error:

[pbabbicola@arch-thinkpad keto]$ go get github.com/ory/keto
# github.com/ory/x/metricsx
../../ory/x/metricsx/middleware.go:101:10: undefined: analytics.Config

Running with -u will still keep the same issue.

Reproducing the bug

Steps to reproduce the behavior:

Run $ go get github.com/ory/keto

Expected behavior

Expected the code to be copied to $GOPATH/src/github.com/ory/keto

Environment

• go version go1.12.6 linux/amd64

ABAC with Groups and Sub Groups.

I need to implement an ABAC system with the following requirements:

• there will be groups
• groups can have subgroups and subgroups can have other subgtoups
• each group will have its own roles.
• a role called "Admin" can be in groups A and in Group B too, and they can have completely different permission (policies).
• An Admin of the Group A can be assigned/removed permissions to view all children groups or just a few of them.
• A role inside a group can be granted the permission to see resources of that group and some resources of the children groups.

Following the keto tutorial I was able to set up the docker container and start adding roles and policies. However It seems that a role is just an id field. No additional information can be added. In such a case what is the best approach to assign roles to a group and how to create groups. I was looking for some field like context or attribute but there's nothing like that.

Is it possible to use keto for a system like the one I described above ?
Thanks.

Add JSON Web Token Authenticator

Show success message after importing things

Right now there is no output after successful creation.

Unexpected DoOryAccessControlPoliciesAllow results in Go SDK

Describe the bug

Performing a DoOryAccessControlPoliciesAllow check with the Go SDK, where the expected allowed value is false, returns a nil DoOryAccessControlPoliciesAllowOK and an error that can be type asserted into a DoOryAccessControlPoliciesAllowForbidden.

I am not sure if this is by design but it felt awkward to me.

Reproducing the bug

Run Keto locally and run the following code:

package main

import (
	"fmt"
	"log"
	"os"
	"time"

	"github.com/ory/keto/sdk/go/keto/client"
	"github.com/ory/keto/sdk/go/keto/client/engines"
	"github.com/ory/keto/sdk/go/keto/models"
)

func main() {
	// Connect to Keto
	config := &client.TransportConfig{
		Host:     "localhost:4466",
		BasePath: client.DefaultBasePath,
		Schemes:  []string{"http"},
	}
	client := client.NewHTTPClientWithConfig(nil, config)

	// Check connection
	ok, err := client.Health.IsInstanceAlive(nil)
	if err != nil {
		panic(err)
	}
	log.Println(ok.Payload.Status)

	// Create policy thats expected to not be allowed
	input := &models.OryAccessControlPolicyAllowedInput{
		Subject:  "foo",
		Action:   "bar",
		Resource: "baz",
	}
	params := &engines.DoOryAccessControlPoliciesAllowParams{
		Body:   input,
		Flavor: "regex",
	}
	params.WithTimeout(30 * time.Second) // We have to do this every time?

        // Check Allowed (should be false)
	allowOk, err := client.Engines.DoOryAccessControlPoliciesAllow(params)
	if err != nil {
                // Cast error?
		t, ok := err.(*engines.DoOryAccessControlPoliciesAllowForbidden)
		if !ok {
			panic("can't cast err to DoOryAccessControlPoliciesAllowForbidden")
		} else {
			log.Println("success? casted err to struct", *t.Payload.Allowed) // Why???
			os.Exit(0)
		}
		panic(fmt.Sprintf("some other error occured %+v\n", err))
	}
	log.Println("everything is ok", *allowOk.Payload.Allowed)
}

Expected behavior

I would expect DoOryAccessControlPoliciesAllow to return a DoOryAccessControlPoliciesAllowOK with a Payload (models.AuthorizationResult) Allowed value to be false.

Environment

• Version: master/571b447ed3a02f43623ef5c5adc09682b5f379bd
• Environment: go version go1.12.1 darwin/amd64

Support for nginx http_auth_request_module

Describe the solution you'd like
https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/
nginx can authenticate each request to your website with an external server or service. To perform authentication, NGINX makes an HTTP subrequest to an external server where the subrequest is verified. If the subrequest returns a 2xx response code, the access is allowed, if it returns 401 or 403, the access is denied. Such type of authentication allows implementing various authentication schemes, such as multifactor authentication, or allows implementing LDAP or OAuth authentication.

Document backslash behavior in glob patterns.

Is your feature request related to a problem? Please describe.

When trying to match the literal string foo\bar, a glob pattern of foo\\bar will match foobar (because \\b is interpreted as a literal b). To match the literal string foo\bar, a glob pattern of foo\\\\bar is required.

This is currently not sufficiently documented.

Describe the solution you'd like

Document the behavior in the ORY Keto docs.

Conditional function based on Context Conditions does not Apply

Bug or Unclear Documentation
Conditional function based on Context Conditions does not Apply
Tested with EqualsSubjectCondition and StringEqualCondition

To Reproduce
Steps to reproduce the behavior:

1. Create a exact Policie with the resulting Json Response on: /engines/acp/ory/exact/policies

[{
   "id": "test-users-alice",
   "description": "",
   "subjects": ["users:alice"],
   "resources": ["resources:users:alice"],
   "actions": [ "read", "update"],
   "effect": "allow",
   "conditions": [ {
      "subject": {
         "type": "EqualsSubjectCondition"
      }
   }]
}]

2. Request on /engines/acp/ory/exact/allowed with the Request Body:

{
    "action": "read",
    "subject": "users:alice",
    "resource": "resources:users:alice",
    "context": {
        "subject": "users:alice"
    }
}

3. Results in following Result

{
    "allowed": false
}

Expected behavior
Expected is to following Result:

{
    "allowed": true
}

Version:

• Environment: Docker
• Version: oryd/keto:v0.2.2-sandbox_oryOS.10

Unable to initialize compiler: lstat /build: no such file or directory

Describe the bug
keto serve does not start

To Reproduce

1. keto serve
   Unable to initialize compiler: lstat /build: no such file or directory

Expected behavior
The server running

Version:

• Environment: [Nixos]
• Version [7798442]

Additional context
I am writing a nixos service for ORY, was already successful in creating one for Hydra and Oathkeeper but ran into this, that looks like #73

errors under load with policy creation(ladon_action table)

Hello!
I made a load tests on policy creation and find some bugs with "duplicated pk error" on pg side.

After that I looked in the sql part of the source and find this:

QueryInsertPolicy: `INSERT INTO ladon_policy(id, description, effect, conditions) SELECT $1::varchar, $2, $3, $4 WHERE NOT EXISTS (SELECT 1 FROM ladon_policy WHERE id = $1);`,

So, it's bad to use GO WHERE NOT EXISTS it's need to replace on ON CONFLICT

If you wanna to repeat it, just try to create many policies in N threads by send POST to "/policies" handler