dapr/python-sdk

Dapr SDK for Python

Any language, any framework, anywhere

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

Dapr codifies the best practices for building microservice applications into open, independent, building blocks that enable you to build portable applications with the language and framework of your choice. Each building block is independent and you can use one, some, or all of them in your application.

We are a Cloud Native Computing Foundation (CNCF) incubation project.

Goals

• Enable developers using any language or framework to write distributed applications
• Solve the hard problems developers face building microservice applications by providing best practice building blocks
• Be community driven, open and vendor neutral
• Gain new contributors
• Provide consistency and portability through open APIs
• Be platform agnostic across cloud and edge
• Embrace extensibility and provide pluggable components without vendor lock-in
• Enable IoT and edge scenarios by being highly performant and lightweight
• Be incrementally adoptable from existing code, with no runtime dependency

How it works

Dapr injects a side-car (container or process) to each compute unit. The side-car interacts with event triggers and communicates with the compute unit via standard HTTP or gRPC protocols. This enables Dapr to support all existing and future programming languages without requiring you to import frameworks or libraries.

Dapr offers built-in state management, reliable messaging (at least once delivery), triggers and bindings through standard HTTP verbs or gRPC interfaces. This allows you to write stateless, stateful and actor-like services following the same programming paradigm. You can freely choose consistency model, threading model and message delivery patterns.

Dapr runs natively on Kubernetes, as a self hosted binary on your machine, on an IoT device, or as a container that can be injected into any system, in the cloud or on-premises.

Dapr uses pluggable component state stores and message buses such as Redis as well as gRPC to offer a wide range of communication methods, including direct dapr-to-dapr using gRPC and async Pub-Sub with guaranteed delivery and at-least-once semantics.

Why Dapr?

Writing highly performant, scalable and reliable distributed application is hard. Dapr brings proven patterns and practices to you. It unifies event-driven and actors semantics into a simple, consistent programming model. It supports all programming languages without framework lock-in. You are not exposed to low-level primitives such as threading, concurrency control, partitioning and scaling. Instead, you can write your code by implementing a simple web server using familiar web frameworks of your choice.

Dapr is flexible in threading and state consistency models. You can leverage multi-threading if you choose to, and you can choose among different consistency models. This flexibility enables you to implement advanced scenarios without artificial constraints. Dapr is unique because you can transition seamlessly between platforms and underlying implementations without rewriting your code.

Features

• Event-driven Pub-Sub system with pluggable providers and at-least-once semantics
• Input and output bindings with pluggable providers
• State management with pluggable data stores
• Consistent service-to-service discovery and invocation
• Opt-in stateful models: Strong/Eventual consistency, First-write/Last-write wins
• Cross platform virtual actors
• Secret management to retrieve secrets from secure key vaults
• Rate limiting
• Built-in Observability support
• Runs natively on Kubernetes using a dedicated Operator and CRDs
• Supports all programming languages via HTTP and gRPC
• Multi-Cloud, open components (bindings, pub-sub, state) from Azure, AWS, GCP
• Runs anywhere, as a process or containerized
• Lightweight (58MB binary, 4MB physical memory)
• Runs as a sidecar - removes the need for special SDKs or libraries
• Dedicated CLI - developer friendly experience with easy debugging
• Clients for Java, .NET Core, Go, Javascript, Python, Rust and C++

Get Started using Dapr

See our Getting Started guide over in our docs.

Quickstarts and Samples

• See the quickstarts repository for code examples that can help you get started with Dapr.
• Explore additional samples in the Dapr samples repository.

Community

We want your contributions and suggestions! One of the easiest ways to contribute is to participate in discussions on the mailing list, chat on IM or the bi-weekly community calls. For more information on the community engagement, developer and contributing guidelines and more, head over to the Dapr community repo.

Contact Us

Reach out with any questions you may have and we'll make sure to answer them as soon as possible!

Platform	Link
💬 Instant Message Chat (preferred)
📧 Mailing List	https://groups.google.com/forum/#!forum/dapr-dev
🐤 Twitter	@daprdev

Community Call

Every two weeks we host a community call to showcase new features, review upcoming milestones, and engage in a Q&A. All are welcome!

📞 Visit https://aka.ms/dapr-community-call for upcoming dates and the meeting link.

Videos and Podcasts

We have a variety of keynotes, podcasts, and presentations available to reference and learn from.

📺 Visit https://docs.dapr.io/contributing/presentations/ for previous talks and slide decks.

Contributing to Dapr

See the Development Guide to get started with building and developing.

Repositories

Repo	Description
Dapr	The main repository that you are currently in. Contains the Dapr runtime code and overview documentation.
CLI	The Dapr CLI allows you to setup Dapr on your local dev machine or on a Kubernetes cluster, provides debugging support, launches and manages Dapr instances.
Docs	The documentation for Dapr.
Quickstarts	This repository contains a series of simple code samples that highlight the main Dapr capabilities.
Samples	This repository holds community maintained samples for various Dapr use cases.
Components-contrib	The purpose of components contrib is to provide open, community driven reusable components for building distributed applications.
Dashboard	General purpose dashboard for Dapr
Go-sdk	Dapr SDK for Go
Java-sdk	Dapr SDK for Java
JS-sdk	Dapr SDK for JavaScript
Python-sdk	Dapr SDK for Python
Dotnet-sdk	Dapr SDK for .NET
Rust-sdk	Dapr SDK for Rust
Cpp-sdk	Dapr SDK for C++
PHP-sdk	Dapr SDK for PHP

Code of Conduct

Please refer to our Dapr Community Code of Conduct

python-sdk's Issues

Set up GitHub Actions workflow for build and publish pkg

Describe the proposal

Set up GitHub Actions workflow for build and publish pkg

• Create workflow for building pkg
• Create workflow for publishing pkg

related to #40

Add requirement.txt to install grpcio package

Need requirement.txt to install grpcio package

Release dapr-python v0.5.0a0

Describe the proposal

This is the release plan to ship dapr-python v0.5.0b.

• Package name: dapr
  • python-sdk includes actor sdk implemenation with the generated proto client so that it makes sense to rename from dapr-clients to dapr
• Package version: 0.5.0b
• Release work items.
  • Merge features/actor to master
  • Set up CI/CD
  • Improve README.md
  • validate with dotnet-sdk
  • Release dapr-python 0.5.0a

Implement actor service via fastapi

Describe the proposal

Implement actor service via aiohttp which is similar to flask_dapr extension implementation.

Add dev-container for python-sdk dev environment

Describe the proposal

Add dev-container for python-sdk dev environment

Unable to subscriber to a topic in pubsub-simple example

Expected Behavior

Service should spinn up with hitting dapr run --app-id python-subscriber --protocol grpc --app-port 50051 python subscriber.py

Actual Behavior

It throws following error when we try to spin up subscriber.py

== APP == Exception calling application: Protocol message ListTopicSubscriptionsResponse has no "topics" field.

== APP == Traceback (most recent call last):

== APP ==   File "/home/girish/Documents/dapr/dapr-samples/python-sdk/examples/pubsub-simple/venv/lib/python3.8/site-packages/grpc/_server.py", line 435, in _call_behavior

== APP ==     response_or_iterator = behavior(argument, context)

== APP ==   File "subscriber.py", line 21, in ListTopicSubscriptions

== APP ==     return appcallback_v1.ListTopicSubscriptionsResponse(topics=['TOPIC_A'])

== APP == ValueError: Protocol message ListTopicSubscriptionsResponse has no "topics" field.

Complete logs
dapr_logs.txt

Steps to Reproduce the Problem

Run dapr run --app-id python-subscriber --protocol grpc --app-port 50051 python subscriber.py from examples/pubsub-simple

Settings & configurations

• using standalone dapr
• tried with both dapr & dapr-dev
• CLI version: 0.8.0
• Runtime version: 0.8.0

Auto-generate python docs via sphinx

Describe the proposal

• Add more code level comments
• Generate sphinx docs

related to #40

Replace all deprecated cli commands and k8s annotations

Describe the proposal

Replace all deprecated cli commands and k8s annotations. Part of this parent epic.

Support default json serializer in DaprClient and add .json helper

Describe the proposal

In the convenient layer spec, DaprClient doesn't have the default json serializer, but let users to handle JSON serialization. Like the other python pkgs, such as flask, requests, etc, DaprClient can have default json serializer to serialize and deserialize python dict type or primitive objects and provide .json() helper to DaprResponse and DataContent in AppCallback Layer. It will improve development experience instead of serializing and deserializing data object.

Use flask-dapr pkg in actor example

Describe the proposal

Use flask-dapr pkg in actor example

Add reminder to actor sdk

Describe the proposal

feature branch: https://github.com/dapr/python-sdk/tree/features/actor

Add reminder to actor sdk.

Split flask_dapr package from dapr package

Describe the proposal

Split flask_dapr package from dapr package

• create setup.py and setup.cfg for flask_dapr
• configure ci to build flask_dapr package

Add scenario based application examples

Describe the proposal

Add more realistic application examples which can show the capabilities of the convenient layer.

Example - Invoke/OnInvoke Method

Describe the proposal

Adding an example on how to utilize the OnInvoke method from the proto bindings. This example helps users on how to create a gRPC server that can listen on direct method invocations.

Improve README to include more detail information

Describe the proposal

Improve README.md and convert it to rst format.

related to #40

Improve code comments to generate sphinx doc

Describe the proposal

Improve code comments

Add K8s deployment yaml for demo actor example

Describe the proposal

Add K8s deployment yaml for demo actor example

Test compatibility between python sdk and dotnet sdk

Describe the proposal

Test compatibility between python sdk and dotnet sdk

Add a script to generate gRPC client code/stubs from dapr proto files

Describe the proposal

Add a script file to generate gRPC client code/stubs from dapr proto files by installing necessary tooling, plugins for python.

Please mark Dapr-dev Python >= 3.7 instead of 3.8

Describe the proposal

I would like to see Python >= 3.7 supported instead of 3.8 (which is cutting edge), seeing that a lot of libraries are still depending on 3.7 at the moment.

Note: Currently I manually clone and install it locally and it's working on Python 3.7 for me

Support DAPR_API_TOKEN

Describe the proposal

This is to support dapr api token authentication. - Reference: https://github.com/dapr/docs/blob/master/howto/enable-dapr-api-token-based-authentication/README.md

1. read DAPR_API_TOKEN environment variable
2. set dapr-api-token metadata in each api call.

Use grpc client interceptor.

Convenient layer for gRPC appcallback

Describe the proposal

Create AppCallback wrapper and decorator.

pip for dapr-client still in version 0.1, need to publish 0.2 and then 0.3

Expected Behavior

pip install dapr-client should install the latest version (0.2 now)

Actual Behavior

installs the first version 0.1

Steps to Reproduce the Problem

pip search dapr-client

or

open https://pypi.org/project/dapr-client/

Update auto-generated python sdk for changes in proto files(updated for multi store feature)

Describe the proposal

Update auto-generated sdk for changes in proto files(multi store feature)

[Proposal] Using Dapr and Python-sdk for ML/DL model deployment

Describe the proposal

The ultimate goal of ML and DL model development is to integrate with the production services. Typically, ML and DL engineers focus on model development rather than how they will integrate model with their services.

Many articles and tutorials introduces how to serve the model with Flask. Once they have Flask service with their model to their production, the backend and frontend services need to implement client code to talk to model server.

Dapr and Python-sdk will be able to resolve this problem easier. So ML engineer creates micro service and deploy it to production cluster without the knowledge of http restful api server and microservice deployment. Backend engineer can use service invocation api to use the model without model server client implementation.

Add Statestore apis to convenient layer

Describe the proposal

Add Statestore apis to convenient layer.

Parent epic : #67

• Implementation
• Unit-tests
• Documentation in code
• update examples

Update SDK for activate changes

Changes are needed in the SDK corresponding to runtime changes here (see bottom of that issue for proposal):
dapr/dapr#1038

Reference for corresponding change in dapr/dotnet-sdk:
dapr/dotnet-sdk#313

Convenient layer for gRPC APIs

Describe the proposal

Create convenient layer for gRPC API. This is to create the convenient layer for gRPC API.

Reference:
JAVA SDK - https://github.com/dapr/java-sdk/blob/master/sdk/src/main/java/io/dapr/client/DaprClient.java
https://github.com/dapr/java-sdk/blob/master/sdk/src/main/java/io/dapr/client/DaprClientGrpc.java

Location: dapr/clients/grpc

Add timer to actor

Describe the proposal

feature branch: https://github.com/dapr/python-sdk/tree/features/actor

Add timer to actor sdk.

Add one requirments.txt file in the root of python-sdk

Describe the proposal

Add one requirments.txt file in the root of python-sdk
It is required for python pkg installation.

Update proto client for 0.8.0

Describe the proposal

Update proto client for 0.8.0

Serialize state value without base64 encoding

Describe the proposal

Serialize state value without base64 encoding

Publish python sdk doc to github page

Describe the proposal

Publish auto-generated Python SDK doc to GitHub page.

Add publish api to convenient layer

Describe the proposal

Add publish api to convenient layer

Add Secret api to convenient layer

Describe the proposal

Add Secrets apis to convenient layer.

Parent epic : #67

• Implementation
• Unit-tests
• Documentation in code
• update or add examples

Allow to pass W3C trace headers to Dapr Api calls

Allow to pass W3C trace header(traceparent) or grpc-trace-bin to Dapr Api call.

0.8.0 supports traceparent and grpc-trace-bin. Python-sdk needs the way to pass traceparent for http or grpc-trace-bin metadata for grpc.

RELEASE NOTE: ADD W3C trace headers to Dapr API calls.

Add Invoke service api to convenient layer

Describe the proposal

Add Invoke service api to convenient layer.

Parent epic : #67

• Implementation
• Unit-tests
• Documentation in code
• update examples

Add state management to python actor-sdk

Describe the proposal

Add state management to python actor-sdk

Sphinx Auto-generation of Documentation

Describe the proposal

Sphinx can be used to generate documentation automatically with the autodoc extension. Napoleon extension can be used to process the Google comment style.

Protobuf client update for python-sdk

Describe the proposal

This is to update protobuf client along with dapr/dapr#1409

Update pubsub example as per new proto apis

Describe the proposal

Current Pub/Sub example doesn't use update proto APIs. I propose to update the example and adapt it to the changes.

(A PR will follow shortly 😃 )

Add a Pubsub Example

Proposal

Add an example (similar to invoke-simple ) showing pubsub pattern.

Why to add?

Pubsub is a very essential feature and it's hard to find a template to make a subscriber.

Add Kubernetes Example

Add an example for deploying on Kubernetes

Implement Dapr http callback to fastapi extension

Describe the proposal

This is to implement the similar attributes in asp.net integration for http appcallback as a flask extension.

Cannot import module "daprclient" in `daprclient_pb2_grpc.py`

When trying to import daprclient_pb2_grpc.py to access the OnInvoke method, an error is thrown. Without this, a correct example utilizing the OnInvoke method can not be created, disallowing the E2E communication of 2 RPC microservices (client and server).

It seems that on the source repo the following proto files are found:

• https://github.com/dapr/dapr/blob/master/pkg/proto/daprclient/daprclient.proto
• https://github.com/dapr/dapr/blob/master/pkg/proto/dapr/dapr.proto

It might be that there is some wrong packaging behavior happening?

Expected Behavior

This should work to allow the creation of the OnInvoke method.

Actual Behavior

== APP == Traceback (most recent call last):
== APP ==   File "server.py", line 7, in <module>
== APP ==     from dapr import daprclient_pb2_grpc as servicesClient
== APP ==   File "C:\Python37\lib\site-packages\dapr\daprclient_pb2_grpc.py", line 4, in <module>
== APP ==     from daprclient import daprclient_pb2 as daprclient_dot_daprclient__pb2
== APP == ModuleNotFoundError: No module named 'daprclient'

Steps to Reproduce the Problem

1. Create a new Python file (e.g. server.py)
2. Add line from dapr import daprclient_pb2_grpc as services
3. Run the application (e.g. dapr run --app-id python-grpc --port 3500 --protocol grpc --grpc-port 50001 python server.py)

Add appcallback versioning to HTTP APIs

Expected Behavior

Add appcallback versioning to HTTP APIs. Make necessary changes to python-sdk and samples
Related to: dapr/dapr#1678

Actor Implementation

I currently do not see a way to implement actor methods, is this already supported in the Python SDK? 😊

Add binding apis to convenient layer

Describe the proposal

Add binding apis to convenient layer.

Parent epic : #67

• Implementation
• Unit-tests
• Documentation in code
• update examples

Add support for async gRPC in Dapr SDK

Describe the proposal

grpc_asyncio is still WIP. We may need to support it later when grpc_asyncio is stable.
https://grpc.github.io/grpc/python/grpc_asyncio.html

Update: grpc_asyncio is now available, however we should publish an asynchronous Dapr Python SDK as a separate SDK, or at least sub package - like dapr.aio (from dapr.aio.clients import DaprClient)

There were concerns from several users about the performance overhead of calling asynchronous methods and running them in a synchronous context, so simply switching over to grpc.aio is not an option unfortunately.

Remove extra return statement

Description

There is an unnecessary return statement in line 32 of /invoke-simple/invoke-receiver.py

python-sdk/examples/invoke-simple/invoke-receiver.py

Line 32 in 2a6701d

return response

How to fix

Removing lines 31-32 will do just fine.