Giter Site home page Giter Site logo

mcyph / speedysvc Goto Github PK

View Code? Open in Web Editor NEW
7.0 3.0 1.0 1.57 MB

Python client-server microservices using the fastest means possible - shared memory. Communication 10-30x faster than almost any other similar python library

Shell 0.09% Python 88.21% JavaScript 1.99% HTML 1.88% CSS 0.10% Cython 7.72%
shared-memory ipc python rpc microservice-framework client-server

speedysvc's Introduction

About

NOTE: speedysvc's status is alpha - it should be functional and don't intend to make major changes, but the web management and configuration interface API might be changed later. It's actively maintained and am using it for my https://langlynx.com service but there may well be bugs!

Mac OSX is not currently supported, though there I plan to investigate whether this is feasible.

This module for python 3.6+ on Linux and Windows allows separating larger, more complex applications into smaller components (microservices). For example, you could build a text-to-speech service, an optical character recognition service, etc. Then, you could have web frontends or other clients communicate with these services. This can often require less RAM, as there is no need to load the components in every web worker process.

Unlike other similar modules for client-server communication (which typically use sockets or HTTP REST), this module uses local shared memory, which typically performs around 10-30 times faster with much lower latency. speedysvc servers can also be remotely communicated with over the network using TCP sockets. The TCP protocol is fast and efficient, and optionally compresses traffic using snappy/zlib to minimise bandwidth.

In order to work around limitations of python's Global Interpreter Lock which normally restricts applications from using more than a single CPU core, services can have multiple worker processes. They optionally can increase or decrease worker processes as needed, depending on CPU usage. This helps to make sure server resources are more effectively used. While this is possible with the multiprocessing module, multiprocessing on Linux uses pipe2 for communication between processes, which is much slower than using shared memory.

There is a service management web interface that shows logs/performance data for each service and allows stopping/starting services individually. Each service is started separately in individual processes with individual logs so as to be able to track down events and errors without being overwhelmed with too much information.

docs/web_index_screenshot.png

See also Implementation Considerations.

Requirements

The following are needed prior to installation:

  • OS: Windows or Linux (tested on Ubuntu 18.04 LTS, other POSIX systems other than Mac OS X may also be supported)
  • Python: version 3.6 or above
  • Dependencies for Windows only: You must first download the "Visual Studio Build Tools 2019" from https://visualstudio.microsoft.com/downloads/

speedysvc also has the below dependencies, but these should be installed automatically with the pip3 install command in the "install" section.

  • Module Dependencies: cython, msgpack, cherrypy, psutil
  • Module Dependencies for Linux/POSIX only: posix_ipc

Install

Then, for both Windows or Linux, type the following commands into a command prompt/terminal window:

pip3 install --upgrade pip
pip3 install Cython
pip3 install git+https://github.com/mcyph/speedysvc

See also Install/Dependencies.

Quick Example

echoserver.py:

from speedysvc import ServerMethodsBase, json_method
from speedysvc.client_server.base_classes.ServerMethodsBase import ServerMethodsBase

class EchoServer(ServerMethodsBase):
    port = 5555
    name = 'echo_serv'

    @json_method
    def echo_json(self, data):
        return data

echoclient.py:

from speedysvc import connect
from speedysvc.client_server.base_classes.ClientMethodsBase import ClientMethodsBase
from echoserver import EchoServer

class EchoClient(ClientMethodsBase):
    def echo_json(self, data):
        return self.send(EchoServer.echo_json, [data])

if __name__ == '__main__':
    # Note: Replace 'shm://' with 'tcp://(host)' for remote services
    methods = EchoClient(connect(EchoServer, 'shm://'))
    print(methods.echo_json("Hello World!"))

service.ini:

[defaults]
# Uncomment this line to listen on the network
#bind_tcp=(host adaptor)
log_dir=/tmp/test_server_logs/

[EchoServer]
import_from=echoserver
max_proc_num=3
min_proc_num=3

Then type python3 -m speedysvc.service service.ini & from the same directory to start the server. The web management interface will start on http://127.0.0.1:5155, where you can monitor the status and logs of the server.

Then, type python3 echoclient.py to test a connection to the server.

See Example for a a more complex example.

See Also

Similar projects:

  • Ray (for running distributed applications)

TODO

  • Allow running services as Docker containers
  • Better log searching/filtering
  • REST requests

More info

License

Licensed under the MIT License.

Copyright 2020 Dave Morrissey

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

speedysvc's People

Contributors

mcyph avatar

Stargazers

Gil Martins avatar Lucas Campos avatar Breeze Chen avatar  avatar Julian Brendl avatar  avatar  avatar

Watchers

Gil Martins avatar  avatar  avatar

Forkers

webclinic017

speedysvc's Issues

Great work - some OS concerns though

Honestly, this looks seriously awesome, I'm surprised I couldn't find anything else anywhere like this. My only feedback is that it only works on Linux. People might deploy their apps on Linux, but more often then not they develop on mac/windows. I would probably use this on our own project, but I don't want to make our project solely possible to work with on linux :/ . I've been looking into making it buildable on OSX at least.

Alternately, making a TCP only version such that it will work on windows / OSX could already be all that is needed, since then at the very least the code works (or building the pip-package in a TCP only version).

I realize it's obviously not easy to get shm to run on windows, but it might be nice to have an example for how to set it up with the windows subsystem for linux for those guys.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.