Giter Site home page Giter Site logo

dcrf-client's Introduction

dcrf-client

npm version

This package aims to provide a simple, reliable, and generic interface to consume Django Channels REST Framework powered WebSocket APIs.

NOTE: This library is a TypeScript port of channels-api-client to support Django Channels v2, and @hishnash's port of linuxlewis's channels-api: djangochannelsrestframework and channelsmultiplexer.

Features

  • Promises encapsulating the request/response cycle
  • Subscribe to updates with a callback
  • Automatically reconnect when connection is broken (with backoff — thanks to reconnecting-websocket)
  • Automatically restart subscriptions on reconnection
  • Requests are queued until a connection is made (no need to wait for connection before sending requests)

Install

npm install dcrf-client

Usage

const dcrf = require('dcrf-client');
const client = dcrf.connect('wss://example.com');

client.create('people', {name: 'Alex'}).then(person => {
  console.info('Created:', person);
});

client.retrieve('people', 4).then(person => {
  console.info('Retrieved person 4:', person);
});

client.update('people', 4, {name: 'Johannes', address: '123 Easy St'}).then(person => {
  console.info('Overwrote person 4. Properties after change:', person);
});

client.patch('people', 4, {name: 'Jefe'}).then(person => {
  console.info('Changed name of person 4. Properties after change:', person);
});

client.delete('people', 4).then(() => {
  console.info('Deleted person 4. No one liked them, anyway :)');
});


// Subscribe to updates to person 1
const personalSubscription = client.subscribe('people', 1, (person, action) => {
  if (action === 'update') {
    console.info('Person 1 was updated:', person);
  }
  else if (action === 'delete') {
    console.info('Person 1 was deleted!');
  }
});

// Stop listening for updates
personalSubscription.cancel();


// Make a generic request to a multiplexer stream
client.request('mystream', {key: 'value'}).then(response => {
  console.info('Got mystream response, yo:', response);
});

// Subscribe using a custom action
const customSubscription = client.subscribe(
  'people',
  {},  // Additional arguments may be passed to action
  (person, action) => {
    if (action === 'create') {
      console.info(`Person ${person.pk} was created:`, person);
    }
    else if (action === 'update') {
      console.info(`Person ${person.pk} was updated:`, person);
    }
    else if (action === 'delete') {
      console.info(`Person ${person.pk} was deleted!`);
    }
  },
  {
    includeCreateEvents: true,
    subscribeAction: 'subscribe_all',
    unsubscribeAction: 'unsubscribe_all',
  },
);

Configuration

The client can be customized by passing an object as the second argument to connect() or createClient(). The available options are described below.

const dcrf = require('dcrf-client');

const client = dcrf.connect('wss://example.com', {
  /**
   * Options to pass along to ReconnectingWebsocket
   *
   * See https://github.com/pladaria/reconnecting-websocket#available-options for more info
   */
  websocket: {
    WebSocket?: any; // WebSocket constructor, if none provided, defaults to global WebSocket
    maxReconnectionDelay?: number; // max delay in ms between reconnections
    minReconnectionDelay?: number; // min delay in ms between reconnections
    reconnectionDelayGrowFactor?: number; // how fast the reconnection delay grows
    minUptime?: number; // min time in ms to consider connection as stable
    connectionTimeout?: number; // retry connect if not connected after this time, in ms
    maxRetries?: number; // maximum number of retries
    maxEnqueuedMessages?: number; // maximum number of messages to buffer until reconnection
    startClosed?: boolean; // start websocket in CLOSED state, call `.reconnect()` to connect
    debug?: boolean; // enables debug output
  },

  /**
   * Name of serializer field is used to identify objects in subscription event payloads.
   *
   * Default: 'pk'
   */
  pkField: 'id',

  /**
   * Whether to ensure subscription delete event payloads store the primary key of the object
   * in the configured `pkField`, instead of the default 'pk'.
   *
   * Because subscription delete payloads aren't run through the configured serializer (as the
   * objects do not exist), the DCRF backend must pick a field to store the primary key of the
   * object in the payload. DCRF chooses 'pk' for this field. If `pkField` is *not* 'pk' (and is
   * instead, say, 'id'), then subscription update payloads will return `{id: 123}`, but delete
   * payloads will return `{pk: 123}`.
   *
   * To address the potential inconsistencies between subscription update and delete payloads,
   * setting this option to true (default) will cause dcrf-client to replace the 'pk' field with
   * the configured `pkField` setting.
   *
   * Default: true
   */
  ensurePkFieldInDeleteEvents: true,

  /**
   * Customizes the format of a multiplexed message to be sent to the server.
   *
   * In almost all circumstances, the default behaviour is usually desired.
   *
   * The default behaviour is reproduced here.
   */
  buildMultiplexedMessage(stream: string, payload: object): object {
    return {stream, payload};
  },

  /**
   * Customizes the selector (a pattern matching an object) for the response to an API request
   *
   * In almost all circumstances, the default behaviour is usually desired.
   *
   * The default behaviour is reproduced here.
   */
  buildRequestResponseSelector(stream: string, requestId: string): object {
    return {
      stream,
      payload: {request_id: requestId},
    };
  },

  /**
   * Customizes the selector (a pattern matching an object) matching a subscription update event for
   * an object.
   *
   * In almost all circumstances, the default behaviour is usually desired.
   *
   * The default behaviour is reproduced here.
   */
  buildSubscribeUpdateSelector(stream: string, pk: number, requestId: string): object {
    return {
      stream,
      payload: {
        action: 'update',
        data: {[this.pkField]: pk},
        request_id: requestId,
      },
    };
  },

  /**
   * Customizes the selector (a pattern matching an object) matching a subscription delete event for
   * an object.
   *
   * In almost all circumstances, the default behaviour is usually desired.
   *
   * The default behaviour is reproduced here.
   */
  buildSubscribeDeleteSelector(stream: string, pk: number, requestId: string): object {
    return {
      stream,
      payload: {
        action: 'delete',
        data: {pk},
        request_id: requestId,
      },
    };
  },

  /**
   * Customizes the payload sent to begin subscriptions
   *
   * In almost all circumstances, the default behaviour is usually desired.
   *
   * The default behaviour is reproduced here.
   */
  buildSubscribePayload(pk: number, requestId: string): object {
    return {
      action: 'subscribe_instance',
      request_id: requestId,
      pk,  // NOTE: the subscribe_instance action REQUIRES the literal argument `pk`.
           //       this argument is NOT the same as the ID field of the model.
    };
  },

  preprocessPayload: (stream, payload, requestId) => {
    // Modify payload any way you see fit, before it's sent over the wire
    // For instance, add a custom authentication token:
    payload.token = '123';
    // Be sure not to return anything if you modify payload

    // Or, you can overwrite the payload by returning a new object:
    return {'this': 'is my new payload'};
  },

  preprocessMessage: (message) => {
    // The "message" is the final value which will be serialized and sent over the wire.
    // It includes the stream and the payload.

    // Modify the message any way you see fit, before its sent over the wire.
    message.token = 'abc';
    // Don't return anything if you modify message

    // Or, you can overwrite the the message by returning a new object:
    return {stream: 'creek', payload: 'craycrayload'};
  },
});

Development

There are two main test suites: unit tests (in test/test.ts) to verify intended behaviour of the client, and integration tests (in test/integration/tests/test.ts) to verify the client interacts with the server properly.

Both suites utilize Mocha as the test runner, though the integration tests are executed through py.test, to provide a live server to make requests against.

The integration tests require separate dependencies. To install them, first install pipenv, then run pipenv install --dev.

To run both test suites: npm run test

To run unit tests: npm run test:unit or mocha

To run integration tests: npm run test:integration or pipenv run py.test

How do the integration tests work?

pytest provides a number of hooks to modify how tests are collected, executed, and reported. These are utilized to discover tests from a Mocha suite, and execute them on pytest's command.

Our pytest-mocha plugin first spawns a subprocess to a custom Mocha runner, which collects its own TypeScript-based tests and emits that test info in JSON format to stdout. pytest-mocha reads this info and reports it to pytest, allowing pytest to print out the true names from the Mocha suite. Using deasync, the Mocha process waits for pytest-mocha to send an acknowledgment (a newline) to stdin before continuing.

pytest-mocha then spins up a live Daphne server for the tests to utilize. Before each test, the Mocha suite emits another JSON message informing pytest-mocha which test is about to run. pytest-mocha replies with the connection info in JSON format to the Mocha runner's stdin. The Mocha suite uses this to initialize a DCRFClient for each test.

At the end of each test, our custom Mocha runner emits a "test ended" message. pytest-mocha then wipes the database (with the help of pytest-django) for the next test run.

(Note that technically, Mocha's "test end" event is somewhat misleading, and isn't used directly to denote test end. Mocha's "test end" demarcates when the test method has completed, but not any afterEach hooks. Since we use an afterEach hook to unsubscribe all subscriptions from the DCRFClient, care must be taken to ensure the DB remains unwiped and test server remains up until the afterEach hook has culminated. To this end, we actually emit our "test ended" message right before the next test starts, or the suite ends. See mochajs/mocha#1860. The logic is inspired by the workaround used in JetBrains's mocha-intellij)

NOTE: this is sorta complicated and brittle. it would be nice to refactor this into something more robust. at least for now it provides some assurance the client interacts with the server properly, and also sorta serves as an example for properly setting up a Django Channels REST Framework project.

dcrf-client's People

Contributors

jhillacre avatar they4kman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

dcrf-client's Issues

Problem running tests

I was looking at making a pull request with a feature I'd like to see in dcrf-client. I started by looking at following the development section from the readme to get tests running. I am running into some minor issues I was able to deal with and one last issue I'm not sure how to work around :

  • The readme states to start with pipenv install. pipenv does not install dev packages by default, which seem to be required to run tests. This could be addressed by directing contributors to pipenv install --dev. Not sure what use dev-packages are in the context of dcrf-client, is it seperated for example reasons?

  • Running npm run test next as in the readme doesn't work because py.test is not in the path yet. pipenv shell needs to be run first or instead run pipenv run npm run test.

  • Running npm run test in a pipenv shell for python 3.7 doesn't work with a traceback of

    > [email protected] test:integration /home/joel/WebstormProjects/dcrf-client
    > py.test || true
    
    Traceback (most recent call last):
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/bin/py.test", line 5, in <module>
        from pytest import console_main
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pytest/__init__.py", line 3, in   <module>
        from . import collect
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pytest/collect.py", line 8, in   <module>
        from _pytest.deprecated import PYTEST_COLLECT_MODULE
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/deprecated.py", line 11,   in <module>
        from _pytest.warning_types import PytestDeprecationWarning
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/warning_types.py", line 7, in <module>
        from _pytest.compat import final
      File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/compat.py", line 57, in   <module>
        import importlib_metadata  # noqa: F401
    ModuleNotFoundError: No module named 'importlib_metadata'
    

    This seems related to this pytest issue: pytest-dev/pytest#7114. The resolution seems to be to regenerate the Pipfile.lock per: pytest-dev/pytest#7114 (comment). I did this using pipenv lock; pipenv install --dev.

  • At this point, the unit tests now run for me, but there is an issue I'm not familiar with running the integration tests. Is it possible a pytest version that is too new is specified in the Pipfile.lock file?

    (dcrf-client-CumnqgWw) [joel@adamant dcrf-client]$ npm run test:integration
    > > [email protected] test:integration /home/joel/WebstormProjects/dcrf-client
    > py.test || true
    
    =================================================================================== test session starts ====================================================================================
    platform linux -- Python 3.7.9, pytest-6.2.1, py-1.10.0, pluggy-0.13.1 -- /home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/bin/python3
    cachedir: .pytest_cache
    django: settings: dcrf_client_test.settings (from ini)
    rootdir: /home/joel/WebstormProjects/dcrf-client, configfile: pytest.ini
    plugins: django-4.1.0, pythonpath-0.7.3
    collecting ... Using selector: EpollSelector
    Expecting event from Mocha of type(s): collect
    Read event from Mocha: {'type': 'collect', 'tests': [{'title': 'returns created values', 'parents': ['DCRFClient', 'things', 'create', 'returns created values'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'imbues retrieve with data', 'parents': ['DCRFClient', 'things', 'create', 'imbues retrieve with data'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'returns empty set', 'parents': ['DCRFClient', 'things', 'list', 'returns empty set'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'returns created rows', 'parents': ['DCRFClient', 'things', 'list', 'returns created rows'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'invokes callback on change', 'parents': ['DCRFClient', 'things', 'subscribe', 'invokes callback on change'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'invokes callback on delete', 'parents': ['DCRFClient', 'things', 'subscribe', 'invokes callback on delete'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'returns created values', 'parents': ['DCRFClient', 'things_with_id', 'create', 'returns created values'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'imbues retrieve with data', 'parents': ['DCRFClient', 'things_with_id', 'create', 'imbues retrieve with data'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'returns empty set', 'parents': ['DCRFClient', 'things_with_id', 'list', 'returns empty set'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'returns created rows', 'parents': ['DCRFClient', 'things_with_id', 'list', 'returns created rows'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'invokes callback on change', 'parents': ['DCRFClient', 'things_with_id', 'subscribe', 'invokes callback on change'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}, {'title': 'invokes callback on delete', 'parents': ['DCRFClient', 'things_with_id', 'subscribe', 'invokes callback on delete'], 'file': '/home/joel/WebstormProjects/dcrf-client/test/integration/tests/test.ts'}]}
    INTERNALERROR> Traceback (most recent call last):
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/main.py", line 269, in wrap_session
    INTERNALERROR>     session.exitstatus = doit(config, session) or 0
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/main.py", line 322, in _main
    INTERNALERROR>     config.hook.pytest_collection(session=session)
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    INTERNALERROR>     return outcome.get_result()
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    INTERNALERROR>     raise ex[1].with_traceback(ex[2])
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    INTERNALERROR>     res = hook_impl.function(*args)
    INTERNALERROR>   File "/home/joel/WebstormProjects/dcrf-client/test/integration/tests/mocha.py", line 179, in pytest_collection
    INTERNALERROR>     session=session,
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/nodes.py", line 91, in __call__
    INTERNALERROR>     fail(msg, pytrace=False)
    INTERNALERROR>   File "/home/joel/.local/share/virtualenvs/dcrf-client-CumnqgWw/lib/python3.7/site-packages/_pytest/outcomes.py", line 153, in fail
    INTERNALERROR>     raise Failed(msg=msg, pytrace=pytrace)
    INTERNALERROR> Failed: Direct construction of MochaFile has been deprecated, please use MochaFile.from_parent.
    INTERNALERROR> See https://docs.pytest.org/en/stable/deprecations.html#node-construction-changed-to-node-from-parent for more details.
    
    ================================================================================== no tests ran in 0.00s ===================================================================================
    sys:1: ResourceWarning: unclosed file <_io.BufferedWriter name=15>
    sys:1: ResourceWarning: unclosed file <_io.BufferedReader name=16>
    (dcrf-client-CumnqgWw) [joel@adamant dcrf-client]$
    

Pagination Support

A merged djangochannelsrestframework pull request, NilCoalescing/djangochannelsrestframework#71, adds pagination support. It would be nice if dcrf-client got updated to provide handling of this. (Also, see NilCoalescing/djangochannelsrestframework#76)

With a StreamedPaginatedListMixin based Consumer, the response is broken up into pages, but the server does not wait for us to request pages. Therefore, to accommodate listing a StreamedPaginatedListMixin based consumer, the list operation should take a callback, similar to subscribe, where the library user can handle one page at a time. Additionally, we may need users, or djangochannelsrestframework, to have a custom paginator that returns the expected number of pages, so dcrf-client knows when to clean up the listener.

Listing a PaginatedListMixin based Consumer should work with dcrf-client as is, although it leaves going through all the pages to the library user. The listing callback could provide a function for library users to get the next page if there is one.

🆘 Unable to catch subscription payload

Greeting !

i am trying to log my stream subscription payload to the console but i can't seem to make it work,

class CacheSocket {
    static client: DCRFClient;

    static getClient(path?: string) {
            if (!path) {
                path = 'ws://localhost:8000/cache/';
            }
            if (!CacheSocket.client) {
                CacheSocket.client = dcrf.connect(path, {
                    websocket: { minReconnectionDelay: 2000 }
                });
            }
    
            return CacheSocket.client;
        }
    }
}

export default class CacheComponent extends React.Component {

    loadSocket = async () => {
        const _client = CacheSocket.getClient();
        const subscription = _client.subscribe(
            'agent',
            {
                // Additional arguments may be passed to action
            },
            (payload, action) => console.log({ payload, action }), // Not Working ...
            {
                requestId: String(uuidv4()),
                includeCreateEvents: true,
                subscribeAction: 'subscribe_to_agent_activity',
                unsubscribeAction: 'unsubscribe_to_agent_activity',
            },
        );

    };

    componentDidMount() {
        this.loadSocket();
    };

    render() {
        return <>{this.props.children}</>;
    }
}

Everything is working perfectly, i can see the stream response in the browser in the Network tab, but i can't log the payload onto the console ...

Serializers without pk field do not trigger subscription callbacks

I'm trying to subscribe to a specific model instance using something similar to this:

const taskSubscription = this.$dcrf.subscribe('url', this.id, () => {
    console.log('called!')
})

I can see that a subscribe request is sent

{
  "stream":"url",
  "payload":{
    "action":"subscribe_instance",
    "request_id":"25f8361e-c627-4b1e-8142-e0214b8b3999",
    "pk":"some-pk-value"
  }
}

and the server sends a reply as follows:

{
  "stream":"url",
  "payload":{
    "errors":[

    ],
    "data":null,
    "action":"subscribe_instance",
    "response_status":201,
    "request_id":"25f8361e-c627-4b1e-8142-e0214b8b3999"
  }
}

Updates are sent down by the server similar to this message:

{
  "stream":"task-status",
  "payload":{
    "errors":[
    ],
    "data":{
--- snip ---
    },
    "action":"update",
    "response_status":200,
    "request_id":"25f8361e-c627-4b1e-8142-e0214b8b3999"
  }
}

To me it seems the messages are correctly formatted, yet the callback is never triggered. It seems that this only happens for update requests as delete requests correctly trigger the callback.

edit: There seem to be several issues at play here.

  1. request_id seem to be changing between the payload and selector in the FifoDispatcher so they are never isMatch-ed to each other. Generating request_ids before calling subscribe does solve this issue.
  2. A general design issue: DCRFClient uses pk as the lookup value when REST framework will usually use id. Adding pk and id to Meta.fields solves this in a hacky way.

Tree shake support

Split functions as build components to enable tree shake, I'm currently working on recreating the library to support tree shake, will you prefer that?

Framework does not work with default or customized id fields

The problem

I just had the issue in a django project with a few defined models, some with an explicitly set primary key (pk), some without one (which have then the default pk field, which is just the auto incrementing integer field then). The corresponding serializer was just set to serialize all fields by stating fields = "__all__".
I could then connect via this and the djangochannelrestframework framework and set the callback in the subscribe call on the client. When updating the entity related to the subscribe call, I could see in the network tab, that the update was sent correctly, but the callback was never called.

The solution

After hours and hours of reading the documentation first and then crawling through the source code, I discovered that the problem was the isMatch(payload, selector) call in the fifo.ts file.
The call requires, as one can see in the build of the selector

protected static buildSubscribeSelector(stream: string, pk: number, requestId: string) {
    return {
      stream,
      payload: {
        data: {pk},
        request_id: requestId,
      },
    };
  }

the primary key of the entity to be named pk. However, apart from not mentioning that anywhere in the README file, the default name for the primary key when using the django rest framework is not pk, but id. Moreover, this is a huge inflexibility as this requires the models to be shaped in a special way, without supporting customized primary key fields.
Therefore, I'd suggest

  1. using id as the default name for comparing the primary key, so that it works out of the box,
  2. make the name of the id field customizable for the developer and
  3. mention all of that in the documentation/README

Cannot import module

I am having issues using the module with Vue+Quasar+Vite.
If I try importing the module with:
import { DCRFPlugin } from 'plugins/djangochannelsrestframework'

I get the following error:

has-flag.js:30 Uncaught TypeError: Cannot read properties of undefined (reading 'indexOf')
    at module.exports (has-flag.js:30:28)
    at node_modules/@colors/colors/lib/system/supports-colors.js (supports-colors.js:34:5)
    at __require (chunk-OL3AADLO.js?v=55207a85:9:50)
    at node_modules/@colors/colors/lib/colors.js (colors.js:41:24)
    at __require (chunk-OL3AADLO.js?v=55207a85:9:50)
    at node_modules/@colors/colors/safe.js (safe.js:9:14)
    at __require (chunk-OL3AADLO.js?v=55207a85:9:50)
    at node_modules/logform/dist/colorize.js (colorize.js:9:14)
    at __require (chunk-OL3AADLO.js?v=55207a85:9:50)
    at node_modules/logform/dist/levels.js (levels.js:3:16)
module.exports	@	has-flag.js:30
node_modules/@colors/colors/lib/system/supports-colors.js	@	supports-colors.js:34
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/@colors/colors/lib/colors.js	@	colors.js:41
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/@colors/colors/safe.js	@	safe.js:9
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/logform/dist/colorize.js	@	colorize.js:9
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/logform/dist/levels.js	@	levels.js:3
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/logform/dist/browser.js	@	browser.js:17
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/winston/dist/winston.js	@	winston.js:9
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/dcrf-client/lib/logging.js	@	logging.js:4
__require	@	chunk-OL3AADLO.js?v=55207a85:9
node_modules/dcrf-client/lib/index.js	@	index.js:35
__require	@	chunk-OL3AADLO.js?v=55207a85:9
(anonymous)	@	dep:dcrf-client:1

The project uses there dependencies:

{
  "scripts": {
    "lint": "eslint --ext .js,.ts,.vue ./",
    "format": "prettier --write \"**/*.{js,ts,vue,scss,html,md,json}\" --ignore-path .gitignore",
    "test": "echo \"No test specified\" && exit 0",
    "dev": "quasar dev",
    "build": "quasar build"
  },
  "dependencies": {
    "@quasar/extras": "^1.0.0",
    "axios": "^1.2.1",
    "dcrf-client": "^1.1.0",
    "logform": "^2.5.1",
    "pinia": "^2.0.11",
    "quasar": "^2.6.0",
    "vue": "^3.0.0",
    "vue-i18n": "^9.2.2",
    "vue-router": "^4.0.0"
  },
  "devDependencies": {
    "@intlify/vite-plugin-vue-i18n": "^3.3.1",
    "@quasar/app-vite": "^1.0.0",
    "@types/node": "^12.20.21",
    "@typescript-eslint/eslint-plugin": "^5.10.0",
    "@typescript-eslint/parser": "^5.10.0",
    "autoprefixer": "^10.4.2",
    "eslint": "^8.10.0",
    "eslint-config-prettier": "^8.1.0",
    "eslint-plugin-vue": "^9.0.0",
    "prettier": "^2.5.1",
    "typescript": "^4.5.4"
  },
  "engines": {
    "node": "^18 || ^16 || ^14.19",
    "npm": ">= 6.13.4",
    "yarn": ">= 1.21.1"
  }
}

Any idea what this could be and how to solve it? It seems to be related to the logging functionality and winston. I am using yarn for package management.

Add support for generic model subscriptions

Though DCRF doesn't have a standardized way of subscribing to all updates to any instance in a stream, it does offer support for this type of subscription, handing off implementation responsibility to the developer.

It would be nice if dcrf-client could support these types of arbitrary subscriptions, i.e.

  • Send an arbitrary request (e.g. subscribe_all, or whatever action name is chosen)
  • Add a dispatch listener, using a selector of the developer's choosing (e.g. {action: 'update', data: {...}})
  • Automatically resubscribe upon reconnection

subscribe() callback not working

I have this code in a VueJS project:

this.subscription = dcrf.subscribe("projects", 1, (project, action) => { if (action === "update") { console.info("Project 1 was updated:", project); } else if (action === "delete") { console.info("Project 1 was deleted!"); } });

But console doesnt return anything when changes are made to the Project's Model (trough Django admin)

And the connection is made and it returns data in the Network's Tab whenever i made a change

image

But i cant use that data inside my Vue because it dont triggers the callback function

Find resources unique field that is not the primary key?

tl;dr is it possible to subscribe to objects with another field than the private key?

I'm trying to connect an existing database model to the asynchronous websockets.
I tried the pkField setting in connect, like this:

client = dcrf.connect(`${ws}:${window.location.host}${path}`, {
    pkField: "uuid",
});

But when I try to subscribe to an object I can still only refer to it by the primary key in the table, not uuid.

I would like to use UUID instead of private key in my subscriptions, as UUID are no sequences, and impossible to guess.
Also the rest of the application works with UUID as the unique identifier.

I tried the 'pkField' setting in the configuration block of the connection, but this didn't seem to make any difference.

I'm subscribing with the following code, this fails:

const prom = client.subscribe('user_node', '932895c1-573e-4552-8703-8a9ca9e958ef', (msg, action) => {
    console.log("msg: ", msg, " action: ", action);
}

What does work is the pk, even though pkField is set to 'uuid':

const prom = client.subscribe('user_node', '1', (msg, action) => {
    console.log("msg: ", msg, " action: ", action);
}

Subscriptions to multiple objects only triggers one handler

When multiple subscriptions are made on a single stream, update/delete events will always use the request ID of the last subscription. This breaks subscription selectors, which expect the request ID used to create the subscription to be the same request ID returned in its event payloads.

Ref: https://github.com/hishnash/djangochannelsrestframework/blob/61369b4418805c6521636401e523a5f634a8cd75/djangochannelsrestframework/observer/generics.py#L75

dcrf only responds to 1 request, ignores the others (or worse breaks them)

Greeting !

after successfully integrating dcrf-client & djangochannelrestframework, i have found myself with another problem. (Disregard my last issue because things just decided to stop working)

i have re-wrote my app the same way the test integration does to the T;

Since this morning, i have been testing with 3 browsers (Firefox, Opera, Chromium), to watch real-time data transfer, and i have noticed that djangochannelrestframework does not respond very well to multiple connections to the same stream

Here's a log from logging:

HTTP b'DELETE' request for ['127.0.0.1', 46992]
Using selector: EpollSelector
Creating tcp connection to ('127.0.0.1', 6379)

// group is added here (Opera receives the updates, the others are ignored)
Wocket Message:  {'type': 'on.department.activity', 'body': {'id': 23, 'name': 'rWtni6KCMgAm'}, 'action': 'delete', 'group': 'DCRF-48fb53c7613a6679f841b954b7054ded0bf71d637bac6962cb8980d74bb2d2f0'}
Sent WebSocket packet to client for ['127.0.0.1', 34502]

// group missing here
Wocket Message:  {'type': 'on.department.activity', 'body': {'id': 23, 'name': 'rWtni6KCMgAm'}, 'action': 'delete'}
// also here
Wocket Message:  {'type': 'on.department.activity', 'body': {'id': 23, 'name': 'rWtni6KCMgAm'}, 'action': 'delete'}

Waiter future is already done <Future cancelled> ()
Closed 1 connection(s)
Closed 1 connection(s)
Exception inside application: 'group'
Traceback (most recent call last):
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/staticfiles.py", line 44, in __call__
    return await self.application(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/routing.py", line 71, in __call__
    return await application(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/sessions.py", line 47, in __call__
    return await self.inner(dict(scope, cookies=cookies), receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/sessions.py", line 263, in __call__
    return await self.inner(wrapper.scope, receive, wrapper.send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/auth.py", line 185, in __call__
    return await super().__call__(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/middleware.py", line 26, in __call__
    return await self.inner(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/routing.py", line 150, in __call__
    return await application(
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 94, in app
    return await consumer(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channelsmultiplexer/demultiplexer.py", line 61, in __call__
    await future
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 94, in app
    return await consumer(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 58, in __call__
    await await_many_dispatch(
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/utils.py", line 51, in await_many_dispatch
    await dispatch(result)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 73, in dispatch
    await handler(message)
  File "/home/john/Desktop/folder/dcrf/observers.py", line 21, in __call__
    group = message.pop('group')
KeyError: 'group'
Exception inside application: 'group'
Traceback (most recent call last):
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/staticfiles.py", line 44, in __call__
    return await self.application(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/routing.py", line 71, in __call__
    return await application(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/sessions.py", line 47, in __call__
    return await self.inner(dict(scope, cookies=cookies), receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/sessions.py", line 263, in __call__
    return await self.inner(wrapper.scope, receive, wrapper.send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/auth.py", line 185, in __call__
    return await super().__call__(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/middleware.py", line 26, in __call__
    return await self.inner(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/routing.py", line 150, in __call__
    return await application(
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 94, in app
    return await consumer(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channelsmultiplexer/demultiplexer.py", line 61, in __call__
    await future
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 94, in app
    return await consumer(scope, receive, send)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 58, in __call__
    await await_many_dispatch(
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/utils.py", line 51, in await_many_dispatch
    await dispatch(result)
  File "/home/john/Desktop/folder/.env/lib/python3.8/site-packages/channels/consumer.py", line 73, in dispatch
    await handler(message)
  File "/home/john/Desktop/folder/dcrf/observers.py", line 21, in __call__
    group = message.pop('group')
KeyError: 'group'
WebSocket closed for ['127.0.0.1', 34960]
WebSocket DISCONNECT /cache/ [127.0.0.1:34960]
WebSocket DISCONNECT /cache/ [127.0.0.1:34960]
WebSocket closed for ['127.0.0.1', 34964]
WebSocket DISCONNECT /cache/ [127.0.0.1:34964]
WebSocket DISCONNECT /cache/ [127.0.0.1:34964]
HTTP 204 response started for ['127.0.0.1', 46992]
HTTP close for ['127.0.0.1', 46992]
HTTP response complete for ['127.0.0.1', 46992]
HTTP DELETE /api/department/23/ 204 [0.83, 127.0.0.1:46992]
HTTP DELETE /api/department/23/ 204 [0.83, 127.0.0.1:46992]
WebSocket HANDSHAKING /cache/ [127.0.0.1:47038]
WebSocket HANDSHAKING /cache/ [127.0.0.1:47038]
Upgraded connection ['127.0.0.1', 47038] to WebSocket
WebSocket HANDSHAKING /cache/ [127.0.0.1:47042]
WebSocket HANDSHAKING /cache/ [127.0.0.1:47042]
Upgraded connection ['127.0.0.1', 47042] to WebSocket
WebSocket ['127.0.0.1', 47038] open and established
WebSocket CONNECT /cache/ [127.0.0.1:47038]
WebSocket CONNECT /cache/ [127.0.0.1:47038]
WebSocket ['127.0.0.1', 47038] accepted by application
WebSocket ['127.0.0.1', 47042] open and established
WebSocket CONNECT /cache/ [127.0.0.1:47042]
WebSocket CONNECT /cache/ [127.0.0.1:47042]
WebSocket ['127.0.0.1', 47042] accepted by application
WebSocket incoming frame on ['127.0.0.1', 47042]
WebSocket incoming frame on ['127.0.0.1', 47038]
Creating tcp connection to ('127.0.0.1', 6379)
Sent WebSocket packet to client for ['127.0.0.1', 47042]
Sent WebSocket packet to client for ['127.0.0.1', 47038]

i have followed (again), the integration to the T, and now i'm getting this behaviour.

is it a problem related to djangochannelrestframework or dcrf-client ?

Thank You!

-- packages pip
channels 3.0.4
channels-redis 3.3.1
channelsmultiplexer 0.0.3
djangochannelsrestframework 0.3.0

-- packages npm
├─┬ [email protected]

// consumers.py

import logging
from functools import partial
from typing import Iterable, Union

from dcrf.observers import model_observer
from djangochannelsrestframework.decorators import action
from djangochannelsrestframework.generics import GenericAsyncAPIConsumer
from djangochannelsrestframework.mixins import (CreateModelMixin,
                                                DeleteModelMixin,
                                                ListModelMixin,
                                                PatchModelMixin,
                                                UpdateModelMixin)
from djangochannelsrestframework.observer.generics import \
    ObserverModelInstanceMixin
from office.groups.models import Department
from rest_framework import status
from rest_framework.exceptions import NotFound

from .serializers import DepartmentSerializer

logger = logging.getLogger(__name__)


class DepartmentConsumer(
    ListModelMixin,
    CreateModelMixin,
    UpdateModelMixin,
    PatchModelMixin,
    DeleteModelMixin,
    ObserverModelInstanceMixin,
    GenericAsyncAPIConsumer,
):
    queryset = Department.objects.all()
    serializer_class = DepartmentSerializer
    lookup_field = 'id'

    def _unsubscribe(self, request_id: str):
        request_id_found = False
        to_remove = []
        for group, request_ids in self.subscribed_requests.items():
            if request_id in request_ids:
                request_id_found = True
                request_ids.remove(request_id)
            if not request_ids:
                to_remove.append(group)

        if not request_id_found:
            raise KeyError(request_id)

        for group in to_remove:
            del self.subscribed_requests[group]

    @action()
    async def unsubscribe_instance(self, request_id=None, **kwargs):
        try:
            return await super().unsubscribe_instance(request_id=request_id, **kwargs)
        except KeyError:
            raise NotFound(detail='Subscription not found')

    @model_observer(Department)
    async def on_department_activity(
        self, message, observer=None, action: str = None, request_id: str = None, **kwargs
    ):
        try:
            reply = partial(self.reply, action=action, request_id=request_id)

            if action == 'delete':
                await reply(data=message, status=204)
                # send the delete
                return

            # the @action decorator will wrap non-async action into async ones.
            response = await self.retrieve(
                request_id=request_id, action=action, **message
            )

            if isinstance(response, tuple):
                data, status = response
            else:
                data, status = response, 200
            await reply(data=data, status=status)
        except Exception as exc:
            await self.handle_exception(exc, action=action, request_id=request_id)

    @on_department_activity.groups_for_signal
    def on_department_activity(self, instance: Department, **kwargs):
        yield f'-pk__{instance.pk}'
        yield f'-all'

    @on_department_activity.groups_for_consumer
    def on_department_activity(self, departments: Iterable[Union[Department, int]] = None, **kwargs):
        if departments is None:
            yield f'-all'
        else:
            for department in departments:
                department_id = department.pk if isinstance(
                    department, Department) else department
                yield f'-pk__{department_id}'

    @on_department_activity.serializer
    def on_department_activity(self, instance: Department, action: str, **kwargs):
        return DepartmentSerializer(instance).data

    @action()
    async def subscribe_many(self, request_id: str = None, departments: Iterable[int] = None, **kwargs):
        await self.on_department_activity.subscribe(request_id=request_id, departments=departments)
        return None, status.HTTP_201_CREATED

    @action()
    async def unsubscribe_many(self, request_id: str = None, departments: Iterable[int] = None, **kwargs):
        await self.on_department_activity.unsubscribe(request_id=request_id, departments=departments)
        return None, status.HTTP_204_NO_CONTENT

// routing.py

import os

from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from dcrf.demultiplexer import AsyncJsonWebsocketDemultiplexer
from django.core.asgi import get_asgi_application
from django.urls import path
from office.groups.asgi.consumers import DepartmentConsumer

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')


application = ProtocolTypeRouter({
    "websocket": AuthMiddlewareStack(
        URLRouter([
            path("cache/", AsyncJsonWebsocketDemultiplexer(
                departments=DepartmentConsumer()
            )),
        ])
    ),
    "http": get_asgi_application(),
})

Browser breaks websocket connection with 1011 status code

Compiled JS

Hi bro....
Can you share one compiled .js module

setImmediate is not defined in browsers

Seems like Winston uses setImmediate which isn't available in browser environments.

Uncaught ReferenceError: setImmediate is not defined
    at Console.log (webpack-internal:///./node_modules/winston/dist/winston/transports/console.js:88)
    at Console._write (webpack-internal:///./node_modules/winston-transport/dist/index.js:90)
    at doWrite (webpack-internal:///./node_modules/readable-stream/lib/_stream_writable.js:428)
    at writeOrBuffer (webpack-internal:///./node_modules/readable-stream/lib/_stream_writable.js:417)
    at Console.Writable.write (webpack-internal:///./node_modules/readable-stream/lib/_stream_writable.js:334)
    at DerivedLogger.ondata (webpack-internal:///./node_modules/winston/node_modules/readable-stream/lib/_stream_readable.js:681)
    at DerivedLogger.emit (webpack-internal:///./node_modules/events/events.js:153)
    at addChunk (webpack-internal:///./node_modules/winston/node_modules/readable-stream/lib/_stream_readable.js:298)
    at readableAddChunk (webpack-internal:///./node_modules/winston/node_modules/readable-stream/lib/_stream_readable.js:280)
    at DerivedLogger.Readable.push (webpack-internal:///./node_modules/winston/node_modules/readable-stream/lib/_stream_readable.js:241)

I was able to work around the issue using a polyfill, as mentioned in this issue: winstonjs/winston#1354 (comment). Not sure that there is anything to do about this in dcrf-client, other than adding this workaround to documentation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.