processone/xmpp

Erlang/Elixir XMPP parsing and serialization library on top of Fast XML

Erlang/Elixir XMPP library

The library provides comprehensive representation of XMPP elements as well as tools to work with them. Every such element is represented by an Erlang record. Most of the library's code is auto generated and thus considered to be bug free and efficient.

The approach is very similar to ASN.1, Google Protocol Buffers or Apache Thrift: an XML element is transformed into internal language structure (an Erlang record in our case) - the process known as "decoding". During decoding, validation is also performed, thus well-typed structures are generated, potentially decreasing bugs related to handcrafted parsing. A reverse process known as "encoding" is applied for transforming an Erlang record into an XML element.

The library should be used along with fast_xml library, because it is only able to decode from and encode to structures generated by that library (that is, xmlel() elements).

Table of Contents:

1. Status
2. Compiling
3. API
4. Usage
   1. Initialization and header files
   2. XMPP elements
   3. Decoding and encoding
   4. Stanzas
      1. Common fields
      2. Constructing stanza responses
   5. Error elements
   6. Text elements
   7. Pretty printer
   8. Namespaces
   9. XMPP addresses
   10. Language translation
5. Supported XMPP elements

Status

The library is considered as production ready and has been used in ejabberd XMPP server since version 16.12. However, the API is quite unstable so far and incompatibilities may be introduced from release to release. The stable API will be denoted by 2.x tag in the future.

Dependency

You need at least Erlang OTP 19.0.

Compiling

As usual, the following commands are used to obtain and compile the library:

$ git clone https://github.com/processone/xmpp.git
$ cd xmpp
$ make

API

The full API is documented in doc/API.md

Usage

Initialization and header files

Before calling any function from the library, xmpp application should be started.

Although there are several header files which a developer might find useful to look into, they should not be included directly in the code. Only include/xmpp.hrl file should be included, because it already includes all needed headers and also defines some useful macros. So the typical code should look like:

%% file: foo.erl
-module(foo).
-include_lib("xmpp/include/xmpp.hrl").
...
start() ->
    application:start(xmpp),
    ...

XMPP elements

All XMPP elements (records) are defined in include/xmpp_codec.hrl file. For convenience, every record has the corresponding type spec. There is also predefined xmpp_element() type which is a container for all defined record types: so sometimes we will refer to an arbitrary XMPP element as xmpp_element() in the rest of this document. These records are generated automatically by XML generator from specification file specs/xmpp_codec.spec. The specification file contains information about XML elements defined within XMPP related namespace.

TODO: writing specs for new elements will be documented later. For now you can learn by example: pick up a spec of any element which is very close to the one you want to add a spec for and use it as a pattern

WARNING: you should not include xmpp_codec.hrl in your erlang code. Include xmpp.hrl instead. xmpp_codec.hrl should be only used to consult definitions of existing XMPP elements.

Decoding and encoding

Once an xmlel() element is obtained (either using fxml_stream:parse_element/1 function or by any other means), it can be decoded using either decode/1 or decode/3 functions. The result will be an XMPP element. Note that decoding might fail if there is no known XMPP element for the provided xmlel() element, or xmlel() is invalid, so you should call these functions inside try ... catch. Exceptions returned during decoding can be formatted using format_error/1 or io_format_error/1 functions.

Example:

handle(#iq{type = get, sub_els = [El]} = IQ) ->
    try xmpp:decode(El) of
        Pkt -> handle_iq_child_element(Pkt)
    catch _:{xmpp_codec, Reason} ->
        Txt = xmpp:format_error(Reason),
	io:format("iq has malformed child element: ~s", [Txt]),
	handle_iq_with_malformed_child_element(IQ)
    end.

encode/1 and encode/2 functions can be used for reverse transformation, i.e. to encode an XMPP element into xmlel() element. Encoding would never fail as long as provided XMPP element is valid.

Stanzas

Amongst all XMPP elements defined in the library, the most notable ones are stanzas: message(), presence() and iq() elements. A large part of xmpp module deals with stanzas.

Common fields

Records of all stanzas have several common fields: id, type, lang, from, to, sub_els and meta. Although it's acceptable to manipulate with these fields directly (which is useful in pattern matching, for example), xmpp module provides several functions to work with them:

• id: get_id/1 and set_id/2 can be used
• type: get_type/1 and set_type/2 can be used
• lang: get_lang/1 and set_lang/2 can be used
• from and to: get_from/1, get_to/1, set_from/2, set_to/2 and set_from_to/3 can be used
• sub_els: get_els/1 and set_els/2 can be used; additional functions to work with subtags directly are: set_subtag/2, get_subtag/2, get_subtags/2, try_subtag/2, try_subtags/2, remove_subtag/2, has_subtag/2, and append_subtags/2.
• meta: this field is intended for attaching arbitrary Erlang terms to stanzas; the field is represented by a map() and can be manipulated using standard maps module, but also using the following functions: get_meta/1, get_meta/2, get_meta/3, set_meta/2, put_meta/3, update_meta/3 and del_meta/2

Constructing stanza responses

For creating iq() replies the following functions exist: make_iq_result/1 and make_iq_result/2.

These two functions are iq() specific and create iq() elements of type result only. To create an error response from an arbitrary stanza (including iq()) make_error/2 function can be used.

Error elements

A series of functions is provided by xmpp module for constructing stanza_error() or stream_error() elements. To construct stanza_error() elements the functions with err_ prefix can be used, such as err_bad_request/0 or err_internal_server_error/2. To construct stream_error() elements the functions with serr_ prefix can be used, such as serr_not_well_formed/0 or serr_invalid_from/2.

Text elements

The text element is represented by #text{} record (of text() type). Some record fields, such as #message.body or #presence.status, contain a list of text elements (i.e. [text()]). To avoid writing a lot of extracting code the following functions can be used to manipulate with text() elements: get_text/1, get_text/2, mk_text/1 and mk_text/2.

Pretty printer

For pretty printing of XMPP elements (for example, for dumping elements in the log in a more readable form), pp/1 function can be used.

Namespaces

There are many predefined macros for XMPP specific XML namespaces defined in include/ns.hrl such as ?NS_CLIENT or ?NS_ROSTER.

WARNING: you should not include ns.hrl in your erlang code. Include xmpp.hrl instead. Consult this file only to obtain a macro name for the required namespace.

There is also get_ns/1 function which can be used to obtain a namespace of xmpp_element() or from xmlel() element.

XMPP addresses

An XMPP address (aka Jabber ID or JID) can be represented using two types:

• jid(): a JID is represented by a record #jid{}. This type is used to represent JIDs in XMPP elements.
• ljid(): a JID is represented by a tuple {User, Server, Resource} where User, Server and Resource are stringprepped version of a nodepart, namepart and resourcepart of a JID respectively. This representation is useful for JIDs comparison and when a JID should be used as a key (in a Mnesia database, ETS table, etc.)

Both types and the record are defined in include/jid.hrl.

WARNING: you should not include jid.hrl in your erlang code. Include xmpp.hrl instead.

Functions for working with JIDs are provided by jid module.

Language translation

The library uses a callback function in order to perform language translation. The translation is applied when constructing text or error elements. To set the callback one can use set_tr_callback/1 function. By default, no translation is performed.

Supported XMPP elements

XMPP elements from the following documents are supported. For more details check the file xmpp.doap and its nice display in Erlang/Elixir XMPP at xmpp.org.

• RFC 6120: XMPP Core
• RFC 6121: XMPP Instant Messaging and Presence
• RFC 9266: Channel Bindings for TLS 1.3
• XEP-0004: Data Forms
• XEP-0012: Last Activity
• XEP-0013: Flexible Offline Message Retrieval
• XEP-0016: Privacy Lists
• XEP-0022: Message Events
• XEP-0023: Message Expiration
• XEP-0030: Service Discovery
• XEP-0033: Extended Stanza Addressing
• XEP-0039: Statistics Gathering
• XEP-0045: Multi-User Chat
• XEP-0047: In-Band Bytestreams
• XEP-0048: Bookmarks
• XEP-0049: Private XML Storage
• XEP-0050: Ad-Hoc Commands
• XEP-0054: vcard-temp
• XEP-0055: Jabber Search
• XEP-0059: Result Set Management
• XEP-0060: Publish-Subscribe
• XEP-0065: SOCKS5 Bytestreams
• XEP-0066: Out of Band Data
• XEP-0077: In-Band Registration
• XEP-0078: Non-SASL Authentication
• XEP-0084: User Avatar
• XEP-0085: Chat State Notifications
• XEP-0092: Software Version
• XEP-0114: Jabber Component Protocol
• XEP-0115: Entity Capabilities
• XEP-0131: Stanza Headers and Internet Metadata
• XEP-0138: Stream Compression
• XEP-0153: vCard-Based Avatars
• XEP-0158: CAPTCHA Forms
• XEP-0166: Jingle
• XEP-0172: User Nickname
• XEP-0184: Message Delivery Receipts
• XEP-0191: Blocking Command
• XEP-0198: Stream Management
• XEP-0199: XMPP Ping
• XEP-0202: Entity Time
• XEP-0203: Delayed Delivery
• XEP-0215: External Service Discovery
• XEP-0220: Server Dialback
• XEP-0221: Data Forms Media Element
• XEP-0231: Bits of Binary
• XEP-0234: Jingle File Transfer
• XEP-0249: Direct MUC Invitations
• XEP-0260: Jingle SOCKS5 Bytestreams Transport Method
• XEP-0261: Jingle In-Band Bytestreams Transport Method
• XEP-0264: Jingle Content Thumbnails
• XEP-0279: Server IP Check
• XEP-0280: Message Carbons
• XEP-0297: Stanza Forwarding
• XEP-0300: Use of Cryptographic Hash Functions in XMPP
• XEP-0313: Message Archive Management
• XEP-0319: Last User Interaction in Presence
• XEP-0328: JID Prep
• XEP-0333: Chat Markers
• XEP-0334: Message Processing Hints
• XEP-0352: Client State Indication
• XEP-0355: Namespace Delegation
• XEP-0356: Privileged Entity
• XEP-0357: Push Notifications
• XEP-0359: Unique and Stable Stanza IDs
• XEP-0363: HTTP File Upload
• XEP-0369: Mediated Information eXchange (MIX)
• XEP-0377: Spam Reporting
• XEP-0386: Bind 2
• XEP-0388: Extensible SASL Profile
• XEP-0402: PEP Native Bookmarks
• XEP-0403: MIX: Presence Support
• XEP-0405: MIX: Participant Server Requirements
• XEP-0417: E2E Authentication in XMPP: Certificate Issuance and Revocation
• XEP-0421: Anonymous unique occupant identifiers for MUCs
• XEP-0422: Message Fastening
• XEP-0424: Message Retraction
• XEP-0425: Message Moderation
• XEP-0430: Inbox
• XEP-0440: SASL Channel-Binding Type Capability
• XEP-0474: SASL SCRAM Downgrade Protection
• draft-cridland-xmpp-session-01: XMPP Session Establishment

As well as some proprietary extensions from ProcessOne

xmpp's Issues

How to use this project as a client?

As https://github.com/processone/exmpp has been deprecated. How to use this project as a full Erlang client?

Implement XEP-0474: SASL SCRAM Downgrade Protection

The exact rationale and multiple examples why this specification fills an important gap in our XMPP security infrastructure is detailed in the XEP.

The XEP is really simple, though, and should be easy to implement in ejabberd.

It is already implemented for prosody at https://modules.prosody.im/mod_sasl_ssdp.html

Bug in SCRAM implementation

RFC 5802 section 5.1 (last bullet point) states:
Unknown optional extensions MUST be ignored upon receipt.
Only mandatory options should result in an error, if unsupported by one party. To quote the RFC again:

Mandatory extensions sent by one peer but not understood by the
other MUST cause authentication failure (the server SHOULD send
the "extensions-not-supported" server-error-value).

The bug is here:

xmpp/src/xmpp_sasl_scram.erl

Line 97 in 9b926aa

{error, unsupported_extension};

Example of an optional attribute that should be ignored by ejabberd:
p=tls-exporter,,n=user,r=12C4CD5C-E38E-4A98-8F6D-15C38F51CCC6,d=deadbeef

Example of a mandatory attribute that should result in an unsupported_extension error:
p=tls-exporter,,n=user,r=12C4CD5C-E38E-4A98-8F6D-15C38F51CCC6,m=deadbeef

make spec - error

How to add custom stanza

What version of ejabberd are you using?
17.01.45

What operating system (version) are you using?
deb

How did you install ejabberd (source, package, distribution)?
source

new deps/xmpp project has spec file which then is used to generate the xmpp_codec.erl
%% Created automatically by XML generator (fxml_gen.erl)
%% Source: xmpp_codec.spec

We have few custom stanzas and i would like to include them for encoding and decoding.
for this

I need any pointers on how to best write the spec for stanza in xmpp_codec.spec
once written how and what to execute to generate the xmpp_codec.erl file.

there is no enough doc for https://github.com/processone/xmpp
Please help

Merci.

Write a DOAP file for xmpp.org software page

The xmpp.org website has a list of software (servers, clients, libraries), which includes among others this library.

That page can get information for software either:

• from DOAP files (like it does for ejabberd.doap)
• or from hosted DOAP files, like erlang-elixir-xmpp.doap

Right now, this library does not have a DOAP file, and the hosted DOAP file is outdated. The result is that the xmpp library is very badly listed in xmpp.org, see https://xmpp.org/software/erlang-elixir-xmpp/

The task here is:

• write a static DOAP file for xmpp and submit it to replace erlang-elixir-xmpp.doap
• or write a DOAP file and submit here in this git, then reference to this DOAP file in sofware.json
• consider writing a script to generate that DOAP file from the source code, as ejabberd does in generate-doap.sh

ejabberd source compile throws deps/xmpp/include/xmpp.hrl:28: can't find include file "fxml.hrl"

Development environment:
Ubuntu 20.04
Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 10.6.4

Using the source check out from the master (21.01), throws

 deps/xmpp/include/xmpp.hrl:28: can't find include file "fxml.hrl" 

when doing make process.

The xmpp.hrl does include with the file and can be found in the defined directory.

-include_lib("fast_xml/include/fxml.hrl").

For it to compile without error, I need to change the statement to

-include_lib("../../fast_xml/include/fxml.hrl").

make spec fails with Erlang/OTP 24

"make specs" works perfectly when using 23.2, but just after switching to 24.0, it fails with:

erl -noinput +B -pa ebin -pa deps/*/ebin -eval \
'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
                                            {function_clause,
                                             [{erl_types,initial_typenames,
                                               [{type,{mod,foo,1}}],
                                               [{file,"erl_types.erl"},
                                                {line,4584}]},
                                              {erl_types,t_from_form1,6,
                                               [{file,"erl_types.erl"},
                                                {line,4564}]},
                                              {fxml_gen,t_from_form,1,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,4167}]},
                                              {fxml_gen,get_fun_return_type,
                                               3,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,3511}]},
                                              {fxml_gen,prepare_default,4,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,3765}]},
                                              {fxml_gen,prepare_cdata,5,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,3809}]},
                                              {fxml_gen,prepare_elem,6,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,3727}]},
                                              {fxml_gen,'-compile/4-fun-4-',
                                               6,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,336}]}]}}
make: *** [Makefile:28: spec] Error 1

Can't compile xmpp

I followed these steps

`

    git clone https://github.com/processone/xmpp.git

        Cloning into 'xmpp'...
        remote: Enumerating objects: 30, done.
        remote: Counting objects: 100% (30/30), done.
        remote: Compressing objects: 100% (22/22), done.
        remote: Total 1967 (delta 13), reused 18 (delta 8), pack-reused 1937
        Receiving objects: 100% (1967/1967), 1.30 MiB | 19.05 MiB/s, done.
        Resolving deltas: 100% (1469/1469), done.
        [user@ip]$ cd xmpp
         [user@ip xmpp]$ make
         ./rebar get-deps compile
          ==> xmpp (get-deps)
         Pulling p1_utils from {git,"https://github.com/processone/p1_utils",
                       {tag,"1.0.16"}}
         Cloning into 'p1_utils'...
         Pulling fast_xml from {git,"https://github.com/processone/fast_xml",
                       {tag,"1.1.37"}}
         Cloning into 'fast_xml'...
         Pulling fast_tls from {git,"https://github.com/processone/fast_tls",
                       {tag,"1.1.2"}}
         Cloning into 'fast_tls'...
         Pulling ezlib from {git,"https://github.com/processone/ezlib",{tag,"1.0.6"}}
         Cloning into 'ezlib'...
         Pulling stringprep from {git,"https://github.com/processone/stringprep",
                         {tag,"1.0.17"}}
         Cloning into 'stringprep'...
          ==> p1_utils (get-deps)
          ==> fast_xml (get-deps)
          ==> fast_tls (get-deps)
          ==> ezlib (get-deps)
          ==> stringprep (get-deps)
          ==> p1_utils (compile)
          Compiled src/p1_server.erl
          Compiled src/p1_utils.erl
          Compiled src/p1_utils_sup.erl
          Compiled src/treap.erl
          Compiled src/p1_queue.erl
          Compiled src/p1_nif_utils.erl
          Compiled src/p1_file_queue.erl
          Compiled src/p1_shaper.erl
          Compiled src/p1_options.erl
          Compiled src/p1_time_compat.erl
          Compiled src/p1_http.erl
          Compiled src/p1_rand.erl
          Compiled src/p1_edoc_layout.erl
           Compiled src/p1_prof.erl
           Compiled src/p1_fsm.erl
            ==> fast_xml (compile)
          Compiled src/fxml_gen_pt.erl
          Compiled src/fxmlrpc_codec_external.erl
          Compiled src/fxmlrpc.erl
          Compiled src/fxml.erl
          Compiled src/fxml_stream.erl
          Compiled src/fxml_sup.erl
          Compiled src/fast_xml.erl
          Compiled src/fxmlrpc_codec.erl
          Compiled src/fxml_gen.erl
          Compiling c_src/fxml.c
          Compiling c_src/fxml_stream.c
           ==> fast_tls (compile)
           Compiled src/fast_tls_app.erl
           Compiled src/fast_tls_sup.erl
           Compiled src/p1_sha.erl
           Compiled src/fast_tls.erl
          Compiling c_src/fast_tls.c
          Compiling c_src/ioqueue.c
          Compiling c_src/p1_sha.c
          ==> ezlib (compile)
          Compiled src/ezlib_app.erl
          Compiled src/ezlib_sup.erl
          Compiled src/ezlib.erl
          Compiling c_src/ezlib_drv.c
          ==> stringprep (compile)
          Compiled src/stringprep_sup.erl
          Compiled src/stringprep_app.erl
          Compiled src/stringprep.erl
          Compiling c_src/stringprep.cpp
           ==> xmpp (compile)
          Compiled asn1/XmppAddr.asn1
          Compiled src/xmpp_sasl.erl
          Compiled src/xep0249.erl
          Compiled src/xep0355.erl
          Compiled src/xep0405.erl
          Compiled src/xmpp_config.erl
          Compiled src/xep0055.erl
          Compiled src/scram.erl
          Compiled src/xep0115.erl
          Compiled src/xep0280.erl
          Compiled src/xep0377.erl
           Compiled src/xep0114.erl
            Compiled src/pubsub_publish_options.erl
           Compiled src/xep0221.erl
           Compiled src/xep0078.erl
           Compiled src/pubsub_get_pending.erl
           Compiled src/xep0369.erl
           Compiled src/http_upload.erl
            Compiled src/xep0065.erl
            Compiled src/xep0363.erl
             Compiled src/xdata_codec.erl
             Compiled src/jid.erl
             Compiled src/xep0022.erl
              Compiled src/xep0050.erl
              Compiled src/xep0047.erl
               Compiled src/xep0012.erl
               Compiled src/xep0013.erl
                Compiled src/xmpp_util.erl
                Compiled src/xmpp_sasl_digest.erl
                Compiled src/xep0138.erl
                Compiled src/xmpp_codec.erl
                Compiled src/rfc6121.erl
                Compiled src/xep0352.erl
                 Compiled src/xep0049.erl
                Compiled src/xep0045.erl
                 Compiled src/xep0039.erl
                 Compiled src/muc_register.erl
                   src/xmpp_stream_out.erl:49: syntax error before: ':='
                   src/xmpp_stream_out.erl:1388: type state() undefined
                    src/xmpp_stream_out.erl:78: Warning: type stream_state() is unused
                    ERROR: compile failed while processing /home/ec2-user/xmpp: rebar_abort
                    make: *** [src] Error 1_`

I am using erlang-18.0 .

Issue modifying subtags

I am able to use append_subtags to add my custom XML elements+namespaces to a Stanza.

However, the set_subtag, remove_subtag and has_subtag only seem to work with xmpp_element types so I am unable to do anything more with my settings (when i try i get "terminated with reason: {'module could not be loaded',[{undefined,do_get_name....").

The doco says "all sub-elements of Stanza matching Tag (i.e. xmpp_element() or xmlel() elements with the same tag name and namespace as Tag)" but the signatures say differently... eg.

-spec remove_subtag(Stanza :: iq(), Tag :: xmpp_element()) -> iq();

Is this an oversight or should i be doing things differently?

[SCRAM] SCRAM-SHA-1-PLUS + SCRAM-SHA-256(-PLUS) + SCRAM-SHA-512(-PLUS) supports

To have compatibility with XMPP Servers and after:

• SCRAM-SHA-1

Can you add supports of :

• SCRAM-SHA-1-PLUS
• SCRAM-SHA-224
• SCRAM-SHA-224-PLUS
• SCRAM-SHA-256
• SCRAM-SHA-256-PLUS
• SCRAM-SHA-384
• SCRAM-SHA-384-PLUS
• SCRAM-SHA-512
• SCRAM-SHA-512-PLUS
• SCRAM-SHA3-512
• SCRAM-SHA3-512-PLUS

"When using the SASL SCRAM mechanism, the SCRAM-SHA-256-PLUS variant SHOULD be preferred over the SCRAM-SHA-256 variant, and SHA-256 variants [RFC7677] SHOULD be preferred over SHA-1 variants [RFC5802]".

• SCRAM-SHA-1(-PLUS):
  -- https://tools.ietf.org/html/rfc5802
  -- https://tools.ietf.org/html/rfc6120

• SCRAM-SHA-256(-PLUS):
  -- https://tools.ietf.org/html/rfc7677 since 2015-11-02
  -- https://tools.ietf.org/html/rfc8600 since 2019-06-21: https://mailarchive.ietf.org/arch/msg/ietf-announce/suJMmeMhuAOmGn_PJYgX5Vm8lNA

• SCRAM-SHA-512(-PLUS):
  -- https://tools.ietf.org/html/draft-melnikov-scram-sha-512

• SCRAM-SHA3-512(-PLUS):
  -- https://tools.ietf.org/html/draft-melnikov-scram-sha3-512

https://xmpp.org/extensions/inbox/hash-recommendations.html

-PLUS variants:

• RFC5056: On the Use of Channel Bindings to Secure Channels: https://tools.ietf.org/html/rfc5056
• RFC5929: Channel Bindings for TLS: https://tools.ietf.org/html/rfc5929
• Channel-Binding Types: https://www.iana.org/assignments/channel-binding-types/channel-binding-types.xhtml
• RFC 9266: Channel Bindings for TLS 1.3: https://tools.ietf.org/html/rfc9266

LDAP:

• RFC5803: Lightweight Directory Access Protocol (LDAP) Schema for Storing Salted: Challenge Response Authentication Mechanism (SCRAM) Secrets: https://tools.ietf.org/html/rfc5803

HTTP:

• RFC7804: Salted Challenge Response HTTP Authentication Mechanism: https://tools.ietf.org/html/rfc7804

2FA:

• Extensions to Salted Challenge Response (SCRAM) for 2 factor authentication: https://tools.ietf.org/html/draft-melnikov-scram-2fa

IANA:

• Simple Authentication and Security Layer (SASL) Mechanisms: https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml

Note, after SCRAM-SHA-1(-PLUS):

• GNU SASL (gsasl) supports SCRAM-SHA-256(-PLUS) since 1.9.1: http://git.savannah.gnu.org/gitweb/?p=gsasl.git;a=blob;f=NEWS;hb=HEAD
  I have informed about missing informations in website/manual/docs.
• Dovecot supports SCRAM-SHA-256(-PLUS) since 2.3.10: https://doc.dovecot.org/configuration_manual/authentication/authentication_mechanisms/
• Cyrus SASL/IMAP supports SCRAM-SHA-256(-PLUS) and more since 2.1.27: https://www.cyrusimap.org/sasl/sasl/authentication_mechanisms.html
• ...

Linked to:

• scram-sasl/info#1
• processone/ejabberd#2742
• processone/ejabberd-contrib#279

Recommend to commit automatic files separately

This xmpp library has:

• human-modified files like xmpp_codex.spec
• and many files that are automatically generated runing make spec, like xmpp_codex.hrl and all the src/xep*.erl files
• of course, after compilation, there are *.beam and dependencies, but those are not tracked in git.

When committing to the xmpp git repository, some people first adds commits with his manual changes, and later adds another final commit with only the automatic changes. An example of this is the commit 1b438dd that fixes a small bug, and later the commit 3f18c9e with all the automatically-generated changes.

But in other cases, commits include both the human changes and the automatic changes in one single commit. That's a little annoying, because when reading that commit, you must know/remember which files were automatically generated, to not bother reading them, and focus only on the ones that the original author really modified.

Some ideas to alleviate this:

• A) recommend in README.txt to not mix in the same commit human and automatic changes
• B) add git hook that would check before commit if those files were updated, then reject the commit. The problem is that each user would need to enable it
• C) add github action that runs "make spec" and commits result if necessary: we don't need to commit those anymore ourselves, and nothing breaks if we forget and commit them
• D) ... some other ideas
• Z) remove automatic files completely from git, and change Makefile so they are generated when compiling *.beam files. However, if xmpp was designed that way, surely there was a good reason. Changing this without considering the reason could produce unexpected problems.

The easiest solution, probably good enough, that doesn't break anything else, is A+C: recommend to not commit changes in automatic files, and add github action that updates and commits those automatic files.

XMPP codec should be modular

XMPP codec should allow third-party developers to create their own submodules and plug them into xmpp_codec.
This is implemented in 3976a85
Here is a brief description for developers about how they could use it:

• Clone xmpp library somewhere:
  $ git clone https://github.com/processone/xmpp
• Compile it:
  $ make
• Add new elements to specs/xmpp_codec.spec with proper module attribute set
• Recompile the codec:
  $ make spec
• The generator will create new file(s) in src directory. Those files are your submodules. Grab them to the working directory of your project.
• The generator also will generate new records (with specs) inside include/xmpp_codec.hrl. Grab those too in your project (either put them in your *.hrl files or in your *.erl files directly).
• In runtime, call to xmpp:register_codec/1 in order to register your submodule within xmpp_codec.

Done!
Not very clear, eh? OK, here is an example.

Let's say we want to add submodule for the following element:

<foo x='1' xmlns='ns:foo'/>

Clone and compile the repo, open specs/xmpp_codec.spec and put the following spec somewhere inside it (the order doesn't matter):

-xml(foo,
     #elem{name = <<"foo">>,
           xmlns = <<"ns:foo">>,
           module = foo,
           result = {foo, '$x'},
           attrs = [#attr{name = <<"x">>}]}).

Now type make spec in order to recompile the specification. New file will be created inside src directory:

$ git status
...
Untracked files:
   ...
   src/foo.erl

Copy this foo.erl into your working directory.
Now, consult the recompiled xmpp_codec.hrl:

$ git diff include/xmpp_codec.hrl
...
+-record(foo, {x = <<>> :: binary()}).
+-type foo() :: #foo{}.
+
...

Put this record and type definition somewhere in your files.
Now, you need to register your submodule (foo) during startup. This should look something like this:

-module(mod_foo).
...
start(Blah, ...) ->
    ...
    xmpp:register_codec(foo),
    ...

Note, that you need to call xmpp:register_codec/1 again, if you have your submodule (foo) recompiled and reloaded in runtime.
Also, you may want to unregister the submodule during shutdown procedure, although, strictly speaking, this is not required:

stop(...) ->
    xmpp:unregister_codec(foo)

Now you have everything done to use your new "subcodec".
Have fun ;)

Don't compiles spec for xmpp

I tries compile xmpp_codec.spec manually.
So i just changed this file to add one new empty line.
After that i ran: make spec
And got:

erl -noinput +B -pa ebin -pa deps/*/ebin -eval \
'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
                                            {{badmap,
                                              {dict,1,16,16,8,80,48,
                                               {[],[],[],[],[],[],[],[],[],[],
                                                [],[],[],[],[],[]},
                                               {{[],[],[],[],[],[],[],[],[],
                                                 [[{opaque,xmlel,[]}|
                                                   {{fxml,1,2,[]},type}]],
                                                 [],[],[],[],[],[]}}}},
                                             [{maps,filter,
                                               [#Fun<erl_types.11.61440706>,
                                                {dict,1,16,16,8,80,48,
                                                 {[],[],[],[],[],[],[],[],[],
                                                  [],[],[],[],[],[],[]},
                                                 {{[],[],[],[],[],[],[],[],[],
                                                   [[{opaque,xmlel,[]}|
                                                     {{fxml,1,2,[]},type}]],
                                                   [],[],[],[],[],[]}}}],
                                               [{file,"maps.erl"},{line,205}]},
                                              {erl_types,
                                               t_opaque_from_records,1,
                                               [{file,"erl_types.erl"},
                                                {line,770}]},
                                              {fxml_gen,t_remote,2,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,2744}]},
                                              {fxml_gen,get_label_type,5,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,2192}]},
                                              {fxml_gen,
                                               '-get_types/3-fun-0-',5,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,2160}]},
                                              {lists,map,2,
                                               [{file,"lists.erl"},
                                                {line,1239}]},
                                              {fxml_gen,
                                               '-get_types/3-fun-1-',5,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,2158}]},
                                              {lists,foldl,3,
                                               [{file,"lists.erl"},
                                                {line,1263}]}]}}

Erlang is:
Erlang/OTP 19 [erts-8.3] [source-d5c06c6] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]

What is wrong ?

Cannot find erl_nif.h on make command

I'm running windows 10 and attempting to run the install commands (git clone, cd, make), but upon running the make command, I come across the following error:

.../xmpp/deps/fast_xml/c_src/fxml.c(18): fatal error C1083: Cannot open include file: 'erl_nif.h': No such file or directory
ERROR: compile failed while processing .../xmpp/deps/fast_xml: rebar_abort
make: *** [Makefile:7: src] Error 1

I managed to find erl_nif.h inside my erlang installation folder under erl9.3/usr and I tried copying all the header files to my deps/fast_xml/c_str/ folder and also tried executing export PATH=$PATH:/c/Program\ Files/erl9.3/usr/include but the error persists.

Below is my entire output:

$ make
./rebar get-deps compile
==> p1_utils (get-deps)
==> fast_xml (get-deps)
==> stringprep (get-deps)
==> xmpp (get-deps)
==> p1_utils (compile)
==> fast_xml (compile)
Compiling c:/Users/Arise/Documents/Programming/Erlang/xmpp2/xmpp/deps/fast_xml/c_src/fxml.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24225.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-g'
cl : Command line warning D9024 : unrecognized source file type 'Files/erl9.3/lib/erl_interface-3.10.1/include', object file assumed
cl : Command line warning D9027 : source file 'Files/erl9.3/lib/erl_interface-3.10.1/include' ignored
cl : Command line warning D9024 : unrecognized source file type 'Files/erl9.3/erts-9.3/include', object file assumed
cl : Command line warning D9027 : source file 'Files/erl9.3/erts-9.3/include' ignored
fxml.c
c:/Users/Arise/Documents/Programming/Erlang/xmpp2/xmpp/deps/fast_xml/c_src/fxml.c(18): fatal error C1083: Cannot open include file: 'erl_nif.h': No such file or directory
ERROR: compile failed while processing c:/Users/Arise/Documents/Programming/Erlang/xmpp2/xmpp/deps/fast_xml: rebar_abort
make: *** [Makefile:7: src] Error 1

Error make spec

Hi,
I have tried (tag 1.8.1) to run make spec in my local. it throws the below error.

user@contus:/var/opt/ejabberd-24.02/deps/xmpp$ make spec
erl -noinput +B -pa ebin -pa deps/*/ebin -eval
'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
{function_clause,
[{erl_types,initial_typenames,
[{type,{mod,foo,1},"mod.erl"}],
[{file,"erl_types.erl"},
{line,4579}]},
{erl_types,t_from_form1,6,
[{file,"erl_types.erl"},
{line,4559}]},
{fxml_gen,t_from_form,1,
[{file,"src/fxml_gen.erl"},
{line,4173}]},
{fxml_gen,get_fun_return_type,
3,
[{file,"src/fxml_gen.erl"},
{line,3504}]},
{fxml_gen,prepare_default,4,
[{file,"src/fxml_gen.erl"},
{line,3758}]},
{fxml_gen,prepare_cdata,5,
[{file,"src/fxml_gen.erl"},
{line,3802}]},
{fxml_gen,prepare_elem,6,
[{file,"src/fxml_gen.erl"},
{line,3720}]},
{fxml_gen,'-compile/4-fun-4-',
6,
[{file,"src/fxml_gen.erl"},
{line,336}]}]}}
make: *** [Makefile:26: spec] Error 1

Error on make

xmpp@devops:~/xmpp_server/deps$ git clone https://github.com/processone/xmpp.git
Cloning into 'xmpp'...
remote: Counting objects: 476, done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 476 (delta 19), reused 0 (delta 0), pack-reused 431
Receiving objects: 100% (476/476), 705.13 KiB | 0 bytes/s, done.
Resolving deltas: 100% (290/290), done.
Checking connectivity... done.
xmpp@devops:~/xmpp_server/deps$ cd xmpp/
xmpp@devops:~/xmpp_server/deps/xmpp$ make
./rebar get-deps compile
==> xmpp (get-deps)
Pulling fast_xml from {git,"https://github.com/processone/fast_xml",
                           {tag,"1.1.21"}}
Cloning into 'fast_xml'...
Pulling stringprep from {git,"https://github.com/processone/stringprep",
                             {tag,"1.0.7"}}
Cloning into 'stringprep'...
==> fast_xml (get-deps)
Pulling p1_utils from {git,"https://github.com/processone/p1_utils",
                           {tag,"1.0.6"}}
Cloning into 'p1_utils'...
==> p1_utils (get-deps)
==> stringprep (get-deps)
==> p1_utils (compile)
Compiled src/treap.erl
Compiled src/p1_edoc_layout.erl
Compiled src/p1_server.erl
Compiled src/p1_nif_utils.erl
Compiled src/p1_time_compat.erl
Compiled src/p1_http.erl
Compiled src/p1_fsm.erl
==> fast_xml (compile)
Compiled src/fxml_gen_pt.erl
Compiled src/fxml_stream.erl
Compiled src/fxmlrpc.erl
Compiled src/fxml_sup.erl
Compiled src/fxml.erl
Compiled src/fast_xml.erl
Compiled src/fxmlrpc_codec.erl
Compiled src/fxml_gen.erl
Compiling c_src/fxml.c
Compiling c_src/fxml_stream.c
==> stringprep (compile)
Compiled src/stringprep_sup.erl
Compiled src/stringprep_app.erl
Compiled src/stringprep.erl
Compiling c_src/stringprep.cpp
==> xmpp (compile)
Compiled src/rfc3921.erl
Compiled src/xep0203.erl
Compiled src/xep0078.erl
Compiled src/muc_roominfo.erl
Compiled src/xep0138.erl
Compiled src/xep0153.erl
Compiled src/xep0016.erl
Compiled src/xep0264.erl
Compiled src/xep0198.erl
Compiled src/xep0085.erl
Compiled src/xep0115.erl
Compiled src/xep0191.erl
Compiled src/xep0131.erl
Compiled src/xep0158.erl
Compiled src/xep0045.erl
Compiled src/xep0221.erl
Compiled src/xep0279.erl
Compiled src/xep0352.erl
Compiled src/xep0048.erl
Compiled src/pubsub_publish_options.erl
Compiled src/xmpp_codec.erl
Compiled src/xep0004.erl
Compiled src/muc_request.erl
Compiled src/xep0055.erl
Compiled src/xmpp_util.erl
Compiled src/xep0231.erl
Compiled src/xep0249.erl
Compiled src/xep0060.erl
Compiled src/xep0013.erl
Compiled src/xep0297.erl
Compiled src/xep0359.erl
Compiled src/xdata_codec.erl
Compiled src/flex_offline.erl
Compiled src/p1_stream.erl
Compiled src/xep0065.erl
Compiled src/xep0172.erl
Compiled src/pubsub_subscribe_options.erl
Compiled src/rfc6121.erl
Compiled src/mam_query.erl
Compiled src/rfc6120.erl
Compiled src/xep0030.erl
Compiled src/xep0012.erl
Compiled src/xep0356.erl
Compiled src/pubsub_node_config.erl
Compiled src/xep0355.erl
Compiled src/xep0280.erl
Compiled src/xep0199.erl
Compiled src/xmpp_codec_external.erl
Compiled src/xep0039.erl
Compiled src/xep0202.erl
Compiled src/xep0369.erl
Compiled src/xep0049.erl
Compiled src/pubsub_subscribe_authorization.erl
Compiled src/xep0022.erl
Compiled src/xep0114.erl
Compiled src/xep0077.erl
Compiled src/xep0066.erl
Compiled src/xep0059.erl
Compiled src/xep0334.erl
Compiled src/xep0023.erl
Compiled src/jid.erl
Compiled src/xep0050.erl
Compiled src/pubsub_get_pending.erl
Compiled src/xmpp.erl
Compiled src/xep0313.erl
Compiled src/xep0054.erl
Compiled src/xep0363.erl
Compiled src/muc_register.erl
mCompiled src/p1_mucsub.erl
Compiled src/xep0033.erl
aCompiled src/xep0092.erl
Compiled src/xep0220.erl
Compiled src/muc_roomconfig.erl
kxmpp@devops:~/glabbr_server/deps/xmpp$ make spec
erl -noinput +B -pa ebin -pa deps/*/ebin -eval \
'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
                                            {function_clause,
                                             [{orddict,find,
                                               [0,{0,[{0,[]}]}],
                                               [{file,"orddict.erl"},
                                                {line,88}]},
                                              {erl_types,t_to_string,2,
                                               [{file,"erl_types.erl"},
                                                {line,4068}]},
                                              {erl_types,
                                               '-t_to_string/2-lc$^0/1-12-',2,
                                               [{file,"erl_types.erl"},
                                                {line,4140}]},
                                              {erl_types,t_to_string,2,
                                               [{file,"erl_types.erl"},
                                                {line,4140}]},
                                              {fxml_gen,
                                               '-record_to_string/5-fun-0-',6,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,662}]},
                                              {lists,map,2,
                                               [{file,"lists.erl"},
                                                {line,1237}]},
                                              {fxml_gen,record_to_string,5,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,639}]},
                                              {fxml_gen,
                                               '-make_records/5-fun-0-',8,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,597}]}]}}
Makefile:10: recipe for target 'spec' failed
make: *** [spec] Error 1

compiling error

i did what the manual said but only got this error messege:
root@DESKTOP-MHFT6P1:/home/senpa1/tidos-framework# cd xmpp
root@DESKTOP-MHFT6P1:/home/senpa1/tidos-framework/xmpp# ls
asn1 CODE_OF_CONDUCT.md c_src include Makefile rebar rebar.config.script src
CHANGELOG.md CONTRIBUTING.md doc LICENSE.txt README.md rebar.config specs
root@DESKTOP-MHFT6P1:/home/senpa1/tidos-framework/xmpp# make
./rebar get-deps compile
/usr/bin/env: ‘escript’: No such file or directory
make: *** [Makefile:7: src] Error 127
root@DESKTOP-MHFT6P1:/home/senpa1/tidos-framework/xmpp#

can you please help me

Failed to compile version 1.1.16 with Erlang OTP 20.1

Hi,
Trying to compile xmpp-1.1.16 from an installation with the last Erlang OTP (20.1), I get the following error message:

xmpp_codec: function get_mod/2+2690:
  Internal consistency check failed - please report this bug.
  Instruction: {test,bs_test_tail2,{f,862},[{x,5},0]}
  Error:       {no_bsm_context,{x,5}}:

Compiling src/xmpp_codec.erl failed:
ERROR: compile failed while processing /path/to/xmpp-1.1.16: rebar_abort

Any thought?

Cannot compile with rebar or rebar3

It is currently not possible to compile the module using rebar or rebar3 when neither ../fast_xml/include nor deps/fast_xml/include exist by coincidence.

There is some logic in ejabberd's rebar.config.script that replaces deps/fast_xml/include to _build/default/lib/fast_xml/include so it will work when compiled with ejabberd. However, third-party ejabberd modules would like to also use it. Therefore, deps/fast_xml/include should be replaced by _build/default/lib/fast_xml/include or the rebar.config.script should be updated to also resolve it accordingly.

$ rebar3 compile 
===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"3.1.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/rebar3_hex-3.1.0.tar
===> Compiling rebar3_hex
===> Verifying dependencies...
===> Fetching fast_xml ({pkg,<<"fast_xml">>,<<"1.1.18">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/fast_xml-1.1.18.tar
===> Fetching pc ({pkg,<<"pc">>,<<"1.4.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/pc-1.4.0.tar
===> Compiling pc
===> Fetching stringprep ({pkg,<<"stringprep">>,<<"1.0.7">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/stringprep-1.0.7.tar
===> Fetching p1_utils ({pkg,<<"p1_utils">>,<<"1.0.6">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/p1_utils-1.0.6.tar
===> Compiling p1_utils
===> Compiling fast_xml
===> Compiling /Users/marcp/Repositories/xmpp/_build/default/lib/fast_xml/c_src/fxml.c
===> Compiling /Users/marcp/Repositories/xmpp/_build/default/lib/fast_xml/c_src/fxml_stream.c
===> Linking /Users/marcp/Repositories/xmpp/_build/default/lib/fast_xml/priv/lib/fxml.so
===> Linking /Users/marcp/Repositories/xmpp/_build/default/lib/fast_xml/priv/lib/fxml_stream.so
===> Compiling stringprep
===> Compiling /Users/marcp/Repositories/xmpp/_build/default/lib/stringprep/c_src/stringprep.cpp
===> Linking /Users/marcp/Repositories/xmpp/_build/default/lib/stringprep/priv/lib/stringprep.so
===> Compiling xmpp
===> Compiling src/xmpp_util.erl failed
include/xmpp.hrl:29: can't find include file "fxml.hrl"

---

Original Issue Description

I'm trying to compile a module that uses this new library. I've a very simple rebar.config file:

{deps, [
    {xmpp, ".*", {git, "https://github.com/processone/xmpp", {tag, "1.1.4"}}}
]}.

When I try to run rebar3 eunit or rebar3 compile I get an error message:

$ rebar3 compile
===> Verifying dependencies...
===> Fetching xmpp ({git,"https://github.com/processone/xmpp",
                                {ref,"c010141487632db1666bcbf64f90f42c0fde50aa"}})
===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"3.1.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/rebar3_hex-3.1.0.tar
===> Compiling rebar3_hex
===> Fetching fast_xml ({pkg,<<"fast_xml">>,<<"1.1.18">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/fast_xml-1.1.18.tar
===> Fetching pc ({pkg,<<"pc">>,<<"1.4.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/pc-1.4.0.tar
===> Compiling pc
===> Fetching stringprep ({pkg,<<"stringprep">>,<<"1.0.7">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/stringprep-1.0.7.tar
===> Fetching p1_utils ({pkg,<<"p1_utils">>,<<"1.0.6">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/p1_utils-1.0.6.tar
===> Compiling p1_utils
===> Compiling fast_xml
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/fast_xml/c_src/fxml.c
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/fast_xml/c_src/fxml_stream.c
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/fast_xml/priv/lib/fxml.so
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/fast_xml/priv/lib/fxml_stream.so
===> Compiling stringprep
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/stringprep/c_src/stringprep.cpp
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/stringprep/priv/lib/stringprep.so
===> Compiling xmpp
===> Compiling _build/default/lib/xmpp/src/xmpp_util.erl failed
_build/default/lib/xmpp/include/xmpp.hrl:29: can't find include file "fxml.hrl"

However, when I compile this module separately, it works:

$ cd _build/default/lib/xmpp
$ rebar3 compile
===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"3.1.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/rebar3_hex-3.1.0.tar
===> Compiling rebar3_hex
===> Verifying dependencies...
===> Fetching fast_xml ({pkg,<<"fast_xml">>,<<"1.1.18">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/fast_xml-1.1.18.tar
===> Fetching pc ({pkg,<<"pc">>,<<"1.4.0">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/pc-1.4.0.tar
===> Compiling pc
===> Fetching stringprep ({pkg,<<"stringprep">>,<<"1.0.7">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/stringprep-1.0.7.tar
===> Fetching p1_utils ({pkg,<<"p1_utils">>,<<"1.0.6">>})
===> Downloaded package, caching at /Users/marcp/.cache/rebar3/hex/default/packages/p1_utils-1.0.6.tar
===> Skipping p1_utils (from {pkg,<<"p1_utils">>,<<"1.0.6">>,
                                     <<"EF0951DDF38E92B7E479AF4B8DC950DF76AF8C1030432EF68B7FD7AD17C436FE">>}) as an app of the same name has already been fetched
===> Compiling p1_utils
===> Compiling fast_xml
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/fast_xml/c_src/fxml.c
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/fast_xml/c_src/fxml_stream.c
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/fast_xml/priv/lib/fxml.so
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/fast_xml/priv/lib/fxml_stream.so
===> Compiling stringprep
===> Compiling /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/stringprep/c_src/stringprep.cpp
===> Linking /Users/marcp/Desktop/erlang-test/myapp/_build/default/lib/xmpp/_build/default/lib/stringprep/priv/lib/stringprep.so
===> Compiling xmpp

I've also posted this question on https://www.rebar3.org/discuss/587d1b6673e9731b00889294. They told me the rebar.config.script is somehow responsible for this. Any help would be greatly appreciated.

Travis CI doesn't work

As travis is not configured, remove it from Readme file also.

Missing supervisor

Hi. Why Here you did not start a supervisor instead ? I'm using some monitoring tools for monitoring Ejabberd and they get supervisor's children in runtime and here (self()) does not send response for supervisor API functions and causes some errors and timeouts on those Tools. for example Wobserver in here tries to make graph for supervision tree of an OTP application.

how to add a tag

I have created below spec

-record(sync, {id = <<>> :: binary(),
               c = <<>> :: sync_type(),
               from :: undefined | jid:jid(),
               to :: undefined | jid:jid(),
               json :: <<>> :: binary(),
               sub_els = [] :: [xmpp_element() | fxml:xmlel()]
               }).


-xml(sync,
     #elem{name = <<"sync">>,
     xmlns =  [<<"jabber:client">>, <<"jabber:server">>,
               <<"jabber:component:accept">>],
     cdata = #cdata{label = '$json'},
     result = {sync, '$id','$c','$from', '$to', '$json', '$_els'},
     attrs = [ #attr{name = <<"id">>},
               #attr{name = <<"c">>,
                     default = <<"0">> },
               #attr{name = <<"from">>,
             	   		  dec = {dec_jid, []},
             		   	  enc = {enc_jid, []}},
               #attr{name = <<"to">>,
               			  dec = {dec_jid, []},
               			  enc = {enc_jid, []}}
             ]}).

Which parses below request

<<"<sync id='6383950948998274048-13' c='-902781519'>{\"t\":\"sync1\",\"id\":{\"t\":\"UserDets\",\"hash\":2102,\"user_id\":5533240}}</sync>">>

however my reponse to the result is with addtion of response tag as below

<<"<sync id='6383950948998274048-13' c='-902781519'> <response>JSON_RESPONSE </response> </sync>">>

when preparing response packet how do i add response tag ?
in old version i used to

Children = [#xmlel{name = <<"response">>,
        attrs = [],
        children = [{xmlcdata, ResponseData}]}],

Please help
Thanks

Change presence unavailable status message

<presence xmlns="jabber:client" to="user" from="[email protected]/xxx" type="unavailable">
  <x xmlns="http://jabber.org/protocol/muc#user">
    <item affiliation="none" role="none" />
  </x>
  <status xml:lang="en">Stream closed by us: Timed out waiting for stream resumption (connection-timeout)</status>
</presence>

status is a text that should be displayed by the client to the user receiving this presence, if my client displays to me "Stream closed by us:" i wonder, why did my client close the stream with that contact.

If the goal was to make clear which side did close a stream, the word "us" fails terribly in this setting.

Implement XEP-0480: SASL Upgrade Tasks

This is the last missing piece for modern SASL2 authentication: XEP-0480: SASL Upgrade Tasks is needed to make sure clients can update the old SHA-1 password hashes to more secure alternatives like SHA-256.

This isn't as urgent as the other SASL2 related stuff you just implemented, but needed to make sure we can transition from SHA-1 to something more secure before SCRAM-SHA-1 becomes insecure.
This transition will take quite some time, so it is good to start early with this.

I promise this is the last SASL-related implementation request I'm doing ;)

BTW: This XEP was originally developed inside the main SASL2 XEP (XEP-0388) but later factored out to not create another of these large XEPs like MUC or PubSub.

logger doesn't work on my module

Environment

• ejabberd version: 20.02.06
• Erlang version: 10.6.4
• OS: Linux (Debian)
• Installed from: source

I created a module in the local repository directory .ejabberd-modules and put in the code the syntax:

-include("logger.hrl").
-include("xmpp.hrl").
-include("ejabberd_commands.hrl").
-include("mod_privacy.hrl").
-include("mod_last.hrl").
-include("translate.hrl").
-include("ejabberd_sql_pt.hrl").

...

start(Host, Opts) ->
  ?DEBUG("module xxxx started", []),
  xmpp:register_codec(some_codec),
  gen_iq_handler:add_iq_handler(ejabberd_local, Host,
    ?NS_WB_XROSTER, ?MODULE, process_sm_iq),
  gen_iq_handler:add_iq_handler(ejabberd_sm, Host,
    ?NS_WB_XROSTER, ?MODULE, process_sm_iq).


stop(Host) ->

  xmpp:unregister_codec(some_codec),
  gen_iq_handler:remove_iq_handler(ejabberd_local, Host,
    ?NS_WB_XROSTER),
  gen_iq_handler:remove_iq_handler(ejabberd_sm, Host,
    ?NS_WB_XROSTER).

depends(_Host, _Opts) ->
  [].

mod_options(_Host) ->
  [].

process_sm_iq() ->
    ?ERROR_MSG("some message", []),
    xmpp:make_iq_result(IQ, Somestuff).

And I see nothing in the ejabberd.log file

1.1.16 hex release

Is someone willing to make a hex package release for 1.1.16 (linked to my request on fast_xml for a hex package release) to smooth out ejabberd build from source.

having other issues building with rebar hence the request.

Question - How to read custom Tag in Message.

Sorry to report an issue here, I need help and don't know where to go.

I am trying to implement a custom module to log all the messages to a webhook. I've found a good reference to do that, https://github.com/PH-F/mod_offline_http_post/blob/master/src/mod_offline_http_post.erl.

There is a custom tag in the message xml, and I want to pass this to the webhook too.

But I couldn't figure out how to read the custom field out. Please help and tell me which API I should use to read the text in the "extra" tag, showing below.

Thanks in advance.

The Ejabber Server is latest v18.04.

The Xml format of the message:

<message type="chat" to="[email protected]" id="7521387E-D6D1-41B4-A909-E9AD3251FB84">
 <body>Image</body>
 <thread>[email protected]</thread>
 <x xmlns="jabber:x:event"><offline/></x>
 <extra xmlns="ocp:extra">&lt;extra type="image" thumbnail="thumbnailUrl" url="url"&gt;&lt;/extra&gt;</extra>
</message>

The Packet format I got from Ejabberd log:

{message,
<<"7521387E-D6D1-41B4-A909-E9AD3251FB84">>,
chat,
<<"en">>,
{jid,<<"dichen">>,<<"123.123.123.123">>,<<>>,<<"dichen">>,<<"123.123.123.123">>,<<>>},
{jid,<<"johndoe">>,<<"123.123.123.123">>,<<>>,<<"johndoe">>,<<"123.123.123.123">>,<<>>},
[],
[{text,<<>>,<<"Image">>}],
{message_thread,<<>>,<<"[email protected]">>},
[
  {xmlel,<<"x">>,[{<<"xmlns">>,<<"jabber:x:event">>}],[{xmlel,<<"offline">>,[],[]}]},
  {xmlel,<<"extra">>, [{<<"xmlns">>,<<"ocp:extra">>}], [{xmlcdata,<<"<extra type=\"image\" thumbnail=\"thumbnailUrl\"  url=\"url\"></extra>">>}]}
]
...
}

make spec error occured

Could not create spec

OS: MAC OSX High Sierra
Erlang: 19.2 version

erl -noinput +B -pa ebin -pa deps/*/ebin -eval \
	'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
                                            {{no_abstract_code_found,fxml_gen},
                                             [{fxml_gen,
                                               make_builtin_codec_funs,0,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,530}]},
                                              {fxml_gen,compile,4,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,321}]},
                                              {fxml_gen,compile,2,
                                               [{file,"src/fxml_gen.erl"},
                                                {line,56}]},
                                              {erl_eval,do_apply,6,
                                               [{file,"erl_eval.erl"},
                                                {line,670}]},
                                              {erl_eval,expr,5,
                                               [{file,"erl_eval.erl"},
                                                {line,269}]},
                                              {init,start_it,1,[]},
                                              {init,start_em,1,[]},
                                              {init,do_boot,3,[]}]}}
make: *** [spec] Error 1

XEP-0009 ?

Hi Process-One,
I've been running ejabberd for some years now.
At one point I had xml-rpc enabled and usable.
With a ruby client.
Within a client stream (bosh), 5280 and suchlike.
I could nominate a call and send the stanza within.
Looks like this functionality has been moved to service ejabberd commands using the ejabberd_xmlrpc module?
Which is actived on a separate listener? (Understandable).
https://github.com/processone/xmpp shows no entry for XEP-0009.
I thought I might find a module in the contribs. Could not so far.
So my server is responding with 'No module to service this request'.
My frontend is js: Strophe.addNamespace("RPC", "jabber:iq:rpc")

Am I up for a refactoring week/month?

Thanks for ejabberd. Amaze.

Wrong channel-bindings announced

The tls-unique channel-binding is only specified for TLS <= 1.2, but not for TLS 1.3 or greater.
The tls-exporter channel-binding on the other hand is only defined for TLS 1.3 or greater, but not for older TLS versions.

--> Ejabberd should only announce one of them, but never both, bug location:

xmpp/src/xmpp_sasl_scram.erl

Line 221 in 9b926aa

cbind_valid(#state{cb_socket = Sock}, <<"p=tls-unique">>) when Sock /= none ->

Relevant quote from RFC 9266:

The specifications for Salted Challenge Response Authentication
Mechanism (SCRAM) [RFC5802] [RFC7677] and Generic Security Service
Application Program Interface (GSS-API) over Simple Authentication
and Security Layer (SASL) [RFC5801] define "tls-unique" as the
default channel binding to use over TLS.  As "tls-unique" is not
defined for TLS 1.3 (and greater), this document updates [RFC5801],
[RFC5802], and [RFC7677] to use "tls-exporter" as the default channel
binding over TLS 1.3 (and greater).

How to use in Elixir client

Hello,

I'm trying to use this library on an Elixir client but I can't find any examples or documentation on how to setup it and use it...

Can someone help me on this?

error while encoding #muc_subscription record

I want to return extra attribute when user retrieves subscribed groups.

I added new column in muc_room_subscribers, and changed

?SQL("select @(room)s from muc_room_subscribers where jid=%(JidS)s"
	     " and host=%(Host)s")

with this

?SQL("select @(room)s, @(unread)s from muc_room_subscribers where jid=%(JidS)s"
	     " and host=%(Host)s")

and

[jid:make(Room, Host, <<>>) || {Room} <- Subs];
with this

[#muc_subscription{jid = jid:make(Room, Host, <<>>)), unread = binary_to_integer(Unread)} || {Room, Unread} <- Subs];
Also i changed xmpp_spec file and transfer new p1_mucsub.erl to ejabebrd deps/xmpp/src directory

i think i'm doing everything right, but when i retrieve subscribed groups i get this error.

RFC 9266: Channel Bindings for TLS 1.3 support

Can you add the support of RFC 9266: Channel Bindings for TLS 1.3?

• https://datatracker.ietf.org/doc/html/rfc9266

Little details, to know easily:

• tls-unique for TLS =< 1.2
• tls-server-end-point
• tls-exporter for TLS = 1.3

Thanks in advance.

Linked to:

• #39
• processone/ejabberd#2742
• processone/ejabberd-contrib#279
• processone/ejabberd#3971
• processone/ejabberd#3972
• processone/ejabberd#4112
• processone/ejabberd#4105

Allow fxml.hrl to be found from system libs

For reference on previous discussion: #3

Hex module does not include /priv/lib/*.so files

This breaks using the hex module with ejabberd in Elixir. It needs to be updated to truly fix processone/ejabberd#1822

Xmpp_idna was removed - breaks ejabberd builds using hex.pm

Hi,

we are using ejabberd in Elixir project - when 19.8.0 version is installed, the certificate loading is broken. The reason is missing xmpp_idna, that was removed from this project.

f3517a9

IMHO changes like this should not be made when major version of lib didn't change as well...

20:31:04.354 [error] Hook ejabberd_started crashed when running ** (UndefinedFunctionError) function :xmpp_idna.domain_utf8_to_ascii/1 is undefined (module :xmpp_idna is not available)
    :xmpp_idna.domain_utf8_to_ascii("localhost")
    (ejabberd) src/ejabberd_pkix.erl:84: :ejabberd_pkix.get_certfile_no_default/1
    (ejabberd) src/ejabberd_pkix.erl:267: anonymous fn/1 in :ejabberd_pkix.check_domain_certfiles/1
    (stdlib) lists.erl:1338: :lists.foreach/2
    (ejabberd) src/ejabberd_pkix.erl:148: :ejabberd_pkix.handle_call/3
    (stdlib) gen_server.erl:661: :gen_server.try_handle_call/4
    (stdlib) gen_server.erl:690: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3```

Improve codec generation from third-party specs

This is a follow-up for #9
The generator should generate separate files (both *.erl and *.hrl) in the third-party project directory. So there should be no hassle to copy anything and compare diffs.
After this is implemented the documentation should be written on how to use the generator. Existing one on the issue mentioned above is not a very good source of information.

RFCs and XEPs support page with VERSION

On the README, it is possible to have a XEPs and RFCs support page with version (XEP-XXXX v1.2)?

• https://github.com/processone/xmpp/blob/master/README.md

Currently there are only XEP-XXXX, there are not version and not RFCs.

Examples:

• https://github.com/UWPX/UWPX-Client/blob/master/README.md
• https://github.com/telldus/xmppjs-client-plugins/blob/master/README.md
• https://github.com/ortuman/jackal/blob/master/README.md
• https://profanity-im.github.io/xeps.html
• https://metronome.im/supported-xeps-list
• https://dev.gajim.org/gajim/gajim/wikis/help/GajimXEPSupport
• https://github.com/jappix/jappix/blob/master/PROTOCOL.md

How i can define equal TAG with different XMLNS and different ATTRIBUTES ?

I need determine custom tag named as one from xmpp_codec.hrl, but with other attributes and other xmlns ?
There is conflict with original record from xmpp_codec.hrl.

19.09.1 does not load custom module

I'm trying to update my setup from 19.05 to 19.09.1, but get an error:
[critical] <0.110.0>@ejabberd_app:start:71 Failed to start ejabberd application: Invalid value of option auth_method->1: unsupported database: something.

Everything works as expected with 19.05.

Should be auth modules deployed in a different way or is it a bug?

spec for custom stanza

Could someone show how the spec is written for more than two child elements? I would like to write a spec for something like this:

<iq to="YourServer" type="set">
  <register xmlns="https://myurl.com/push" >
    <token>TOKEN</token>
     <device>Android/Ios</device>
  </register>
</iq>

Crash running "make spec" with Erlang 26

This xmpp library uses fast_xml when running make spec.

Since Erlang/OTP 26.0, this fails:

$ erl -version
Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 14.0.2

$ REBAR=rebar3 make spec
erl -noinput +B -pa ebin -pa _build/default/lib/*/ebin -eval \
'case fxml_gen:compile("specs/xmpp_codec.spec", [{add_type_specs, xmpp_element}, {erl_dir, "src"}, {hrl_dir, "include"}]) of ok -> halt(0); _ -> halt(1) end.'
failed to compile "specs/xmpp_codec.spec": {'EXIT',
                                            {badarg,
                                             [{ets,lookup_element,
                                               [{dict,0,16,16,8,80,48,
                                                 {[],[],[],[],[],[],[],[],[],
                                                  [],[],[],[],[],[],[]},
                                                 {{[],[],[],[],[],[],[],[],[],
                                                   [],[],[],[],[],[],[]}}},
                                                jid,2,error],
                                               [{error_info,
                                                 #{cause => type,
                                                   module =>
                                                    erl_stdlib_errors}}]},
                                              {erl_types,lookup_module_types,
                                               3,
                                               [{file,"erl_types.erl"},
                                                {line,5196}]},

The function called by fast_xml that worked previously and now fails:

This worked:

$ erl
Erlang/OTP 25 [erts-13.2.2.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit:ns]

Eshell V13.2.2.1  (abort with ^G)
1>     erl_types:t_from_form(
1>        {remote_type,5184,[{atom,5184,jid},{atom,5184,jid},[]]},
1>                sets:new(),
1>                {type, {mod, foo, 1}, "mod.erl"},
1>                dict:new(),
1>        erl_types:var_table__new(),
1>                erl_types:cache__new()).
{any,{cache,#{},
            {mrecs,{dict,0,16,16,8,80,48,
                         {[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
                         {{[],[],[],[],[],[],[],[],[],[],[],[],...}}}}}}

But now it fails:

$ erl
Erlang/OTP 26 [erts-14.0.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit:ns]

Eshell V14.0.1 (press Ctrl+G to abort, type help(). for help)
1>     erl_types:t_from_form(
..        {remote_type,5184,[{atom,5184,jid},{atom,5184,jid},[]]},
..                sets:new(),
..                {type, {mod, foo, 1}, "mod.erl"},
..                dict:new(),
..        erl_types:var_table__new(),
..                erl_types:cache__new()).
** exception error: bad argument
     in function  ets:lookup_element/4
        called as ets:lookup_element({dict,0,16,16,8,80,48,
                                           {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                                           {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
                                     jid,2,error)
        *** argument 1: not an atom or a table identifier
     in call from erl_types:lookup_module_types/3 (erl_types.erl, line 5196)
     in call from erl_types:remote_from_form/8 (erl_types.erl, line 4623)
     in call from erl_types:t_from_form2/5 (erl_types.erl, line 4287)

The problem seems to be that fast_xml calls an internal, undocumented module erl_types from Erlang/OTP dialyzer, and that one changed in Erlang 26 its data storage from dict to map in erlang/otp@5732196

Which calls an ets BIF function added in erlang/otp@e528e26

I took a look attempting to improve fast_xml to support the new erl_types.erl, but I didn't succeed. As a temporary workaround, I'll disable running make spec with newer Erlang 26 in the CI tests.

Internal Consistency check failed

Hi there,

I'm having an issue with this library on the latest version of OSX (10.13.1) and erlang 20.1:

Erlang/OTP 20 [erts-9.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false]

<snip>

===> Fetching rebar3_hex ({pkg,<<"rebar3_hex">>,<<"4.1.0">>})
===> Downloaded package, caching at /Users/john/.cache/rebar3/hex/default/packages/rebar3_hex-4.1.0.tar
===> Compiling rebar3_hex
===> Fetching pc ({pkg,<<"pc">>,<<"1.6.0">>})
===> Downloaded package, caching at /Users/john/.cache/rebar3/hex/default/packages/pc-1.6.0.tar
===> Compiling pc
===> Compiling xmpp
===> Compiling /Users/john/Public/fap/api/2.1/deps/xmpp/c_src/jid.c
===> Linking /Users/john/Public/fap/api/2.1/deps/xmpp/priv/lib/jid.so
src/xep0191.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0153.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0231.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0033.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0352.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0030.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0356.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0357.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0023.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0221.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0355.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0369.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0220.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0022.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0092.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0045.erl:6: Warning: export_all flag enabled - all functions will be exported

src/rfc6121.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0131.erl:6: Warning: export_all flag enabled - all functions will be exported

src/rfc6120.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0050.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0078.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0085.erl:6: Warning: export_all flag enabled - all functions will be exported

src/xep0279.erl:6: Warning: export_all flag enabled - all functions will be exported

===> Compiling src/xmpp_codec.erl failed
xmpp_codec: function get_mod/2+2657:
  Internal consistency check failed - please report this bug.
  Instruction: {test,bs_test_tail2,{f,856},[{x,5},0]}
  Error:       {no_bsm_context,{x,5}}:


src/xmpp_codec.erl:6: Warning: export_all flag enabled - all functions will be exported

I have no idea what this means or how to go about debugging it!

RFC7622 support

RFC7622 defines XMPP address format (i.e. JID). Currently jid.erl doesn't support this and, what's worse, treats a string with blanks (e.g. server . com) as a valid JID. This should be improved.

How to decode with var

Hello i want to write a custom spec in xmpp_codec.spec which will decode json cdata

_-xml(sync,
#elem{name = <<"sync">>,
xmlns = <<"mycustom:xmlsns:sync">>,
cdata = #cdata{label = '$json',
dec = {jsonutil,dec_json, []}, ### how to pass the attribute variable here
enc = {jsonutil,enc_json, []}}, ### how to pass the attribute variable here
result = {sync, '$c','$json', '$els'},
attrs = [ #attr{name = <<"c">>,
default = 0}
]}).

in dec = dec_json , [] i want to pass the label '$c' so that my decode will be based on '$c'

how can i achieve this right now it just accepts the atom or constants is it possible to pass a attribute var ?
any other method i can achieve this?

Merci