restcomm/smscgateway

RestComm SMS Gateway (SMSC) to send/receive SMS from/to Operators Network (GSM)

Note

On 29 June 2023, Mobius Software LTD signed an agreement with Mavenir to assume development and support of the RestcommOne on prem products. For more details regarding open source / commercial differentiation, roadmap, licensing and other details please visit the Mobius Website.

RestComm SMS Gateway (SMSC)

RestComm SMS Gateway (SMSC) to send/receive SMS from/to Mobile Operator Networks (GSM, SS7 MAP) , SMS aggregators (SMPP) and Internet Telephony Service Providers (SIP, SMPP).

Introduction

RestComm SMS Gateway (SMSC) is built on RestComm SS7, SMPP and RestComm JSLEE Server.

When a user sends a text message (SMS message) to another user, the message gets stored in the SMSC (short message service center) which delivers it to the destination user when they are available. This is a store and forward option.

An SMS center (SMSC) is responsible for handling the SMS operations of a wireless network.

When an SMS message is sent from a mobile phone, it will reach an SMS center first.

2. The SMS center then forwards the SMS message towards the destination.

3. The main duty of an SMSC is to route SMS messages and regulate the process. If the recipient is unavailable (for example, when the mobile phone is switched off), the SMSC will store the SMS message.

4. It will forward the SMS message when the recipient is available.

smscgateway's Issues

smpp esme show <ESME name>

We have now only "smpp esme show" that shows all ESMEs together.
We need a special command to see only one ESME.

Adding of SMPP statistics into regular SMSC statistics

Cloud hopper stack has its own native SMPP statistics.

We need to think of can we include this statistics results into regukar SMSC statistic (SmscStatProviderJmx).

Update Advanced server settings SMSC doc

Update Advanced server settings SMSC doc

We need to assemble splitted messages that we are sending to SIP / ESME

Long SMS when it comes from GSM is splitted, but at SIP / some ESMEs (that are configured for this) destinations side we need to have such message as a solid one.

So we need to assemble spliited message before we are sending them into SIP / some ESMEs.

We need to establiosh some extra memory cache that will keep message parts till all of them have come.

Simple maven project example that can help for developing of customized mproc rules

We need to create a simple maven project example that can help for developing of customized mproc rules

support for cassandra database versions 2.0, 2.1, 2.2, 3.0

Now SMSC GW is based on top of cassandra database versions 1.2.
We need to add support for versions 2.* and 3.0.

ENQUIRE messages also for SERVER type ESMEs

We have the following functionality for CLIENT type ESMEs:
a possibility of sending periodically ENQUIRE messages to a peer and if 3 responses was not achieved back - SMSC GW will break a connection.

We need to make the same for SERVER type ESMEs. This option must be configurable (turnung on/off and sending intervals).

Custom (private) extra AVP fields for diameter requests

Standartized diameter AVP fields (from a specification) does not cover all customer needs for providing some info like networkID and others.

We need to think to introduce optional AVP fields for customers needs.

Reporting via Graylog

SMSC should log all stats in server.log

Use Graylog to export logs and create reports

Reduced sending rate when very many messages to the one dest address - SIP

When we are sending very many messages into one dest address (this is possible for SMPP and SIP) the next message will be sent only when the receipt has come. This can reduce of sending rate for messages for the single destination.

Solution: We need to improve algo at SIP Rx SBB for the case when the message delivery count for the dest address is big

Add Do Not Disturb MProc

Add Do Not Disturb MProc

Expose API to store and search SMS

[Ivelin]
The long term store and search are a good added value for our cloud offering.
Designing and maintaining a scalable data center is too much pain for
operators. They are used to buying a box that lives forever. So we can
offer the temporary storage on their copy of SMSC and long term
storage if they want to plug-in our cloud services. Further added
value extensions can be integration with Dropbox, Google Drive, S3,
etc.

Add support for "priority_flag" for SUBMIT_SM

See section 5.2.14 of SMPP 3.4 Specs

Corresponed issue:
#27

Check also database update issue: #95

Gather basic usage stats

Many users and prospects ask what is the scale of use of Mobicents products. We should add a simple counter in the server to report delivered messages, along with public IP of the server. The aggregated stats from around the world can be reported live back to the community.

This issue is related to
RestComm/sip-servlets#60

Releasing update for diameter part

when binaries creating we need:
1)
/server/default/deploy/TelScale-diameter-mux-6.2.0.GA.sar/lib/sctp-api-1.6.0.FINAL.jar
/server/default/deploy/TelScale-diameter-mux-6.2.0.GA.sar/lib/sctp-impl-1.6.0.FINAL.jar
replace with SCTP release 1.7.0.FINAL when we switch to JSS7 with netty support

copy
/resources/ocs/smsc-part/jdiameter-config.xml
to
/server/simulator/deploy/TelScale-diameter-mux-6.2.0.GA.sar/config
to have already configured SMPP profile

3. check SMSC GW general diameter settings to fit jdiameter-config.xml (for simulator profile)

4. update manuals for this update

Installation issue

Hi,
There is dependencies problem while trying to install (mvn install)
Find the error below :

[ERROR] Failed to execute goal on project smpp: Could not resolve dependencies for project org.mobicents.smsc:smpp:jar:2.0.0-SNAPSHOT: Failure to find org.mobicents.protocols.ss7.management:shell-server-api:jar:2.1.0.FINAL

Regards,

Bypass of SRI request from SMSC GW to HLR for home routing procedure

We need to have a configurable for each networkID option to bypass of SRI request from SMSC GW to home HLR.
This is needed for the case when SMSC GW accepts messages that will be routed further to some SMPP destination and therefor a SRI request to a local HLR is useless and we do not have an access to such HLR.

Timeout for ESMEs after which SMSC GW will drop a TCP connection without message activity

We need to add a configurable timeout for SERVER type ESMEs. If during this time no messageflow in the ESME then SMSC GW must drop TCP connection because TCP connection may be already dropped at TCP level.
The proposal from: https://telestax.zendesk.com/agent/tickets/33003

A note:
This option can be dangerous in case of outgoing traffic and when we have no traffic periodically. The connection can be dropped and outgoing messages will be dropped in a datagramm mode and postponed in a storeAndForward mode.
For CLIENT mode - the problem is because SMSC GW does not establish a connection just before it has a message to send. In CLIENT mode and no traffic case SMSC GW will periodically drop connection and auto reconnect. If we want this in the CLIENT mode we need to add the functionality to create a CLIENT TCP connection when we have a message to send before message sending. So that is the reason for we will introduce this behaviour only for SERVER mode.

So this way can be recommended in SERVER mode and no outgoing traffic. We need to update manual with the mentioning that:

• the best solution is ENQUIRE messages soulution
• only if a peer does not support it and only for SERVER mode we can suggest this solution. And if we have outgoing traffic, we have a risk to drop it in case of datagramm messaging mode.

Docs: describe "SRI only request" application interface

We need to describe the case when an application wants to generate some SRI request (via SMSC GW) to HLR without the following message sending but getting back response to the application with IMSI and NNN (VLR number).

This is achieavable in the last SNAPSHOT version (and next 6.2.4.GA) by default mproc rules. We can configure the rule like:
smsc mproc add 1 origesmenamemask XXX dropaftersri true
in this case all messages that fits to the mproc rules conditions (in this example == all messages that have come from ESME with name "XXX") will be dropped after a SRI request without futher message sending ("dropaftersri true" says for it). A condition can be any of possible (not exactly "origesmenamemask XXX"), this condition is good if you want to send all messages for SRI request only via a dedicated for this purpose ESME.

We can then get SRI response results by one of the following ways (pay attention that it may be also a MAP error response from HLR):

• CDR will contain extra fields for IMSI / VLR (if the response is successfull)
• you can request a receipt that will be routed back to your ESME and this receipt will contain also extra fields (IMSI / VLR).

We need also describe the response examples (CDR and Receipts) + error codes (CDR and Receipts) to differentiate the succeded SRI resopnse and MAP error messages (and message delivery cases).

Docs: adding description for MAP error messages / messageflow

*** This is a draft comment - we need to update it ***

SYSTEM_FAILURE message can mean two things:

1. receiving of SYSTEM_FAILURE MAP error message from a peer (some internal error at peer side)
2. some internal errors at SMSC GW side. These error are logged into sever.log log file and can be observed

MESSAGE_QUEUE_FULL means that the message stroge at a subscribers's SIM card is full

"UNKNOWN_SUBSCRIBER" - unknown subscribe MAP error message is recieved after SRI request
"UNDEFINED_SUBSCRIBER" - undefined subscribe MAP error message is recieved after MTForwarsSM request

How can we reduce the failure rate

For most of error you will hardly reduce the failure rate. If subscriber is absent, his message query is full or it is makred as barred in HLR - you can not affect it. 20 % rate is not high.

If you want to investigate it you need to capture traces and try understand failure reasons. What you can affect is a MAP transport network to HLR's / MSC/s. SCCP routing can be badly configured at your or peer's sides for example or there may be some restrictyions at a peer side or SMSC routing rules are bad.

Bundle DataStax in SMSC binary

SMSC Binary doesn't include DataStax and we request users to download it separately.
However now TeleStax is official partner with DataStax and it makes sense to bundle DataStax in SMSC binary.

hardcode sender name for esme

if esme.xml, has sender_name configured for an esme, then irrespective of what is in the submit_sm, smsc should overwrite this ...

optional: make this a regex field, so multiple options can be set

Support for delivering messages via SGSN (GPRS support)

This feature must be configurable, possible values:

• not supported
• supported, first delivery address is MSC
• supported, first delivery address is SGSN

To support it we should set gprsSupportIndicator parameter to true (in SRI request) and then properly process possible two NetworkNodeNumber addresses in SRI response

Docs: JAVA version in the manual

We need to specify in the manual (Installation Pre-Requisites) that the only supported JAVA version is JAVA7 (+Oralce/64 is prefered)

Docs+code: how to accept SRI requests with SSN==6

In home routing procedure incoming SRI requests are routed into SSN==6 (to HLR) instead of other
incoming message (MO, MT). We have a problem hier because SMSC GW TCAP stack listerns only SSN==8 and does not listern SSN==6.

Support of LMSI delivering in MTForwardSM request

This feature must be configurable.
If SRI response contains LMSI parameter and this feature is turned on than in the first MTForwardSM request we need:

• SM_RP_DA should contains LMSI
• IMSI must be put as DestReference parameter of MAPDialog (AddressNature=international_number, NumberingPlan=land_mobile)

C-cedilla capital / small in GSM7 encoding

http://www.unicode.org/Public/MAPPINGS/ETSI/GSM0338.TXT
and
3GPP TS 23.038
have different ideas which C-cedilla character (capital / small) "Ç" / "ç".
We have a ticket: https://telestax.zendesk.com/tickets/31880
We need to decide which of character will we encode / decode (may be configurable, not sure). The customer propose encode both characters, I am not sure here.

here is some unit test for it:
org.mobicents.protocols.ss7.map.datacoding.GsmEncodingTest

Big CLI response gives BufferOverflowException

1. https://telestax.zendesk.com/tickets/31430
   smsc esme show
   When the response string is more than 8000 byte we have an Exception at server side in CLI.
   in
   org.mobicents.ss7.management.transceiver.ShellChannel :
   we have hardcored length:
   private ByteBuffer txBuffer = ByteBuffer.allocateDirect(8192);
   We need increase this limit but it is better to make some flexible solutions like - if a respond String is too long we will split it and return a response part by part.
2. We can also think of split response inself for SMSC request "smsc esme show" like:

• "smsc esme show" returns first XXX values
• "smsc esme show YY" returns next XXX values after esme YY

3. I have also the bug at CLI cliect size - when a user press "row up" after multistring response from a server the Exception occurs I do not yet know why.

Remove advertisement from GUI

When a user presses a menu link in GUI, an advertisment goes down and up before the user will be switched into the next topic. Let's remove it.

White/Black List MProc

Add White/Black List MProc

Docs: ESME cluster clarifications

1. Let's add ESME cluster fundamentals clarification into the chapter "5.3.2. The procedure of routing messages to destinations inside subnetworks."

2. Let's explain in "6.3. External Short Messaging Entities (ESMEs)" chapter the rule that we have to specify the same "routing address" for every ESME in the single cluster.

PS: may be we need to implement more imtelegent ESME configuring to escape a possibility like configuring ESMEs in the cluster with different routing addresses etc.

cassandra.cql dublicate SM_STATUS

The SM_STATUS filed is duplicated in the following tables:
SLOT_MESSAGES_TABLE_YYYY_MM_DD
MESSAGES_YYYY_MM_DD

Caring of messageId in case of delivery report transit

When SMSC GW is a transit point for SMPP messages (ESME1 -> SMSC GW -> ESME2), there is a possibility for delivery report transit back to originator (ESME2 -> SMSC GW -> ESME1). In this case we need to replace messageId field in the message content of receipt to the value that SMSC GW has returned to ESME1 in the submit_sm_response.

To achieve it we need to setu extra tables in cassandra database (a table per a day). If SMSC GW is configured to provide a receipt transit, at the step of delivering messages into ESME (RxSmpp) SMSC GW will check if a message contains a receipt request and store message data into a database.
At the step of receiving a reciept from SMPP (TxSmpp) SMSC GW will check if it is a receipt and then
look at the database and update the message content.

https://telestax.zendesk.com/tickets/31975#

Do not return messages into the SIP back

In SMPP ESME we have now behavior that we will not route a message back to the same ESME SMPP.

Let's make the same behavior for SIP connector. We will not route a message back to SIP if
the messages has come from SIP.

Making if some MAP error responses as a temporary failure

Some users needs that some MAP error responses were treated by SMSC GW as temporary failure (not permanent as we have now) and resend messages for them. We need to make such functionality configurable so users can select if such errors will be processed as temporary or permanent.
Errors that can be:

• SystemFailure
• sm-DeliveryFailure with cause EquipmentProtocolError

Docker Image for SMSC

Model it similarly to https://github.com/RestComm/Restcomm-Docker and https://github.com/RestComm/load-balancer-docker

GUI: Adding originatorSccpAddress into mproc rules

We need to update GUI after https://telestax.atlassian.net/browse/SMSC-285:

adding "originatorSccpAddress" parameter into mproc create/update

Docs: Routing to an ESME when ESME is down / cluster routing issues

https://telestax.zendesk.com/agent/tickets/33019 :

1. The current algo for routing is the following. We have two steps:

1 step - selecting ESME cluster for delivering. Each cluster can contain one or more ESME. The idea of cluster usage is to have load sharing between nodes (ESMEs) inside a cluster and/or if some cluster node are unbound then SMSC GW will send messages to another cluster.
SMSC GW selects the first cluster that ESME fits to the conditions:

• bindType==transiever or receiver (for SERVER type) or transmitter (for CLIENT type)
• the ESME is not the ESME from which the message has come to SMSC GW
• networkID of a message == networkID of an ESME
• dest address, ton and npi of a message fit to routing address, ton and npi of ESME
  The fact if this ESME is bound or not is not taken into account at this step

2 step - selecting of ESME in the cluster

• selecting of ESME in the cluster that is bound (and not stopped) now. If all ESMEs are not bound in the culster then the message will be dropped

2. We need to describe in the manual that "all routing addresses templates, npi and ton in all ESMEs in the same cluster must be equal for messages will be routed properly". May be we also need to add some functionality that will prevent of ESME creating if this condiotion is not fit.
   See also: #48

MO messages : getting of StatusReportRequest field - generating of SMS-STATUS-REPORT

A) For MO and HR (?) incoming messages there is a field:
"smsSubmitTpdu.getStatusReportRequest()"
that is not taken into account. The meaning of this field is that SMSC GW will need to send a delivery receipt to the originator (no standard of a receipt format).

1. we need to insert this value into Sms class and MProcMessage interface
2. add a default mproc rule that can send a response (+ possibility of custom mproc rules)

B) we need to persist into cassandra database this field (StatusReportRequest) and field (originatorSccpAddress from issue 289)

Check also database update issue: #95

We need to think also for "SMS-STATUS-REPORT for MO originated messages". This needs some investigations.

Mobile Number Portability MProc

Add Mobile Number Portability MProc

Spam filtering (MProc) based on SMS content and called/calling party address

Add Spam filtering (MProc) based on SMS content and called/calling party address

GUI: for commands smsc set hrhlrnumber, smsc set hrsribypass

1. We need to add GUI for 2 general SMSC GW options:

• smsc set hrhlrnumber
• smsc set hrsribypass

Both have "networkID" parameter like we have for "scgt" parameter.

hrhlrnumber is already present without "networkID" parameter
hrsribypass is completely new;

Description is there:
#29
#45

2. I have tested "HR Global Title Indicator and HR SRI Bypass Network Id" - hrhlrnumber with 0, non 0 and a new networkID. It does not work properly. Let's fix it. We need also be sure that removing of values also works and is proper documented.

Delivery algorithm improvement

Now SMSC GW uses the following delivery retry Algo. It depends on the reasin of teh failure.

1. busy_subscriber reason: - next retry with SubscriberBusyDueDelay parameter delay (a short delay)

2. other temporary reasons (like absent subscriber or some SCCP part failure or some other reasons): some increasing algo using SecondDueDelay, DueDelayMultiplicator, MaxDueDelay parameters. In this case is also validityPeriod of the message used, no more retries ofter it is over

3. permanent reasons - no more retries
   All messages for an SS7 destination are scheduled to one time point and if a new more message to dest address has come, this message will be scheduled to the same time as all other pending messages.

At every delivery attempt ValidityPeriod message value is checked and if it over, this message is dropped.

Corresponsed issue:
#12

Check also database update issue: #95

After Alert new coming messages to that address can be stored for a too far due_slot

If:

• a message is stored in the database for the future delivery after a delivery failure
• then comes Alert from HLR and the message is successfully delivered
• after this delivery DST_SLOT_TABLE_YYYY_MM_DD table for this dest address still contains the due_slot for the far future
• if a new message comes for this address it will be sheduled for the far future

Solution: we need to update the due_slot when Alert request (after delivery success only !!!) for some due_slot in the past so new messages will be scheduled in the proper way.

Session-based charging for Diameter CCR

From ticket 32979:
At this time, CCR only support EVENT_REQUEST, we need to support INITIAL (type 1), UPDATE_REQUEST, TERMINATION_REQUEST and it need to be configured:
https://github.com/RestComm/smscgateway/blob/master/core/slee/services/chargingsbb/src/main/java/org/mobicents/smsc/slee/services/charging/ChargingSbb.java

Possible unbind() and close() Exception

When SMSC GW is unbinding / closing SMPP connection then it may occurs that the base TCP connection is already down. This leads the Exception that is not caught and next "close() / destroy()" method is not invoked.

This leads than this ESME will not start again.

found methods for this case:
SmppClientManagement.stopSmppClientSession(Esme esme)
EsmeManagement.stopWrappedSession(Esme esme)

we need to find out also ALL methods with "close()" and "unbind()" in smpp part and
enclose them with try - catch section with just logging.

Multitenancy for hrhlrnumber

We have this parameter: "6.1.2.9. Pre-configured HLR address for SRI messages" that is global for whole SMSC GW. Sometimes we need to have different hrhlrnumber for diffeent networkId.

Let's ad multi-tenancy for this parameter like we have for "SMSC Gateway Global Title".

smsc set hrhlrnumber <hlr GT digits> networkid <networkId>

If no "hlr GT digits" is configured for nextworkId this means that we need to use MSISDN as CalledPartyAddress.

Restriction in ESME creation / updating time: routing-address, ton and npi must be the same in the cluster

When we are creation or updating ESME then we need to check that:
routing-adress,ton and npi must be same for all ESMEs in the cluster (when we are adding 2-nd, 3-d, etc ESME in the cluster).

Issue for a manual for it: #46

See also the issue: #143 (to do all together)

BindTimeout option for ESME

We need to add a configuring for BindTimeout option for CLIENT ESMEs (and pass them to cloud hopper).

https://telestax.zendesk.com/tickets/32969#

ErrorCode values must be propely used

We must create a proper set of ErrorCode values and then use them in proper way in all SMSC parts.