ostelco/ostelco-core

Cloud-native Telco BSS hosted in GCP K8s with standalone Diameter to gRPC gateway. Rule Engine using Neo4j graphs. Analytics Events sent to GCP BigData (Dataflow+BigQuery) via PubSub. It's awesome!

ostelco-core

Mono Repository for core protocols and services around a OCS/BSS for packet data. For each service please see the individual Readme.md files.

• The big picture (diagram) of current work-flow
• General Documentation
• admin-api
• auth-server
• client-api
• dataflow-pipelines
• diameter-stack
• diameter-test
• exporter
• neo4j-store
• ocs-grpc-api
• ocsgw
• payment-processor
• prime
• infra
• pseudonym-server
• seagull
• neo4j-admin-tools

ostelco-core's Issues

[OCS-gw] Tcc timer still triggered although session is removed

If a session is moved from one OCS gateway to another then the TCC timer is not cleared from the first instance. If the user then terminate the session the data kept in Redis is cleared.

Then when the timer triggers it will try to fetch the data from Redis and since it is gone it throws an IllegalStateException.

java.lang.IllegalStateException: null
at org.ostelco.diameter.ha.common.AppSessionDataReplicatedImpl.getApplicationId (AppSessionDataReplicatedImpl.kt:57)
at org.jdiameter.common.impl.app.AppSessionImpl.getSessionAppId (AppSessionImpl.java:130)
at org.jdiameter.common.impl.app.AppSessionImpl.release (AppSessionImpl.java:140)
at org.jdiameter.common.impl.app.cca.AppCCASessionImpl.release (AppCCASessionImpl.java:90)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.release (ServerCCASessionImpl.java:494)
at org.jdiameter.common.impl.app.cca.CCASessionFactoryImpl.sessionSupervisionTimerExpired (CCASessionFactoryImpl.java:435)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl$TccScheduledTask.run (ServerCCASessionImpl.java:356)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.onTimer (ServerCCASessionImpl.java:438)
at org.ostelco.diameter.ha.timer.ReplicatedTimerTask.runTask (ReplicatedTimerTaskData.kt:99)
at org.ostelco.diameter.ha.timer.ReplicatedTimerTask.run (ReplicatedTimerTaskData.kt:77)

[Prime] one subscriber should be able to have more then one msisdn

Currently it is only possible for one subscriber to have one SIM card. We need to redesign the DB so that we can have 1:N instead

[Prime]Implement Redirect in FinalUnitIndication

https://tools.ietf.org/html/rfc4006#section-5.6.3

[Prime] Low balance notifications is no longer sent

After the refactor to ocs-ktc there is no longer any notifications sent to handsets on low balance.

[OCS-gw] Handle timeouts on pubsub

There is no timeout in ocs gateway for CCR-I messages on pubsub. This is an issue since the P-GW will have a timeout and after that it will go over to the mode set fo Credit-Control-Failure-Handling. If this is set to "continue" the subscriber will get unlimited access until handset re-attach to network. We should have a default policy for how to handle this.

[OCSgw] [Prime] Handle Roaming

From the SGSN-MNC-MCC value in the CCR we can figure out if the subscriber is roaming.
This value should be sent all the way from OCSgw -> Prime.

For Roaming we could charge in value instead of units and therefor make a reservation of units from the amount the user can spend.

[OCS-gw] pubsub match messages on request identifier

Currently the pubsub messages are matched on session Id but not on CC-Request-Number. This is not ideal in case of resends from the P-GW where we might send answer with incorrect CC-Request-Number.

[Prime] Send push notification on low balance

When a subscriber is about to run out of balance we should notify this with a push notification.

[Prime][OCSgw]CCR graceful shutdown does not seem to work correctly

Currently I send the following Credit-Control-Answer when the user has no more credit:

09 Apr 2018 10:34:16,946 INFO CreditControlContext - Credit-Control-Answer
09 Apr 2018 10:34:16,946 INFO DiameterUtilities - 263 : Session-Id (UTF8String) . . . . . . . . . . 0004-sessmgr.Gy_Diameter;15020002;516;5acb41a8-402
09 Apr 2018 10:34:16,946 INFO DiameterUtilities - 258 : Auth-Application-Id (Unsigned32). . . . . . 4
09 Apr 2018 10:34:16,946 INFO DiameterUtilities - 268 : Result-Code (Unsigned32). . . . . . . . . . 2001
09 Apr 2018 10:34:16,947 INFO DiameterUtilities - 416 : CC-Request-Type (Integer32) . . . . . . . . 1
09 Apr 2018 10:34:16,947 INFO DiameterUtilities - 415 : CC-Request-Number (Unsigned32). . . . . . . 0
09 Apr 2018 10:34:16,948 INFO DiameterUtilities - 264 : Origin-Host (OctetString) . . . . . . . . . 0004-sessmgr.Gy_Diameter
09 Apr 2018 10:34:16,948 INFO DiameterUtilities - 296 : Origin-Realm (OctetString). . . . . . . . . loltel
09 Apr 2018 10:34:16,949 INFO DiameterUtilities - 456 : Multiple-Services-Credit-Control (Grouped).
09 Apr 2018 10:34:16,949 INFO DiameterUtilities - 432 : Rating-Group (Unsigned32) . . . . . . . . 10
09 Apr 2018 10:34:16,949 INFO DiameterUtilities - 439 : Service-Identifier (Unsigned32) . . . . . 1
09 Apr 2018 10:34:16,949 INFO DiameterUtilities - 431 : Granted-Service-Unit (Grouped). . . . . .
09 Apr 2018 10:34:16,949 INFO DiameterUtilities - 412 : CC-Input-Octets (Unsigned64). . . . . . 0
09 Apr 2018 10:34:16,952 INFO DiameterUtilities - 414 : CC-Output-Octets (Unsigned64) . . . . . 0
09 Apr 2018 10:34:16,952 INFO DiameterUtilities - 420 : CC-Time (Unsigned32). . . . . . . . . . 0
09 Apr 2018 10:34:16,953 INFO DiameterUtilities - 421 : CC-Total-Octets (Unsigned64). . . . . . 0
09 Apr 2018 10:34:16,953 INFO DiameterUtilities - 417 : CC-Service-Specific-Units (Unsigned64). 0
09 Apr 2018 10:34:16,953 INFO DiameterUtilities - 430 : Final-Unit-Indication (Grouped) . . . . .
09 Apr 2018 10:34:16,955 INFO DiameterUtilities - 449 : Final-Unit-Action (Integer32) . . . . . 0
09 Apr 2018 10:34:16,965 INFO DiameterUtilities - 268 : Result-Code (Unsigned32). . . . . . . . . 4012
09 Apr 2018 10:34:16,965 INFO DiameterUtilities - 448 : Validity-Time (Unsigned32). . . . . . . . 86400

I am not sure this is correct. As the P-GW keep on hammering with Credit-Control-Request Update after this.

09 Apr 2018 10:34:16,974 INFO OcsApplication - << Received Credit-Control-Request [ UPDATE ]
09 Apr 2018 10:34:16,975 INFO DiameterUtilities - 263 : Session-Id (UTF8String) . . . . . . . . . . 0004-sessmgr.Gy_Diameter;15020002;516;5acb41a8-402
09 Apr 2018 10:34:16,977 INFO DiameterUtilities - 264 : Origin-Host (OctetString) . . . . . . . . . 0004-sessmgr.Gy_Diameter
09 Apr 2018 10:34:16,977 INFO DiameterUtilities - 296 : Origin-Realm (OctetString). . . . . . . . . loltel
09 Apr 2018 10:34:16,977 INFO DiameterUtilities - 283 : Destination-Realm (OctetString) . . . . . . loltel
09 Apr 2018 10:34:16,977 INFO DiameterUtilities - 258 : Auth-Application-Id (Unsigned32). . . . . . 4
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 461 : Service-Context-Id (UTF8String) . . . . . . [email protected]
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 416 : CC-Request-Type (Integer32) . . . . . . . . 2
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 415 : CC-Request-Number (Unsigned32). . . . . . . 1
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 293 : Destination-Host (OctetString). . . . . . . 0004-sessmgr.Gy_Diameter
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 1 : User-Name (UTF8String). . . . . . . . . . . . 4790300157@loltel-test
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 278 : Origin-State-Id (Unsigned32). . . . . . . . 1518698721
09 Apr 2018 10:34:16,978 INFO DiameterUtilities - 55 : Event-Timestamp (Time) . . . . . . . . . . . Mon Apr 09 10:34:16 GMT 2018
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 443 : Subscription-Id (Grouped) . . . . . . . . .
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 450 : Subscription-Id-Type (Integer32). . . . . 0
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 444 : Subscription-Id-Data (UTF8String) . . . . 4790300157
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 443 : Subscription-Id (Grouped) . . . . . . . . .
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 450 : Subscription-Id-Type (Integer32). . . . . 1
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 444 : Subscription-Id-Data (UTF8String) . . . . 242017100000122
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 456 : Multiple-Services-Credit-Control (Grouped).
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 446 : Used-Service-Unit (Grouped) . . . . . . .
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 420 : CC-Time (Unsigned32). . . . . . . . . . 0
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 417 : CC-Service-Specific-Units (Unsigned64). 0
09 Apr 2018 10:34:16,979 INFO DiameterUtilities - 439 : Service-Identifier (Unsigned32) . . . . . 1
09 Apr 2018 10:34:16,980 INFO DiameterUtilities - 432 : Rating-Group (Unsigned32) . . . . . . . . 10
09 Apr 2018 10:34:16,980 INFO DiameterUtilities - 872 : Reporting-Reason (Integer32). . . . . . . 2

Notice that the server set the Reporting-Reason to 2 in this update request. Need to figure out when I am supposed to do when I get this.

CVE-2012-6708 (Medium) detected in jquery-1.7.1.min.js

CVE-2012-6708 - Medium Severity Vulnerability

Vulnerable Library - jquery-1.7.1.min.js

JavaScript library for DOM operations

Library home page: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.1/jquery.min.js

Path to dependency file: /tmp/ws-scm/ostelco-core/houston/node_modules/sockjs/examples/hapi/html/index.html

Path to vulnerable library: /ostelco-core/houston/node_modules/sockjs/examples/hapi/html/index.html,/ostelco-core/houston/node_modules/winchan/example/jquery.js,/ostelco-core/houston/node_modules/vm-browserify/example/run/index.html,/ostelco-core/houston/node_modules/sockjs/examples/multiplex/index.html,/ostelco-core/houston/node_modules/sockjs/examples/express/index.html,/ostelco-core/houston/node_modules/winchan/complex_example/jquery.js,/ostelco-core/houston/node_modules/sockjs/examples/express-3.x/index.html,/ostelco-core/houston/node_modules/winchan/test/stress/jquery.js,/ostelco-core/houston/node_modules/sockjs/examples/echo/index.html

Dependency Hierarchy:

• ❌ jquery-1.7.1.min.js (Vulnerable Library)

Found in HEAD commit: ee14cf0e70f50371f3f37d823ad293487ab4a002

Vulnerability Details

jQuery before 1.9.0 is vulnerable to Cross-site Scripting (XSS) attacks. The jQuery(strInput) function does not differentiate selectors from HTML in a reliable fashion. In vulnerable versions, jQuery determined whether the input was HTML by looking for the '<' character anywhere in the string, giving attackers more flexibility when attempting to construct a malicious payload. In fixed versions, jQuery only deems the input to be HTML if it explicitly starts with the '<' character, limiting exploitability only to attackers who can control the beginning of a string, which is far less common.

Publish Date: 2018-01-18

URL: CVE-2012-6708

CVSS 3 Score Details (6.1)

Base Score Metrics:

• Exploitability Metrics:
  • Attack Vector: Network
  • Attack Complexity: Low
  • Privileges Required: None
  • User Interaction: Required
  • Scope: Changed
• Impact Metrics:
  • Confidentiality Impact: Low
  • Integrity Impact: Low
  • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://nvd.nist.gov/vuln/detail/CVE-2012-6708

Release Date: 2018-01-18

Fix Resolution: jQuery - v1.9.0

---

Step up your Open Source Security Game with WhiteSource here

Make sure that test can be run automatically.

Currently the test is run automatically when you run docker-compose up in the base folder of the ostelco-core project.

But what we would like is an end to end test that can be triggered that would

• run docker-compose up
• expose the results from the test
• do docker-compose down

[OCS] BundleId missing in PrimeEvent for CCR

https://github.com/ostelco/ostelco-core/blob/develop/ocs/src/main/kotlin/org/ostelco/prime/events/EventProcessor.kt#L34

requires a bundle id. But it is not set in

https://github.com/ostelco/ostelco-core/blob/develop/ocs/src/main/kotlin/org/ostelco/prime/ocs/OcsState.kt#L45

[Prime] Implement proper zero-rating

Currently the Rating-Groups are hardcoded. This should be moved to configuration. Non specified Rating-Groups and Service-Identifiers should be blocked.

[OCSgw] Conditional logging filter

It would be nice if there was a way to enable more verbose logging for a msisdn.

This can be achieved by a filter in logback. We can make this dynamic by setting :

for in the logback configration file, which will make detect if the configuration changed. To replace the configuration file ocs gateway will check for a new file regularly.

[P-GW][OCSgw] restart does not re-init all session

It seems that when we restart the OCSgw it is possible that not all diameter sessions are recreated.
We have seen that for some msisdn we no longer get CCR after ocsgw restart.

[OCSgw] HA failed to fetch STATE from Redis

Getting the STATE value from Redis fail in ServerCCASessionDataReplicatedImpl

val value = getValue(STATE) return null

I see the following stack.

Caught exception from session object!
java.lang.IllegalStateException: null
at org.ostelco.diameter.ha.server.ServerCCASessionDataReplicatedImpl.getServerCCASessionState(ServerCCASessionDataReplicatedImpl.kt:50)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.setState(ServerCCASessionImpl.java:472)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.setState(ServerCCASessionImpl.java:467)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl$TccScheduledTask.run(ServerCCASessionImpl.java:361)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.onTimer(ServerCCASessionImpl.java:438)
at org.ostelco.diameter.ha.timer.ReplicatedTimerTask.runTask(ReplicatedTimerTaskData.kt:99)
at org.ostelco.diameter.ha.timer.ReplicatedTimerTask.run(ReplicatedTimerTaskData.kt:77)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:835)"

[OCSgw] origin host seems to be incorrect in CCA

ocsgw | 11 Apr 2018 11:06:04,415 INFO CreditControlContext - Credit-Control-Answer
ocsgw | 11 Apr 2018 11:06:04,416 INFO DiameterUtilities - 263 : Session-Id (UTF8String) . . . . . . . . . . 0004-sessmgr.Gy_Diameter;15020003;772;5acdec1b-402
ocsgw | 11 Apr 2018 11:06:04,417 INFO DiameterUtilities - 258 : Auth-Application-Id (Unsigned32). . . . . . 4
ocsgw | 11 Apr 2018 11:06:04,418 INFO DiameterUtilities - 268 : Result-Code (Unsigned32). . . . . . . . . . 2001
ocsgw | 11 Apr 2018 11:06:04,420 INFO DiameterUtilities - 416 : CC-Request-Type (Integer32) . . . . . . . . 1
ocsgw | 11 Apr 2018 11:06:04,421 INFO DiameterUtilities - 415 : CC-Request-Number (Unsigned32). . . . . . . 0
ocsgw | 11 Apr 2018 11:06:04,422 INFO DiameterUtilities - 264 : Origin-Host (OctetString) . . . . . . . . . 0004-sessmgr.Gy_Diameter
ocsgw | 11 Apr 2018 11:06:04,424 INFO DiameterUtilities - 296 : Origin-Realm (OctetString). . . . . . . . . loltel

[OCS-gw] [Prime] Promethues should scrape ocsgw for metrics

We have moved OCSgw to the cloud. so the GRPC endpoint for metrics is no longer needed. Instead Promethues should scrape the metric from OCSgw directly.

This might also be an opportunity to add Dropwizard to OCSgw

[Prime] Top up is broken

Probably cause by the conversion to Kotlin

prime | ERROR [2018-04-19 07:33:42,999] com.google.firebase.database.core.ThreadPoolEventTarget: Event handler threw an exception
prime | ! java.lang.IllegalArgumentException: Parameter specified as non-null is null: method org.ostelco.prime.firebase.FbDatabaseFacade$newProductDefChangedListener$1.onChildAdded, parameter previousChildName
prime | ! at org.ostelco.prime.firebase.FbDatabaseFacade$newProductDefChangedListener$1.onChildAdded(FbDatabaseFacade.kt)
prime | ! at com.google.firebase.database.core.ChildEventRegistration.fireEvent(ChildEventRegistration.java:81)
prime | ! at com.google.firebase.database.core.view.DataEvent.fire(DataEvent.java:65)
prime | ! at com.google.firebase.database.core.view.EventRaiser$1.run(EventRaiser.java:58)
prime | ! at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
prime | ! at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
prime | ! at java.lang.Thread.run(Thread.java:748)

[Prime][OCS-gw] Pubsub broadcast is not working for activate

When the user top up balance then there should be an activate message sent from Prime to all instances of ocsgw. We planned to use pubsub with broadcast for this, and then ocsgw that has an active session for this msisdn should send a RAR message.

The current implementation does not work. The Active message is sent on the topic but it is not broadcast, so only one instance gets the message.

[Prime] Create "CDR" for usage

There should be a way to get the usage per subscriber.

[OCS-gw] Handle failures on pubsub

OCS-gw currenly does not support errors on pubsub.

We should consider having gRPC as a backup channel in case there is an error detected on pubsub.

We should handle malformed messages so that we do not get stuck in a loop where the top message can not get parsed.

[Prime] [OCSgw] make complete integration test

The current integration test rely on Firebase to have users configured and have a balance.

We should make this run without Firebase. The test should be like:

* Create a user
* Top-up the balance for this user
* Generate some CCR traffic

So that we can create tests where the user is running out of credit etc.

Test RESTRICT_ACCESS in Final-Unit-Action

There is an option in graceful service termination that will let you set a list of IP addresses that should still be available when a user is blocked from the network. We should try this out as it can be useful.

[Prime] Support Roaming

Currently all traffic is handled the same. The information about network and location is sent from OCS gateway to Prime but there is no differentiation.

Create pack tasks in gradle to create installable bundle

[OCSgw] [Prime] Reconnect gRPC

It seems like we are loosing the gRPC connection between OCSgw and Prime sometimes and when that happen it is not re established again until ocsgw is restarted.

Last item in logs seems to be an error on the streamobserver:

 https://github.com/ostelco/ostelco-core/blob/develop/ocsgw/src/main/java/org/ostelco/ocsgw/data/grpc/GrpcDataSource.java#L85

We also have a dependency that prime needs to be started before ocsgw.

Remove hardcoded product Ids used in prime, move them to configuration.

e.g.

neo4j-store/src/main/kotlin/org/ostelco/prime/storage/graph/Neo4jStore.kt
                                }
                            }.bind()
                    // Give 100 MB as free initial balance
                    var productId: String = "100MB_FREE_ON_JOINING"

@la3lma says : This is a business rule that is now encoded as code. That is in general an anti pattern, at lest update the comment to contain an XXX or TODO so that it is easer to pick up this thing and amend later.

I would prefer this particular function to be added either via configuration or via a separate API, but that's not required for this PR. This is technical debt for sure.

Implement Re-Auth Requests in ocsgw

The Charging Server needs to be able to unblock a user that has used up his quota and topped up his account. To be able to do this the server needs to communicate to the P-GW that it should try to query the OCS for a new bucket. This is done in Diameter through a Re-Auth Request. We should look into how this can be implemented in Prime and OcsGw.

[Prime] Storage should have an api to reserve/consume multiple rating groups

The Credit-Control-request can contain requests to reserve buckets in multiple rating groups in one request. It would be convenient if the Storage api had the option to send multiple reservation and consumptions in one api call.

[OCS-gw] Docker is using up diskspace on GCE host

Each time we update OCS gateway the local docker on the host system in GCP will fetch a new image from the Docker registry. But it is not deleting the old images. The host will slowly use up more and more space until it runs out.

Now I am manually cleaning up the local registry on the host when it runs out but this should be an automated job.

[OCSgw] Make sure Graceful shutdown works correctly

It seems like the P-GW sometimes keeps on sending CCR-U for new buckets. Although we reply with with Final-Unit-Indication terminate. The RFC is not telling me how to handle this so we need to hunt down a pcap or find some more doc on how this is supposed to work.

[OCS] Support for dynamic bucket size based on rate of data consumption.

Problem description

We end up configuring a large bucket size on PGW, OCSGW or OCS which can handle fastest data consumption.
But most of the users do not consume data at a peak rate.
Due to the large bucket size, the consumption happens in large steps (of 100 MB, for e.g.).
There is also a risk of large data to be unaccounted for due to technical issues like restart in components or delayed/lost messages between them.

Solution

Instead of having a large bucket size to handle the fastest data consumption, the bucket size should be proportionate for the moderate data consumption, but which can dynamically increase if someone is consuming at higher rates.

[Prime] Detect pubsub duplicate messages from OCS gateway

Cloud Pub/Sub guarantees at-least-once message delivery, which means that occasional duplicates are to be expected. In the OCS gateway we detect duplicates as the request-Id is tracked and if we get a duplicate then it is ignored. But on the Prime side there is no detection, this means that there is a possibility for double charging in the case where a message from Ocs gateway to Prime is duplicated.

[Question] Does ES2+ client usable with commercial SMDP+?

Hello,

Does ES2+ client usable out of the box with commercial GSMA grade SMDP+?
We're very excited to test it with Gemalto/THALES SMDP+ server...

Thank you for your work!
Guillaume Ballet

[OCS-gw] Add structured logging

The logs in Stackdriver from OCS-gw does not have the correct Level set and multiline logs are getting logged as separate entries. We should start using JSON based logging to get this correct.

Add roaming information to Credit-Control-Requests

In the Credit-Control-Request -> Service-Information -> PS-Information -> 3GPP-SGSN-MCC-MNC there is information that we should use for Roaming. This should be sent to Prime.

[OCSgw] [P-GW] investigate threshold for CCR

Currently the P-GW will not request a new bucket until it has used up the currently allocated bucket. This will have an impact on the speed to the UE as packets are buffered while the P-GW is awaiting a new bucket from the OCS. There is a concept where the P-GW is asking for a new bucket early and we should be able to configure this. Need to investigate how this is supposed to work and how to set this up.

[OCSgw] Re-Auth test is not stable

I see this test sometimes failing with following output:

26 Apr 2018 10:53:45,234 3696 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 30 : Called-Station-Id (UTF8String) . . . . . panacea
26 Apr 2018 10:53:45,234 3696 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 18 : 3GPP-SGSN-MCC-MNC (UTF8String) . . . . . 24201
26 Apr 2018 10:53:45,234 3696 [ApplicationSession-1] INFO org.ostelco.ocsgw.data.local.LocalDataSource - Sending Credit-Control-Answer
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.CreditControlContext - Credit-Control-Answer
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 263 : Session-Id (UTF8String) . . . . . . . . . . BadCustomSessionId;1524740025205;0
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 258 : Auth-Application-Id (Unsigned32). . . . . . 4
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 268 : Result-Code (Unsigned32). . . . . . . . . . 2001
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 416 : CC-Request-Type (Integer32) . . . . . . . . 1
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 415 : CC-Request-Number (Unsigned32). . . . . . . 0
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 264 : Origin-Host (OctetString) . . . . . . . . . ocs
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 296 : Origin-Realm (OctetString). . . . . . . . . loltel
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 456 : Multiple-Services-Credit-Control (Grouped).
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 432 : Rating-Group (Unsigned32) . . . . . . . . 10
26 Apr 2018 10:53:45,235 3697 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 439 : Service-Identifier (Unsigned32) . . . . . 1
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 431 : Granted-Service-Unit (Grouped). . . . . .
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 412 : CC-Input-Octets (Unsigned64). . . . . . 0
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 414 : CC-Output-Octets (Unsigned64) . . . . . 0
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 421 : CC-Total-Octets (Unsigned64). . . . . . 500000
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 268 : Result-Code (Unsigned32). . . . . . . . . 2001
26 Apr 2018 10:53:45,236 3698 [ApplicationSession-1] INFO org.ostelco.diameter.util.DiameterUtilities - 448 : Validity-Time (Unsigned32). . . . . . . . 86400
26 Apr 2018 10:53:45,249 3711 [FSM-SPeer{Uri=aaa://ocs; State=null; con=null; incConnull }_3-2] INFO org.ostelco.diameter.test.TestClient - Received Answer: 272
E2E:2069889051
HBH:26632034
AppID:4
26 Apr 2018 10:53:45,269 3731 [FSM-SPeer{Uri=aaa://ocs; State=null; con=null; incConnull }_3-2] INFO org.ostelco.diameter.test.TestClient - AVPS[8]:
26 Apr 2018 10:53:45,274 3736 [TCPReader-1] INFO org.jdiameter.client.impl.transport.tcp.TCPTransportClient - Read thread is stopped for socket [Socket[addr=/127.0.0.1,port=60612,localport=3868]]
26 Apr 2018 10:53:45,274 3736 [ApplicationSession-1] INFO org.jdiameter.common.impl.app.cca.CCASessionFactoryImpl - Diameter CCA SessionFactory :: stateChanged :: source[ServerCCASessionImpl [sessionData=org.jdiameter.server.impl.app.cca.ServerCCASessionDataLocalImpl@fe2b7deb]], oldState[IDLE], newState[OPEN]
26 Apr 2018 10:53:45,718 4180 [main] ERROR org.jdiameter.client.impl.controller.PeerTableImpl - Can not send message
java.lang.NullPointerException
at org.jdiameter.client.impl.controller.PeerTableImpl.sendMessage(PeerTableImpl.java:233)
at org.jdiameter.client.impl.StackImpl.sendMessage(StackImpl.java:471)
at org.jdiameter.client.impl.BaseSessionImpl.genericSend(BaseSessionImpl.java:202)
at org.jdiameter.client.impl.BaseSessionImpl.genericSend(BaseSessionImpl.java:168)
at org.jdiameter.client.impl.SessionImpl.send(SessionImpl.java:110)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.dispatchEvent(ServerCCASessionImpl.java:518)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.handleEvent(ServerCCASessionImpl.java:318)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.send(ServerCCASessionImpl.java:508)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.sendReAuthRequest(ServerCCASessionImpl.java:137)
at org.ostelco.ocsgw.OcsServer.sendReAuthRequest(OcsServer.java:72)
at org.ostelco.ocsgw.OcsApplicationTest.testReAuthRequest(OcsApplicationTest.java:161)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:436)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:170)
at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:166)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:113)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
at org.junit.platform.console.tasks.ConsoleTestExecutor.executeTests(ConsoleTestExecutor.java:65)
at org.junit.platform.console.tasks.ConsoleTestExecutor.lambda$execute$0(ConsoleTestExecutor.java:57)
at org.junit.platform.console.tasks.CustomContextClassLoaderExecutor.invoke(CustomContextClassLoaderExecutor.java:33)
at org.junit.platform.console.tasks.ConsoleTestExecutor.execute(ConsoleTestExecutor.java:57)
at org.junit.platform.console.ConsoleLauncher.executeTests(ConsoleLauncher.java:84)
at org.junit.platform.console.ConsoleLauncher.execute(ConsoleLauncher.java:74)
at org.junit.platform.console.ConsoleLauncher.execute(ConsoleLauncher.java:47)
at org.junit.platform.console.ConsoleLauncher.main(ConsoleLauncher.java:39)
26 Apr 2018 10:53:50,743 9205 [main] INFO org.jdiameter.client.impl.StackImpl - (-)(-)(-)(-)(-) Stopping Mobicents DIAMETER Stack v1.7.1-123 (-)(-)(-)(-)(-)
26 Apr 2018 10:53:52,942 11404 [main] INFO org.jdiameter.client.impl.StackImpl - (-)(-)(-)(-)(-) Stopped Mobicents DIAMETER Stack v1.7.1-123 (-)(-)(-)(-)(-)
Failures (1):
JUnit Jupiter:OcsApplicationTest:testReAuthRequest()
MethodSource [className = 'org.ostelco.ocsgw.OcsApplicationTest', methodName = 'testReAuthRequest', methodParameterTypes = '']
=> java.lang.AssertionError: expected: but was:
org.junit.Assert.fail(Assert.java:88)
org.junit.Assert.failNotEquals(Assert.java:834)
org.junit.Assert.assertEquals(Assert.java:118)
org.junit.Assert.assertEquals(Assert.java:144)
org.ostelco.ocsgw.OcsApplicationTest.waitForRequest(OcsApplicationTest.java:261)
org.ostelco.ocsgw.OcsApplicationTest.testReAuthRequest(OcsApplicationTest.java:162)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
[...]
Test run finished after 12237 ms
[ 3 containers found ]
[ 0 containers skipped ]
[ 3 containers started ]
[ 0 containers aborted ]
[ 3 containers successful ]
[ 0 containers failed ]
[ 2 tests found ]
[ 0 tests skipped ]
[ 2 tests started ]
[ 0 tests aborted ]
[ 1 tests successful ]
[ 1 tests failed ]

Logo Design Offer as Open Source Contribution

Hello Sir. I'm an UI/UX and Graphics Designer. I want to provide a logo for your project. Would you mind if I propose a new logo design for your project as my Open Source Contribution?

Thanks before.

[OCSgw] [Prime] Reliable restart of Prime gRPC

There is still some issue with restart of gRPC connections. We need to look into this so that we can restart the cluster without breaking the gRPC connection for OCSgw.

This is the log from the OCSgw

ocsgw | 11 Jun 2018 09:38:48,982 ERROR GrpcDataSource - ActivateObserver error
ocsgw | io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
ocsgw | at io.grpc.Status.asRuntimeException(Status.java:526)
ocsgw | at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
ocsgw | at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
ocsgw | at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
ocsgw | at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
ocsgw | at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
ocsgw | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
ocsgw | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
ocsgw | at java.lang.Thread.run(Thread.java:748)
ocsgw | Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: ocs.endpoints.pantel-2decb.cloud.goog/35.195.49.238:80
ocsgw | at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
ocsgw | at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
ocsgw | at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
ocsgw | at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
ocsgw | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
ocsgw | at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
ocsgw | ... 1 more
ocsgw | 11 Jun 2018 09:38:48,984 INFO GrpcDataSource - reconnectActivate called
ocsgw | 11 Jun 2018 09:38:48,984 INFO GrpcDataSource - Schedule new Callable initActivate
ocsgw | 11 Jun 2018 09:38:48,984 ERROR GrpcDataSource - CreditControlRequestObserver error
ocsgw | io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
ocsgw | at io.grpc.Status.asRuntimeException(Status.java:526)
ocsgw | at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
ocsgw | at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
ocsgw | at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
ocsgw | at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
ocsgw | at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
ocsgw | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
ocsgw | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
ocsgw | at java.lang.Thread.run(Thread.java:748)
ocsgw | Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: ocs.endpoints.pantel-2decb.cloud.goog/35.195.49.238:80
ocsgw | at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
ocsgw | at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
ocsgw | at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
ocsgw | at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
ocsgw | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
ocsgw | at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
ocsgw | ... 1 more
ocsgw | 11 Jun 2018 09:38:48,985 INFO GrpcDataSource - reconnectCreditControlRequest called
ocsgw | 11 Jun 2018 09:38:48,986 INFO GrpcDataSource - Schedule new Callable initCreditControlRequest
ocsgw | 11 Jun 2018 09:38:53,984 INFO GrpcDataSource - Calling initActivate
ocsgw | 11 Jun 2018 09:38:53,992 ERROR GrpcDataSource - ActivateObserver error
ocsgw | io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
ocsgw | at io.grpc.Status.asRuntimeException(Status.java:526)
ocsgw | at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
ocsgw | at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
ocsgw | at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
ocsgw | at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
ocsgw | at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
ocsgw | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
ocsgw | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
ocsgw | at java.lang.Thread.run(Thread.java:748)
ocsgw | Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: ocs.endpoints.pantel-2decb.cloud.goog/35.195.49.238:80
ocsgw | at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
ocsgw | at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
ocsgw | at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
ocsgw | at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
ocsgw | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
ocsgw | at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
ocsgw | ... 1 more
ocsgw | 11 Jun 2018 09:38:53,998 INFO GrpcDataSource - reconnectActivate called
ocsgw | 11 Jun 2018 09:38:54,000 INFO GrpcDataSource - Schedule new Callable initActivate
ocsgw | 11 Jun 2018 09:38:53,996 INFO GrpcDataSource - Calling initCreditControlRequest
ocsgw | 11 Jun 2018 09:38:54,001 ERROR GrpcDataSource - CreditControlRequestObserver error
ocsgw | io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
ocsgw | at io.grpc.Status.asRuntimeException(Status.java:526)
ocsgw | at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
ocsgw | at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
ocsgw | at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
ocsgw | at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
ocsgw | at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
ocsgw | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
ocsgw | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
ocsgw | at java.lang.Thread.run(Thread.java:748)
ocsgw | Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: ocs.endpoints.pantel-2decb.cloud.goog/35.195.49.238:80
ocsgw | at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
ocsgw | at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
ocsgw | at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
ocsgw | at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
ocsgw | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
ocsgw | at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
ocsgw | ... 1 more
ocsgw | 11 Jun 2018 09:38:54,008 INFO GrpcDataSource - reconnectCreditControlRequest called
ocsgw | 11 Jun 2018 09:38:54,005 INFO GrpcDataSource - Calling initKeepAlive
ocsgw | 11 Jun 2018 09:38:59,006 INFO GrpcDataSource - Calling initActivate
ocsgw | 11 Jun 2018 09:38:59,015 ERROR GrpcDataSource - ActivateObserver error
ocsgw | io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
ocsgw | at io.grpc.Status.asRuntimeException(Status.java:526)
ocsgw | at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
ocsgw | at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
ocsgw | at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
ocsgw | at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
ocsgw | at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
ocsgw | at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
ocsgw | at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
ocsgw | at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
ocsgw | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
ocsgw | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
ocsgw | at java.lang.Thread.run(Thread.java:748)
ocsgw | Caused by: io.netty.channel.ConnectTimeoutException: connection timed out: ocs.endpoints.pantel-2decb.cloud.goog/35.195.49.238:80
ocsgw | at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:267)
ocsgw | at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
ocsgw | at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:120)
ocsgw | at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403)
ocsgw | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
ocsgw | at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
ocsgw | at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
ocsgw | ... 1 more
ocsgw | 11 Jun 2018 09:38:59,016 INFO GrpcDataSource - reconnectActivate called
ocsgw | 11 Jun 2018 09:39:45,452 INFO OcsApplication - << Received Request

[Prime] Activation no longer sent to P-GW on topup

Since the refactor to ocs-kt the activation message is no longer sent to OCS gateway on topup. This means that the Re-Auth Request is never sent to the P-GW, hence the user is no longer auto enabled again if they topup a zero balance bundle.

This should be fixed in Prime and only send activation messages in the case where the balance goes from zero.

How to handle different configuration files for unit test, integration test and production

[OCSgw] ProxySource should only send one reply back to P-GW

it looks like both the LocalSource and GRPC Source is sending CCA back to P-GW for the same request

Dependabot couldn't find a build.gradle for this project

Dependabot couldn't find a build.gradle for this project.

Dependabot requires a build.gradle to evaluate your project's current Java dependencies. It had expected to find one at the path: /build.gradle.

If this isn't a Java project, or if it is a library, you may wish to disable updates for it from within Dependabot.

You can mention @dependabot in the comments below to contact the Dependabot team.

[OCS-gw][Prime] Clear reservation

Each CCR/CCA has a validity time for the requested bucket. There should be a timer to clear the reservation incase the P-GW does not update after the time has passed.

So we need to include this time in the gRPC requests and make sure timer is set in Prime.

[Prime] Does not handle Terminate correctly

There is no return from Prime to OCSGW when the P-GW sends Credit-Control-Request Terminate.
Need to verify if this is actually correct.

ocsgw | 09 Apr 2018 11:46:15,924 INFO GrpcDataSource - [>>] Requesting bytes for 4790300157
ocsgw | 09 Apr 2018 11:46:15,926 DEBUG NettyClientHandler - [id: 0x7e647cc8, L:/172.16.238.1:34767 - R:172.16.238.4/172.16.238.4:8082] OUTBOUND DATA: streamId=3 padding=0 endStream=false length=95 bytes=000000005a08031232303030332d736573736d67722e47795f4469616d657465723b31303032303030313b3235393b35616362346136662d3330321a0a343739...
prime | INFO [2018-04-09 11:46:15,934] org.ostelco.prime.ocs.OcsGRPCService: Received Credit-Control-Request request :: for MSISDN: 4790300157 with request id: 0003-sessmgr.Gy_Diameter;10020001;259;5acb4a6f-302
prime | ERROR [2018-04-09 11:46:15,935] org.ostelco.prime.disruptor.PrimeEventProducer: Received empty list

[OCSgw] Fail to send CCA

When HA module fail to fetch the apiId it will throw an illegalStateException. This cause the CCA to failed to get send.

The log will have an attempt log to get:

"Got key : apiId , value : null"

Then the stacktrace :

org.jdiameter.api.InternalException: java.lang.IllegalStateException
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.handleEvent(ServerCCASessionImpl.java:325)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.sendCreditControlAnswer(ServerCCASessionImpl.java:132)
at org.ostelco.ocsgw.datasource.protobuf.ProtobufDataSource.handleCcrAnswer(ProtobufDataSource.kt:58)
at org.ostelco.ocsgw.datasource.protobuf.PubSubDataSource$1.invoke(PubSubDataSource.kt:64)
at org.ostelco.ocsgw.datasource.protobuf.PubSubDataSource$1.invoke(PubSubDataSource.kt:30)
at org.ostelco.ocsgw.datasource.protobuf.PubSubDataSource$setupPubSubSubscriber$receiver$1.receiveMessage(PubSubDataSource.kt:157)
at com.google.cloud.pubsub.v1.MessageDispatcher$4.run(MessageDispatcher.java:379)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:835)
caused by: java.lang.IllegalStateException: null
at org.ostelco.diameter.ha.common.AppSessionDataReplicatedImpl.getApplicationId(AppSessionDataReplicatedImpl.kt:57)
at org.jdiameter.common.impl.app.AppSessionImpl.getSessionAppId(AppSessionImpl.java:130)
at org.jdiameter.common.impl.app.AppSessionImpl.release(AppSessionImpl.java:140)
at org.jdiameter.common.impl.app.cca.AppCCASessionImpl.release(AppCCASessionImpl.java:90)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.release(ServerCCASessionImpl.java:494)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.setState(ServerCCASessionImpl.java:482)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.setState(ServerCCASessionImpl.java:467)
at org.jdiameter.server.impl.app.cca.ServerCCASessionImpl.handleEvent(ServerCCASessionImpl.java:310)
... 12 common frames elided"

For streaming gRPC, do load balancing on requests instead of connections.

Problem description

The load balancing on streaming gRPC happens on connections.
So, a single gRPC streaming connection is established with a single instance of prime.
Hence, that one instance of prime gets all the CCR gRPC requests.
#Instead, the load balancing should happen on requests instead of connections.

Reference

This article clearly explains the issue:
https://kubernetes.io/blog/2018/11/07/grpc-load-balancing-on-kubernetes-without-tears/

#715 has a dependency on this issue.

[OCSgw] [Prime] Implement Multiple-Services-Credit-Control correctly

There has been some shortcuts in the implementation of Multiple-Services-Credit-Control where I only pick the first Service and rating group. This should be handled as a list and individual requests all the way from OCSgw <-> Prime <-> DB