hazelcast/hazelcast-tomcat-sessionmanager

Tomcat Based Web Session Replication

Table of Contents

• Tomcat Based Web Session Replication
• Features and Requirements
• How Tomcat Session Replication Works
  • Deploying P2P for Tomcat
  • Deploying Client-Server for Tomcat
• Configuring Manager Element for Tomcat
• Controlling Session Caching with deferredWrite
• Setting Session Expiration Checks
• Enabling Session Replication in Multi-App Environments
• Sticky Sessions and Tomcat
• Tomcat Failover and the jvmRoute Parameter
• Spring Boot Auto-configuration

Tomcat Based Web Session Replication

Sample Code: Please see our sample application for Tomcat Based Web Session Replication.

You can also check Hazelcast Guides: Spring Boot Tomcat Session Replication using Hazelcast.

Features and Requirements

Hazelcast Tomcat Session Manager is a container specific module that enables session replication for JEE Web Applications without requiring changes to the application.

Features

• Seamless Tomcat 7, 8, 8.5 & 9 integration.
• Support for sticky and non-sticky sessions.
• Tomcat failover.
• Deferred write for performance boost.
  
  

Supported Containers

Tomcat Web Session Replication Module has been tested against the following containers.

• Tomcat 7.0.x - It can be downloaded here.
• Tomcat 8.0.x - It can be downloaded here.
• Tomcat 9.0.x - It can be downloaded here.

The latest tested versions are 7.0.40, 8.0.36, 8.5.9, and 9.0.27.

Requirements

• Tomcat instance must be running with Java 1.6 or higher.
• Session objects that need to be clustered have to be serializable on Hazelcast cluster. Please see here for how you can configure and implement serialization for Hazelcast.
• Hazelcast 4.0+ is supported by Hazelcast Tomcat Session Manager v2.0+.

How Tomcat Session Replication Works

Hazelcast Tomcat Session Manager is a Hazelcast Module where each created HttpSession Object is kept in the Hazelcast Distributed Map. If configured with Sticky Sessions, each Tomcat Instance has its own local copy of the session for performance boost.

Since the sessions are in Hazelcast Distributed Map, you can use all the available features offered by Hazelcast Distributed Map implementation, such as MapStore and WAN Replication.

Tomcat Web Sessions run in two different modes:

• P2P: all Tomcat instances launch its own Hazelcast Instance and join to the Hazelcast Cluster and,
• Client/Server: all Tomcat instances put/retrieve the session data to/from an existing Hazelcast Cluster.

Deploying P2P for Tomcat

P2P deployment launches an embedded Hazelcast member in each server instance.

This type of deployment is simple: just configure your Tomcat and launch. There is no need for an external Hazelcast cluster.

The following steps configure a sample P2P for Hazelcast Session Replication.

1. Go to hazelcast.org and download the latest Hazelcast.

2. Unzip the Hazelcast zip file into the folder $HAZELCAST_ROOT.

3. Go to hazelcast-tomcat-sessionmanager repository and download the latest version.

4. Put $HAZELCAST_ROOT/lib/hazelcast-all-<version>.jar, and hazelcast-tomcat<tomcatversion>-sessionmanager-<version>.jar and hazelcast.xml (if you want to change defaults) in the folder $CATALINA_HOME/lib/.

5. Put a <Listener> element into the file $CATALINA_HOME$/conf/server.xml as shown below.

       <Server>
       	...
           <Listener className="com.hazelcast.session.P2PLifecycleListener" configLocation="/path/to/hazelcast.xml"/>
           ...
       </Server>

6. Put a <Manager> element into the file $CATALINA_HOME$/conf/context.xml as shown below.

       <Context>
       	...
           <Manager className="com.hazelcast.session.HazelcastSessionManager"/>
           ...
       </Context>

7. Start Tomcat instances with a configured load balancer and deploy the web application.

Optional Attributes for Listener Element

Optionally, you can add a configLocation attribute into the <Listener> element. If not provided, hazelcast.xml in the classpath is used by default. URL or full filesystem path as a configLocation value is supported.

Classloader for Hazelcast Instance

Hazelcast instance sets and uses context classloader (WebAppClassLoader) whilst the initialization. However, Hazelcast's classloader configuration cannot be changed dynamically after the instance startup. Thus, if the context's classloader changes during the application runtime, it won't be reflected to the Hazelcast instance.

Deploying Client-Server for Tomcat

In this deployment type, Tomcat instances work as clients on an existing Hazelcast Cluster.

Features

• The existing Hazelcast cluster is used as the Session Replication Cluster.
• Offloading Session Cache from Tomcat to the Hazelcast Cluster.
• The architecture is completely independent. Complete reboot of Tomcat instances.
  
  

The following steps configure a sample Client/Server for Hazelcast Session Replication.

1. Go to hazelcast.org and download the latest Hazelcast.

2. Unzip the Hazelcast zip file into the folder $HAZELCAST_ROOT.

3. Go to hazelcast-tomcat-sessionmanager repository and download the latest version.

4. Put $HAZELCAST_ROOT/lib/hazelcast-all-<version>.jar, and hazelcast-tomcat<tomcatversion>-sessionmanager-<version>.jar and hazelcast.xml (if you want to change defaults) in the folder $CATALINA_HOME/lib/.

5. Put a <Listener> element into the $CATALINA_HOME$/conf/server.xml as shown below.

       <Server>
       	...
           <Listener className="com.hazelcast.session.ClientServerLifecycleListener" configLocation="/path/to/hazelcast-client.xml"/>
           ...
       </Server>

6. Update the <Manager> element in the $CATALINA_HOME$/conf/context.xml as shown below.

       <Context>
            <Manager className="com.hazelcast.session.HazelcastSessionManager"
             clientOnly="true"/>
       </Context>

7. Launch a Hazelcast Instance using $HAZELCAST_ROOT/bin/server.sh or $HAZELCAST_ROOT/bin/server.bat.

8. Start Tomcat instances with a configured load balancer and deploy the web application.

Optional Attributes for Listener Element

Optionally, you can add configLocation attribute into the <Listener> element. If not provided, hazelcast-client-default.xml in hazelcast-client-<version>.jar file is used by default. Any client XML file in the classpath, URL or full filesystem path as a configLocation value is also supported.

An example client config that connects directly (i.e. doesn't use multicast) to a specified cluster is:

XML Configuration

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast-client xmlns="http://www.hazelcast.com/schema/client-config"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  xsi:schemaLocation="http://www.hazelcast.com/schema/client-config
                  http://www.hazelcast.com/schema/client-config/hazelcast-client-config-4.0.xsd">    

    <network>
      <cluster-members>
        <address>HAZELCAST_MEMBER</address>
      </cluster-members>
    </network>
</hazelcast-client>

YAML Configuration

hazelcast:
  network:
    cluster-members:
      - HAZELCAST_MEMBER

Configuring Manager Element for Tomcat

<Manager> element is used both in P2P and Client/Server mode. You can use the following attributes to configure Tomcat Session Replication Module to better serve your needs.

• Add mapName attribute into <Manager> element. Its default value is default Hazelcast Distributed Map. Use this attribute if you have a specially configured map for special cases like WAN Replication, Eviction, MapStore, etc.
• Add sticky attribute into <Manager> element. Its default value is true.
• Add processExpiresFrequency attribute into <Manager> element. It specifies the frequency of session validity check, in seconds. Its default value is 6 and the minimum value that you can set is 1.
• Add deferredWrite attribute into <Manager> element. Its default value is true.
• In P2P mode, add hazelcastInstanceName attribute into <Manager> element. It specifies an existing Hazelcast instance to use for session replication. The same can be achieved by setting instanceName property in Hazelcast configuration. If no instance name is configured, Hazelcast instance starts with a default instance name (SessionManager.DEFAULT_INSTANCE_NAME).

Controlling Session Caching with deferredWrite

Tomcat Web Session Replication Module has its own nature of caching. Attribute changes during the HTTP Request/HTTP Response cycle is cached by default. Distributing those changes to the Hazelcast Cluster is costly. Because of that, Session Replication is only done at the end of each request for updated and deleted attributes. The risk in this approach is losing data if a Tomcat crash happens in the middle of the HTTP Request operation.

You can change that behavior by setting deferredWrite=false in your <Manager> element. By disabling it, all updates that are done on session objects are directly distributed into Hazelcast Cluster.

Setting Session Expiration Checks

Based on Tomcat configuration or sessionTimeout setting in web.xml, sessions are expired over time. This requires a cleanup on the Hazelcast Cluster since there is no need to keep expired sessions in the cluster.

processExpiresFrequency, which is defined in <Manager>, is the only setting that controls the behavior of session expiry policy in the Tomcat Web Session Replication Module. By setting this, you can set the frequency of the session expiration checks in the Tomcat Instance.

Starting with v2.1, the session expiration in Hazelcast cluster is not controlled by Tomcat anymore. Instead, Hazelcast's own expiry/eviction mechanism should be configured. In order to to configure the expiration settings on the Hazelcast cluster, you need to configure eviction for the related session map. Please see the Hazelcast's own eviction mechanism details here. Please note that the sessions might be kept alive longer than the sessionTimeout setting in the cluster if you don't configure the Hazelcast expiration/eviction properly.

Enabling Session Replication in Multi-App Environments

Tomcat can be configured in two ways to enable Session Replication for deployed applications.

• Server Context.xml Configuration
• Application Context.xml Configuration

Server Context.xml Configuration

By configuring $CATALINA_HOME$/conf/context.xml, you can enable session replication for all applications deployed in the Tomcat Instance.

Application Context.xml Configuration

By configuring $CATALINA_HOME/conf/[enginename]/[hostname]/[applicationName].xml, you can enable Session Replication per deployed application.

Sticky Sessions and Tomcat

Sticky Sessions (default)

Sticky Sessions are used to improve the performance since the sessions do not move around the cluster.

Requests always go to the same instance where the session was firstly created. By using a sticky session, you mostly eliminate session replication problems, except for the failover cases. In case of failovers, Hazelcast helps you to not lose existing sessions.

Non-Sticky Sessions

Non-Sticky Sessions are not good for performance because you need to move session data all over the cluster every time a new request comes in.

However, load balancing might be super easy with Non-Sticky caches. In case of heavy load, you can distribute the request to the least used Tomcat instance. Hazelcast supports Non-Sticky Sessions as well.

Tomcat Failover and the jvmRoute Parameter

Each HTTP Request is redirected to the same Tomcat instance if sticky sessions are enabled. The parameter jvmRoute is added to the end of session ID as a suffix, to make Load Balancer aware of the target Tomcat instance.

When Tomcat Failure happens and Load Balancer cannot redirect the request to the owning instance, it sends a request to one of the available Tomcat instances. Since the jvmRoute parameter of session ID is different than that of the target Tomcat instance, Hazelcast Session Replication Module updates the session ID of the session with the new jvmRoute parameter. That means that the Session is moved to another Tomcat instance and Load Balancer will redirect all subsequent HTTP Requests to the new Tomcat Instance.

NOTE: If stickySession is enabled, jvmRoute parameter must be set in $CATALINA_HOME$/conf/server.xml and unique among Tomcat instances in the cluster.

 <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat-8080">

Spring Boot Auto-configuration

Starting with v2.2, Hazelcast Tomcat Session Manager supports auto-configuration when used with Spring Boot. The only thing you need to do is to add Hazelcast Tomcat Session Manager (for Tomcat 9) and Hazelcast IMDG libraries to the classpath. This will set Hazelcast Session Manager as the session manager of the Tomcat. If you would like to configure the session manager properties, you can setup the following properties in your application.properties file:

• tsm.autoconfig.enabled: Allows to enable/disable Spring Boot Auto-configuration for Hazelcast Tomcat Session Manager. Default is true, you need to set it to false if you would like to configure Hazelcast Tomcat Session Manager manually with Spring Boot.
• tsm.config.location: Allows to provide Hazelcast member or client configuration. If not provided, hazelcast.xml in the classpath is used by default.
• tsm.map.name: Use this property if you have a specially configured map for special cases like WAN Replication, Eviction, MapStore, etc.
• tsm.sticky: Allows to set sticky mode. Its default value is true.
• tsm.process.expires.frequency: It specifies the frequency of session validity check, in seconds. Its default value is 6 and the minimum value that you can set is 1.
• tsm.deferred.write: Allows to set deferred write mode. See this section for details.
• tsm.hazelcast.instance.name: It specifies an existing Hazelcast instance to use for session replication. The same can be achieved by setting instanceName property in Hazelcast configuration. If no instance name is configured, Hazelcast instance starts with a default instance name (SessionManager.DEFAULT_INSTANCE_NAME).

Notes about Spring Boot Auto-configuration

• By default, a Hazelcast member instance is initialized when starting Hazelcast Tomcat Session Manager with Spring Boot. If you would like to initialize a client instance instead, then you need to provide a Hazelcast client configuration with tsm.config.location or initialize a ClientConfig bean in Spring Boot.
• If a com.hazelcast.client.config.ClientConfig bean is configured explicitly, then both instanceName setting in the ClientConfig bean and tsm.hazelcast.instance.name property should be set.
• If a com.hazelcast.config.Config bean is configured explicitly, then both instanceName setting in the Config bean and tsm.hazelcast.instance.name property should be set.

License

Hazelcast Tomcat Session Manager is available under the Hazelcast Community License.

hazelcast-tomcat-sessionmanager's Issues

Too many sessions gives 'java.lang.IllegalStateException: null

When the maxActiveSessions limit is reached on tomcat 8.5 we get

java.lang.IllegalStateException: null
at com.hazelcast.session.HazelcastSessionManager.checkMaxActiveSessions(HazelcastSessionManager.java:354) ~[hazelcast-tomcat85-sessionmanager-1.1.4-SNAPSHOT.jar!/:na]
at com.hazelcast.session.HazelcastSessionManager.createSession(HazelcastSessionManager.java:136) ~[hazelcast-tomcat85-sessionmanager-1.1.4-SNAPSHOT.jar!/:na]
at org.apache.catalina.connector.Request.doGetSession(Request.java:3075) ~[tomcat-embed-core-8.5.34.jar!/:8.5.34]

this is because the wrong String reference is used, standardManager.createSession.ise, should be managerBase.createSession.ise

hazelcast-tomcat-sessionmanager/tomcat85/src/main/java/com/hazelcast/session/HazelcastSessionManager.java

Line 348 in 1eb0d79

throw new IllegalStateException(sm.getString("standardManager.createSession.ise"));

Cuncurrent calls lead to no session found

I understand there is a fix for the concurrent calls but it still persists, however less frequent

tomcat85-sessionmanager:1.1.5
tomcat:8.5
hazelcast:3.9

[http-nio-8080-exec-4] com.hazelcast.session.HazelcastSessionManager.findSession Attempting to find sessionId: 303B2C10337479C99450A40D2B5E3FB6
[http-nio-8080-exec-4] com.hazelcast.session.HazelcastSessionManager.findSession Session found for: 303B2C10337479C99450A40D2B5E3FB6
[http-nio-8080-exec-5] com.hazelcast.session.HazelcastSessionManager.findSession Attempting to find sessionId: 303B2C10337479C99450A40D2B5E3FB6
[http-nio-8080-exec-5] com.hazelcast.session.HazelcastSessionManager.findSession No Session found for: 303B2C10337479C99450A40D2B5E3FB6

Why is Tomcat 7 no longer supported?

I wanted to download the latest release 1.1.4 for Tomcat 7 and found none. From the commit history I saw it was disabled by #64 because tests failed. Unfortunately I do not see any more details.

For me the current snapshot causes no issues - see my changes for details. The same applies to the tagged version 1.1.4 (just needed to change the parent version in Tomcat 7's pom.xml).

What am I missing here? If Tomcat 7 is not going to be supported in the future it would also be good to change the README.md.

HazelcastSessionManager.findSession can return null for an existing session under load

I believe the issue is in this method: https://github.com/hazelcast/hazelcast-tomcat-sessionmanager/blob/master/tomcat8/src/main/java/com/hazelcast/session/HazelcastSessionManager.java#L226

from findSession method

HazelcastSession hazelcastSession = sessionMap.get(id);
if (hazelcastSession == null) {
  log.info("No Session found for:" + id);
  return null;
}
.....
// call remove method to trigger eviction Listener on each node to invalidate local sessions
sessionMap.remove(id);
sessionMap.set(id, hazelcastSession);

After checking sessionMap, if the session is present, first remove & then set called. After remove & before set call, if you call the same method, you can get null from sessionMap.get(id) call. This could cause users getting null for an existing session under load, as described in #41

ClassNotFoundException can occur when using parallel deployments

Background:
• Tomcat Parallel Deployments are in use
• The session object contains a custom data type only available on the web-application class-path
• Requests are occurring whilst the application is being redeployed
• Hazelcast is configured to use non-sticky sessions and deferred write.

Description:
When using parallel deployments whilst an application is processing requests, a new deployment can trigger a class not found exception in the HazelcastSession deserializeMap method. I believe this is because the web-application classpath has begun to unload whilst the session manager runs in the tomcat common classloader.

Impact:
Once Tomcat has entered this state, the instance becomes unstable and will only respond with a 500 - InternalServerError until the tomcat instance is restarted.

More Details

An example to reproduce can be found here: https://github.com/edwardsmatt/hazelcast-cnfe-parallel-deployment-example.

Set hazelcast session custom attribute

Hi, is it possible to set an attribute to hazelcast session, smth like that:

IMap<String, HazelcastSession> map = hazelMap.get("empty_session_replication");
HazelcastSession session = map.get(request.getSession.getId());
session.setAttribute("currentUserId", userId);

Because when I'm trying to set an attribute I got NullPointerException

java.lang.NullPointerException
	org.apache.catalina.session.StandardSession.setAttribute(StandardSession.java:1464)
	org.apache.catalina.session.StandardSession.setAttribute(StandardSession.java:1376)
	com.hazelcast.session.HazelcastSession.setAttribute(HazelcastSession.java:47)
	ru.mts.smartfarming.rs.resources.TESTResource.testSettingCurrentUserId(TESTResource.java:69)
	sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(Method.java:498)
	org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
	org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
	org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
	org.glassfish.jersey.server.model.internal.VoidVoidDispatcherProvider$VoidToVoidDispatcher.doDispatch(VoidVoidDispatcherProvider.java:83)
	org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
	org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
	org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
	org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
	org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
	org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
	org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
	org.glassfish.jersey.internal.Errors.process(Errors.java:316)
	org.glassfish.jersey.internal.Errors.process(Errors.java:298)
	org.glassfish.jersey.internal.Errors.process(Errors.java:268)
	org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
	org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
	org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
	org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
	org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
	org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
	org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
	org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
	org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)

Inconsistent sessionId in the first request after a failover

I am not confident that this is really an issue so this is more of a question. When a failover happens with stickiness enabled, the first request of a session assigned to the server that failed (let .host1) is expected to have a suffix (i.e. jvmRoute) that will not match the jvmRoute of the healthy host (let .host2). After digging a bit I have noticed that the HazelcastSessionChangeValve will call an internal Tomcat API request.changeSessionId() to change the sessionId so that it matches the new jvmRoute. This means that after passing from the valve the value returned from the servlet API call request.getRequestedSessionId() will have the suffix of the healthy host .host2. In addition, examining the headers of the response one can see that this call also adds a new Set-Cookie header with the updated sessionId. getSession().getId() however still returns the id ending in .host1!

I believe that this is confusing given that the Servlet spec states about HttpServletRequest.getRequestedSessionId():

Returns the session ID specified by the client. This may not be the same as the ID of the current valid session for this request. If the client did not specify a session ID, this method returns null.

I understand that maybe this is the best way that Tomcat provides to change the sessionId using its internal API but shouldn't it at least be consistent with the id stored in the session returned by getSession(). This might be important for any servlet filters that need the sessionId of the validated session and not the information passed by the client (i.e., returned by getRequestedSessionId).

Session manager in client only mode doesn't handle HZ client SHUTDOWN

This issue is submitted after investigating some reconnection issue on the Hazelcast client as described here :
hazelcast/hazelcast#17062

Since HZ 3.12, it seems that the client will not handle automatic reconnection whenever the cluster members are restarted with different IP addresses (always the case in the docker world).
In such a scenario, whenever a cluster member is restarted, the client is not capable of reconnecting to its new IP address, and if at some point all the members have been restarted, the client will completely be unable to connect to any member and will SHUTDOWN.
As proposed in the linked issue and in the google group here : https://groups.google.com/forum/#!msg/hazelcast/6OM_msgJvzg/kDi06y29CQAJ
The solution is to register a LifecycleListener on the client to be notified whenever such a SHUTDOWN condition is encountered, and then fully re-instanciate/restart the HZ client from scratch so that it resolves again the member names provided in the configuration.
In the tomcat replication manager, this can be implemented in the session manager.

As stated in the linked issue, I am finishing a patch that I will propose for such automatic reconnection whenever a client SHUTDOWN is encountered.

Move LifecycleListener Logic to Hazelcast Session Manager

P2PLifecycleListener/ClientServerLifecycleListener are responsible for some initializations which makes Hazelcast Deployments complex and error-prone.

The idea is to move this logic into HazelcastSessionManager to encapsulate all SessionManager related logic in one place.

related #53

tomcat-core artifacts missing from Maven Central

http://central.maven.org/maven2/com/hazelcast/hazelcast-tomcat-sessionmanager/1.1.2/ is present, but the tomcat-core dependencies are missing in central.maven.org.

tomcat 6 commit log message always writes "commited key:null"

When saving the session back to the sessionMap in the tomcat6 flavor of com.hazelcast.session.HazelcastSessionManager, the log message writes the value of hazelcastSession.getAttribute("key") which looks to be a relic from testing. The other implementations for Tomcat7, Tomcat8 and Tomcat85 all log the expected value of the session id. Will create a pull request and submit shortly.

Session Synchronization Fails

The same phenomenon occurs with the following issues.
#10

In my case, Apache(1EA) - ajp - tomcat(4EA) with hazelcast-tomcat-sessionmanager.

If you click after 2 or 3 seconds after Tomcat is shut down, it works normally.
If you click on Tomcat at the time of shut down, the session ID is changed and session clustering does not work.

===== Normal Case =====
Tomcat 1> SESSION ID : A16FAD3A97C4FC28312CAC005CA80488.worker4009
(Tomcat 1 shut down)
(Click after 2~3 seconds --> Session Synchronization)
Tomcat 2> SESSION ID : A16FAD3A97C4FC28312CAC005CA80488.worker5009

but,
===== Abnormal Case =====
Tomcat 1> SESSION ID : 2E593562D6A17FA8236EADDBAD7706FB.worker5009
(Tomcat 1 shut down)
(If click Immediately --> Maybe, No Session Synchronization??? so new session)
Tomcat 2> SESSION ID : 86527A93CAEF4A2D380AB90FBC1F6A02.worker4009

Of course, backup-count = 1 in hazelcast-default.xml

Thank you in advance.

hazelcast-all-3.8.5
hazelcast-tomcat8-sessionmanager-1.1.2.jar

---------------------------- Normal log detail ----------------------------

20-Sep-2017 20:05:50.531 INFO [hz.SESSION-REPLICATION-INSTANCE.cached.thread-4]
com.hazelcast.transaction.TransactionManagerService.null [XX.XX.XXX.XXX]:5702
[nip] [3.8.5] Committing/rolling-back alive transactions of Member [XX.XX.XXX.XXX]:5701
7171e37e-711c-43d1-ac8f-649393b02a20, UUID: 7171e37e-711c-43d1-ac8f-649393b02a20

20-Sep-2017 20:05:50.700 INFO [hz.SESSION-REPLICATION-INSTANCE.migration]
com.hazelcast.internal.partition.InternalPartitionService.null [XX.XX.XXX.XXX]:5702
[nip] [3.8.5] Fetching most recent partition table! my version: 950

20-Sep-2017 20:05:50.703 INFO [hz.SESSION-REPLICATION-INSTANCE.migration]
com.hazelcast.internal.partition.InternalPartitionService.null [XX.XX.XXX.XXX]:5702
[nip] [3.8.5] Most recent partition table version: 950

20-Sep-2017 20:05:50.733 INFO [hz.SESSION-REPLICATION-INSTANCE.migration]
com.hazelcast.internal.partition.impl.MigrationManager.null [XX.XX.XXX.XXX]:5702
[nip] [3.8.5] Partition balance is ok, no need to re-partition cluster data...

20-Sep-2017 20:07:50.796 INFO [ajp-apr-5009-exec-2]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.
Some failover occured so reading session from Hazelcast map:null

2017-09-20 20:07:51,213 234925 [ajp-apr-5009-exec-2] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|A16FAD3A97C4FC28312CAC005CA80488.worker4009|

2017-09-20 20:07:53,878 237590 [ajp-apr-5009-exec-3] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|A16FAD3A97C4FC28312CAC005CA80488.worker5009|

2017-09-20 20:07:55,346 239058 [ajp-apr-5009-exec-4] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|A16FAD3A97C4FC28312CAC005CA80488.worker5009|

---------------------------- Abnormal log detail ----------------------------

20-Sep-2017 19:41:06.104 INFO [hz.SESSION-REPLICATION-INSTANCE.cached.thread-2]
com.hazelcast.transaction.TransactionManagerService.null [XX.XX.XXX.XXX]:5701
[nip] [3.8.5] Committing/rolling-back alive transactions of Member [XX.XX.XXX.XXX]:5702
47c4c6b6-ae25-4b30-a000-587fca5c52f2, UUID: 47c4c6b6-ae25-4b30-a000-587fca5c52f2

20-Sep-2017 19:41:06.103 INFO [hz.SESSION-REPLICATION-INSTANCE.migration]
com.hazelcast.internal.partition.impl.MigrationManager.null [XX.XX.XXX.XXX]:5701
[nip] [3.8.5] Partition balance is ok, no need to re-partition cluster data...

20-Sep-2017 19:41:07.134 INFO [hz.SESSION-REPLICATION-INSTANCE.migration]
com.hazelcast.internal.partition.impl.MigrationThread.null [XX.XX.XXX.XXX]:5701
[nip] [3.8.5] All migration tasks have been completed, queues are empty.

20-Sep-2017 19:41:07.930 INFO [ajp-apr-4009-exec-4]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.
Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:07.958 INFO [ajp-apr-4009-exec-3]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.
Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:07.960 INFO [ajp-apr-4009-exec-3]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

20-Sep-2017 19:41:07.962 INFO [ajp-apr-4009-exec-3]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.
Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:07.965 INFO [ajp-apr-4009-exec-3]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

2017-09-20 19:41:08,354 97676 [ajp-apr-4009-exec-4] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|2E593562D6A17FA8236EADDBAD7706FB.worker5009|

2017-09-20 19:41:08,354 97676 [ajp-apr-4009-exec-3] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|537B161FE5E5BD20CDBF45B64F32997B.worker4009|

20-Sep-2017 19:41:09.202 INFO [ajp-apr-4009-exec-7]
com.hazelcast.session.HazelcastSessionManager.findSession Sticky
Session is currently enabled.Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:09.206 INFO [ajp-apr-4009-exec-7]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

20-Sep-2017 19:41:09.208 INFO [ajp-apr-4009-exec-7]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.
Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:09.209 INFO [ajp-apr-4009-exec-7]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

2017-09-20 19:41:09,215 98537 [ajp-apr-4009-exec-7] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|67B86BCA65097729135D9F4FEDCB6A46.worker4009|

20-Sep-2017 19:41:09.790 INFO [ajp-apr-4009-exec-10]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:09.791 INFO [ajp-apr-4009-exec-10]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

20-Sep-2017 19:41:09.793 INFO [ajp-apr-4009-exec-10]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:09.796 INFO [ajp-apr-4009-exec-10]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

2017-09-20 19:41:09,799 99121 [ajp-apr-4009-exec-10] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|0F190873930FDC897BA78382B05992B0.worker4009|

20-Sep-2017 19:41:10.556 INFO [ajp-apr-4009-exec-1]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:10.560 INFO [ajp-apr-4009-exec-1]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

20-Sep-2017 19:41:10.563 INFO [ajp-apr-4009-exec-1]
com.hazelcast.session.HazelcastSessionManager.findSession
Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null

20-Sep-2017 19:41:10.566 INFO [ajp-apr-4009-exec-1]
com.hazelcast.session.HazelcastSessionManager.findSession
No Session found for:2E593562D6A17FA8236EADDBAD7706FB.worker5009

2017-09-20 19:41:10,570 99892 [ajp-apr-4009-exec-1] WARN
yongs.temp.interceptor.LoginInterceptor ::: SESSION ID ==>|86527A93CAEF4A2D380AB90FBC1F6A02.worker4009|

Session data loss during failover

During failover next faulty behavior was observed:

• Node 1 Session was created. SessionID AAA.node1 was assigned
• Session variable A with data 111 was created
• Node 1 was killed
• Node 2 shows session variable A with correct data and same sessionID as before (AAA.node1)
• Trying to add session variable B with data 222 -> ERROR
  New sessionID is assigned (a bit too late) ex. AAA.node1 and session variable B is lost. Session variable A is still present

Unable to deserialize session after adding CrawlerSessionManagerValve valve into server.xml

It works fine before adding CrawlerSessionManager

 com.hazelcast.session.HazelcastSession.deserializeMap Unable to deserialize object in session
 com.hazelcast.nio.serialization.HazelcastSerializationException: There is no suitable de-serializer for type 1633907817. This exception is likely to be caused by differences in the serialization configuration between members or between clients and members.
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.newHazelcastSerializationException(AbstractSerializationService.java:238)
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:265)
        at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:574)
        at com.hazelcast.session.HazelcastSession.deserializeMap(HazelcastSession.java:171)
        at com.hazelcast.session.HazelcastSession.readData(HazelcastSession.java:158)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
        at com.hazelcast.client.spi.ClientProxy.toObject(ClientProxy.java:102)
        at com.hazelcast.client.proxy.ClientMapProxy.get(ClientMapProxy.java:318)
        at com.hazelcast.session.HazelcastSessionManager.findSession(HazelcastSessionManager.java:182)
        at org.apache.catalina.connector.Request.doGetSession(Request.java:2977)
        at org.apache.catalina.connector.Request.getSession(Request.java:2434)
        at org.apache.catalina.valves.CrawlerSessionManagerValve.invoke(CrawlerSessionManagerValve.java:162)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

HttpSessionListener.sessionCreated calls twice

com.hazelcast.session.HazelcastSessionManager.createSession has a call of session.setId(newSessionId) from class org.apache.catalina.session.StandardSession

org.apache.catalina.session.StandardSession internally calls setId(id, true); where true is notify. If notify is true then it calls tellNew(); methodd.

Now back to com.hazelcast.session.HazelcastSessionManager.createSession after calling session.setId(newSessionId) it calls session.tellNew();

This creates tellNew() call twice which eventually results in calling sessionCreated call twice.

Tomcat 8.5.x support for Hazelcast tomcat session manager

Can't start Tomcat 8.5.11 or 8.5.4 with P2P Hazelcast tomcat session manager. Works on Tomcat 8.0.41.
Tomcat fails to start with errors:
25-Jan-2017 16:43:24.664 SEVERE [localhost-startStop-1] org.apache.tomcat.util.digester.Digester.startElement Begin event threw erro
r
java.lang.NoClassDefFoundError: org/apache/catalina/util/LifecycleSupport
at com.hazelcast.session.HazelcastSessionManager.(HazelcastSessionManager.java:36)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:117)
at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1190)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.ja
va:1338)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScan
nerImpl.java:2781)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:5
04)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1458)
at org.apache.catalina.startup.ContextConfig.processContextConfig(ContextConfig.java:537)
at org.apache.catalina.startup.ContextConfig.contextConfig(ContextConfig.java:475)
at org.apache.catalina.startup.ContextConfig.init(ContextConfig.java:738)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:310)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:94)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:395)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:108)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:140)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.catalina.util.LifecycleSupport
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 34 more

25-Jan-2017 16:43:24.667 SEVERE [localhost-startStop-1] org.apache.tomcat.util.digester.Digester.startElement Begin event threw erro
r
java.lang.NoClassDefFoundError: org/apache/catalina/util/LifecycleSupport
at com.hazelcast.session.HazelcastSessionManager.(HazelcastSessionManager.java:36)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.tomcat.util.digester.ObjectCreateRule.begin(ObjectCreateRule.java:117)
at org.apache.tomcat.util.digester.Digester.startElement(Digester.java:1190)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:182)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.ja
va:1338)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScan
nerImpl.java:2781)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:5
04)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1458)
at org.apache.catalina.startup.ContextConfig.processContextConfig(ContextConfig.java:537)
at org.apache.catalina.startup.ContextConfig.contextConfig(ContextConfig.java:475)
at org.apache.catalina.startup.ContextConfig.init(ContextConfig.java:738)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:310)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:94)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:395)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:108)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:140)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

`Principal` is not replicated

Tomcat store Principal in the session object. Although it is said in StandardSession:

/**
* The authenticated Principal associated with this session, if any.
* IMPLEMENTATION NOTE: This object is not saved and
* restored across session serializations!
*/
protected transient Principal principal = null;

but Request method getUserPrincipal or getRemoteUser takes principal from field which is set inside AuthenticatorBase.

        Principal principal = request.getUserPrincipal();
        if (principal == null) {
            Session session = request.getSessionInternal(false);
            if (session != null) {
                principal = session.getPrincipal();
                if (principal != null) {
                    if (log.isDebugEnabled()) {
                        log.debug("We have cached auth type " + session.getAuthType() +
                                " for principal " + principal);
                    }
                    request.setAuthType(session.getAuthType());
                    request.setUserPrincipal(principal);
                }
            }
        }

I suggest that userPrincipal and authType should be replicated

NPE when starting Hazelcast Client in embedded Tomcat 8

NPE when starting Hazelcast Client in embedded Tomcat 8

Embedded Tomcat Version: 8.0.30
Hazelcast version: 3.1.1
HazelcastSessionManager version: hazelcast-tomcat8-sessionmanager-1.1.3

Running embedded tomcat and setting hazelcast via:

      // ...
      // Tomcat creation and context setting
      // ...
      // Hazelcast Config
      HazelcastSessionManager hazelcastSessionManager = new HazelcastSessionManager();
      hazelcastSessionManager.setClientOnly(true);
      hazelcastSessionManager.setSticky(true);
      standardContext.setManager(hazelcastSessionManager);
      standardContext.addLifecycleListener(new ClientServerLifecycleListener());
      tomcat.start();

Results in following NPE:

com.hazelcast.session.HazelcastSessionManager startInternal
SEVERE: Hazelcast Client could not be created. 
java.lang.NullPointerException
	at com.hazelcast.session.HazelcastSessionManager.startInternal(HazelcastSessionManager.java:92)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5272)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Nov 14, 2018 3:12:39 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: The session manager failed to start
org.apache.catalina.LifecycleException: Failed to start component [com.hazelcast.session.HazelcastSessionManager[/pbrc]]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5272)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1408)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1398)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.catalina.LifecycleException
	at com.hazelcast.session.HazelcastSessionManager.startInternal(HazelcastSessionManager.java:96)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	... 9 more

After running through debugger in IntelliJ:

When in client mode HazelcastSessionManager attempts to retrieve client config from ClientServerLifecycleListener within the startInternal method.

ClientConfig is only created in ClientServerLifecycleListener on the Tomcat "start" LifeCycle event, which occurs after the state change so is never created in time for the session manager to use.

Tomcat 6 dependency in 8 module

It appears the core module depends on Tomcat 6, which has an attributes field in org.apache.catalina.session.StandardSession, but that field is gone in Tomcat 8, leading to this:

java.lang.NoSuchFieldError: attributes
       at com.hazelcast.session.HazelcastSession.writeData(HazelcastSession.java:85)

Use "IMap.set" instead of "IMap#put" to prevent unnecessary deserialization

An example https://github.com/hazelcast/hazelcast-tomcat-sessionmanager/blob/master/tomcat6/src/main/java/com/hazelcast/session/HazelcastSessionManager.java#L203

Spring Boot support

Hi,

Is it possible to use this project together with Spring Boot and Java configuration?
If so, can you please provide a sample or instructions on how to configure it?

Thanks.

Hazelcast Enabled Tomcat Docker Image

We can release a docker images where we pre-configure a tomcat server with hazelcast as session replication provider. That would be useful for cloud native applications with caching requirements.

Sessions don't expire!

If using sticky sessions and a node in the cluster fails, all sessions on that failing node will not expire unless the client makes a new request handled by another node.

The problem is that the findSessions() method is not implemented in HazelcastSessionManager which the expire thread in tomcat depends on. So it's falling back to the one implemented in ManagerBase which only returns the local sessions, but if no new request are made for a session then that session will never end up in a local session map.

So that method needs to return all sessions in the cluster for the sessions to be expired correctly. And I'm guessing other functionality in tomcat that uses that method also needs all sessions.

If sessions aren't expired they will stay in the hazelcast map forever, e.i. a memory leak. Also sessionDestroyed events are never triggered for those sessions.

I see two possible implementations. Either do something simple and just return all sessions in the distributed map:

@Override
public Session[] findSessions() {
	return sessionMap.values().toArray(new Session[0]);
}

Or try to do something clever to avoid deserializing sessions which are locally available:

@Override
public Session[] findSessions() {
	// Get all local sessions
	Set<Session> allSessions = new HashSet(sessions.values());

	// Get all non-local sessions ids
	Set<String> keys = sessionMap.keySet();
	keys.removeAll(sessions.keySet());

	// Add all non-local sessions
	allSessions.addAll(sessionMap.getAll(keys).values());

	return allSessions.toArray(new Session[allSessions.size()]);
}

Your thoughts about this?

configuration problem

SEVERE: Error while connecting to cluster!
java.lang.IllegalStateException: Unable to connect to any address in the config!

i add

    <cluster-members>
        <address>cacheserver-dev-3.atm:5701</address>
        <address>cacheserver-dev-3.atm:5702</address>
    </cluster-members>

into hazelcast.xml and put it in bin/

but nothing happen.

Any sugesstions?

Why session manager doesn't use replicated map?

In my project we use embedded instances of hazelcast, as I understand from codebase session manager uses regular distributed map for session replication, but what if I execute rolling restart of my application (with embedded hazelcast)?

I expect session pick up on any available instance, but session manager uses distributed map. It leads to session data lost.

Deserialization problem when doing aggregate on a map

I've got a problem with deserialization when doing aggregate on a map in a P2P configuration.

map.aggregate(Supplier.fromPredicate(filter::test), Aggregations.count());

This will throw an HazelcastSerializationException caused by a ClassNotFoundException. The problem is similar to this issue: #13

The hazelcast thread doing the aggregate has a context ClassLoader with only the tomcat libraries, but it needs to have the webapp context classloader to be able to deserialize objects from the webapp. So I'm guessing the classloader in the config needs to be set to the webapp context classloader before creating the hazelcast instance (and the hazelcast threads). But currently the instance is created in the P2PLifecycleListener where the webapp context is not available. Should the instance be created in the SessionManager instead, like with the server-client case?

Or is there some other way to set the classloader in the webapp after the hazelcast instance has been created? Executing instance.getConfig().setClassLoader(...) seems to have no effect.

Hazelcast does not startup with tomcat9 sessionmanager

Currently we are using the Tomcat8.5 with the tomcat85-sessionamanger-1.1.3.jar and hazelcast-all-3.12.1. Everything runs smoothly and the Hazelcast cluster is formed on Tomcat startup. Session replication works.

Now we need to upgrade the Tomcat-9.0.33. So, I have tried bringing up the hazelcast cache with the following combinations of tomcat9-sessionmanager and hazelcast-all jars. However, the hazelcast cache does not startup on tomcat startup. No errors are logged in the tomcat logs.

Tried the following combinations:

tomcat9-sessionmanager-2.0 with hazelcast-all-4.0
tomcat9-sessionmanager-1.1.5 with hazelcast-all-3.12.1

However, strangely, if I start the tomcat 9 with the following combination it starts up fine:
tomcat85-sessionmanager-1.1.3 with hazelcast-all-3.12.1

I do not want to use the 8.5 combination on Tomcat 9.0 if it is not recommended.
Is there a particular version of tomcat 9.x that the tomcat9-sessionmanager jar works with?

Concurrent access

I have a lot of concurrent calls inside my application and I receive the following logs:

Nov 24, 2017 6:35:16 AM com.hazelcast.session.HazelcastSessionManager findSession
INFO: No Session found for:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:16 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:16 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager findSession
INFO: No Session found for:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager findSession
INFO: No Session found for:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-1 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-1 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager findSession
INFO: No Session found for:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:17 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-7 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager findSession
INFO: No Session found for:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-3 committed key:ADC7938F491576F2703AFEA344EB0A3F
Nov 24, 2017 6:35:18 AM com.hazelcast.session.HazelcastSessionManager commit
INFO: Thread name:http-bio-8080-exec-6 committed key:ADC7938F491576F2703AFEA344EB0A3F

As you can see I always have the same session id, but for some reason session manager cannot find it.

I think the root cause is the following code:

this.sessionMap.remove(id);
this.sessionMap.set(id, hazelcastSession);

Can somebody explain me why we cannot just execute set without remove to prevent concurrency issue?

If it's required how I can prevent problem like this.

Simplify Tomcat Session Manager with Spring-Boot

Initial prototype is located here.

Tomcat Session Manager should be purely operational so adding it to the classpath should easily switch on/off session replication.

JSP Session Issues

Currently I`m facing an issue with JSP beans property management. Please follow the source code below:

...
<jsp:useBean id="user" class="...User" scope="session"/>
<jsp:useBean id="list" class="...LinkedList" scope="session"/>
...

Previously, all the JSPs were handled by Oracle Weblogic but now I had to migrate the project to Spring Boot with Embedded Tomcat and Hazelcast Session Management. So the issue occurs if we do smth like this:

user.setUsername("test");
list.add(new Object());
response.setRedirect(...);

After the redirect is made I`m trying to reach the username or list values but what I actually receive is:

user.getUsername() => null
list.size() => 0
This one was working using the Weblogic.

Note: But there is a easy but not very effective solution for this issue:

session.putValue("user", user);
session.putValue("list", list);
response.setRedirect(...);

I have a lot of JSP pages so I can`t cover all of this issues. Could you please help me with this one? Thanks!

Tomcat 8.5.16 Error while starting

I followed the steps to run hazelcast with my tomcat 8.5.16 in P2P mode, once I changed server.xml, tomcat doesn't start anymore

My configue :

<Server>
...
  <!-- hazelcast listener -->
  <Listener className="com.hazelcast.session.P2PLifecycleListener"/>
...
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

			<Manager className="com.hazelcast.session.HazelcastSessionManager"/>

...
</Cluster>

....
</Server>

PS : with "DeltaManager", tomcat cluster works.

The error I get is :

24-Jul-2017 17:06:37.046 GRAVE [main] org.apache.tomcat.util.digester.Digester.endElement End event threw exception
 java.lang.IllegalArgumentException: argument type mismatch
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.util.IntrospectionUtils.callMethod1(IntrospectionUtils.java:377)
	at org.apache.tomcat.util.digester.SetNextRule.end(SetNextRule.java:145)
	at org.apache.tomcat.util.digester.Digester.endElement(Digester.java:971)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
	at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:183)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1339)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1472)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:579)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:630)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:311)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:494)

24-Jul-2017 17:06:37.052 AVERTISSEMENT [main] org.apache.catalina.startup.Catalina.load Catalina.start using conf/server.xml: Error at (142, 72) : argument type mismatch
24-Jul-2017 17:06:37.055 GRAVE [main] org.apache.tomcat.util.digester.Digester.endElement End event threw exception
 java.lang.IllegalArgumentException: argument type mismatch
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.tomcat.util.IntrospectionUtils.callMethod1(IntrospectionUtils.java:377)
	at org.apache.tomcat.util.digester.SetNextRule.end(SetNextRule.java:145)
	at org.apache.tomcat.util.digester.Digester.endElement(Digester.java:971)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
	at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:183)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1339)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2784)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:841)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:770)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at org.apache.tomcat.util.digester.Digester.parse(Digester.java:1472)
	at org.apache.catalina.startup.Catalina.load(Catalina.java:579)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:644)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)

24-Jul-2017 17:06:37.056 AVERTISSEMENT [main] org.apache.catalina.startup.Catalina.load Catalina.start using conf/server.xml: Error at (142, 72) : argument type mismatch
24-Jul-2017 17:06:37.057 GRAVE [main] org.apache.catalina.startup.Catalina.start Cannot start server. Server instance is not configured.

Excessive logging on info level

Tomcat uses info as default logging level. And in HazelcastSessionManager there are a couple of messages which should be on debug level, especially in the findSession method which is heavily used.

I propose to switch those log messages to debug level. Otherwise everyone needs to suppress them in the tomcat config. It's no good for performance to log messages on each request and several thousands of messages when doing a failover on a system with a lot of sessions, especially since the tomcat logging framework (juli) actually creates a stacktrace for each logged message (see log-method in DirectJDKLog).

Custom serialization is not supported on Session objects

Right now, it is a requirement that the session objects have to be Serializable. Other custom serializers should also be supported.

Please see the following community request as a sponsor: https://stackoverflow.com/questions/48428131/hazelcast-with-global-serializer-kryo-there-is-no-suitable-de-serializer-for

com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-X commited key:null #

Hi guys,

I did a long search but unfortunately I did not find anything and I do not know if it can be a problem or configuration.

Everything seems to be work normal, session replication and so on but, every time a Session is created in tomcat, the message below is written in the log ...

15-May-2017 19:34:45.961 INFO [ajp-nio-8009-exec-1] com.hazelcast.session.HazelcastSessionManager.findSession Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null
15-May-2017 19:34:45.961 INFO [ajp-nio-8009-exec-2] com.hazelcast.session.HazelcastSessionManager.findSession Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null
15-May-2017 19:34:45.962 INFO [ajp-nio-8009-exec-1] com.hazelcast.session.HazelcastSessionManager.findSession No Session found for:A738487E57F23BF5C3C042E8EBA467E6.tomcat1
15-May-2017 19:34:45.962 INFO [ajp-nio-8009-exec-2] com.hazelcast.session.HazelcastSessionManager.findSession No Session found for:A738487E57F23BF5C3C042E8EBA467E6.tomcat1

15-May-2017 19:35:40.399 INFO [ajp-nio-8009-exec-3] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-3 commited key:null
15-May-2017 19:36:09.526 INFO [ajp-nio-8009-exec-7] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-7 commited key:null
15-May-2017 19:36:20.598 INFO [ajp-nio-8009-exec-1] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-1 commited key:null
15-May-2017 19:36:26.951 INFO [ajp-nio-8009-exec-9] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-9 commited key:null
15-May-2017 19:36:36.814 INFO [ajp-nio-8009-exec-6] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-6 commited key:null
15-May-2017 19:36:56.453 INFO [ajp-nio-8009-exec-6] com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-6 commited key:null
...

And each page the user navigates, the message:
com.hazelcast.session.HazelcastSessionManager.commit Thread name:ajp-nio-8009-exec-6 commited key:null

continue all the time....
Is that some problem ?

My env 64 bits: Linux RedHat 7.3, Tomcat 8.5, hazelcast-tomcat85-sessionmanager-1.1.jar, hazelcast-all-3.8.jar

Thank you very much!

Hazelcast client for session replication conflicts with webapp

At the moment we're running Tomcat 8.5 with built-in session replication. Our webapps include a Hazelcast client for application related caching. This all works without issues.

Now I adjusted the Tomcat config to use Hazelcast as session store and for replication. Unfortunately this seems to conflict with our apps. I already assured that webapp and Tomcat use the same Hazelcast version jars.

This is the Tomcat startup with configured hazelcast-tomcat-sessionmanager:

30-Apr-2018 22:40:22.844 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.5.30
30-Apr-2018 22:40:22.847 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Apr 3 2018 20:04:09 UTC
30-Apr-2018 22:40:22.847 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.5.30.0
30-Apr-2018 22:40:22.847 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
30-Apr-2018 22:40:22.847 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            3.10.0-693.17.1.el7.x86_64
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/java/jdk1.8.0_172-amd64/jre
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_172-b11
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /opt/tomcat
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /opt/tomcat
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
30-Apr-2018 22:40:22.848 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -javaagent:/usr/java/default/jre/lib/ext/jmx_prometheus_javaagent.jar=1234:/etc/jmx_exporter/tomcat.yaml
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms6g
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx6g
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+AlwaysPreTouch
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseConcMarkSweepGC
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCDateStamps
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -verbose:gc
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+PrintGCDetails
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xloggc:/opt/tomcat/logs/gc.log
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseGCLogFileRotation
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:NumberOfGCLogFiles=10
30-Apr-2018 22:40:22.849 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:GCLogFileSize=100M
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DDOMAIN_CONFIG_HOME=/appconfig
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DDOMAIN_CONFIG_LOCATION=/appconfig/config/application/it
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhazelcast.server=hazelcast:5801
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhazelcast.name=dev
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhazelcast.password=dev-pass
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -DmemcachedServerAddresses=webserver:11212
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8453
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dspring.profiles.active=IT,dev
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dlogback.configurationFile=/appconfig/config/application/it/logback.xml
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/opt/tomcat
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/opt/tomcat
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/opt/tomcat/temp
30-Apr-2018 22:40:22.850 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based Apache Tomcat Native library [1.2.14] using APR version [1.6.3].
30-Apr-2018 22:40:22.851 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
30-Apr-2018 22:40:22.851 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
30-Apr-2018 22:40:22.856 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.0.2k-fips  26 Jan 2017]
30-Apr-2018 22:40:23.009 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
30-Apr-2018 22:40:23.013 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
30-Apr-2018 22:40:23.019 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-8009"]
30-Apr-2018 22:40:23.020 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
30-Apr-2018 22:40:23.020 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 633 ms
30-Apr-2018 22:40:23.041 WARNING [main] org.apache.catalina.users.MemoryUserDatabase.createUser Null or zero length user name specified. The user will be ignored.
30-Apr-2018 22:40:23.042 INFO [main] org.apache.tomcat.util.digester.FactoryCreateRule.begin [FactoryCreateRule] Create exception ignored: Null or zero length user name specified. The user will be ignored.
30-Apr-2018 22:40:23.152 WARNING [main] com.hazelcast.config.AbstractXmlConfigHelper.null Name of the hazelcast schema location incorrect using default
30-Apr-2018 22:40:23.246 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
30-Apr-2018 22:40:23.247 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.30
30-Apr-2018 22:40:23.254 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deploying configuration descriptor [/opt/tomcat/conf/Catalina/localhost/manager.xml]
30-Apr-2018 22:40:24.430 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
30-Apr-2018 22:40:24.486 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_0 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is STARTING
30-Apr-2018 22:40:24.856 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_0 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is STARTED
30-Apr-2018 22:40:24.887 INFO [localhost-startStop-1] com.hazelcast.client.spi.impl.ClusterListenerSupport.null hz.client_0 [dev] [3.8.9] Trying to connect to [hz-tomcat-it]:5701 as owner member
30-Apr-2018 22:40:24.973 INFO [hz.client_0.internal-2] com.hazelcast.client.connection.ClientConnectionManager.null hz.client_0 [dev] [3.8.9] Setting ClientConnection{alive=true, connectionId=1, socketChannel=DefaultSocketChannelWrapper{socketChannel=java.nio.channels.SocketChannel[connected local=/172.24.0.4:40092 remote=hz-tomcat-it/172.24.0.3:5701]}, remoteEndpoint=[172.24.0.3]:5701, lastReadTime=2018-04-30 22:40:24.964, lastWriteTime=2018-04-30 22:40:24.961, closedTime=never, lastHeartbeatRequested=never, lastHeartbeatReceived=never, connected server version=3.9.4} as owner  with principal ClientPrincipal{uuid='6456c805-5954-4ff2-bb20-25cb90158acb', ownerUuid='8cb66370-9641-429e-89b8-bb14af094e95'}
30-Apr-2018 22:40:24.973 INFO [hz.client_0.internal-2] com.hazelcast.client.connection.ClientConnectionManager.null hz.client_0 [dev] [3.8.9] Authenticated with server [172.24.0.3]:5701, server version:3.9.4 Local address: /172.24.0.4:40092
30-Apr-2018 22:40:24.978 INFO [hz.client_0.event-2] com.hazelcast.client.spi.impl.ClientMembershipListener.null hz.client_0 [dev] [3.8.9] 

Members [1] {
        Member [172.24.0.3]:5701 - 8cb66370-9641-429e-89b8-bb14af094e95
}

30-Apr-2018 22:40:24.979 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_0 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is CLIENT_CONNECTED
30-Apr-2018 22:40:25.027 INFO [localhost-startStop-1] com.hazelcast.session.HazelcastSessionManager.startInternal contextPath:/manager
30-Apr-2018 22:40:25.036 INFO [localhost-startStop-1] com.hazelcast.session.HazelcastSessionManager.startInternal HazelcastSessionManager started...
30-Apr-2018 22:40:25.059 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDescriptor Deployment of configuration descriptor [/opt/tomcat/conf/Catalina/localhost/manager.xml] has finished in [1,804] ms
30-Apr-2018 22:40:25.060 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/opt/tomcat/webapps/jolokia.war]
30-Apr-2018 22:40:26.120 INFO [localhost-startStop-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
30-Apr-2018 22:40:26.122 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_1 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is STARTING
30-Apr-2018 22:40:26.152 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_1 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is STARTED
30-Apr-2018 22:40:26.162 INFO [localhost-startStop-1] com.hazelcast.client.spi.impl.ClusterListenerSupport.null hz.client_1 [dev] [3.8.9] Trying to connect to [hz-tomcat-it]:5701 as owner member
30-Apr-2018 22:40:26.172 INFO [hz.client_1.internal-2] com.hazelcast.client.connection.ClientConnectionManager.null hz.client_1 [dev] [3.8.9] Setting ClientConnection{alive=true, connectionId=1, socketChannel=DefaultSocketChannelWrapper{socketChannel=java.nio.channels.SocketChannel[connected local=/172.24.0.4:40110 remote=hz-tomcat-it/172.24.0.3:5701]}, remoteEndpoint=[172.24.0.3]:5701, lastReadTime=2018-04-30 22:40:26.171, lastWriteTime=2018-04-30 22:40:26.166, closedTime=never, lastHeartbeatRequested=never, lastHeartbeatReceived=never, connected server version=3.9.4} as owner  with principal ClientPrincipal{uuid='45d420fe-980d-4fa5-a82b-0ee56509eef9', ownerUuid='8cb66370-9641-429e-89b8-bb14af094e95'}
30-Apr-2018 22:40:26.172 INFO [hz.client_1.internal-2] com.hazelcast.client.connection.ClientConnectionManager.null hz.client_1 [dev] [3.8.9] Authenticated with server [172.24.0.3]:5701, server version:3.9.4 Local address: /172.24.0.4:40110
30-Apr-2018 22:40:26.177 INFO [hz.client_1.event-10] com.hazelcast.client.spi.impl.ClientMembershipListener.null hz.client_1 [dev] [3.8.9] 

Members [1] {
        Member [172.24.0.3]:5701 - 8cb66370-9641-429e-89b8-bb14af094e95
}

30-Apr-2018 22:40:26.178 INFO [localhost-startStop-1] com.hazelcast.core.LifecycleService.null hz.client_1 [dev] [3.8.9] HazelcastClient 3.8.9 (20180103 - 1b184fb) is CLIENT_CONNECTED
30-Apr-2018 22:40:26.178 INFO [localhost-startStop-1] com.hazelcast.session.HazelcastSessionManager.startInternal contextPath:/jolokia
30-Apr-2018 22:40:26.184 INFO [localhost-startStop-1] com.hazelcast.session.HazelcastSessionManager.startInternal HazelcastSessionManager started...
30-Apr-2018 22:40:26.384 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/tomcat/webapps/jolokia.war] has finished in [1,324] ms
30-Apr-2018 22:40:26.417 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
30-Apr-2018 22:40:26.475 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
30-Apr-2018 22:40:26.487 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3466 ms

Deploying the webapp does not succeed anymore with hazelcast-tomcat-sessionmanager:

30-Apr-2018 22:44:50.826 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
30-Apr-2018 22:44:50.836 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext
30-Apr-2018 22:44:54.125 SEVERE [localhost-startStop-2] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [org.springframework.web.context.ContextLoaderListener]
 org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'messageSource' defined in class path resource [applicationContext-web-analytics.xml]: Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cache.annotation.ProxyCachingConfiguration': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cacheManager' defined in class path resource [redacted/common/configuration/CommonConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cache.CacheManager]: Factory method 'cacheManager' threw exception; nested exception is javax.cache.CacheException: Error opening URI [hazelcast]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.context.support.AbstractApplicationContext.initMessageSource(AbstractApplicationContext.java:713)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:531)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4776)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5240)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1856)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cache.annotation.ProxyCachingConfiguration': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cacheManager' defined in class path resource [redacted/common/configuration/CommonConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cache.CacheManager]: Factory method 'cacheManager' threw exception; nested exception is javax.cache.CacheException: Error opening URI [hazelcast]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:372)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
        at org.springframework.aop.framework.autoproxy.BeanFactoryAdvisorRetrievalHelper.findAdvisorBeans(BeanFactoryAdvisorRetrievalHelper.java:92)
        at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findCandidateAdvisors(AbstractAdvisorAutoProxyCreator.java:102)
        at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.findEligibleAdvisors(AbstractAdvisorAutoProxyCreator.java:88)
        at org.springframework.aop.framework.autoproxy.AbstractAdvisorAutoProxyCreator.getAdvicesAndAdvisorsForBean(AbstractAdvisorAutoProxyCreator.java:70)
        at org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.wrapIfNecessary(AbstractAutoProxyCreator.java:346)
        at org.springframework.aop.framework.autoproxy.AbstractAutoProxyCreator.postProcessAfterInitialization(AbstractAutoProxyCreator.java:298)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:423)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1633)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
        ... 23 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cacheManager' defined in class path resource [redacted/common/configuration/CommonConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cache.CacheManager]: Factory method 'cacheManager' threw exception; nested exception is javax.cache.CacheException: Error opening URI [hazelcast]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.obtainBeanInstanceFromFactory(ConfigurationClassEnhancer.java:389)
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361)
        at com.redacted.common.configuration.CommonConfiguration$$EnhancerBySpringCGLIB$$e229c9cb.cacheManager(<generated>)
        at org.springframework.cache.annotation.AbstractCachingConfiguration.useCachingConfigurer(AbstractCachingConfiguration.java:84)
        at org.springframework.cache.annotation.AbstractCachingConfiguration.setConfigurers(AbstractCachingConfiguration.java:77)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:701)
        at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
        at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
        ... 46 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cache.CacheManager]: Factory method 'cacheManager' threw exception; nested exception is javax.cache.CacheException: Error opening URI [hazelcast]
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
        ... 68 more
Caused by: javax.cache.CacheException: Error opening URI [hazelcast]
        at com.hazelcast.cache.impl.AbstractHazelcastCachingProvider.getCacheManager(AbstractHazelcastCachingProvider.java:97)
        at com.hazelcast.cache.impl.AbstractHazelcastCachingProvider.getCacheManager(AbstractHazelcastCachingProvider.java:126)
        at com.redacted.common.configuration.CommonConfiguration.cacheManager(CommonConfiguration.java:44)
        at com.redacted.common.configuration.CommonConfiguration$$EnhancerBySpringCGLIB$$e229c9cb.CGLIB$cacheManager$0(<generated>)
        at com.redacted.common.configuration.CommonConfiguration$$EnhancerBySpringCGLIB$$e229c9cb$$FastClassBySpringCGLIB$$e0121c59.invoke(<generated>)
        at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
        at com.redacted.common.configuration.CommonConfiguration$$EnhancerBySpringCGLIB$$e229c9cb.cacheManager(<generated>)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
        ... 69 more
Caused by: com.hazelcast.core.HazelcastException: java.lang.ClassCastException: Cannot cast com.hazelcast.client.impl.DefaultClientExtension to com.hazelcast.client.ClientExtension
        at com.hazelcast.util.ServiceLoader$NewInstanceIterator.next(ServiceLoader.java:331)
        at com.hazelcast.client.impl.HazelcastClientInstanceImpl.createClientInitializer(HazelcastClientInstanceImpl.java:357)
        at com.hazelcast.client.impl.HazelcastClientInstanceImpl.<init>(HazelcastClientInstanceImpl.java:209)
        at com.hazelcast.client.HazelcastClient$1.createHazelcastInstanceClient(HazelcastClient.java:55)
        at com.hazelcast.client.HazelcastClientManager.newHazelcastClient(HazelcastClientManager.java:74)
        at com.hazelcast.client.HazelcastClientManager.newHazelcastClient(HazelcastClientManager.java:60)
        at com.hazelcast.client.HazelcastClient.newHazelcastClient(HazelcastClient.java:68)
        at com.hazelcast.client.cache.impl.HazelcastClientCachingProvider.getInstanceThroughDefaultInstanceIfItIsDefault(HazelcastClientCachingProvider.java:119)
        at com.hazelcast.client.cache.impl.HazelcastClientCachingProvider.getOrCreateInstance(HazelcastClientCachingProvider.java:109)
        at com.hazelcast.client.cache.impl.HazelcastClientCachingProvider.createHazelcastCacheManager(HazelcastClientCachingProvider.java:56)
        at com.hazelcast.client.cache.impl.HazelcastClientCachingProvider.createHazelcastCacheManager(HazelcastClientCachingProvider.java:36)
        at com.hazelcast.cache.impl.AbstractHazelcastCachingProvider.getCacheManager(AbstractHazelcastCachingProvider.java:94)
        ... 81 more
Caused by: java.lang.ClassCastException: Cannot cast com.hazelcast.client.impl.DefaultClientExtension to com.hazelcast.client.ClientExtension
        at java.lang.Class.cast(Class.java:3369)
        at com.hazelcast.util.ServiceLoader$NewInstanceIterator.next(ServiceLoader.java:329)
        ... 92 more

30-Apr-2018 22:44:54.190 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log Closing Spring root WebApplicationContext

Looks like there currently is no way of isolating the Hazelcast client for the Tomcat session manager from the app’s client.

Still classloading issues

When two applications in Tomcat share a class definition the latter one will not be able to deserialize the attribute from the session because I assume the first application will hold the reference to this class. A ClassCastException will be thrown then.

To illustrate the issue I have set up a project based on your Hazelcast's docker images. See https://github.com/helge79/hazelcast-session-classloading-issue for details. Instead of putting a simple String in the session I will use a class called StringHolder for it. I will built an application and deploy it as example.war and example2.war. If example2.war is deployed before example.war I will not be able to get the session attribute back in /example.

And I think this is intended as described in #55 and #56 because it is not possible to change the class loader after initialisation. But how is it possible to have one instance per Tomcat in a P2P scenario at all then? How is this supposed to work? What am I missing here? Are there any configuration issue on my end?

Publish jars to Maven Repository

Basically, we need to include those libs into our maven project, however they are not publichsed yet. Which is making some inconvenience. Are you planning to publish them to maven?

Serialization exception when using Container Managed Security (Tomcat Realms)

Environment:
Tomcat 8.0.20
hazelcast-all-3.6.4
hazelcast-tomcat8-sessionmanager-1.0
with „Container Managed Security” (Apache Tomcat Realm)

Caused by: java.io.NotSerializableException: org.apache.catalina.authenticator.SavedRequest
      at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
      at java.util.Hashtable.writeObject(Hashtable.java:1157)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
      at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
      at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
      at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
      at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.write(JavaDefaultSerializers.java:242)
      at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:41)
      at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:197)
      ... 49 more

Test failure: Tomcat85SessionExpireTest

Tests in error:  

Tomcat85SessionExpireTest>
AbstractSessionExpireTest.testSessionExpireAfterFailoverAndSessionTimeout:49->HazelcastTestSupport.sleepSeconds:463 ? TestTimedOut

Jenkins Log: http://jenkins.hazelcast.com/job/TomcatSessionManager-Master/409/console

Session Access timestamps are inconsistent when using non sticky sessions

When using non-sticky sessions, the lastAccessedTime and thisAccessedTime may be inconsistent between nodes.

See #49

Thread context ClassLoader change causes serialization problem

When using client-server mode, if someone changes Thread's context ClassLoader, HazelcastSessionManager fails with serialization errors. We need to set the context ClassLoader to WebAppClassLoader before any serialization/deserialization occurs (similar to this).

Related SO question: http://stackoverflow.com/questions/41381817/hazelcast-tomcat-session-not-loading-classes-from-app-context

Default session timeout is 30sec (not 30min)

On Tomcat 85 (at least, not tried the others) there is a mismatch between seconds and minutes in the session timeouts.

HazelcastSessionManager contains:
session.setMaxInactiveInterval(getContext().getSessionTimeout());

However getContext().getSessionTimeout() returns the value in minutes, but session.setMaxInactiveInterval() expects the value in seconds. This means that the default timeout of 30min is turned into 30sec!

sessionDestroyed is not called after a failover

Due to the recent removal of the implementation of findSessions() method in HazelcastSessionManager, Tomcat will only expire local sessions.

So whenever a node has been shutdown the sessions created on that node will never be expired by Tomcat, only removed from the global session map by Hazelcast. And that means that the application won't get notified through the HttpSessionListener.sessionDestroyed() for those sessions.

I've created a patch to fix this issue by introducing a EntryExpiredListener on the session map which calls session.expire() if it's still valid. I also extended the AbstractSessionExpireTest class with test cases for this issue.

Have I understood this correctly? Is there a better way of fixing the issue?

YAML configuration is parsed as XML (com.hazelcast.session.P2PLifecycleListener)

The project README has a sample hazelcast.yaml file, which gives the impression that yaml is supported when using hazelcast 4. However, P2PLifecycleListener uses com.hazelcast.internal.config.ConfigLoader, which seems to always parse the config file as XML, giving this error on startup:

INFO [main] com.hazelcast.config.UrlXmlConfig.null Configuring Hazelcast from 'file:/usr/local/tomcat/lib/hazelcast.yaml'.
[Fatal Error] :1:1: Content is not allowed in prolog.
SEVERE [main] com.hazelcast.config.XmlConfigBuilder.null Failed to parse the inputstream
Exception: Content is not allowed in prolog.
Hazelcast startup interrupted.

Serialization improvement for attributes field inside HazelcastSession

ClassNotFoundException when deserializing session

I'm running Tomcat 8 within a container in a Kubernetes cluster.

Under /usr/local/tomcat/lib/ I have the following (other than the standard JARs):

• hazelcast-3.11.1.jar
• hazelcast-kubernetes-1.3.1.jar
• hazelcast-tomcat8-sessionmanager-1.1.3.jar

Under /usr/local/tomcat/webapps/ROOT/WEB-INF/lib/ I have the dependencies of my webapp.

I'm using NGINX Ingress Controller for Kubernetes with Sticky sessions enabled. To simulate a failover, I'm removing my INGRESSCOOKIE. This gives me a new backend, and the session manager notices it and tries to fetch the session on the new backend. I get the following logs:

16-Jan-2019 15:14:45.237 INFO [http-apr-8080-exec-1] com.hazelcast.session.HazelcastSessionManager.findSession Sticky Session is currently enabled.Some failover occured so reading session from Hazelcast map:null
16-Jan-2019 15:14:45.244 SEVERE [http-apr-8080-exec-1] org.apache.coyote.http11.AbstractHttp11Processor.process Error processing request
 com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: org.acegisecurity.context.SecurityContextImpl
        at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:86)
        at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:75)
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:269)
        at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:574)
        at com.hazelcast.session.HazelcastSession.deserializeMap(HazelcastSession.java:141)
        at com.hazelcast.session.HazelcastSession.readData(HazelcastSession.java:127)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:160)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:106)
        at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:51)
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
        at com.hazelcast.map.impl.proxy.MapProxySupport.toObject(MapProxySupport.java:1245)
        at com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:120)
        at com.hazelcast.session.HazelcastSessionManager.findSession(HazelcastSessionManager.java:231)
        at org.apache.catalina.connector.Request.doGetSession(Request.java:2951)
        at org.apache.catalina.connector.Request.getSession(Request.java:2343)
        at org.apache.catalina.valves.CrawlerSessionManagerValve.invoke(CrawlerSessionManagerValve.java:189)
        at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1156)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
        at org.apache.tomcat.util.net.AprEndpoint$SocketWithOptionsProcessor.run(AprEndpoint.java:2464)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.acegisecurity.context.SecurityContextImpl
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:288)
        at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:252)
        at com.hazelcast.nio.IOUtil$ClassLoaderAwareObjectInputStream.resolveClass(IOUtil.java:646)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
        at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:82)
        ... 26 more

It does look like an incorrect class loader is being used. The session manager exists in the generic Tomcat class loader and the classes that is put into the session exists in the webapp class loader. Pure guesses, but how do I make sure that the correct class loader is being used?

Async Put when deferredWrite is enabled

IMap.putAsync can be used in session sync stage as deferredWrite is enabled by default. When deferredWrite is disabled then IMap.put will be the option.

Question about sessions sharing in K8s

I am using K8s hazelcast config, and my multiple instances of tomcat servers find each other, a cluster is created. I can also see that session are shared ok. However, there's one use case that fails for me. A new member does not seem to get sessions from existing members. So, for instance, when there are 2 replicas running, I log into my webapp, and then hit F5 to see that I am landing on different containers - everything works as expected, and I logged it and both Tomcats have my session. However, when I add a new Tomcat instance, this one does not seem to have those sessions, though I clearly see in log that I have 3 members now:

Members {size:3, ver:5} [
	Member [10.244.4.121]:5701 - 4e9c2665-670c-4422-b5fd-86499d7bc68d
	Member [10.244.8.22]:5701 - f65a86f6-a49f-4f94-8705-6220d8ca7bce this
	Member [10.244.7.117]:5701 - 07a34f13-676d-4f46-b416-b696e006a441
]

Is this expected or there's some configuration that I am missing?

Thanks!

HazelcastSessionManager.findSessions can lead to OOME

HazelcastSessionManager.findSessions is used by Tomcat's session expiration task, and it is calling IMap.getAll().values() under the hood.
This can cause an OOME on caller side if the sessionMap is large enough. If our investigation of this method's usage does not indicate any reason to keep this implementation, we can improve this behavior.

at com.hazelcast.client.proxy.ClientMapProxy.getAll(ClientMapProxy.java:1196) [hazelcast-client-3.12.6.jar!/:3.12.6]
at com.hazelcast.session.HazelcastSessionManager.findSessions(HazelcastSessionManager.java:223) [hazelcast-tomcat9-sessionmanager-1.1.5.jar!/:?]
at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:567) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:558) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5539) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1353) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1357) [tomcat-embed-core-9.0.27.jar!/:9.0.27]
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1335) [tomcat-embed-core-9.0.27.jar!/:9.0.27]

ZD 6145

Session Synchronization Fails - Older session is updated in failed-over request

I am trying to implement a sticky session based load balancing across two Tomcat instances using the Hazelcast Tomcat Web Session Replication. For testing purposes, I have deployed the application on two different Tomcat instances and the load balancing is handled through Apache HTTPD (tried both mod_proxy and mod_jk). The jvmroute parameters and the mod-proxy settings are fine and the load balancing has no issues.

The problem is with the session replication across the two instances. When the first server (currently serving the request) goes down the request is sent to the second server. The Hazelcast cluster identifies that the session is through fail-over and is copying the session with the new session id (suffixed with the jvmroute parameter of the second server) - as described in the Hazelcast documentation https://github.com/hazelcast/hazelcast-tomcat-sessionmanager#sticky-sessions-and-tomcat) . However for the failed-over request, the session attributes are getting updated in the older session(failed over jvmroute) and not getting replicated resulting in the failure of the subsequent request.

I have gone through the documentation but unable to find a resolution at this point.

After tracing the flow, able to determine that the handleTomcatSessionChange in com.hazelcast.session.HazelcastSessionChangeValve is being called correctly. The request.changeSessionId(newSessionId) call happens and post this if I display the value of the requestedsession id, the value is updated. However, the session id is not updated and this is resulting in the older id in a request.getSession().getId() call and thereby the application setting the values in the older session. The first request fails as one of the session variables are not present, but once this is handled, the subsequent requests goes through fine. However, we would not be able to trace this across the whole application.

Version Used : Hazelcast 3.7.1
Tomcat Plugin: 1.0.2
Tomcat Version: 7.0.40
Apache HTTPD: 2.4