Comments (24)
I don't know this issue. Could you provide me more details? "Signals stop responding" means that whole server is down, or there is a problem with setting up connection for a while? I'll write some performance tests and then check memory. But previously I wanted to have some point to start
from nextrtc-signaling-server.
It happened again and now I see it seems to be blocking in ConversationRepository findBy.
I have a CustomJoinConversation SignalHandler :
package app.services.webrtc;
import org.eclipse.jetty.util.log.Log;
import org.nextrtc.signalingserver.cases.CreateConversation;
import org.nextrtc.signalingserver.cases.SignalHandler;
import org.nextrtc.signalingserver.domain.Conversation;
import org.nextrtc.signalingserver.domain.InternalMessage;
import org.nextrtc.signalingserver.domain.Signals;
import org.nextrtc.signalingserver.exception.Exceptions;
import org.nextrtc.signalingserver.exception.SignalingException;
import org.nextrtc.signalingserver.repository.ConversationRepository;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import javax.inject.Inject;
import java.util.Optional;
import static org.nextrtc.signalingserver.exception.Exceptions.CONVERSATION_NOT_FOUND;
import static org.nextrtc.signalingserver.exception.Exceptions.MEMBER_IN_OTHER_CONVERSATION;
@Component("joinwithreject")
public class CustomJoinConversation implements SignalHandler {
static Logger log = LoggerFactory.getLogger(CustomJoinConversation.class);
public ConversationRepository conversations;
private CreateConversation createConversation;
@Inject
public CustomJoinConversation(ConversationRepository conversations,
CreateConversation createConversation) {
this.conversations = conversations;
this.createConversation = createConversation;
}
public void execute(InternalMessage context) {
log.debug("joinwithreject start: " + context.toString());
conversations.findBy(context.getFrom())
.map(Conversation::getId)
.map(MEMBER_IN_OTHER_CONVERSATION::exception)
.ifPresent(SignalingException::throwException);
log.debug("joinwithreject: " + context.toString());
Optional<Conversation> conversation = findConversationToJoin(context);
if (conversation.isPresent()) {
log.debug("joinwithreject: found conversation and joining");
conversation.get().join(context.getFrom());
} else {
Exception exception = new Exception();
log.debug("joinwithreject: conversation not found " + context.getContent());
Exceptions conversationNotFound = CONVERSATION_NOT_FOUND;
CustomJoinSignalingException customJoinSignalingException = new CustomJoinSignalingException(CONVERSATION_NOT_FOUND);
customJoinSignalingException.setConversation(context.getContent());
throw customJoinSignalingException;
}
}
private Optional<Conversation> findConversationToJoin(InternalMessage message) {
return conversations.findBy(message.getContent());
}
}
And the logs shoe that it reached the line :
log.debug("joinwithreject start: " + context.toString());
But not :
log.debug("joinwithreject: " + context.toString());
Maybe findBy is blocking ?
from nextrtc-signaling-server.
from nextrtc-signaling-server.
Any news on this ?
Do you have an idea how I can debug and find the reason ?
from nextrtc-signaling-server.
seems to be blocked here:
return member != null && members.contains(member); ( MeshConversation.class line 63 )
from nextrtc-signaling-server.
from nextrtc-signaling-server.
Tested also with 1.0.0-RC3-SNAPSHOT, same issue
Seems that for some reason the ConcurrentHashSet 'contains' is blocking.
I have written
a python script to test , and can easily reproduce the issue.
`#!/usr/bin/env python
import asyncio
import websockets
import json
async def hello(i,uri,isCreate):
async with websockets.connect(uri) as websocket:
if isCreate:
x = {
'from':'',
'to':'',
'signal':'create',
'content':str(i)
}
else:
x = {
'from':'',
'to':'',
'signal':'join',
'content':str(i)
}
s = json.dumps(x);
await websocket.send(s)
loop = asyncio.get_event_loop()
tasks = []
for i in range(200):
t = asyncio.ensure_future(hello(i,'ws://localhost:8080/signaling',True))
tasks.append(t)
t = asyncio.ensure_future(hello(i,'ws://localhost:8080/signaling',False))
tasks.append(t)
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
`
from nextrtc-signaling-server.
from nextrtc-signaling-server.
Can you think of a reason for that line to block?
from nextrtc-signaling-server.
ok, I have some more clues, I think what is blocking is the entire function as its synchronized on the object, and there is another thread in 'join' method that is blocked because of the RemoteEndpoint flushBatch
in 'sendJoinedFrom' Conversation.class line 58
If I remove 'getRemotePeer().flushBatch();' line 51, from InternalMessage the deadlock doesnt happen.
Do you see any reason why not to remove it ?
from nextrtc-signaling-server.
from nextrtc-signaling-server.
yes, I remember that issue.
What if we run the flush from a thread pool ?
something like this:
public void send() {
if (signal != Signal.PING) {
log.info("Outgoing: " + toString());
}
getRemotePeer().sendObject(transformToExternalMessage());
threadExecutor.execute(new FlushRunnable(this));
// try {
// getRemotePeer().flushBatch();
// } catch (IOException e) {
// log.debug("Unable to send message: " + transformToExternalMessage() + " error on flush!");
// }
}
This also seems to solve the deadlock.
( how can I pass the ExecutorService to InternalMessage ? for testing I used a new local CachedThreadPool )
from nextrtc-signaling-server.
from nextrtc-signaling-server.
I've read about websocket session and I'll use basic endpoint instead of async. In that configuration metod send
must be synchronized and some retry mechanism will be nice to have. I'll prepare a branch with that changes. It's nice point to introduce performance tests (gatling)
from nextrtc-signaling-server.
Could you please check is it working for you?
https://github.com/mslosarz/nextrtc-signaling-server/tree/fixed_signals_stop_working_issue_27
from nextrtc-signaling-server.
this also hangs, on 'sendObject'
Why not send on new thread? that seems to work... ( just need to use the same ExecutorService for all operations )
from nextrtc-signaling-server.
I've check it. I've created some code to test lib in real environment. I found issue described by you. But after modification I wasn't able to reproduce issue. I'll finish that performance tests and I'll release version RC3. I suspect that you've used old version during second testing session
from nextrtc-signaling-server.
I retested, I am using the new version ( commit b8d2e2a ) and still getting the deadlock.
It is blocking on
this.getRemotePeer().sendObject(this.transformToExternalMessage()); ( InternalMessage.java line 49)
from nextrtc-signaling-server.
btw, I am testing with tomcat 8 , maybe that's the difference ?
from nextrtc-signaling-server.
from nextrtc-signaling-server.
tomcat 8.0.39
nextrtc with spring
from nextrtc-signaling-server.
I've release version 1.0.0-RC3 with a lot of changes according to your issue. Could you please test it in your environment. (I've introduced PerformanceTest where you can test your local server setup)
from nextrtc-signaling-server.
1.0.0-RC3 looks great so far, I will continue testing
from nextrtc-signaling-server.
Hi , I know this is an old issue, but it seems to still be a problem in RC6
from nextrtc-signaling-server.
Related Issues (20)
- nextrtc Support multiplayer video? HOT 8
- In MeshConversation sometimes newJoined is not sent to all HOT 4
- Join or create? HOT 15
- Often video connection is not work HOT 4
- Lost signals HOT 1
- How to use this library to create a java client HOT 4
- Is it possible to use nextRTC in mobile using for example ionic or native developing? HOT 1
- Nextrtc in a distributed context? HOT 2
- Text chat support ? HOT 1
- org.nextrtc.signalingserver.DaggerNextRTCComponent not found HOT 2
- STUN/TURN support? HOT 1
- Hope NextRTCEndpoint can extend Endpoint HOT 2
- Project build error HOT 2
- custom JoinConversation HOT 3
- close conversation when creator leaves HOT 2
- Is nextRTC use a single thread to handle connections ? HOT 2
- Close websocket after webrtc connection estasblished HOT 1
- Use modern logging framework HOT 3
- Invalid bean definition with name 'nextRTCEventBus' defined in org.nextrtc.signalingserver.NextRTCConfig HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nextrtc-signaling-server.