segs/segs

:muscle: SEGS - Super Entity Game Server

SEGS - Super Entity Game Server

SEGS is the Super Entity Game Server. It is a free, open source (under the terms of the BSD License), cross-platform, and object-oriented suite of protocols and services designed to interface with popular super hero themed MMORPG clients. SEGS is written in C++ and facilitates common MMORPG functionality such as account and character storage to a database, account and character retrieval from a database, secure client authentication, client movement in a virtual world, and client chat.

For legal information, please see LICENSE.md.

For a list of SEGS authors and contributors, please see AUTHORS.md.

Some other useful links:

• Our Discord
• Our wiki
• The Issue Queue
• How Can I Help? Specific Tasks to Help
• Our target coding style

DOCUMENTATION

• Contribute
• Compile and install SEGS on:
  - MacOS
  - CentOS/RHEL
  - Fedora
  - FreeBSD
  - Ubuntu
  - Windows
• Docker
• Install SEGS from Package:
  - Windows
• Configure SEGS using SEGSADMIN:
  - Linux
  - Windows
• CoH Client installation:
  - Windows
  - Ubuntu
• FAQs

CONTRIBUTE TO DEVELOPMENT

SEGS needs your help! From editing README's like this one, to coding the MapServer, there are tasks that can be tackled by contributors of all skill level!

Basic instructions for compiling SEGS in Linux and Windows are above, however for more detailed visual instructions, visit https://segs.dev/developers

Please read CONTRIBUTING.md and see the links below to begin:

• Our GitHub: https://github.com/Segs/Segs
• The Issue Queue: https://github.com/Segs/Segs/issues
• How Can I Help Issue: #706
• Our target coding style
• Join us on Chat!

HELP AND MORE INFORMATION

For Help with installation and configuration of your local SEGS, please see visit us on discord at https://discord.segs.dev/

FAQs

Visit our FAQ at https://github.com/Segs/Segs/blob/master/docs/FAQ.md

segs's Issues

Add explicit way of marking CRUDLink_Event as unreliable.

When unreliable CRUDLink_Event is sent over the link we will not wait for it's id to be ACK'ed from the other side - the packet should be removed from unsent_packets list immediately after it is sent over the underlying transport link (udp)

---

Consider splitting unsent_packets into unsent and unacked ?

Provide a error reporting component

Maybe use breakpad ?

Remove unused cmake scripts from CMakeScripts

Only 3rdparty_support.cmake and SEGS_Components.cmake are used.

Implement collision detection in map server

We need to:

• move scene and model loaders to a common library accessible from mapserver and mapviewer both
• select a scene graph format for mapserver ( maybe use existing engine in headless mode ? )
• implement scene graph conversion to mapserver format
• add map loading/conversion in MapInstance class
• implement Entity-> collision cylinder conversion
• finally implement the actual collision detection :)

Handle client input

Client input packets should by transformed into input events, and posted into appropriate Entity's event queue.
For this version only things we should worry about are:

• direction keys + jump
• camera control

Server doesn't AFK (or disconnect?) idle clients

I accidentally left the client running on my server all night and day, totaling over 18hrs. This shows how rock-solid the server is! However, it probably isn't desired behavior for a live server? Should the server disconnect idle clients?

Regardless, the server should detect when a client goes idle for a period of time, and set the character to /afk status

Can't find MapServerData.h

Can't find MapServerData.h

Maybe it missed the commit?

Segs/Projects/CoX/Servers/MapServer/MapServerData.cpp

Line 1 in f53acb9

#include "MapServerData.h"

Client/Server Patching tool

It would be very helpful to have some tool/component that would check user's files, compare with latest versions, update as needed.
Crazy use-cases:

Preparing asset lists.
Handle project-specific archives (patching 2 100kb files instead of 10GB zip archive)
Update queries (start-up)
Update notifications (live)
Distributing patch through multiple channels (servers/p2p).
Patch reversing (at least back to a working version)

Jumping produces Unhandled event type 116 message in the server console window.

Jumping produces Unhandled event type 116 message in the server console window.

Read/prepare internal map representation

Client needs to be informed which map it's in, what parts/groups/references are used in the map's display, their location, size etc.

Client's connection to DBServer times out during Character Creation

Spending too much time on Character Creation results in the client's connection to the dbserver timing out.

I don't have a precise duration for the timeout, but it looks to be around 1.5 minutes, based upon two tests:

Following:
Log In -> Create New Character -> Choose Powers -> Choose Costume -> Choose Name -> Spawn in Atlas Park

First Test (Log In to Spawn): 2 minutes 20 seconds
Result: Timeout
Sat Oct 21 14:43:17 PDT 2017
Sat Oct 21 14:45:37 PDT 2017

Second Test (Log In to Spawn): 1 minute precisely
Result: Successful Spawn in Atlas Park
Sat Oct 21 14:46:05 PDT 2017
Sat Oct 21 14:47:05 PDT 2017

Spawn NPCs (Contact and Mobs)

Upon Server Start, we need to spawn static NPCs and Mob Spawns.

I believe all of these npcs, along with relevant information is in npcs_client.bin

CoH server should report clearer messages

"Couldn't load Costume!!" and "Couldn't load Supergroup colors!!" are not clear enough.
Should provide a full path of missing files + maybe inform how to get those.

Local chat outputs to Broadcast channel instead, meanwhile Broadcast chat appears to do nothing

Using Local chat (either through UI or with slash commands) outputs message to the Broadcast channel, meanwhile the broadcast channel appears to do nothing.

Test:
/l test local! output: [Broadcast] test local!
/b test broadcast! output: none

I haven't been able to test Team, Supergroup, or Private Messages yet.

Server shuts down immediately

After latest commits, server will not startup, and produces only the following nondescript error message:

main
Reading map data from ./data/bin/ folder
Shutting down AdminServer: "No reason"
Shutting down game server because :  "Full system shutdown"
MapServer has not been started yet.

I'm looking through the various ShutDown calls now.

SEGS version

Clone latest from git

Operating system

Linux (Fedora 26)

C++ compiler and version

gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)

Create account management tool

Minimal functionality:

• Account Creation
• Account Removal
• Account Search/Edit

A clean database schema + access description should allow creation of simple website with this functionality

Day Night Cycle is too fast

The current day/cycle is too fast, transitioning through a full Cycle every 10min or so. It's so quick you can see the moon moving across the sky in jerky phases.

According to ParagonWiki:
One real time hour is equivalent to two in-game days, with midnights occurring on the hour and on the half hour.

They have a handy chart as well: https://paragonwiki.com/wiki/Time

And includes information regarding timed events, which means eventually we'll want to expose the game "clock" to the Lua scripting engine for missions and spawns.

Merge most of CoX/Common libraries into a single library

Until there are separate executables for each service, the separation does not make much sense.

Remove old scripting code.

The files in Components/Scripting sub-directory are not used and should be removed.

Implement SWIG

Need to implement SWIG interface to Ruby and Python scripts. Each server will have its own interface to its own set of scripts.

Create a database initialization/server management tool

This tool should:
Given a database ip and admin login/pass:

• Create segsdb user
• create segs databases ( according to current sql files)
• create database tables ( also using provided sql files)

Given a server address:

• Set server IP's and ports correctly
• If database info is available, put it into server configuration file.

Things to consider:

• Database schema updates.

Server crashes after first client disconnects.

Crash happens on linux/windows on qt5 based version.

Unable to abort disconnect

Expected behavior

When quitting or logging out, a countdown begins allowing a user to abort the disconnect by pressing any key.

Actual behavior/symptoms

Disconnect is not aborted, client disconnects.

Steps to reproduce behavior/symptoms

Start server, log in, select character, log out > try to abort log out by pressing any key.

SEGS version

clone of latest git

Operating system

Test on Windows 10 (also in compat mode for XP-SP3 and Win7)

C++ compiler and version

g++ 5.3.0 (mingw530_32)

Put simple build instructions into top-level Readme.md

Something like:
Windows:

• Install Qt >= 5.8.0 for Windows 32-bit (MinGW 5.3.0) ( with QtCreator )
• Install CMake.
• Open Segs/CMakeLists.txt with QtCreator
• Build.
  Linux:
  ....

ClientManager.h does not exist

ClientManager.h does not exist. This doesn't seem to prevent me from compiling, but did produce this error upon shutdown:

ACE_ASSERT: file /Segs/Projects/CoX/Common/Servers/ClientManager.h, line 96 assertion failed for 'getById(id)!=0'.Aborting...
Aborted (core dumped)

Included by these two files:

Segs/Projects/CoX/Servers/GameServer/GameHandler.h

Line 2 in 9625d58

#include "ClientManager.h"

Segs/Projects/CoX/Servers/MapServer/MapInstance.h

Line 14 in 9625d58

#include "ClientManager.h"

Properly change translucency of entities

Currently all entities have translucency hard-coded to 0.5 this needs to be modified.
Entity::serializeto call is here

Fix user action processing code.

User actions are packed after bits of InputState packets.
current code tries to read as many packets as it can in CRUDLink<EVENT_FACTORY>::event_for_packet.

Since only Map's InputState packets have additional 'following' packets. the logic should be moved closer to InputState processing part ( maybe by storing ByteStream with contents in InputState packet, and processing bits in void MapInstance::on_input_state(InputState *st)

Ctrl+C or Ctrl+Break results in Segmentation fault (core dumped)

There does not appear to be any "safe" way to shutdown the server from the terminal window. Ctrl+C or Ctrl+Break results in Segmentation fault (core dumped)

Perhaps the server should provide a prompt #server: while running? It could accept a few basic commands:

• save would run a world save. Useful if the server needs to be rebooted.
• shutdown would execute a safe shutdown event. Would display press any key to close this window when completed and before closing console window
• status would display stats such as number of active connections, etc.
• debug would display debug information in the console window, in real time. Would return to prompt upon key press. All debug info should be output to output.log anyway.

I'm not sure what "best practices" are for this type of thing.

Other characters do not show correct Alignment, Origin, or Archetype

Expected behavior

When selecting a character, the client shows you their current information, including Alignment, Origin, Archetype, HP, etc.

Actual behavior/symptoms

Default values are shown. For example, Origin/Archetype is always Science/Blaster

This information is already saved in the database, so I'm assuming EntityResponse isn't updating other clients when sending this data.

Movement has a lot of rubberbanding and collision issues

Moving results in a lot of rubberbanding and collision issues. This might be fixed by #44?

Client cannot connect on Linux (Fedora 26 kernel 4.13.8-200.fc26.x86_64)

The CoX client crashes immediately on Linux and shows two dialog boxes with error messages (below). Maybe I'm setting up the database incorrectly? Or the server config?

I'm trying to connect to my local IP, not a loopback address.

It works on Windows, with the same setup.

Let me know what other information you may need.

uname -a
Linux zelda.hyrule 4.13.8-200.fc26.x86_64 #1 SMP Wed Oct 18 15:42:58 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Server Log (output.log)

2017-10-20 02:42:12.671@@11835@LM_ERROR@main
2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Reading game data
2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Reading Costumes ... 2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Couldn't load Costume!!
2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Reading Supergroup colors ... 2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Couldn't load Supergroup colors!!
2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) All game data read
2017-10-20 02:42:12.672@@11835@LM_WARNING@(11835|139669139534976) Filling hashes .. 2017-10-20 02:42:12.672@@11835@LM_WARNING@Hashes filled

Client Error Log

fixme:winediag:start_process Wine Staging 2.18 is a testing version containing experimental patches.
fixme:winediag:start_process Please mention your exact version when filing bug reports on winehq.org.
Running City Of Heroes
Error opening config file (N:/revisions/gimmecfg.txt)                          
Will proceed without source control

You probably have been disconnected from N:, double click on My Computer
and then double click on N: to reconnect it.
project segs
fullscreen 0
Caching relevant folders.. done                                       (0.00)
Error loading required game data files.  (trick load error)

Loading texture headers..                                             (0.00)

Dialog Box 1
Error loading required game data files. (trick load error)

Dialog Box 2
Failed to load texture library.
It is likely that City of Heroes is not properly installed. Re-installing or running CohUpdater may fix this.

Database management tool: step 1.

A simple tool that will create sqlite databases based on the sql template fiiles:
Basic logic should look like

...
if(weAreNotRunningFromServerDir())
    return RUN_US_FROM_SERVER_DIR;
if(databaseFilesAreHere())
    return NOTHING_TO_DO;
// create auth server account db
QString path = SEGS_ACCOUNTS_DB_PATH;
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE");
db.setDatabaseName(path);
db.open();
QSqlQuery query;
QFile db_template("segs.sql");
if(db_template.open(QFile::ReadOnly)
    query.exec(db_template.ReadAll());
db.close();
// create game server character db
...

Migrate relevant issues/milestones from redmine.

When this and #31 task are done, we can remove redmine from project's front page.

Implement /stuck command

It might make sense to implement the /stuck command as the mapserver and collision detection come together.

Official Server behavior was to move you a very small distance to a unobstructed location.

Increase number of character slots per account and allow configuration in settings.cfg

Currently the server allows for only two character slots per account.

I couldn't find where this is regulated in the source, but we should expand this to 12.

Client crashes sporadically on windows

When running CoX on Windows 10, the client will crash randomly. I can't attribute the crash to any specific event(s)

Let me know what kind of log/information you may need.

[Meta] Issue 23+ Client Support

Update as of August 2018

Current consensus is that developing a custom client may actually prove to be faster, and a better use of resources than reverse-engineering the I23+ client. Preliminary work has begun on AuthExerciser (#237), MapViewer (#238), and Client Gfx Patch (#450) which lay a foundation for a custom client. Additional work on a QT based GUI resources should be followed here: Segs/Segs-Assets#6

The below is left here for posterity, and to provide a sort of loose roadmap for feature parity.

Summary

This is a meta issue, for proposed Issue 23+ Client support. As the term "issue" is vague and may refer to either a GitHub issue, or a City of Heroes Client Release Milestone, I'll standardize the usage here by referencing all CoX Client release milestones as such: I## or I24 or I7 or by referring to them as client patches

The Goal here is to provide information and resources should a desire rise to support later clients, or should a developer come to the table interested in forking current development to a later client patch, such as I23+

This meta issue should identify and reference other issues or concerns that may prevent, or create hurdles for development towards I23+ support. Additionally, this should support general discussion revolving around late-game client support viability.

When submitting new issues related to this meta, please tag with the (proposed) Issue23 label for sorting.

Known release-blocker Issues:

• issues to go here

Issues needing more research:

• Blowfish encryption likely changed every time the client was patched, which means that packet calls would have to be rewritten to work with the new client.
• Is there a way to configure the server for a specific CoX patch period? Something like server-issue: 11
• The server should check client version and output a message for versions incompatible with the server issue
• Should the scripting system be able to target a specific release version?
• Target: CoH release (I0)
• Target: CoV release (I6)
• Target: Going Rogue release (I18)
• Target: Sunset Client (I24beta)

Additional Game Subsystems that will need development

• Tailor System (this may be handled mostly client side)
• Exemplaring & Sidekicking
• Respec'ing
• Personal Badges, Exploration Badges, Accolades, and Supergroup Badges
• PvP
• City Info Terminals & Server/Player Stats
• Enhancement Diversification
• Arenas and Arena Combat
• Villian Hero Type, Classes, and Powersets
• Mission Architect
• Wentworth's Auction House
• SuperGroup Bases
• Rogue/Vigilante Hero Types
• Holiday Events (Halloween, Winter, etc.)
• Ski Chalet ski challenge
• Ouroboros (Flashback System)
• Salvage and Invention System
• Incarnate System
• Day Jobs

This is a reminder to fill out SceneGraph wiki page

SceneGraph needs you! :)

Server data folder names are case sensitive

Server looks for CoreData, Data, and data folders during initialization.

This isn't much of an issue on Windows, which does not have a case sensitive file system, but may present problems on Linux during server setup. Additionally, for organization, it may be best to maintain a consistent naming schema for data folders.

My proposal is to enforce a consistent naming format for data files moving forward, and reorganize the current folders as below:

./README.txt       # proposed installation guide
./settings.cfg     # proposed name for local_1.cfg
./data/            # data folder for client data (.pigg) files
./data/bin         # data folder for extracted bin files
./mapviewer/       # data folder for mapviewer utility
./scripts/         # lua scripts folder

If there are considerations I've overlooked, please share them in the comments below.

Chatting features

This needs filling out:

ChatServerFeatures

[Movement] Rubberbanding movement and Interpolation

This issue began with work on orientation, which was not being updated correctly. The scope has since expanded to include all movement

This will need

• Correct Position Update math (math in physicsStep(...))
• (interp-v2, sending junk vals) Correctly send interpolated movement values in EntityUpdateCodec (storeUnknownBinTree(...))
• Magic all of these values together
• Correctly store ControlBits states (keypresses and releases) with keypresses time.
• Do we need MotionStates (surface and physics params) for this to work? Probably.
• Do we need TimeStates (record of when keys were pressed relative to server time) for this to work? Probably.
• (interp-v2) Store InputStates to vector InputStateStorage
• (interp-v2, I think 😜) Correct Velocity calculations -- Currently the velocity the client believes you to have, does not match with the server's expectation
• Verify stops to server/client -- it appears as though the client doesn't always recognize that you've stopped moving (may relate to ControlBits)
• Store Position updates to PosUpdates for interpolation
• Calculate Interpolation values from PosUpdates (return BinTree)
• Does EntityUpdateCodec send m_update_pos_and_cam every time? Or only for special updates?
• Correct Entity Rotation upon spawn (Fixed by #104)
• Correct Entity Rotation during movement in EntityUpdateCodec (addressed by #166)
• Verify Speed calculations
• Verify Drag calculations
• Verify Gravity calculations
• Client does not recognize ABS_TIME, though it is sent by the server

Bonus

• Send Animations
• Send NetFX
• Correctly calculate jumping (this is going to be a whole other set of algos)

Useful testing commands (require Access Level 9)

• /controldebug 1 - outputs to client console and shows markers in the world with position deltas
• /entdebugclient 1 - displays GUI window that shows active data, and displays reticle on entity
• /moveto {x} {y} {z} - moves entity to position
• /postest - Outputs server values for position. Only available on my broxen/segs/interp-v2 branch
• /interp - Runs an interpolation test. Only available on my broxen/segs/interp-v2 branch

SEGS version

78c7fd5 and later

Tracking

Progress taking place in https://github.com/broxen/Segs/tree/movement

Script integration design

We need some kind of ScriptingIntegration designed.

Implement proper EventProcessor shutdowns

Since many core objects now are active event processors, it would behoove us to shut them down properly. This will require creating a shutdown event and handling it properly (sending it to children, etc.)

Should not be able to log into the same Account twice simultaneously.

Summary

Logging in with the same Account and same character results in the CoX client crashing and if tried repeatedly, the server crashing.

Looks like the server needs to handle this code 0x7f5ffc019700 gracefully.

(3274|140050713548544) Connection closed
 Telling map server to expect a client with character Sarim,1
Sending msg to client 0x7f5ffc01c080
(3274|140050713548544) Connection closed
 Telling map server to expect a client with character Sarim,1
Sending msg to client 0x7f5ffc019700
(3274|140050713548544) Connection closed
 Telling map server to expect a client with character Sarim,1
Sending msg to client 0x7f5ffc019700
^CSegmentation fault (core dumped)

SEGS version

clone of latest git

Operating system

Fedora 26

C++ compiler and version

gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2)

Character Select Screen doesn't show current Zone info ($15)

The Character Select Screen during login doesn't show the character's current zone information. It appears to have a static text statement "Some map name"

This is found in CharacterDatabase.cpp on line 146:

Segs/Projects/CoX/Servers/AdminServer/CharacterDatabase.cpp

Line 146 in 90cc9e2

c->setMapName("Some map name"); // "V_City_00_01.txt" STR_OR_EMPTY(r.getColString("current_map"))

Use QSettings in the codebase

using QSettings in the codebase should make retrieving various config variables much easier:

• Use QSettings::setPath to set ini path globally
• Use QSettings::setDefaultFormat to set default format to ini files.
• Check if using QSettings constructor with a single "SEGS" argument whenever we need to access configuration will read/produce the same INI files as it does now, if it doesn't update the default ini files.

Compiling\Installation Guide Needs Updating (and install script?)

The installation guide on the forums and github wiki need updating and/or additional detail.

Specific objectives should be:

• How to Compile in major platforms and the tools or libraries required. See #36
• What the server needs in order to operate properly (data files, piggs, etc), and where to get those files
• How to configure local_1.cfg (might need a name change? proposed: settings.cfg
• How to setup and configure the postgres database
• Include a SQL script to setup the database automagically
• How to acquire the client and what version of the client work
• How to configure the client to access your server
• Common Issues/FAQ

These guides might be best broken up into three separate how-tos: howto-compile howto-setup-server and howto-setup-client.

I'm happy to tackle this task, but as of now, I'm not 100% sure I'm doing it correctly myself. Once I've gotten everything running on both Linux and Windows, I'll feel more confident taking this on.

In the future, an install script that walks a user through the setup/config would be excellent.

Spawn Pedestrians and Misc Entities (cars, blimps, monorail)

Summary

On Server Startup we need to spawn Pedestrians and other moving objects/entities. These are things like cars, blimps (in atlas park), monorails, etc.

Nem's work on NPCGenerator's has laid a foundation for this.

Images

Monorail doesn't work

The Monorail doesn't show up in the station.
Clicking on the door to the monorail has no effect and it doesn't render the gates properly.

cmake fails on libACE.so citing undefined reference to aio_suspend and other aio functions

I'm running into an issue compiling the latest from github. Cmake fails in both QTCreator and Visual Studio Code on Linux (Fedora 26)

[~]$ uname -a Linux fedora.workgroup 4.12.12-300.fc26.x86_64 #1 SMP Mon Sep 11 17:18:56 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Attempted with both ACE lib preinstalled and by compiling. Worked in Sept, but that was several commits ago. Partial output:

/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_suspend'
/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_write'
/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_error'
/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_read'
/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_return'
/home/broxen/CoX/SEGS/build/3rd_party/built/lib/libACE.so: undefined reference to `aio_cancel'
collect2: error: ld returned 1 exit status
gmake[2]: *** [Projects/CoX/Utilities/MapViewer/CMakeFiles/mapviewer.dir/build.make:308: ../out/mapviewer] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:835: Projects/CoX/Utilities/MapViewer/CMakeFiles/mapviewer.dir/all] Error 2
gmake: *** [Makefile:84: all] Error 2
13:38:40: The process "/usr/bin/cmake" exited with code 2.

QTCreator compile output attached (txt file)
consoleoutput-10-16-17.txt