yihaopeng/dragonbornspeaksnaturally

SkyrimVR mod for dialogue and other voice control

Dragonborn Speaks Naturally (NexusMods)

DSN is composed of two projects, a plugin and a background service.

The dsn_plugin is a DLL written in C++ which hooks into SkyrimVR/SE and communicates with the background service.

The background service dsn_service is a Windows application written in C# which uses Microsoft's System.Speech API to do speech recognition. It communicates with the dsn_plugin over stdin/stdout.

Build dsn_service only

You need a Visual Studio with .NET Desktop Development modules.

Then enter dsn_service directory and double-click dsn_service.sln, a Visual Studio C# project will be loaded.

Build dsn_plugin only

You need a Visual Studio (with C++ Desktop Development module) and a CMake.

Then enter dsn_plugin directory and double-click configure.bat, a Visual Studio project will be created by Cmake and loaded automatically.

Auto install dlls to game directory after building

When you run configure.bat for the first time, you are asked about the installation path of SkyrimVR and SkyrimSE. If you want to enable the automatic installation feature, please provide the root directory of your games. If you want to disable the installation, just press Enter.

An example:

Set plugin install directories after building:
SkyrimVR game root path (empty to disable installation): E:/Games/SteamLibrary/steamapps/common/SkyrimVR
SkyrimSE game root path (empty to disable installation): E:/Games/SteamLibrary/steamapps/common/Skyrim Special Edition
CMakeFlags: -DSVR_DIR="E:/Games/SteamLibrary/steamapps/common/SkyrimVR" -DSSE_DIR="E:/Games/SteamLibrary/steamapps/common/Skyrim Special Edition"

And these commands will be execute for the example:

md build
cd build
cmake -A x64 -DSVR_DIR="E:/Games/SteamLibrary/steamapps/common/SkyrimVR" -DSSE_DIR="E:/Games/SteamLibrary/steamapps/common/Skyrim Special Edition" ..
start dsn_plugin.sln

A file named install-path.ini will be created after the first running of configure.bat. You can delete it and run configure.bat again to reset the path, or edit it directly.

How the auto installation works

The installation will be triggered each time you build dsn_plugin_xx.

Notice: The Visual Studio project INSTALL is generated by CMake for the project PACKAGE which is used to generate a FOMod/NMM/Vortex Compatible ZIP MOD file. It will only copy files into the build/package_tmp directory. It is different from the auto install mentioned here.

Directory structure of dsn_plugin

name	description
dsn_plugin/	The code of the plugin itself.
sse/	The SKSE64 codes for linking to dsn_plugin to generate a SkyrimSE-compatible DLL.
svr/	The SKSEVR codes for linking to dsn_plugin to generate a SkyrimVR-compatible DLL.
CMakeLists.txt	A project description file used by the CMake build tool.
configure.bat	A script to create a Visual Studio project in the build directory via CMake and load it.
build/	After you run configure.bat, the directory will be created automatically to hold the Visual Studio project and all build outputs. You can delete this directory at any time and re-run configure.bat to generate it. Files in this directory should not be commited to the repository.

About Visual Studio project dsn_plugin_se and dsn_plugin_vr

They are the same directory and have the same codes. Modifying the code in one place is equivalent to modifying the other.

The only difference between the two is that they linked to different SKSE libraries and header files.

Build dsn_service and dsn_plugin at the same time

Double-click configure.bat in the root directory of the repo, a Visual Studio project will be created by Cmake and loaded automatically.

Generate FOMod/NMM/Vortex Compatible ZIP Package

You can build the Visual Studio project PACKAGE to Generate a ZIP Package for dsn_service and dsn_plugin.

A fomod directory will be added to so the ZIP can be installed via NMM and Vortex.

The package name will be build/DSN-0.1.1-win64.zip.

Cannot build dsn_service because some assemblies missing

3>------ Build started: Project: dsn_service, Configuration: Debug x64 ------
3>C:\work\DragonbornSpeaksNaturally\dsn_service\dsn_service\CommandList.cs(1,7,1,16): error CS0246: The type or namespace name 'IniParser' could not be found (are you missing a using directive or an assembly reference?)
3>C:\work\DragonbornSpeaksNaturally\dsn_service\dsn_service\Configuration.cs(64,24,64,31): error CS0246: The type or namespace name 'IniData' could not be found (are you missing a using directive or an assembly reference?)
...

For technical reasons, CMake can't automatically restore the NuGet package and update references for project dsn_service.

If you met assembly missing, please restore and reinstall NuGet packages manually:

1. open NuGet Package Manager Console first:

   Tools > NuGet Package Manager > Package Manager Console

2. A prompt box will appear:

   Some NuGet packages are missing from this solution. Click to restore from your online package sources.

   Click the restore button. If you can't find it, just skip this step. The restore may be finished automatically.

3. Run this command in your NuGet Package Manager Console:

   Update-Package -reinstall -projectname dsn_service

4. Then you can build dsn_service normally.

License

We have a MIT License. Any reasonable exploitation of this project is welcome, including commercial use and porting to other games.

Except: The SKSE source code embedded in this project (in the dsn_plugin/sae|sse|svr dir) is not suitable for the MIT license. Please refer to the SKSE official website for understand the licensing policies for these codes.

dragonbornspeaksnaturally's Issues

Either incorrect or misleading documentation, or errors in project.

Either your documentation is misleading in that we can not build this project from the source files provided, or your Visual Studio project has a multitude of errors leading to build failure.

These are the errors I have received leading to build failure: https://gist.github.com/yndrelbosch/3a177d97ebb36cdd6c625e0cf02f2637

Uncompatible with modlist "Animonculory Visual Overhaul" for AE (From Wabbajack)

Which kind of mod could perhaps conflict with this one? The mods works fine if it's the only one installed, but in the modlist I get no responses

Debug the issue that crash frequently in SkyrimVR

As the title.

Attach to SkyrimVR.exe to debug.

Filepath Not Found

I completed all steps installing the mod. Docker Desktop starts up just fine but mod is not working. I checked the logs and it shows repeated errors for "filepath not found". The problem is the mod is acting as if the game in on D: drive. It looks like it's checking a directory for an old location of the mod and game. I previously reinstalled Skyrim VR on a different drive which is C: drive. For some reason, the mod is looking for the old location on D: drive.

Add a deselect windows option

For some odd reason, WMR works much better if SkyrimVR isn't selected. While there is a switchwindows commands to input key presses, there doesn't seem to be a deselectwindow command. This would at least help me get smoother performance - I could just say "stutter be gone" or "stutter return" to select or deselect it, as well as adding them into other custom commands.

Automatically disable EngineFixesVR's memory manager, or prompt the user to do so

This simple ini tweak fixed my Crash To Desktop on pressing the 'e' key, now I can interact with people more naturally!
rollingrock/EngineFixesVR#28

No license for source code

Please consider putting this under an open source license so we can update this when you stop. I believe there are some issues for the VR version after we enabled ESL support but we can't fix them without a license.

Add controller button presses

For some reason, when I use DSN to type E I get a CTD in SkyrimVR. If I was able to send controller presses to it, the problem probably wouldn't happen. Perhaps by using vJoy we could get around that problem.
As an example:
Greetsing=switchwindow; controllerpress vivetrackpadmiddle

DragonbornSpeaksNaturally.exe cannot exit after game exited

Biirman 29 March 2019, 2:23AM

I have encountered an issue that is quite annoying. Whenever I close the game, the dragonbornspeaknaturally.exe does not, resulting in a couple things. If I launch the game without closing, there are two instances open. Also, steam thinks the game is still running. Is there some way to make the program close when Skyrim closes, so I don't have to open up task manager?

Support for Windows New Voice Access?

Windows has recently release an extremely accurate low recourse speech recognition program called Voice Access. It's much less resource intensive then voice2json, but is at least as good or better. Is it possible to use it as the backend?

Compatible for SkyrimSE 1.5.73

As title

Add the option for using other speech to text programs

Unfortunately, Window's default speech to text tends to have a lot of trouble understanding me. Adding the ability to hook into other programs and apis would be useful, as there are other more accurate programs such as Mozilla deep speech or https://github.com/Uberi/speech_recognition. Having the ability to have accurate online or slightly less accurate offline speech to text would be a great improvement over the default Windows recognition.

(Suggestion) Toggle Voice Input

It would be nice to have a function where you can toggle the listening function. Use case would be, you have someone talking to you in Real, you toggle the input, and respond. Otherwise, if you respond sometimes DSN thinks you are talking to it, and will execute a command.