surge-synthesizer/shortcircuit-xt

Will be a sampler when its done!

Shortcircuit XT

Welcome to Shortcircuit XT!

Shortcircuit was a popular creative sampler in the mid 2000s from vembertech, the company that originally created surge. In 2021, Claes Johansen, the principle at vembertech (and now at Bitwig), open sourced a version of Shortcircuit2 and gave the source to the Surge Synth Team.

After several false starts getting this code running, we made a decision in early 2023 to preserve much of the design and DSP code from SC2, but to rebuild a sampler in the spirit of the first two versions of Shortcircuit using modern C++ and JUCE. This project, which you are looking at here, is called Shortcircuit XT (SCXT) and is underway.

Right now, you can download our pre-alpha version of the instrument here or you can build it using the instructions below.

What Works, and How Can I Help?

Well first, thanks for asking! As we proceed through summer 2023, more and more stuff is starting to work, but we have a lot to do to get to a working beta. We have a rather complete wireframe of the final product, though, and have made quite a few core decisions about selection, mechanics, playback, and more, some of which are reflected in the code.

If you are going to use the product now, though, many things will be incomplete or mysteriously not work. Really the best way to particpate is to join the surge synth team discord and come say hi in the #sc-development channel.

There's lots of ways you can help. Of course, developers are always welcome. But testers, design partners, and just generally nice people who want to shoot-the-you-know-what about a sample-initiated instrument can join in the fun. Also, folks who loved and used SC1 and 2 are welcome to come and see what we are thinking is different and the same, and help us determine if we are right!

I would love to build it myself. What do I do?

To build, first configure your machine. Basically set up your machien the same way you would to build Surge XT then fork this repo and:

git clone <this repo or your fork>
cd shortcircuit-xt
git submodule update --init --recursive
cmake -Bignore/build -DCMAKE_BUILD_TYPE=Release
cmake --build ignore/build --config Release --target scxt_plugin_All

Our production build uses clang on macos, and gcc on linux. We will test with a wide variety of compilers, including msvc on windows and several gcc versions. If you are using a new compiler and have changes to the CMake or so on, please do send them to us.

How we got here?

Vember Audio, founded by @kurasu/Claes Johanson, released Surge and Shortcircuit in the 2000s. In 2018, Claes, realizing he would not complete Surge 1.6, open sourced it. A community arose around it, leading to Surge we have today.

In late 2020, @baconpaul (one of the maintainers of Surge) sent a thank you note to Claes for open sourcing Surge and they got to chatting about Shortcircuit. That led to Claes sending Paul the source code, and that led to this repository and plan. We had a couple of false starts, but are confident in the path and architecture we have now!

An important note about licensing

Just like with Surge, we welcome any and all contributions from anyone who wants to work on open source music software.

At the initial open-sourcing, the copyright to the Shortcircuit source was held by Claes (on the initial code) and Paul (on his diffs since Claes shared the code). This repository is licensed under GPL3. We know that, if we end up with Shortcircuit XT, we will end up with a GPL3 plugin, since it will depend on JUCE or VST3SDK, which are GPL3 if used in an open source context.

But, Paul and Claes are discussing refactoring the code to be a mix of MIT and GPL3 code, with some critical things like sample format loaders, some DSP code, and some utilities at least being released under MIT in a subordinate library (and in a separate repo). We don't know where that line is yet, but we do want to reserve the right to re-license any or all of the code here under an MIT license at our discretion.

As a result, we are asking individual contributors who want to contribute to Shortcircuit XT to sign a CLA. We have used the Canonical/Harmony 1.0 CLA http://selector.harmonyagreements.org with the following choices:

1. You retain copyright to your code. We do not need you to assign copyright to us.
2. You grant us a license to distribute your code under GPL3 or MIT; and your content under CC3 attribution

You can read the entire document here.

To agree to this document, please add your name to the AUTHORS list in a git transaction where you indicate in the git log message that you consent to the CLA. You only need to do this once, and you can do it in your first commit to the repo.

shortcircuit-xt's Issues

non-ascci-or-utf files

paul ~/dev/music/shortcircuit-xt (plod-away) % find src -exec file -I {} \; | grep -v us-ascii | grep -v binary | grep -v utf-8
src/sampler_state.h: text/x-c; charset=unknown-8bit
src/vembertech/vt_dsp/basic_dsp.h: text/x-c; charset=unknown-8bit
src/vembertech/vt_gui/vt_gui_controls_browser.cpp: text/x-c++; charset=iso-8859-1
src/generator.h: text/x-c++; charset=iso-8859-1

What on earth is pragma pack and why is it warning us all the time?

Format: GIG/DLS

https://musical-artifacts.com/artifacts/77 there's some CC GIG files

They don't load but they don't crash

_mm_free - sigh

Over in #47 @rudeog made things better by explicitly calling a destructor before _mm_free is called on a C++ object

But the right thing is really to use alignas, free, and new, and so on, all properly.

Patch and Sample Browser

SC2 had a patch and sample browser which required things right next to the DLL and had an internal database you had to rebuild by hand. This led to the code now in vt_gui/browserdata and also the associated browser data messages.

That should all go though and be replaced with

1. A sampel browser on the side based on filesystem
2. Where the user can select roots
3. and where ~/Documents/Shortcircuit XT (as determined by sst-plugininfra) is one of the roots available

This will lead to several commits and issues but want to put this here as a sort of master issue.

Stub out the mmio code using RIFFMemMap thing

The MMIO code has the same shape as the riff men map but deals with on-disk

We want win / lin to do same so do a portable file mmap and use the overlay riff code.

mac and linux both use posix mmap so should be doable pretty easily

Typo in README.md: Johnason should be Johanson

Johnason should be Johanson.

Way to reset the synth

Related to #155
We want a user action to be able to reset the synth. Right now you have to use the juce reset in the standalone but there is no way in the plugin.

Move the unused directory out of the source tree; make each file explicitly not compile.

SC2 and SC1 Screenshots

What Are: _object and bdata messages

ip_filter_object and stuff. What do they mean and what should we do with them?

Audio format support: FLAC, OGG Vorbis, Opus

Integrate these as submodules (each of which have cmake support) and get it to link being sure to get a static link

• https://github.com/xiph/flac
• https://github.com/xiph/ogg
• https://github.com/xiph/vorbis
• https://github.com/xiph/opus

then use them to load single shots in add_zone / sampler_fileio for .flac and (whatever extension ogg/vorbis uses - .ogg?)

and add a test sample to the resources/tests

Sample mapping: what it is, how it used to work, and how it should work

SC1 had this mapping editor view (keygroup view), which is way better than SC2 to be honest (and also in line with pretty much any other sampler - pitch horizontally, velocity vertically).

You drop a sample in (or multiple samples), and then you can drag them by the edges to change the key and velocity boundaries. In SC1 shift+dragging a boundary added key or velocity fade. Multiselection of zones was not possible but this is obviously a must.

It also used a kludgey < - + > setup for scrolling and zooming. Let's use horizontal/vertical scrollbars and mousewheel for this instead!

You can see the virtual keyboard there. You could click on a key and it plays that note. You can see that one of the keys in the virtual keyboard there is gray instead of black or white. That's the root key for the currently selected zone. We should be able to drag and drop that gray key elsewhere in order to change the root key (neither SC1 nor SC2 could do this, annoyingly).

You can see the top info line (where < - + > buttons are) has some info about the currently selected zone. This was just text in SC1, but we should make it editable (numfields) in SC3. This is in line with samplers such as Kontakt. Super handy to have it there. It doesn't mention velocity or key fades here but it should. We have enough room.

There are no right-click context menu options here whatsoever, which is a shame. We could improve things a lot by adding useful zone mapping features in here, let's take hints from Kontakt:

Most things in that screenshot should make logical sense even if you've never used Kontakt (disregard the AET stuff, that's very Kontakt specific).

You can see that Kontakt's mapping editor has some buttons in the toolbar. Those are some very handy functions, like auto-stretching the selected zones horizontally or vertically in order to fill the "holes" in the mapping:

resolving overlapped zones horizontally or vertically:

setting the keyrange or velocity range of selected zone via MIDI (by playing two successive notes):

locking the mapping (so that you don't mess it up accidentally by a mouse drag):

and so on.

You can also see the filename of the sample loaded into the selected zone, and also exchange it by jogging prev-next across the folder in which that sample originates:

It shows just the filename, but full path to the sample shows up when you hover above the filename in the tooltip.

I guess this is enough for starters. We can add stuff as we see fit later on through separate issues.

Setup Issues for Sprint 0

Right now the issues are sitting in @baconpaul 's head. Lets try and write some here and do some basic organization.

Create Windows Installer

Create installer for windows using Inno Setup.
Mainly just copy from Stochas and modify to suit. This will be in addition to raw zip installs.

Relace vt_util/vt_string with C++-14 std wchar stuff

fPIC for filesystem for vst3 on linux

Format: SCM

A SC custom format

/Users/paul/dev/music/shortcircuit3/resources/old-patches/shortcircuit patches/SC Presets 1 fixed by Benedict Roff-Marsh/Drums/Drum Kit 2.scm

crashes us right now

Am I compiling empty files?

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libshortcircuit-core.a(load_recycle.cpp.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libshortcircuit-core.a(tinystr.cpp.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libshortcircuit-core.a(halfratefilter_ppc.cpp.o) has no symbols

Replace libs/vembertech/thread with std::thread primitives.

Loading an SC2P file doesn't reset the synth

Right now when you load a patch the synth doesn't reset to empty first. This is confusing and error prone. And just weird.

An Event Subscriber to sampler

the way the old VSt2 and sampler interacted was through the post_events_from and to_ methods which I've basically disabled and moved to unused (but they are still called as no-ops)

My experience from surge is that you really want to subscribe to events. So I propose we add an API like this inside sampler

   struct samplerChangeListener {
      void onZonesChanged() = 0;
      void onGroupsChanged() = 0;
      void onEverythingChanged() = 0;
   };
   void addSamplerChangeListener( samplerChangeListener *l );
   void removeSamplerChangeListener();

etc....

I realized this as I was about to code up my super hacky what's in my zone in the UI so wanted to drop the thought here so I didn't forget it.

Move the dump code to the unused directory

Widget: Disable State

Some widgets support disable state but these don't

• ComboBox
• Toggle Button
• Int MultiSwitch

Test by making sure they are grayed out in a new instance

FindFirstFile stub

inline HANDLE FindFirstFile(const char *, WIN32_FIND_DATA *) { return 0; }
inline HANDLE FindNextFile(HANDLE, WIN32_FIND_DATA *) { return 0; }

this code in the compat lib is only called in one place, that place is a recursive directory scan, and that scan could be rewritten to be 4 lines of portable code with fs::recursive_directory_iterator.

logfile.h/cpp

1. a stream version pls
2. be able to hand it a log target
3. namespace, etc....
4. extern char? ugh.

Consistent Copyright Headers in Source

1. Write a copyright header template which indicates the CLA status
2. Apply it uniformly

A Text Representation of the State

For debugging a full text dump of the current state would be way useful

Make at least some semblance of a downloadable asset

I think only @baconpaul can do this because of step 3

1. cmake moves the JCUE assets into products in the sc3-allproducts stage
2. We make a 'nightly' tag
3. Invite surge-rackupdater to the repo with write
4. setup the azure pipeline to do a GitHub release including a disclaimer in the release notes that is in 300pt blinking red or some such

Zones: What are they and how do they work?

Create mac installer

Use Packages to create a mac installer. Copy from Stochas and modify to suit.

CMake Cleanups (soon)

1. Python builds win pipeline
2. Better use of the sc3core library with interfaces and public compile flags and stuff
3. Some comments
4. sc3-products includes the python and juce stuff too
5. And the pipeline builds all of that

non-OS-indepdent path separators

make a JUCE LookAndFeel subclass and attach it properly

block_size -> BLOCK_SIZE etc

Surge has the convention that constants are ALL_CAPS
But we made that change around 1.6.0 beta5
so I should just use the clion refactor tool to do that here.

Format: SFZ

Reports of crashes on linux. Need some good test cases. @baconpaul has the plogue ones but those are flac inside which I don't think the SC3 code does today.

Big Issue: Configuration Setup and Paths

So there's configuration but it is very windows specific using a quasi-portable approach

I suggest we do something much closer to the surge approach for configuration and startup.

but I need to read all the code and figure out what it is doing.

Loading Files which crash on Mac and Win

• /Users/paul/dev/music/shortcircuit3/resources/old-patches/shortcircuit patches/SC Presets 1 fixed by Benedict Roff-Marsh/Drums/Drum Kit 2.scm

That drive one from @tank-trax which crashes on linux only

Do we have an .kit or .akp files?

etc

WAV file load sometimes loads more than is there

If you load a WAV file on mac or linux (a set) into the plugin, sometimes the third one plays too long

This also happens in the regtest I have written

Code not committed yet and I will commit before I fix this, but definitely some memory or file size bug in the wav load path.

lattice.cpp is over register bound

/Users/paul/dev/music/shortcircuit3/libs/vembertech/vt_dsp/lattice.cpp:168:15: warning: array index 4 is past the end of the array (which contains 4 elements) [-Warray-bounds]
        __m128d in = d.reg[4];
                     ^     ~
/Users/paul/dev/music/shortcircuit3/libs/vembertech/vt_dsp/lattice.h:14:2: note: array 'reg' declared here
        __m128d reg[4];
        ^
/Users/paul/dev/music/shortcircuit3/libs/vembertech/vt_dsp/lattice.cpp:212:2: warning: array index 4 is past the end of the array (which contains 4 elements) [-Warray-bounds]
        d.reg[4] = in;
        ^     ~
/Users/paul/dev/music/shortcircuit3/libs/vembertech/vt_dsp/lattice.h:14:2: note: array 'reg' declared here
        __m128d reg[4];
        ^

is it even used though?

Soundfont 2.0 Initiation out of bounds in note_to_pitch

Trying with Robert C. Allen's Maganda Piano (<2005 version, 9.52MB, this is older than the 25MB version currently floating around).
Win10 x64, Standalone, standard windows audio settings (no ASIO)

1. On Debug build, first note fails note range assertion in tools.cpp note_to_pitch(float x) due to some weird envelope values being passed in. Release build plays fine since it bypasses these. This happens with some but not all soundfonts.

2. With this particular soundfont, holding notes long enough for the entire one-shot sample to play, suddenly plays random notes or other corrupted sound and eventually crashing (Release mode only) with an Access violation. On Debug mode with asserts disabled, the corrupted audio just plays but no crash which is interesting, suggesting a memory bounds or envelope generator issue.

Group/Zone/Wave proposal

I'd like to offer a proposal for how we organize the hierarchy of objects. Hopefully this will start a discussion, as I know there are many good ideas that other people have. This does not have to do with how we map effects or do audio routing across these levels. This is more just for how things could hypothetically be organized and triggered. In SC1, there were groups and zones. Each zone could contain a wave (sample). In addition, each wave could be chopped so that different parts of it play depending on key.
In SC2, it seems like this was altered a bit. Anyway I think we can agree that the SC1 layout was preferable.
I'd like to propose that we have not 2 but 3 levels: group, zone and wave.

• A group could contain multiple zones
• A zone could contain multiple waves
• A group could have associated triggering conditions eventually (kind of like kontakt with key switches)
• A zone could have start key/end key/root note/ upper and lower velocity, mute groups, and also a triggering mode for waves in the zone (discussed below)
• A wave could have its own ADSR, start and end point, loop points, (possibly even fx per wave)
• A single sample in a zone could be split into multiple waves in the zone (kind of like for SC1 where you can split by key)

So triggering for waves in a zone is where it could get interesting. We could have the following modes:

• All the waves are triggered when the zone is triggered (eg you are building a drum sound with different attack and decay portions from different waves, or you are using one wave as an attack sound and another in loop mode as a sustaining sound). Could also extend to different ways to mix the sounds like eg ring mod.
• random/round robin - each wave is alternately triggered
• key mapped - waves are mapped sequentially to keys in the zone (eg you are cutting up a drum loop)

Anyway, just an idea

Remove __asm

So many warnigns

Mac, Windows, Linux
Just full of warnings
Go chase em away

Do it over time and attach to this issue

Close this issue when everyone compiles at least sc3-test and the py at werror

Who needs the hInstance?

The hInstance is used solely on windows code to look for the location of the DLL. So move it to config only code eventually.

mv libs/vembertech to src/vembertech

Delete SampleData - warning is legit

/home/vsts/work/1/s/src/sample.cpp: In member function ‘bool sample::AllocateI16(int, int)’:
/home/vsts/work/1/s/src/sample.cpp:68:34: warning: deleting ‘void* __restrict__’ is undefined [-Wdelete-incomplete]
   68 |         delete SampleData[Channel];
      |                                  ^
/home/vsts/work/1/s/src/sample.cpp: In member function ‘bool sample::AllocateF32(int, int)’:
/home/vsts/work/1/s/src/sample.cpp:85:34: warning: deleting ‘void* __restrict__’ is undefined [-Wdelete-incomplete]
   85 |         delete SampleData[Channel];
      |                                  ^
/home/vsts/work/1/s/src/sample.cpp: In member function ‘void sample::clear_data()’:
/home/vsts/work/1/s/src/sample.cpp:115:28: warning: deleting ‘void* __restrict__’ is undefined [-Wdelete-incomplete]
  115 |         delete SampleData[0];
      |                            ^
/home/vsts/work/1/s/src/sample.cpp:117:28: warning: deleting ‘void* __restrict__’ is undefined [-Wdelete-incomplete]
  117 |         delete SampleData[1];
      |

Cleanup and Test memfile.h

memfile is an in-memory riff traverser.

Review, test cleanup, probably rename.

Goal: Load a simple single wav file and play it with JUCE Python and Test on Mac Win Lin

Investigate VST3 build crash SAVIHost (win10, x64)

So far:

• Debug window loads
• SC3AudioProcessorEditor destructor gets called before the crash occurs (why?)
• JUCE asssertion error observed prior to crash on juce_Component.cpp (noted unitialized string and weird x/y coords - all large negative numbers, for passed in comp object)
• Crashes on memory address violation during juce_AffineTransform.cpp:transformPoints() likely due to an uninitialized or released component object?

TODO: determine premature destructor call.