ultrastar-deluxe/play

Free and open source singing game with song editor for desktop, mobile, and smart TV

play's Issues

Recorded note does not give points when its sentence is not displayed

Actual behaviour

It is possible to set a delay for mic input, such that notes are recorded for positions earlier than the current playback position.
When using a high delay value, before the grading of the last syllables of a sentence happens, it switches already to the next sentence and apparently does not give points for the last syllables of the previous sentence.

Expected behaviour

The notes should give points, even if the notes are not for the currently displayed sentence.

Steps to reproduce

• Set a high mic delay (e.g. 500ms).
• Sing a sentence
• Notice how the last 500ms of the sentence are not counted, even though the notes are recorded for this delay.

Additional Notes

The issue has been noticed in #108

Add a changelog

Do you want to request a feature or report a bug?

Feature

What did you expect to see?

A changelog to see all changes between version.
Would be nice to write it according the 'standard' defined here https://keepachangelog.com
I'll get to this tonight

Screen division algorithm

Do you want to request a feature or report a bug?

Feature

What did you expect to see?

In theory, we could support m different Voices across n Players. In the best case, and assuming Players do not get to choose which Voice they get to sing, you basically have to make a bunch of rectangles and then cluster them in a visually pleasing way inside a bigger rectangle (ie, the screen).

For the case of making this as generic as possible yet at least somewhat doable, we will assume that the actual number of screens, p, is variable as well, but that all screens are the same width and height. Whether menus/background image/video is repeated on all screens or stretched across is outside the scope of this algorithm (could be just a setting), as is the configuration of the screens itself (3x1, 2x2, etc).

So:
Input:

• p screens, each of which is width Sx and height Sy
• m Voices (depends on song)
• n Players

Step 1: assign Players to Voices

• The first Voice will get ceil(n/m) Players assigned to it.
• All subsequent Voices will also get this number of Players assigned to it, until the number of unassigned Players reaches zero.

Step 2: distribute the Voices and their Players across the screens
To make optimal use of screen estate, spreading 6 players across 3 screens for a duet song should be possible. Therefore:

• The optimal number of screens per Voice is SPV = p/m (do not round this number)
• If this number >= 1, each Voice gets floor(SPV) screens. Screen 1 gets ceil(number of Players for this voice / SPV) Players, etc. This is important as we need to repeat the lyrics on the different screens.
• Otherwise, each screen gets ceil(1/SPV) Voices, until there are no more Voices left to assign.
  In both cases, empty screens can happen.

Step 3: compute the optimal distribution per screen
Note: this algorithm will give each Player an equally sized rectangle to put their Notes in, aka if you're with 3 players across 2 screens, the third Player will not get a bigger notes area. If running the algorithm for every screen, for every voice, for every player is okay, then go ahead and do that, otherwise you only need to compute it once for the first screen, the first of its voices, and the first of its players.

Step 3.1: decide where to place each Voice and its players
Essentially, this results in a grid. On a portrait screen, it's probably best to have 4 Voices as a column, 2x2 on square-ish monitors, or even as a row if your screen is mega wide. It's probably a function that depends on the aspect ratio of the screen and the number of Voices it needs to fit.

Just make a function that returns the sort-of heuristic optimal number of rows when given the screen aspect ratio and the number of voices. From there you'll know the number of columns required, and then it's just a simple grid and assigning stuff. This can be tweaked later on if necessary.

Step 3.2: decide where to align the lyrics
By default, lyrics will be placed at the bottom of the Voice, unless the grid has two rows, in that case lyrics for the top row will be placed at the top.
Lyrics will take up some (fixed) amount of space from the rectangle this Voice can place its stuff in.

Step 3.3: decide where the players go
This is where it becomes easy. Essentially, you have some subspace x by y pixels left for drawing notes for some number of players. Either we make this into a grid again (see above) or each note area needs to have a specific aspect ratio (can be changed later on or even made an option) which for now we'll set at 10:4.
Either it's some heuristic, or it's a maximization problem that is easy to solve. In case of the latter:

• A: Compute the optimal width of the rectangle by trying widths of x, x/2, x/3 etc. We can deduce the required height and number of required rows for this to work, and see if height times rows <= y. The maximum division we need to try is the number of Players (for this Voice, on this particular screen). Be aware that on a 4000x300 region with 4 players, this will produce no result!
• B: do exactly the same, but for height.

At the end, you have either one or two possible candidate rectangle sizes, just take the biggest. The optimal rectangle grid will always either cover the entire x or the entire y (or both if x:y is also 10:4).

Todo / Decisions that still need to be made

• Figure out how to store all this information [efficiently]
• Current lyric state/player score/notes should be "a pointer" so that we can respond to window resizes even in the middle of a song

Backgroud Video Pixel

Hello, I have a problem with the display of background videos. I have gray pixel errors / overlays. That happens with mp4 / mpg videos. Not at avi videos. I already have all codecs installed. what can this bug be?

Here is a video about the bug:
Youtube

create initial unity project with proper setup

• 2D
• basic folder structure for songs content (assets), scenes, settings, themes(assets?), game code
• select some current version of unity
• initial structuring of stuff -> architecture ?
• commit to public repo
• gitgnore
• readme

Linebreak algorithm

In #26 I noted that the specific beats at which line breaks occured are (at the time of writing) ignored by the parser. Currently, the song format files have line breaks at specific beats, but other than saving some computation work, is it really that useful? It's probably not hard to add a little algorithm that, given two sentences A and B, computes the optimal line break beat.

Here's some pseudo algorithm, I've explicitly used the notes for start/endbeat here, there's a convenience method on the Sentence object as well which will do just that (the important thing to remember is that a Sentence doesn't have any "padding"):

• If (A.lastnote.endbeat >= B.firstnote.startbeat) return A.lastnote.endbeat
  (we can't really do anything useful in this situation)
• Else:
  • compute the midway (or 2/3 or something else) point between A.lastnote.endbeat and B.firstnote.startbeat, call it C for now. I checked some random songs and most seem to have it at around 60%
  • compute some A.lastnote.endbeat + (x seconds * BPM) and call it D for now. One second is probably fine.
  • If C < D ? return C : return D

Essentially, this will cause two equidistant sentences to always have the same (relative) linebreak beat.

Calculate time in the song (e.g. in seconds) from a given beat and bpm of Ultrastar file

Issue type: Development discussion

I try to calculate note positions based on the BPM and beat in the song that is given in Ultrastar files.
My problem is that the values I calculate are not matching the speed when playing the file in USDX.

Example

I create a simple song file. It has 120 BPM and a single note with a duration of 10 beats:

#TITLE:EmptyTitle
#ARTIST:EmptyArtist
#MP3:EmptyArtist - EmptyTitle.mp3
#BPM:120
#GAP:1000
: 0 10 5 bla
- 15
E

The note of this file when viewed in Yass (or played in USDX) has a duration of 1.25 seconds.
However, with 120 beats per minute, it holds that
1 beat has a duration of 60 [s] / 120 [b] = 0.5 [s/b],
thus 10 beats have a duration of 5 [s] .
This is obviously different from the 1.25 seconds the note has in Ultrastar.

So, what is happening here? Did I make a mistake or is BPM in Ultrastar not "beats per minute"?

set license for project

• support usage of Unity3D / third party stuff
• permissive license
• support future change of license to (for example) GPL or LGPL (if desired)
• open for competition
• don't try to be the "only" / "best" / ... karaoke game, instead target to improve karaoke gaming. Choose a license that supports this

Do pitch detection on the song itself in song editor

The song editor can re-use existing pitch detection features to aid the user in the creation of song files.
The detected notes should go to a dedicated layer, such that the user can easily hide or copy the detected notes without any existing notes beeing affected.

A user can use this feature to compare the detected notes with the current notes in the song to check if the current notes are correct.

Of course, the audio file contains much more than the singers voice, so doing pitch detection on the song will result in a lot of wrong notes beeing shown. Still, I think this feature can help in creating songs.

Play midi notes in song editor and virtual piano keyboard

The song editor could play midi notes of a piano to aid the user in the creation of the song. The acoustic feedback will help in deciding whether a note is set correctly or not.

For example, clicking on a note in the editor could play the piano sound of that midi note.

Similarly, the song in the editor could be started (with or without the original music in the background) and the virtual instrument will play the notes in the song such that the user can check whether they are correct.

A "virtual keyboard" on the side of the editor (like in Yass, Cubase, LMMS, and other digital audio workstations) can be added such that the user has hearable feedback in addition to the pitch labels.

---

The project CSharpSynthForUnity seems to provide an implementation to load and play sound banks and midi files in pure C#. License is MIT.

implement storing and loading game settings

Actual behaviour

Changes to game settings are lost after user closes the game.

Expected behaviour

The game stores changes to settings. Settings survive game updates.

Steps to reproduce

1. Open the game
2. Change a setting in settings menu
3. Restart the game and check if the previous change survived.

Remove voice from songs with Spleeter

Issue type: Development discussion

Problem or question to be solved

There could be an option to remove all voice from an song to get "real" karaoke songs.

Suggested solution/s

-> https://github.com/deezer/spleeter

Your opinion on the topic

Would be a great feature ;)

Add more options (general volume, preview volume, ...)

Actual behaviour

The SoundOptionsScene are empty and also some other options are missing.

Expected behaviour

There should be options for the general volume and preview volume.
Note that at the moment there is no song preview (see #89) but the option can be prepared nonetheless.

It would also be nice to have some options for the SingScene. For example, debug options to enable / disable the grid of beats and lyrics on the notes.
Maybe the debug options do not need an in-game UI. It could be sufficient to enable them in the Settings.json file.

Feature request: Pitch indicator and scrolling notes/lyrics

Could this kind of thing be possible to implement? I know a lot of mainstream singing games that have this feature. I'd be more than happy to work on the graphical aspect (I'm fairly competent in both raster and vector programs) if someone wants to work on the coding aspect of this.

Audio latency detection

Issue type: Feature request

Actual behaviour

The game currently assumes a certain amount of milliseconds as audio output + input device latency. This fixed latency value does not work for many different hardware configurations and different player distances from the speakers (3ms of extra latency per 1 meter).

Expected behaviour

The game should play three medium loud pitch tones, count the time from playback started to pitch detected, average the three intervals and use that as pitch as expected latency.
This process should be done once for each in the game activated soundcard and use different pitch frequencies to distinguish the targeted soundcards/microphones.

new lib: Tween (for easy animations)

There are some easy to use libs for Unity that make animating GameObjects (e.g. move to XY, or fade color to XXX) a lot easier. I want to use one of these for USDP. What I found so far:

iTween: assetstore, github
It is a popular one, I think one of the first (if not THE first) Tween lib for Unity and I used it in another project before.
It is a ~8000 LoC single file. However, it uses an API with key-value maps that are not type safe, thus relatively brittle, and of course without code completion of an IDE. For this reason I don't want to use it.
Furthermore, LeanTween (see below) did a nice performance comparision and iTween seems to be quite slow compared to the rest.

LeanTween: assetstore, github
This looks very good. Simple to use and able to do a lot of stuff out of the box. I tried the examples and you can do simple stuff with understandable and type safe one-liners. More complex stuff is composed of the simple ones using sequences.
Also all relevant platforms are supported. Very good performance compared to other Tween libs (see diagram in the assetstore images).
Also nice documentation. The lib is available as source code or compiled as DLL, such that the source files do not re-compile all the time in the Unity Editor.
License is MIT, thus a perfect match for USDP.

DOTween: assetstore, github
This looks like another good choice, similar to LeanTween. However, there is a pro version, which is paid. For an open project like UltraStar Deluxe Play I think this is no go because it implies that some features are missing in the free version and future development is not in community hands.
Furthermore, LeanTween seemed to perform a bit better.

Tween: assetstore
This is a very simple script. Basically the core concepts of a tween system (i.e. interpolating values for floats, Vectors, Colors) is implemented in ~1000 lines (including comments) in a single file. License is MIT, thus a perfect match for USDP.
This is super easy and what it does, it does well.
However, it does not have all the features of LeanTween or DOTween, e.g. creating sequences of animations, delays and looping will require some manual work.
I still think it should do for USDP, at least for now.

What do you think?
In conclusion, to me LeanTween seems to be the best choice for USDP. It just looks like it gets the job done with little effort and nice out-of-the-box features. No matter if we need simple or more complex animations. These features of course come at the cost of a bigger lib than the simple Tween lib, but I think this is not a problem.
I would give LeanTween a try.

Change the video gap by dragging horizontally on the video area in the song editor

The video and music of a UltraStar song can be out-of-sync (e.g. when the music and video file come from different sources). The #VIDEOGAP tag is used to synchronize these tracks. Thus, it should be possible to edit the #VIDEOGAP in the song editor.

The editor could have a feature to fine-tune the video gap. Adjusting the video gap via drag gesture provides a precise and intuitive control. For this to work, the dragging motion has to change the displayed frame of the video immediately.

When the user hovers over the video, the cursor should change to indicate that a drag gesture is possible. There is already such an icon, which is used for the drag gesture to stretch notes.

I would suggest to implement it as (1 second = width of video area) or maybe (1 second = width of window).

What I imagine is that the user plays the song, then pauses where the video gap is clearly visible (typically this is pretty obvious when people in the video are opening their mouth to start singing). The playback position is set to this point. Then the video area is dragged until the displayed frame matches the playback position of the video.

requirements for first release version

Issue type: Development discussion

Problem or question to be solved

So far, there exists no build of UltraStar Play for people to download and use. The question now is

• when to build the next public release of the game and
• what issues have to be done before we release and
• what other tasks should be done ahead of the release?

Suggested solution/s

Initially, I planned on often releasing new non-stable versions as long as the game works mostly.

Your opinion on the topic

So far, there exists no website for this project - I'd consider having one before release to be mandatory. (there exists the usplay.net domain already, but no content)
I guess we will also need some very basic user-targeting documentation.

What are your opinions on these topics?

Preview of music and video in SongSelectScene

Actual behaviour

The SongSelectScene is showing covers and song title. However, neither the video nor the music is played at the moment.

Expected behaviour

Just like in USDX, the video and music should be started somewhere in the middle of the song when a song is selected.

Note that some songs have tags in the txt file that define "previewstart" and "previewend"

new lib: UniRx (reactive extensions)

I want to make use of some libs in this project.

UniRx
For a fluid, functional, reactive API
Asset store: https://assetstore.unity.com/packages/tools/integration/unirx-reactive-extensions-for-unity-17276
Tutorial on GitHub: https://github.com/neuecc/UniRx

This could make code much more readable. Especially when working with Unity events.

Implement user input via Unity's new input system

Problem or question to be solved

At the moment only keyboard and mouse input is implemented in UltraStar Play.

Suggested solution/s

In Unity 2020.1 the new input system for Unity will be available in a stable version.

User input should be implemented using the new input system and documented.
I prepared a wiki page for this: https://github.com/UltraStar-Deluxe/Play/wiki/Handling-User-Input.

Future Plans? Maintainer contact?

I see UltraStar Play based on Unity as a very neat idea. I tried it with current Unity 2019.02. It is working. Sadly there is not much yet.

Are there active developers or developers that want to contribute? The Gitter seems abandoned. Is there any chat or similar (e.g. Discord)?

Where do you developers live?
I am from Germany and I think you as well. So you can answer in German if you please.

online multiplayer mode

Issue type: Feature request

Actual behaviour

Currently UltraStar Play does not support playing karaoke together with friends who are not in the same room/location.

Expected behaviour

The game should support some multiplayer mode where users can use their own device or share a device to and play together with users of other devices running UltraStar Play.

Topics to consider

• how can different devices find each other? maybe some online matchmaking / party finder website that could be hosted on usplay.net
• we probably don't want to rely on commercial-only Unity online services.
• consider using a network multiplayer framework that supports various platforms that Unity supports. Maybe something like this: https://github.com/MidLevel/MLAPI
• local multiplayer might be nice eventually
• decide on what development path to take. The first iteration of the multiplayer feature should only contain what would usually be considered the minimum viable product (MVP). Once that is implemented, more developers will probably help developing the fancy features.
• people have different song file databases with different variations of song txt files - how should matchmaking work and still be fair? For legal reasons, the online multiplayer mode may not exchange / transfer / store any lyrics or song txt files or similar. Maybe a md5/sha1 hash of the song txt files minus the headers could be used as song "identifier" and to support party match making.

"coyote time" or similar player advantage to make the game feel better

Issue type: Development discussion

Problem or question to be solved

Pitch detection is still not feeling as good as it could be. This is limiting the fun you have when singing.

Suggested solution/s

One way might be to improve the pitch detection algorithm. However, improving the pitch detection using YIN did not work as I expected. The current approach is pretty fine already.

Instead I suggest to implement something similar to "coyote time", which gives players an advantage (see https://www.youtube.com/watch?v=sLXLlJ7FhJU at 5:20). The goal is to make scoring feel fair and the player feel like a boss.

One could, for example, delay the scoring by N beats. If the current N beats to-be-scored were detected as "wrong", but the previous and the following N beats were measured as "correct", then also handle the current N beats as if they were measured as "correct".

Such an approach is not costly, and could make our pitch detection feel more stable (although in reality it is not). It makes use of the additional knowledge that we have about the expected pitch.

Good players are not irritated by single false detected pitches that destroy their perfect sentence, but it also does no hand-holding (so if you do not hit a single note, then this does not help you).

implement microphone input & pitch detection

detect what pitch(es) was "sung" by the player(s) since last time it was checked (polling from some pitch detection service per microphone).

• support multiple input devices
• support multiple channels per device -> #85
• support getting the best pitches guess for certain time lengths (depending on beats per minute of the current song file)
• detect and properly handle sudden input / output devices being unplugged or added or locked
• check if unity actually detects mics being changed
• support linking mic to player by having the player make noise in front of the desired mic
• auto-detect system total audio latency by playing 3 different pitches and checking for each mic how much latency there is until reception of these tones -> #86
• audio input -> pitch -> sung note between two beats has to be framerate independent. Thus, don't just rely on the Update() method
• use Application.RequestUserAuthorization before accessing webcam or microphone

create persistent highscore store

Actual behaviour

Highscores are not tracked, and not stored.

Expected behaviour

The game stores the singers scores depending on difficulty level and thus allows for statistics, comparing with previous tries, competitions, Top 5 Ranking and so on.

Steps to reproduce

1. Open the game, sing and finish a few songs
2. Check and confirm the Top 5 Ranking
3. Restart the game and check the singing statistics

set up automatic static code analysis for commits and pull requests

• check basic code quality analysis
• check compliance with coding convention
• automatic pull request review

Guidelines for Unity3d development

Guidelines for Unity3d development needed

In current code there are following entities:

• Views (unity3d scenes)
• Models
  Static classes of managers f.e "SettingsManager", enums, non static classes f.e GameSetting
• Controllers
  c# scripts that are hooked up in scene to main canvas, they have mostly only OnGui with debug text
• ViewNavigator
  c# script hooked up in scene to main canvas (buttons call methods from this class to navigate between scenes)
• skin (unity3d skin file)
  I guess it should be use for GUI themes but I don't think it is is hooked up anywhere in game

Questions & problems:

• There are no prefabs for UI (for user to change game setting)
  Prefab probably should contain label, input field, buttons(left/decrease,right/increase)
• How should this gameObject that allow player to change setting be connected to scripts.
• Should there be script for prefab (logic of setting specific setting in script for prefab)?
• Should logic for specific view be entirely in controller (set specific setting, get specific setting)?
• We could use some guidelines for "frontend" structure, in "UltraStar Play Development README" there are some guidelines according project structure, but no guideliness for View<=>Controller in Unity3d.
  I'm no expert in this matter so I'm not sure which approach should be the right one in order to link Views,prefabs, scripts(controllers and models)

Lyrics helper: correct placement seems to be dependent on screen resolution

Issue type: Bug report

Actual behaviour

The lyrics helper seems to be correctly placed for e.g. a game resolution of 800x600:

However, it seems to be incorrectly placed for e.g. a game resolution of 2560x1600:

It is difficult to see this in the screenshots, but in the correct case, the lyrics helper is always exactly at the current syllable and never goes beyond the last syllable of a sentence, while in the incorrect case, it seems to be "stretched" and to start left of the actual lyrics and goes beyond the last syllable of the sentence (as is obvious in the second screenshot).

Expected behaviour

The lyrics helper be placed correctly independent of game resolution.

Steps to reproduce

Start UltraStar Play with different resolutions and observe the position of the lyrics helper.

Details

Provide some additional information:

• UltraStar Play version: latest trunk
• Operating System + version: macOS 10.15.2

Mutable song data structure for song editor ?

Problem or question to be solved

For the song editor, there is a need for a song data structure that can be modified and loaded and saved in an unfinished state.
The data structure will have some fields that are specific to the editor, and not for the sing scene (for example, I want to have something like a "layer" for a note that can be toggle visible/invisible).

Because an unfinished or broken song can be saved and loaded (it is just a txt file, anything is possible), errors and warnings of the song have to be collected (instead of rejecting the song file). I want to show errors / warnings of a song in the editor similar to how your IDE is showing issues with your code.

My question is whether the required fields and features should be added to the current SongMeta, Voice, Sentence and Note classes or if dedicated versions of these classes should be created for the editor.

Your opinion on the topic

I would try to re-use the existing data structures.

How does USDX calculate the player score?

Issue type: Development discussion

Problem or question to be solved

When the player sings a note at the correct pitch then he gains points.
Question is, how much points does he get for a given song, note in the song and range of the note that was sung correctly?

I noticed that USDX is giving 10000 points when everything is sung correctly. Let this be MAX_SCORE.
How is this amount distributed over the notes? My intuition would be that golden notes bring twice as much points than normal notes and freestyle notes are not contributing to the score.
Is this true?

Then this would be:
MAX_SCORE = normal_note_points + golden_notes_points
MAX_SCORE = (any_note_points * normal_notes_portion) + (2 * any_note_points * golden_notes_portion)
MAX_SCORE = any_note_points * ((normal_notes_portion) + (2 * golden_notes_portion))
any_note_points = MAX_SCORE / ((normal_notes_portion) + (2 * golden_notes_portion))

normal_notes_portion = beats_of_normal_notes / total_beats_of_song
golden_notes_portion = beats_of_golden_notes / total_beats_of_song

Gaining points when singing a portion of a note would thus be:
score_for_this_portion = (beats_of_note_sung_at_correct_pitch / total_beats_of_note) * any_note_points * (isGolden ? 2.0 : 1.0)

Score labels

Related to this question is the labels that are associated with a certain score.
E.g. score > 9000 is "Ultrastar", score > 8000 is "Superstar" (or similar).
Is there a complete table for this?

create prefabs for singing screen

required prefabs are

• empty note piece
• hit note piece
• golden note piece
• 7 spaced horizontal note lines (background for the notes)
• box that contains the name and/or avatar of the current player
• maybe a panel or border for each "voice"/player to support multiplayer on one screen?

Record notes in song editor

The song editor can re-use existing microphone and pitch detection features to record notes.
The recorded notes should go to a dedicated layer, such that the user can easily hide, modify, accept, or remove the recorded notes without any existing notes beeing affected.

This way, a user can use the recording feature in the editor for two things:
(a) record new notes to add to the song
(b) record notes to compare the recording with the current notes in the song to check if the current notes are correct.

Switch to .NET 4.x

I would propose targeting .NET 4.x for this project to be able to use the Task Parallel Library for easier threading implementation the usage of newer C# language features.

I have already done this in my local clone. I could submit a pull request.

Was there any specific reason for staying with 3.5 or was this just the default setting?

Mic visualization in recording options should consider amplification and noise suppression

Actual behaviour

The mic visualization in the recording options scene does not consider the amplification and noise suppression.

Expected behaviour

It would be nice, if these settings are actually reflected in the visualization.

define basic coding conventions

• define basic coding conventions
• document them
• proof read by other person

Integration of music streaming service (e.g. Spotify) to make MP3 file unnecessary

Motivation

I noticed that one of the larger USDX song hosters "ultrastar-base.com" has stopped providing downloads. I assume this is because of copyright issues with music and videos.

This legal issue when distributing USDX songs has always been present. Distributing the txt files alone should not be a problem but without the music, it is not a satisfying game experience. Similar legal issues arise when USDX players download the music file for a txt file from other sources, such as Youtube.

Beeing able to create a txt file and play every song you want is a main argument for UltraStar. However, legal issues because of the music is a major factor that prevents UltraStar from growing its user base as well as song creators / providers.

How to overcome this obstacle?
Paid streaming services such as Spotify, Deezer, Apply Music, Google Music etc. could be the solution. They are already wide spread and provide pretty much every song that has been made for UltraStar in a legal way.

The idea

It would be super awesome if instead of a MP3 file, a music streaming service could be used to play the music.

What is needed is a way to

• start / pause / stop a track
• query the position in the song with a good resolution and little latency

or

• download the song somehow and play the song data using Unity API

I don't know if it is possible with current API of any streaming service but a quick search on the internet got me to Spotify web API to query the current playback position in song.

There is also Spotify API to get a track. It seems it is also possible to get an album cover this way.

This could be a real game changer ;)

Feature: Song editor, navigation and edition using the numpad

Hello!

I'm used to editing songs the way YASS streamlined it, using the numeric keypad instead of holding key modifiers. This allows for fast, one-handed editing when working on content. Would it be possible to add those bindings to USP's song editor?

The numpad is used as some sort of extension of the usual arrow keys:

• 4 and 6 move to the previous/next note on the current page
• 8 and 2 acts as page up (next), page down (previous)
• 5 plays the current selected note
• 1 and 3 moves the note left and right (by one beat, length unchanged)
• 7 and 9 shortens/lengthens the note (by one beat)
• Minus sign moves a note up a half-tone (due to the key's physical location, this makes sense)
• Plus sign moves a note down a half-tone

Here's a screenshot of YASS' bindings:

Thanks !

Code style

Issue type: Development discussion

In a collaborative project, it makes sense to decide on a code style, such that code from different people is more readable as it uses the same conventions.
I saw the existing conventions of the project. However, in some points I don't concur and in others they are a little vague.

Unity C# guidelines differ a little from other C# styles, because Unity supports others languages as well and mixed styles (e.g. from JS). I found some blog posts with C# code guidelines with Unity in mind:
(a) https://github.com/raywenderlich/c-sharp-style-guide
(b) http://wiki.unity3d.com/index.php?title=Csharp_Coding_Guidelines&oldid=17000
They agree in many points.

(b) suggest to not prefix private members, enums, or classes. (a) also suggest to not prefix private members but is less strict with enums, and other classes. Both suggest to prefix interfaces with I.

Personally, I would like to use (a) (namely raywenderlich) as the basis for our new code style guidelines.
However, I think prefixing enums with E, and maybe abstract classes with A conveys information and is a good thing to do, similar to prefixing interfaces with I.
Furthermore, I would also want to prefix or suffix scenes, e.g., "SingScene", or "OptionScene" with corresponding scripts such as "SingSceneController". In this example, "Controller" would be the suffix for scripts related to a specific scene.

Prefixes for private members and static variables are discuraged in (a) and (b). Instead, they suggest camelCase for members (static or not) and PascalCase for constants, properties, methods, classes, namespaces, etc.
Thus, the only difference to our current style guide is to not use m_ and s_ prefixes for members.

Finally, our current style guide suggests "only strictly typed variables". Personally, I very much like type inference. IMO, reducing redundant type information makes code more readable. Note also that type inference is still strictly typed code. Note also that I am speaking of C#. Stuff like UnityScript should not be used for this project.
In C#, I often use the var keyword inside methods and when iterating over collections. From my experience, the type in most cases is trivial to infere from the right side (e.g. var text = "hi"; or var boxes = new List<Box>(); ), or the type does not matter anyway. Thus, I suggest to remove this rule and allow the var keyword of C# inside methods.

implement synced audio and video playback

%todo%

create UI concept targeting first minimal release version

The goal of the first minimal release version of this game is to get some demo game working, so people get to know about this new project, come up with various use cases and piece by piece contribute to this project.

To be able to reach that goal, someone should come up with a rough UI design / layout for the game. This initial design most likely will be changed regularly later on, but we need some design to start with. That design should not be changed until the first minimal release version (Alpha version of this game) is released, so all devs and artists and testers can help efficiently and do stuff, instead of discussing endlessly 😅

set up automatic builds

• automatic build for each commit & pr
• github integration
• require successful build before merging
• build logs publicly available
• use same code / script for local builds where possible
• upgrade travis to be used from a webhook or as a Github App
• upload windows travis build artifacts upon commit to master.

Support multiple channels in recording device (e.g. left and right microphone) using PortAudio

Actual behaviour

At the moment, the recording options do not allow to select which channel of a device should be used for recording.

Expected behaviour

The recroding options should allow to select a specific channel of a recording device.

Implementation notes

I think AudioClip.channels should contain the number of channels for the recording device. The AudioClip is created using the Microphone API of Unity.

In the MicrophonePitchTracker the call to micAudioClip.GetData(MicData, currentSamplePosition) fills a float array with the samples of the recording. I expect that the samples of all channels are in this float array alternating. For example, when there are three channels, I would expect the float array to contain the data for the channels 0, 1, and 2 as follows: [0,1,2,0,1,2,0,1,2,...]

new lib: FullSerializer (json (de)serialization for persistence)

Persist game settings

I thought about how to persist game settings.
I argue that we do not need a database because we don't have multiple users that concurrently access and modify the same data. So there is no need for transactions.
Further, the amount of data will be rather small. All of it can easily be loaded in few classes into the RAM at program startup. So there is no need for lazy loading via Proxy or anything like that.
If needed, queries can be written in C# using LINQ.

Thus, my approach for persisting game settings (maybe also player scores, etc.) would be to keep it simple and just serialize an object that holds all the data to a file. At program start, the file can be loaded from file via a SettingsManager.

I find the current implementation for settings in the master using a dictionary and enums a bit weird. Why not skip the enums and the dictionary and put the values to be stored directly in some classes?

Example:

public class Settings
{
    public GameSettings GameSettings { get; set; } = new GameSettings();
    public GraphicSettings GraphicSettings { get; set; } = new GraphicSettings();
    public List<PlayerProfile> PlayerProfiles { get; set; } = new List<PlayerProfile>();
}

public class GameSettings
{
    public SystemLanguage language = SystemLanguage.English;
    public List<string> songDirs = new List<string>() { "./Songs" };
}

public class GraphicSettings
{
    public Resolution resolution = GetResolution();
    public FullScreenMode fullScreenMode = FullScreenMode.Windowed;
}

[Serializable]
public class PlayerProfile
{
    public string Name { get; set; } = "New Player";
    public Difficulty Difficulty { get; private set; } = Difficulty.Easy;
    public Color Color { get; set; } = new Color(0, 0.8f, 0, 1);
}

As you see, Settings is the root of the settings hierarchy with more specialized classes that hold various fields.
A SettingsManager can load the Settings when they are needed. They can be saved at scene change or OnDisable() or something like that.

---

Serialize objects to (human readable) strings

I took a look at different ways to serialize such a Settings class and save it to a file.
The requirements:

• serialize basic types including int, string, float, double
• escape strings if needed (e.g. backslashes in paths)
• serialize List and Dictionary
• serialize nested objects (deep serialization)

System.Xml.Serialization

In C# there is System.Xml.Serialization.
This xml Serialization does not serialize Dictionaries. Also, you need to annotate some fields (e.g. for nested object serialization).

Although something like SharpSerializer might work for us, I would rather focus on JSON for now because XML is more verbose and JSON should be perfectly fine for serializing the Settings.

Here and here is a comparison of some JSON serializers for C#.

Unity's JsonUtility

Unity comes with JsonUtility, which can serialize objects to JSON.
Unity's JsonUtility does not serialize Dictionaries and does not serialized nested objects.

LitJson

LitJson is a fast and pretty simple lib for Json serialization.
Sadly, LitJson does not support serialization of floats. Thus, serialization of Vector3, Color and everything else that uses floats does not work either.
It should be relatively simple to implement this missing feature in LitJson.

Json.Net

Json.Net has issues with Unity.
It appears to me as a relatively complex lib (compared with LitJson for instance). So I don't think we want to invest into this.

FullSerializer

FullSerializer meets all requirements and was specifically written to work with Unity.
In the performance comparison, FullSerializer was the slowest. However, I argue that the performace is still sufficient for us. In the test it took 800 milliseconds for 10000 iterations, so a hand full of iterations should finish in few milliseconds.

FullSerializer does not come with a pretty print mode. However, this can be worked around with some lines from StackOverflow.

FullSerializer is MIT license.

---

Conclusion

I don't see the need for a database.
Instead, I suggest a simple POCO structure for the game settings (maybe also for player scores etc.).
And I would persist this POCO structure as a JSON string using FullSerializer.

Change Player Settings to ".NET Standard 2.0" ?

Problem or question to be solved

I noticed that the Player Settings are set to ".NET 4.x". I switched to ".NET Standard 2.0" without any issues. The current code base seems to work perfectly fine.

UltraStar Play is targeting many different factors. Using .NET 2.0 might be advantageous.

Should we change the setting to .NET 2.0 or leave it at .NET 4.x ?

Assets Folder Structure

Issue type: Development discussion

Problem or question to be solved

The current folder structure is mixing different approaches:
by scenes
by asset-type (e.g. material, sounds, scripts... )
by architecture level (e.g. model, persistence, ui, ...)
by feature (e.g. microphone recording, showing highscores)
The current structure, is cluttered IMO.

Suggested solution/

Structure stuff by scenes where possible. I think this naturally connects related assets (e.g. the scene itself, scripts of the scene, prefabs of the scene).

However, this only works for stuff that is related to a single scene. For everything else (e.g. persistence layer, libraries, general graphics for the UI, general sound effects) I would create a "Common" folder, which holds the stuff that is used in more than a single scene.

Of course, special folders (e.g. "Editor") are an exception to the structure.

Furthermore, I suggest to use PascalCase to name folders. This fits with the C# guidelines and other folders inside Unity. Thus, it would be "SongSelection". It would not be "songSelection" or "song-selection" or "Song Selection".

Example of what I have in mind:

                            - Assets
				- Editor
				- Common	
					- Persistence
					- Sounds
					- Graphics
						- Icons
					- Recording
						- PitchTracker
						- MidiUtils.cs
                                                - MicrophoneListener.cs
				- Scenes
					- Settings
						- SSettings.unity
						- SSettingsController.cs
					- SongSelection
						- SSongSelection.unity
						- SSongSelectionController.cs
						- SongButton.cs
						- SongButton.prefab
					- SongEditor
						- SSongEditor.unity
                                                - SSongEditorController.cs
                                                - DeleteSelectedNoteButton.cs
                                                - VerySongEditorSpecificIcon.png

Your opinion on the topic

Structuring information in folders is tricky. I am sure there are various threads and opinions.
For an example that discusses folder structure in Unity, look here

HashCodeBuilder and EqualsBuilder ?

Problem or question to be solved

Defining equality and hash functions for data classes is tedious and repetitive and a good hash code is not trivial to implement.
Codacy shows an issue when not defining equals and hash code on structs because it can be slow (due to reflection, see here).

Suggested solution/s

From Java (more specifically Apache Commons) I know HashCodeBuilder and EqualsBuilder.
I did not find such a consistent library for C#.
However, I think it should be possible to implement the same logic in C#. Due to their wide usage in the Java world, I assume that these implementations are solid and efficient enough for UltraStar Play as well.

Maybe someone finds the time to port the HashCodeBuilder and EqualsBuilder (without the reflection stuff) to C#?

new libs: dependency injection (Zenject, UniInject, ...)

I want to make use of some libs for the UltraStar Play (let's call it USDP ;) ) codebase.
This is a request for comments. Anyone agrees / disagrees?

Zenject
For dependency injection (aka Inversion of Control)
Asset store: https://assetstore.unity.com/packages/tools/integration/zenject-dependency-injection-ioc-17758
Tutorial on GitHub: https://github.com/modesttree/Zenject

This could make wiring up objects more easy. It could also be used to register the manager instances in a central way and inject them where needed.
I want to try it and see if my expectations are met.

implement audio playback

• playing an audio should be as easy as one line of code.
• maybe something like Audio.play(Theme.getResource(ETheme.BackgroundMusic), EAudioBehaviour.StopAnyOther, ELoundness.Max)
• This will be used to play button click sounds, background music, notification sounds, and other background audio.
• method for stopping audio playback (not sure if pause/play capability is required)

Add github templates

Actual behaviour

Currently a template exists for creating an issue.
However there's no template for pull requests, bugs, features, release

Expected behaviour

Add specifc templates:

.github\
---- ISSUE_TEMPLATE\
-------- Bug_report.md
-------- Feature_request.md
---- PULL_REQUEST_TEMPLATE\
-------- mergeback.md
-------- release.md
---- PULL_REQUEST_TEMPLATE.md
---- ISSUE_TEMPLATE.md

Steps to reproduce

Create a new issue, you won't get to choose between feature or bug report.

Details

N/A