MSUPCM++

I just realized I never posted this here, but today I happened to be poking around and fixed a long-standing bug, so along with posting a new release, I figured I'd post it here. This is a tool for converting audio files to MSU-1 .pcm files, and is a full replacement for my old batch-script utility. It is entirely config-file driven, allowing you to not only convert the file, but also apply a fairly flexible effects chain so you can convert directly from the original source file to the final output using only config file options, no external audio editors necessary (except to find your loop point sample numbers, and perhaps if your editor provides more complex or higher quality filters/effects than SoX). It also supports the full command-line functionality of SoX via the -s command-line parameter (i.e. msupcm.exe -s -x -y -z is the equivalent of sox.exe -x -y -z, with the exceptions of a few formats and features that were removed to reduce external dependencies that weren't relevant to this tool).

Anyway, without further ado:

https://github.com/qwertymodo/msupcmplusplus/releases/

-arbitrary input (should accept anything supported by SoX)
-PCM output built directly into LibSoX
-nongnu-normalize effect built directly into LibSoX (written by DarkShock)
-Unicode support

Supported effects:
trim(start, end)
loop(sample)
fade(in, out)
pad(start, end)
cross-fade(length, rate)
normalization(level)
tempo(ratio)
mix(file1, file2)
concatenate(file1, file2)
startoffset(sample) - allow the song to start after the loop point
dither(type)
compressor()

To-do:
arbitrary user-supplied effect strings

Some options are specified globally, but overridable on a per-track basis, such as normalization and output name. I also support some additional options just for display purposes, such as a track title, which is displayed as the track is being processed, or global game name and artist name parameters, which are simply displayed at the beginning of processing.

Switching to a JSON-based config file allowed me to implement recursive track combining, which will help with a handful of special cases. A single track can be made up of any number of sub-tracks or sub-channels (but not both at the top level). A sub-track can be made up of any number of sub-channels and vice versa. Sub-tracks are concatenated, sub-channels are mixed.

Some uses for these combinations:

Chrono Trigger - The Final Battle:
The battle music starts with the sound of Lavos screaming. This is part of the SPC music track, rather than a separate SFX track. So, we have an audio track that doesn't contain this effect, and we want to concatenate the original sound onto the beginning. We can do this with sub-tracks, with the scream as the first sub-track followed by the music file as the second sub-track. If we wanted the scream mixed over top of the audio instead, we could do this with sub-channels instead. Cross-fading sub-tracks won't be implemented, but you can do it manually by using sub-channels instead, then adding a fade-out to the first part, and a pad and fade-in to the second.

Chrono Trigger - Theme
The main Chrono Trigger is used in several places throughout the game played on an endless loop. However, the same theme is used in the "attract mode" intro, and it loops the middle section exactly twice and then continues to the end of the song. We can implement this using 4 sub-tracks. All 4 sub-tracks use the same input file, but with different trim points. Sub-track 1 starts at sample 0 and ends at the song's loop point. Sub-track 2 starts at the loop point, and continues to the end of the loop (which would be defined as the trim end point in the endlessly looping version). Sub-track 3 is an identical copy of sub-track 2. Sub-track 4 is the remaining section starting at the loop end point to the end of the song.

Basically, my end goal is to be able to convert every pack I've ever done directly from the original source files, without any need for pre-processing. There are a few tracks that I might still use pre-processed inputs, for instance I still don't quite know how to use the SoX compressor to achieve the same results that I've gotten from Audacity's compressor, and there are a few that I've used some more advanced effects like hand-tuned volume envelopes for mixing multiple tracks. However, I think this should be capable of nearly everything, once I get the recursive combining implemented. Documentation of some kind will be forthcoming, for the mean time, I'm working on converting my old config files to the new format so I can run my old packs through it to test the output.

Currently building in Visual Studio 2015. Should also be possible to build in Linux, but I have not touched the CMake files to get that working, so you're kind of on your own there...

Usage:

This is a command-line application intended for use with a configuration file, so it may take a bit of getting used to for people without CLI experience.  The two simplest ways to use the application are to name your config file "tracks.json" and place it in the same directory as the .exe (along with the audio files), and just double-click on the .exe.  Or, you can drag-and-drop your .json file onto the .exe, in which case it can be named anything (as long as it still has a .json file extension).

As for the config file itself, probably the best option is to look at the existing example files here: https://github.com/qwertymodo/msupcmplusplus/tree/master/configs

Notice the first line of each example contains a $schema definition.  Including that line in your config file will allow JSON-aware editors such as Visual Studio Code to provide validation and other help like auto-complete and tooltips.

I also posted a pair of tutorials, one for basic looping and another showcasing some more advanced usage of the tool

I've noticed, that when you lower the speed of files generated by MSUPCM++, there is quite a bit of noise.
I thought it may be artifacts generated by SoX, so I generated a .pcm file manually with SoX and then lowered the speed afterwards. There was no noise at all.

Does MSUPCM++ add noise to the generated files?


Examples:

File generated by MSUPCM++
File generated by MSUPCM++ at half speed
Manually generated file
Manually generated file at half speed

Dither is enabled by default, it tends to help clean up imperfect loop points, and it is suggested to use whenever changing sample rates.  You should see the same effect in SoX CLI whenever dither is enabled. The issues with noise in the tempo change effect is really just due to SoX's tempo change filter not being very good.

qwertymodo wrote:Dither is enabled by default, it tends to help clean up imperfect loop points, and it is suggested to use whenever changing sample rates.  You should see the same effect in SoX CLI whenever dither is enabled.  The issues with noise in the tempo change effect is really just due to SoX's tempo change filter not being very good.

Ah okay, thank you for the clarification.

On another note, I've noticed that "cross_fade" and "trim_start" don't get along very well.
If you generate the above mentioned file with this configuration:

Code:

{
   "track_number": 5,
   "title": "Forest Interlude (Uncompressed)",
   "file": "DKC2\\Forest Interlude.flac",
   "trim_start": 7444,
   "fade_in": 3,
   "loop": 38656,
   "trim_end": 6333089,
   "cross_fade": 5000
}

The final output clips at the looppoint: dkc2_msu1-5_Crossfade_Clip.pcm

This doesn't happen if you remove either cross_fade or trim_start.

It's not that the two don't play nice together, my best guess is that it has something to do with trying to cross_fade so close to the start of the track. Give me a minute, I can probably get you a better loop point without the cross_fade.

Ok, try loop: 246743 and trim_end: 6541187, no cross_fade. Also, that fade_in: 3 is weird, 3 samples is essentially nothing.

qwertymodo wrote:Ok, try loop: 246743 and trim_end: 6541187, no cross_fade.  Also, that fade_in: 3 is weird, 3 samples is essentially nothing.

Yes that loop works perfectly, thank you!

The "fade_in: 3" is just something I put on everything, to make sure the output doesn't clip at the start and has completely no volume on the first sample.

I also found a workaround for using cross_fade with this configuration.
If you put everything except the "trim_start" into a sub_track or sub_channel (Should be the same, if you only use one), the loop will work.
(Probalby because it creates the sub_track first and then applies the other effects afterwards)
Though everything outside the sub_track needs to be recalculated to 44,1kHz.

Code:

{
   "track_number": 5,
   "title": "Forest Interlude (Uncompressed)",
   "file": "DKC2\\Forest Interlude.flac",
   "trim_start": 6839,
   "sub_tracks": [
      {
         "loop": 38656,
         "trim_end": 6333089,
         "cross_fade": 50
      }
   ]
}

Yeah, cross_fade creates a second fade track and then combines it with the original, so it's possible that doing that combination may have caused the clipping if the original track was close to 0.0dBFS peak at some point. So, the reason it's working with the sub_track is that it's doing the normalization first, reducing the overall loudness, and then it avoids the clipping when it mixes in the fade bit. Not really anything I can do about that, you just have to be careful cross_fading tracks that are nearly saturated like that.

Okay, will do. Thank you for your tips, you have been a great help!