E-mu disk images: share the audio of a sample between the zones which play it

documentation/CHANGELOG.md

@@ -45,8 +45,12 @@
   * Fixed: The times of the 8-stage envelopes are now converted with the increment table of the sampler instead of an assumed law. The sampler advances an envelope by an amount which it looks up with the rate of the stage, 500 times a second, and those increments span 3 to 32767 - the times are therefore not a curve which halves every 8 rates as was assumed. The error is largest in the middle of the range, where the settings of real voices sit: a rate of 16 is 0.47 seconds and not 15, a rate of 32 is 0.14 seconds and not 3.75, and only at the very top of the range do the two agree. Measured over the FZ-1 factory library (12 disks, 153 programs, 674 envelopes) the amplitude decays drop from a maximum of 2.8 to 0.84 seconds and the releases from 6.3 to 0.85 - the brass ensemble of 'BRASS ENS 1' decays in 0.11 seconds and releases in 0.08, where it was 0.84 and 0.16 before.
 * CWITEC TX16Wx
   * New: Added support for bank output (*.txbank). Select 'Preset Library' as destination output.
+* E-mu Emulator III/IIIX/ESI
+  * Fixed: Reading a CD-ROM or hard disk image kept its own copy of the audio of a sample for every zone which plays it. The presets of a bank map the same samples over and over - the first volume of the 'Emulator Standards' library plays 1480 samples from 25580 zones - so an image was expanded to many times its size while it was read: that 304 MB volume held 3.5 GB of audio instead of the 192 MB it contains, and the conversion ended with 'Java heap space' and no presets at all on a machine whose Java heap is smaller than that. The audio of a sample is now decoded once and shared by all zones which play it, whether they belong to the same preset or not, which is what the samplers themselves do. Reading that volume now holds 192 MB and finishes in a 512 MB heap.
 * Fairlight CMI
   * Fixed: Reading a Voice file whose version is not recognized wrote a copy of that file into a hard-coded folder of the developer's machine which did crash the conversion.
+* ISO File
+  * Fixed: An image of the E-mu disk file system was read twice, once for each of the two E-mu formats which share that file system (Emulator III and EOS) since neither of them can read the banks of the other. Reading the files of an image reads all of their content, so a full library CD-ROM was loaded a second time only to find that it holds no bank of the second format. The image is now read once and both formats are offered its files.
 * Roland MV-8000
   * Fixed: A written patch loaded on the device but was silent, and the device showed a start point, loop start and end point of 0 for every sample. The hardware plays - and displays - the three points which are stored in the SMT slot of a partial, not the ones of the sample parameter block, and only the latter were written. The slot points are written now; the 103 factory patches carry identical values in both places in all of their 1918 slots. Reading prefers the slot points as well, so a patch whose points were edited on the device converts with the points it actually plays.
   * Fixed: The wave data of a written sample now keeps 2 frames behind its end point, as the device does for every sample it writes itself - none of the 1918 samples of the factory patches ends closer to the end of its data. A sample whose end point sat at the last frame of its data (a one-shot which plays to its very end, e.g. every sample of an Akai S1000 program whose end marker is its last frame) is padded with 2 frames of silence.

src/main/java/de/mossgrabers/convertwithmoss/format/emu/emulator3/Emulator3Detector.java

@@ -10,6 +10,7 @@
 import java.nio.file.Files;
 import java.util.ArrayList;
 import java.util.Collections;
+import java.util.EnumMap;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.Iterator;
@@ -70,6 +71,18 @@ public class Emulator3Detector extends AbstractDetector<MetadataSettingsUI>
     private static final int    MAXIMUM_REPORTED_INDICES  = 10;
 
 
+    /** Which channels of a sample the audio data of a zone holds. */
+    private enum ChannelSelection
+    {
+        /** Both channels of a stereo sample. */
+        STEREO,
+        /** The left channel of a stereo sample or the only channel of a mono sample. */
+        LEFT,
+        /** The right channel of a stereo sample. */
+        RIGHT
+    }
+
+
     /** Holds the parsed information of one sample of a bank. */
     private static class Sample
     {
@@ -86,6 +99,14 @@ private static class Sample
         boolean loopInRelease;
         int     loopStart;
         int     loopEnd;
+
+        /**
+         * The audio data of the channel selections which the zones ask for, created on demand. The
+         * presets of a bank reference the same samples over and over - the library banks map a few
+         * dozen samples from hundreds of zones - so the data of a sample is decoded once and shared
+         * by all zones instead of being copied for each of them.
+         */
+        final Map<ChannelSelection, InMemorySampleData> audioData = new EnumMap<> (ChannelSelection.class);
     }
 
 
@@ -178,10 +199,26 @@ private List<IMultisampleSource> parseImage (final File sourceFile) throws IOExc
      * @throws IOException Could not read the image
      */
     public List<IMultisampleSource> readImageBanks (final File sourceFile) throws IOException
+    {
+        return this.readImageBanks (sourceFile, Emu3DiskImage.readFiles (sourceFile));
+    }
+
+
+    /**
+     * Read all Emulator III banks from the already read files of an E-mu disk image. Reading the
+     * files of an image means reading all of their content, so the generic ISO/IMG format reads
+     * them once and offers them to both of the E-mu formats instead of letting each of them read
+     * the whole image on its own.
+     *
+     * @param sourceFile The image file, which the sources refer to as their source
+     * @param imageFiles The files of the image
+     * @return The multi-sample sources, empty if the image holds no EIII bank
+     */
+    public List<IMultisampleSource> readImageBanks (final File sourceFile, final List<Emu3DiskImage.ImageFile> imageFiles)
     {
         final List<IMultisampleSource> results = new ArrayList<> ();
         this.numberOfReadBanks = 0;
-        for (final Emu3DiskImage.ImageFile imageFile: Emu3DiskImage.readFiles (sourceFile))
+        for (final Emu3DiskImage.ImageFile imageFile: imageFiles)
         {
             // Skip the files which are not EIII banks, e.g. the banks of the newer EOS samplers,
             // which share the filesystem but not the format
@@ -684,7 +721,7 @@ private void parseLayers (final byte [] data, final Emulator3BankFormat bankForm
                 final int zone = zoneOffset + zoneIndex * Emulator3Constants.ZONE_SIZE;
                 if (zone + Emulator3Constants.ZONE_SIZE > data.length)
                     continue;
-                final ISampleZone sampleZone = this.parseZone (data, bankFormat, zone, presetOffset, keyLow + Emulator3Constants.KEY_OFFSET, keyHigh + Emulator3Constants.KEY_OFFSET, samplesByIndex, indexRepairs, missingSampleIndices, bankName);
+                final ISampleZone sampleZone = parseZone (data, bankFormat, zone, presetOffset, keyLow + Emulator3Constants.KEY_OFFSET, keyHigh + Emulator3Constants.KEY_OFFSET, samplesByIndex, indexRepairs, missingSampleIndices);
                 if (sampleZone != null)
                 {
                     group.addSampleZone (sampleZone);
@@ -761,10 +798,9 @@ private static void applyVelocityRange (final byte [] data, final int presetOffs
      * @param samplesByIndex The samples of the bank by their 1-based index
      * @param indexRepairs The repaired zone sample indices of the preset
      * @param missingSampleIndices Where to collect the indices of referenced but absent samples
-     * @param bankName The name of the bank
      * @return The zone or null if its sample is not in the bank
      */
-    private ISampleZone parseZone (final byte [] data, final Emulator3BankFormat bankFormat, final int offset, final int presetOffset, final int keyLow, final int keyHigh, final Map<Integer, Sample> samplesByIndex, final Map<Integer, Integer> indexRepairs, final Set<Integer> missingSampleIndices, final String bankName)
+    private static ISampleZone parseZone (final byte [] data, final Emulator3BankFormat bankFormat, final int offset, final int presetOffset, final int keyLow, final int keyHigh, final Map<Integer, Sample> samplesByIndex, final Map<Integer, Integer> indexRepairs, final Set<Integer> missingSampleIndices)
     {
         final int storedIndex = Emulator3Constants.getU16 (data, offset + Emulator3Constants.ZONE_SAMPLE_INDEX) & bankFormat.getZoneSampleIndexMask ();
         if (storedIndex == 0)
@@ -782,7 +818,8 @@ private ISampleZone parseZone (final byte [] data, final Emulator3BankFormat ban
         final boolean disableRight = (flags & Emulator3Constants.ZONE_FLAG_DISABLE_RIGHT) > 0;
 
         final ISampleZone zone = new DefaultSampleZone (sample.name, keyLow, keyHigh);
-        zone.setSampleData (createSampleData (sample, disableLeft, disableRight, bankName, this.notifier));
+        final ChannelSelection channels = getChannelSelection (sample, disableLeft, disableRight);
+        zone.setSampleData (sample.audioData.computeIfAbsent (channels, selection -> createSampleData (sample, selection)));
         zone.setKeyRoot ((data[offset + Emulator3Constants.ZONE_ORIGINAL_KEY] & 0xFF) + Emulator3Constants.KEY_OFFSET);
         zone.setStart (0);
         zone.setStop (sample.numFrames);
@@ -926,23 +963,39 @@ private static IEnvelope parseEnvelope (final byte [] data, final int offset)
     }
 
 
+    /**
+     * Get the channels of a sample which a zone plays.
+     *
+     * @param sample The sample which the zone references
+     * @param disableLeft True if the zone mutes the left channel
+     * @param disableRight True if the zone mutes the right channel
+     * @return The channel selection
+     */
+    private static ChannelSelection getChannelSelection (final Sample sample, final boolean disableLeft, final boolean disableRight)
+    {
+        if (!sample.isStereo)
+            return ChannelSelection.LEFT;
+        if (!disableLeft && !disableRight)
+            return ChannelSelection.STEREO;
+        // A muted left side leaves the right channel
+        return disableLeft ? ChannelSelection.RIGHT : ChannelSelection.LEFT;
+    }
+
+
     /**
      * Create the audio data of a sample. The two channels of a stereo sample are stored one after
      * the other in the bank (in either order) and are interleaved here. A zone which mutes one of
      * the two sides only gets the other one.
      *
      * @param sample The sample
-     * @param disableLeft True if the zone mutes the left channel
-     * @param disableRight True if the zone mutes the right channel
-     * @param bankName The name of the bank
-     * @param notifier Where to report a truncated sample
+     * @param channels The channels of the sample which the zone plays
      * @return The audio data
      */
-    private static InMemorySampleData createSampleData (final Sample sample, final boolean disableLeft, final boolean disableRight, final String bankName, final INotifier notifier)
+    private static InMemorySampleData createSampleData (final Sample sample, final ChannelSelection channels)
     {
         final int numFrames = sample.numFrames;
         final byte [] bankData = sample.bankData;
-        final boolean useBothChannels = sample.isStereo && !disableLeft && !disableRight;
+        final boolean useBothChannels = channels == ChannelSelection.STEREO;
         final int numChannels = useBothChannels ? 2 : 1;
         final byte [] pcm = new byte [numFrames * 2 * numChannels];
 
@@ -959,13 +1012,10 @@ private static InMemorySampleData createSampleData (final Sample sample, final b
         }
         else
         {
-            // A muted left side leaves the right channel
-            final int channelOffset = sample.isStereo && disableLeft ? sample.rightDataOffset : sample.dataOffset;
+            final int channelOffset = channels == ChannelSelection.RIGHT ? sample.rightDataOffset : sample.dataOffset;
             System.arraycopy (bankData, channelOffset, pcm, 0, numFrames * 2);
         }
 
-        if (notifier != null && bankName != null && numFrames <= 0)
-            notifier.logError (IDS_EIII_MALFORMED_SAMPLE, sample.name, bankName);
         return new InMemorySampleData (new DefaultAudioMetadata (numChannels, sample.sampleRate, 16, numFrames), pcm);
     }
 

src/main/java/de/mossgrabers/convertwithmoss/format/emu/emulator4/Emulator4Detector.java

@@ -146,10 +146,26 @@ private List<IMultisampleSource> parseImage (final File sourceFile) throws IOExc
      * @throws IOException Could not read the image
      */
     public List<IMultisampleSource> readImageBanks (final File sourceFile) throws IOException
+    {
+        return this.readImageBanks (sourceFile, Emu3DiskImage.readFiles (sourceFile));
+    }
+
+
+    /**
+     * Read all Emulator IV banks from the already read files of an EOS disk image. Reading the
+     * files of an image means reading all of their content, so the generic ISO/IMG format reads
+     * them once and offers them to both of the E-mu formats instead of letting each of them read
+     * the whole image on its own.
+     *
+     * @param sourceFile The image file, which the sources refer to as their source
+     * @param imageFiles The files of the image
+     * @return The multi-sample sources, empty if the image holds no Emulator IV bank
+     */
+    public List<IMultisampleSource> readImageBanks (final File sourceFile, final List<Emu3DiskImage.ImageFile> imageFiles)
     {
         final List<IMultisampleSource> results = new ArrayList<> ();
         this.numberOfReadBanks = 0;
-        for (final Emu3DiskImage.ImageFile imageFile: Emu3DiskImage.readFiles (sourceFile))
+        for (final Emu3DiskImage.ImageFile imageFile: imageFiles)
         {
             // Skip files which are not Emulator IV banks, e.g. the banks of the older EIII
             // samplers which use the same filesystem

src/main/java/de/mossgrabers/convertwithmoss/format/iso/IsoDetector.java

@@ -15,6 +15,7 @@
 import de.mossgrabers.convertwithmoss.core.settings.MetadataSettingsUI;
 import de.mossgrabers.convertwithmoss.format.akai.mpc2000.AkaiMPC2000Detector;
 import de.mossgrabers.convertwithmoss.format.emu.emulator3.Emulator3Detector;
+import de.mossgrabers.convertwithmoss.format.emu.emulator4.Emu3DiskImage;
 import de.mossgrabers.convertwithmoss.format.emu.emulator4.Emulator4Detector;
 import de.mossgrabers.convertwithmoss.format.ensoniq.epsasr.EnsoniqEpsAsrDetector;
 import de.mossgrabers.convertwithmoss.format.roland.s5xx.S5xxDetector;
@@ -115,13 +116,18 @@ private List<IMultisampleSource> processEmuDisk (final File sourceFile)
     {
         try
         {
+            // Reading the files of an image reads all of their content, which is the whole image
+            // for a full library CD-ROM. Both formats are offered the same files, otherwise the
+            // format which is tried second reads the image a second time
+            final List<Emu3DiskImage.ImageFile> imageFiles = Emu3DiskImage.readFiles (sourceFile);
+
             this.emulator4Detector.setSourceFolder (this.sourceFolder);
             this.emulator4Detector.setSettings (this.settingsConfiguration);
-            final List<IMultisampleSource> results = new ArrayList<> (this.emulator4Detector.readImageBanks (sourceFile));
+            final List<IMultisampleSource> results = new ArrayList<> (this.emulator4Detector.readImageBanks (sourceFile, imageFiles));
 
             this.emulator3Detector.setSourceFolder (this.sourceFolder);
             this.emulator3Detector.setSettings (this.settingsConfiguration);
-            results.addAll (this.emulator3Detector.readImageBanks (sourceFile));
+            results.addAll (this.emulator3Detector.readImageBanks (sourceFile, imageFiles));
 
             if (results.isEmpty ())
                 this.notifier.logError ("IDS_ISO_NO_EMU_BANKS", sourceFile.getName ());