Plan Guide

An Aimi Sync Plan is a JSON object that details the plan that the executor will use to generate audio. The sync_generate endpoint will generate a complete plan that can be used immediately, and will return the plan to the API client for client-side editing if desired.

Sections

Aimi Sync analyzes videos to find suitable points in time to trigger changes in the generated music. These segments of music between these change points are called sections.

Each plan will contain an array of sections. Sections can have their own key, instrumentation, and voiceovers. The timing of sections can be modified if desired, but it is important to keep in mind that changing the tempo or beat length of a section will impact subsequent sections and the total length of the music generated such that it may no longer fit the video provided.

Soundsets

Aimi has curated sets of sounds that are associated with different genres and styles of music. A score can have a single, or multiple soundsets that can be utilized by specific instruments in each section. For example, the Bass in the first section of a plan may draw from a soundset in the House style, while the Melody in that same section is drawing from a soundset in the Deep House style.

Plan Generation

When using the sync_generate endpoint, a range of high-level plan parameters can be set upfront:

• vocals – include sung vocals
• voiceover – include a (generated) spoken audio voiceover
• original_audio – overlay the original audio (from your uploaded video)
• original_audio_voiceover – overlay the original audio as a "voiceover" (i.e. spoken with ducking)
• no_ducking – by default any voiceover (either voiceover or original_audio_voiceover) will include ducking. no_ducking turns this behaviour off
• multitrack – include individual stems in zipped download

Example Plan

{
          "fxstyle": "chill",
          "id": "12345678-1234-1234-1234-123456781234",
          "masterGain": 70,
          "parameterPresets": [
            {
              "id": 0,
              "trackname": "musictracks",
              "value": 0
            },
            {
              "id": 0,
              "trackname": "utilitytracks",
              "value": 1
            }
          ],
          "sections": [
            {
              "audiooverlay": "",
              "automations": [
                {
                  "parameterId": 0,
                  "points": [
                    [0, 0],
                    [8, 1],
                    [24, 1]
                  ],
                  "trackname": "musictracks",
                  "type": "bpec"
                }
              ],
              "beats": 24,
              "bpm": 96,
              "duration": 24,
              "instruments": [
                { "offset": 0, "role": "Rhythm", "soundset": "electronica_set_1", "volume": 1 },
                { "offset": 0, "role": "Bass", "soundset": "electronica_set_1", "volume": 1 }
              ],
              "key": "CM",
              "name": "Introspective Glimmer",
              "order": 1,
              "voiceactor": "echo",
              "voiceover": "In the quiet of a dimly lit room, gentle sounds invite the soul to wander and find a fragile spark amidst memories and soft reflections that stir an inner calm spirit silently guiding our thoughts.",
              "voiceoveroffset": 4,
              "voicevolume": 90
            },
            // ... more sections ...
          ],
          "soundsets": [
            { "id": "electronica_set_1", "style": "electronica" }
          ],
          "variation": 0,
          "withDucking": true,
          "withMultitrack": false,
          "withOriginalAudio": false,
          "withOriginalMusicAsVoiceover": false,
          "withVoiceover": true
        }

For more details, see the official Plan Guide documentation.