Skip to content
Get in touch

Xionia Intelligence. AI that understands how sound is made.

Xionia Intelligence is our research into specialist models trained on Xionia’s own synthesis technology. These models understand how an instrument creates its sound, run on the musician’s own computer and help shape the instrument itself, rather than generating finished audio.

Explore the researchGet in touch

Research in progressA research programme, not a product announcement.

The principle

The intelligence changes the parameters. The instrument’s own engine still makes every sound.

We use machine learning to help musicians discover and control what their instruments can do.

A model proposes changes to parameters, Motion Waves and the relationships between instruments, and the instrument’s engine still produces every sound, with the reliability and editability of everything else in it. Differentiable DSP lets an optimisation see how a change to each parameter changes the sound; where part of an instrument resists that, with switches, discontinuities or feedback, learned approximations and conventional optimisation take over.

It’s the technical side of an idea at the heart of our music research, which sees AI as the instrument maker, not the musician.

“Make this pad darker, but preserve the attack and the sense of movement.”

Asked for that, a system could work like this, step by step. In a star, it works on one instrument. In StarShaper, it can work on several interacting instruments at once.

  1. Analyse

    The sound as it is, and the behaviour of its Motion Waves.

  2. Identify

    The parameters most likely to shape its brightness.

  3. Search

    For adjustments that work together across them.

  4. Compare

    The candidates, rendered and heard side by side.

  5. Propose

    An editable result, for the musician to keep, refine or reject.

Research areas

What we’re researching.

Each area starts from what a musician is trying to do with an instrument, not from what a model can generate.

Sound to instrument

Can a model listen to a sound and rebuild it as a patch you can edit?

Our first experiment is a model that runs on your own computer, listens to a reference sound and proposes a Xionia patch that recreates it. Many different patches can sound almost identical, so we judge the results by how they sound and behave rather than by whether they recover the original settings, and offer several candidates rather than one answer. The same research reaches into spectral resynthesis, separating a recording’s harmonics, noise and transients more cleanly and matching FM and additive settings to it.

Describing sound

Can you find a sound by describing its character?

Learned representations of how sounds relate to one another, combined with search, so that asking for something darker, glassier or more restless leads somewhere useful.

Motion that fits

Which Motion Waves would bring this sound to life?

Analysing how sensitive a sound is to each of its parameters, then ranking the Motion Wave mappings most likely to matter. Models can suggest movement, but Motion Waves never need them to run.

Complementary layers

What would sit well beside this sound?

Timbre analysis combined with constrained optimisation, to suggest layers from other engines that complement a sound rather than crowd it.

Whole instruments

Can intelligence shape the relationships between instruments?

Picture a creation with an analog bass, FM harmonics, a granular atmosphere and a rhythmic percussion system, and a request to make it more aggressive under pressure, more rhythmically complex and still harmonically related. We’re researching how to propose new Motion Wave relationships, pressure mappings across instruments, rhythmic variations that follow the music and changes that keep its character, every one open to inspection. It’s a much harder problem than matching one sound, and a long-term one.

Neural sound

When should a neural model make the sound itself?

Selectively. Lightweight neural models could become optional synthesis or processing components, with predictable processing costs, declared latency and the same timing and stability standards as the rest of Xionia Core, while the ordinary synthesis path stays.

Approach

How we build it.

Specialist models, combined with differentiable DSP, conventional optimisation and deterministic processing, rather than everything built around generative AI. Whatever a model proposes arrives as parameters, mappings and relationships the musician can inspect, adjust or reject, and the instruments stay complete and playable without it.

  • Trained on our own instruments.

    We control the synthesis engines, so we can render thousands of varied configurations with every parameter, note and Motion Wave known exactly, and teach models how sound relates to the structure that made it, with a shared model that understands sound and smaller models for each engine’s parameters.

  • Small, local and optional.

    Models are trained on our own machines, then designed to run in compact form on the musician’s computer, with no powerful graphics card, no hours of training and no cloud connection required. Analysis and suggestions run away from the audio thread, and any model used during performance has to meet the same timing and stability standards as the rest of the instrument.

  • AI only where it earns its place.

    Music theory and deterministic algorithms already give excellent chord mapping and voice leading. We combine trained models with differentiable DSP, conventional optimisation and deterministic processing, and test whether machine learning actually improves a result before relying on it.

Where it’s going

From architecture to instruments.

The research moves in three stages, each built on the one before.

  1. An architecture ready for intelligence.

    Stable identities and meaningful ranges for every parameter, modulation metadata, instrument graphs a model can read and reliable offline rendering, with deterministic analysis and optimisation as the baseline any model has to beat.

  2. Specialist models.

    Beginning with matching a sound to an editable patch, then exploring timbre and recommending Motion Waves, each tested against traditional DSP and optimisation.

  3. Intelligence across instruments.

    Relationships between instruments, coordinated musical behaviour and high-level creative requests, across whole StarShaper creations.

Xionia Core creates the sound. Motion Waves bring it to life. Xionia Intelligence helps you discover and control its possibilities. StarShaper connects it all.

From research to instruments

Research that reaches the instruments.

Every Xionia instrument benefits from the intelligence of Xionia Core. StarShaper brings that intelligence to the relationships between instruments.

Research earns its place in a product when it makes music-making more rewarding. Then it reaches musicians through the seven stars and StarShaper from Xionia Audio, where intelligence is part of Xionia Core.

This page describes research, not products. Anything that becomes a product will be announced by the Xionia division that makes it, with what it does and when it’s available.

Work with us

We’d like to hear from researchers, sound designers and developers working on machine learning for audio, and from musicians with a view on what intelligent sound design should and shouldn’t do.

Get in touchContact Xionia Labs