digital harmony

musical harmony as ternary computing

This is a theory, but it works in practise, and yields surprisingly specific and rich musical results.

We know that binary numbers represent the presence or absence of powers of two:

1910 = 100112 
  = [[(1)×24]+[(0)×23]+[(0)×22]+[(1)× 21]+[(1)×20]
  = 16 + 2 + 1

We might also use binary numbers to represent the presence or absence of powers of three:

8510 = 100113 
  = [[(1)×34]+[(0)×33]+[(0)×32]+[(1)×31]+[(1)×30]
  = 81 + 3 + 1

Using threes in these positions is a step toward ternary computing, using digits 0, 1, and 2.

It also turns out to give a very fruitful representation of musical scales. Twos, in that scheme, turn out to be redundant — or probabalistic — when all is reduced to a single octave. So the system is surprisingly complete. It can be used to analyze audio and music notation, or to create music dynamically.

Circle of Fifths: 3n/2x

Instead of thinking of these powers of three as a sum (or dot product), as in a number system, we could think of them as populating a collection:

10011 = [(1)×34],[(1)×31],[(1)×30]

For musical purposes, it is useful to think of this as a collection of multiples of a base freqency:

10011 = [(freq)×34],[(freq)×31],[(freq)×30]

The circle of fifths is just such a sequence of twelve consecutive powers of three, from 30 to 311. Thus, if the above set of numbers were applied to a base frequency of middle C (as ‘freq’, above), the ones would correspond to C, G, E — a major triad. The sum of the waves at these frequencies sounds like this:

A major triad as powers of three, octave reduced.

The ratios of the complete circle of fifths pattern (starting at F) look like this in a single octave:

F:  3^0/2^0,
C:  3^1/2^1,
G:  3^2/2^3,
D:  3^3/2^4,
A:  3^4/2^6,
E:  3^5/2^7,
B:  3^6/2^9,
F#: 3^7/2^11,
C#: 3^8/2^12,
G#: 3^9/2^14,
Eb: 3^10/2^15,
Bb: 3^11/2^17

Our conventional musical notes correspond to powers of three, divided by whatever power of two is necessary to bring the frequency down into the the right octave. Twos define register: if their exponent changes, their octave changes.

In practical reality, this register change (which doesn’t change the name-identity of a note) may be important. It seems a sort of trivial reduction, but in the end the matter of register may help us identify the separateness of tones within a chord. The overtones of a pitch played on a real instrument multiply quickly beyond the first octave, but if there is a big spike within an octave of a perceived tone, it’s likely that it is the result of a secondary source.

Back to binary

In logical terms, then, we can derive the following: a tonal ‘note’ in the system is defined as the presence or absence of a power of three (with presence/absence marked by 0/1). A power of 2 changes the octave, but not the identity, of the note.

We can thus represent a group of notes as a single binary number, showing the presence or absence of the power of three relative to some core frequency.

Using this system, then, we can represent a C Major scale like this:

The above pattern serves as a kind of identity matrix, through which all other combinations of notes can be filtered, re-organized, and re-conceived.

Two things to notice: 1) this is a rearrangement of the white and black keys on the piano and 2) the order is reversed here to match the piano, ascending left to right. (C Major would be 000001111111 if the numbers went up from left to to right.) I’ve put the highest-exponent-value note at the right; numbers would increase to the left…

Maybe that was a mistake from musical habit.

For finding mathematical patterns (and there are many to find), it might be best to consider it the other way around…

The Audio End

We only come across the chromatic scale insofar as we consider powers of three to be musically meaningful. And it is fair to say that it is meaningful: the ratio of 3:2 lives powerfully in real-world pitches. Here it is on a violin. The lower band is C and neighboring pitches. The upper band is G, the frequency at a ratio of 3:2:

A violin playing a middle C

On Intonation and Tuning

There’s no point insisting on the finer points of intonation at this moment. In fact, I am confident in saying, as a highly competent professional violinist, that intonation is a moving target, with ever-shifting possibilities for ‘rightness’.

It is possible, for example, to tune dominant seventh chords to the overtone series, with a low major third, and a very low seventh. It sounds good – it’s not ‘out of tune’ – but it’s quite static, and not terribly useful for polyphony. The harmonic series functions most powerfully in the area of timbre, giving focus and color rather than dynamism and note-predictions. Chords can function (and can be suggestively tuned) far more flexibly in this binary system, where consonance is defined not only by the harmonic series, but also, simultaneously, by the entropy and patterning of its powers of three.


I write and pursue this musical system for two reasons.

  1. The presence of a large amount of conventional harmony at the root of the computer suggests a durable representation in the digital age. What can show up in binary can continue to exist digitally, because it is maximally efficient in the medium/mechanism.
  2. This system suggests a kind of predictive computing, as I will suggest in upcoming posts. Perhaps it can serve as a simple, tangible example of the basic processes of networks, memory, and even backpropagation. Evidence of its usefulness can be seen in the physical disturbance that every harmony change can cause a player of an instrument.

That is to say, it seems to be real enough.

digital harmony

sine wave blues

I would post this as ‘blues’ only because I know it’s not blues and never should be categorised as such.

It’s just some sort of tightened symbolic rendering, some mathemagical/numerological shadow.

But here in our digital igloos, before our computers, respecting our ‘social distance’, we play with dots and waves, alone. Perhaps, at least, there is some glue in this, for us all to use later, when there are instruments and gatherings.

In any case: for beginning to play along with harmony, there is hardly a better starting point than a simple blues progression. Here’s a little low resolution, lockdown blues, with a digital absence of style as backup.

I’ve tried to keep it in a kind of digital lo-res, trying to be aesthetically clear that we’re working at the bottom of the computer, in binary, and making decisions by hand, in some kind of reality.

The whole of it derives from three numbers: 0, 1, and -1, applied to a travelling harmony state.

Below is a graph of the waveform (barely recognisable as a waveform, in the green-blue strip) of the sinewave-piled blues progression, repeated ten times (GGGG-CCGG-DCGG). Below that a harmonic analysis of the waveform, based on a Fourier analysis. All is based in C Major (the red at the bottom), centered around G dominant (orange).

When I add a bass line and analyse the more complex waveform, the analysis turns to the flat side, centering itself, rather properly, in a B-flattish area. It’s an alternate analysis, equally valid:

If I set one of the hyperparameters higher, to filter out less of the signal, the basic structure returns, even if the extension analysis (the line at the top) disappears:

So to a large extent, the basic structure remains, even in a more complex system. Even my own live blues show through, somewhat:

Thankfully, it’s not so absolutely clear.

Harmony can indicate direction and tension in a near-total absence of timbre and dynamics. Harmony is something that folds us forward.

What we add on our own, in timbre and dynamics, is infinitely complicated, barely succumbing to any analysis, but still showing some structure. At least we know we’re on the flat side, and finding pillars of G major7. In the end, the variation within the analysis is akin to the many directions — the many potential continuities — we hear when we listen closely. It’s just at the edge of categorisation, where it ought to be.


Jill Lepore’s ‘if then’

The story of data analysis and prediction in Jill Lepore’s extraordinary new book IF THEN: How the Simulmatics Corporation Invented the Future commanded my attention. It will serve as a concrete reminder that when we analyze data, we cannot forget that we engage not only with the intended subject (‘human behavior’, in the case of Simulmatics), but also the behaviors of the humans organizing the data, and even the behavior and characteristics of data itself.

Above all, what stood out for me were 1) the positively insane variation of real-world issues even in the preliminary stage of ‘data collection’ and 2) the influence of the characters’ blindness, which at least equalled their vision. (This shows itself most consequentially in the company’s Vietnam work.) Defensive techniques against randomness and bias in data are highly developed by now, of course, and ‘Big data’ at our fingertips provides some stability… but, Jesus, what a mess it all is in real life.

The mechanisms by which technology can undermine and warp its own future show up here with surprising clarity. It is easy to imagine how the studies and conclusions of our own time will similarly erode and rust, sooner or later giving in to the flow of time, events, and memory.

The on-the-fly nature of memory we require for music reminds us that we are constantly adjusting, relying on thin similarities, ducking and weaving, and somehow coming out with larger pictures and structures. These larger pictures cannot accurately arise from brittle, contrived models. Ultimately, even the process of ‘labelling’ data proves an impediment to flexible knowledge, and to action. Music knows this, better than most arts.

Highly recommended, highly readable, worth sharing.

digital harmony

playing along

These days, we musicians are left with a question: what is left?

practise muted violin on minor vi-ii-V-I

What is left when everything is audio? What can performance mean in this infinite, icy context? What can be the message from our fingers, our own digits, our own breath, when touch seems at best superfluous and at worst outright dangerous?

Even as the idea of performance changes radically – and I can see neither limit nor brake in the current trend away from former conventions – live performance will remain a crucial point of contact not only amongst people, but also between people and the tactile, living world.

musical questions and answers

The question even of a digitized message is not only how does it make you feel, but how does it ask you to respond? This second question makes us build music. Music is a thing which asks for a response in kind.

Here is an answer:

Autumn Leaves for violin and sine waves

To the following. question from the computer:

[‘0x6c0’, ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x6c0’,  ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x6c0’,  ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x20b’,  ‘0xd80’,  ‘0x82c’,  ‘0x660’,  ‘0x660’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x6c0’,  ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x6c0’,  ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x6c0’,  ‘0x6c0’,  ‘0x590’,  ‘0x590’,  ‘0x330’,  ‘0x330’,  ‘0x660’,  ‘0x660’,  ‘0x4d0’,  ‘0x4d0’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x20b’,  ‘0xd80’,  ‘0x82c’,  ‘0x660’,  ‘0x660’,  ‘0xb2’,  ‘0xb2’,  ‘0x262’,  ‘0x262’,  ‘0x262’,  ‘0x262’] 

What I hope to be able to give with harmonypartition is not a technique for automation, but rather a language for prediction which allows multiple, flexible and complementary solutions. If we learn to play over chords, we will learn how to accommodate multiple voices, and we can play with each other, with infinite variety, in timbre, timing, register…

The computer can help with this, perhaps.

what is (or was) a concert?

Consider a concert, in the abstract. They were always call-and-response at their core, regardless of genre (wikipedia definition of call-and-response notwithstanding).

One group of people, identifiable as a ‘group’ by costume or location, would make sounds together in an identifiable way, while another group of people listens and watches. As the concert proceeded, the audience also makes sounds, usually of approval or encouragement. Sometimes this came as ‘applause’: handmade pink noise.

The computer sends echoes, but that is not sufficient: we need responses, not echoes. A response is not a right or wrong answer: a response is something which answers affirmatively that someone is indeed out there.

Simulation of a response is perhaps the only wrong answer to a musical question. Simulation… or absence, which are weirdly similar. to one another.

When we play with each other, live, we constantly respond, with infinite complexity. We applaud someone else’s music when we respond to it; we give it a history, a trajectory, a social meaning. We miss a part of it, we take a part of it; we transform it, we preserve it.

It is a miracle that it can be remembered at all, let alone taken further. It was all a memory, even when it happened.

digital harmony

tonality => memory

The question that keeps appearing before me is about what changing keys does to your memory. I’d like to build a framework to seek a more particular answer.

long-term and medium-term tonal memory: ii vs. V/V

To begin down this path, I’m going to start with the vi-ii-V-I sequence I’ve been going on about in the previous posts.

I’d like to contrast vi-ii-V-I with a vi-V/V-V-I sequence, which is only one note away, but makes a real difference in perceived brightness and in orientation.

Here they are to compare as audio files. First, a clean vi-ii-V-I:

Then, something similar, but with V/V instead of ii (the second chord is brighter):

Notice the change to orange in the medium-term memory, below the yellow

The analysis here shows the V/V changing something in the medium-term memory below the yellow box (look closely for the orange stripe below the yellow). It also does not disrupt the longest-term memory (consistently red, C Major, at the very bottom).

memory of major in relative minor

The relative minor is also one parameter away, and we can see that as relative minor, it also contains a memory of C Major (red) in the long-term memory, though the medium-term memory settles in around D minor:

This is a preliminary change in tonality: something that leaves a near-audible trace in the medium term memory. The change could be as short as a V/V, or could settle into a related key.

Here it settles into A minor, which is where the major sequence starts — put them together and you have something like ‘Autumn Leaves’, plus a bit of a new image of how Autumn Leaves descends in harmony as well as melody.

song structure: Autumn Leaves

So here are the first seven chords of Autumn Leaves analysed as audio:

And here as analysed from within the system’s clean taxonomy.

There are some minor differences in the long-term memory of these two versions, which is fantastic. The audio analysis shows that the music must find its way from a ‘real’, unprepared entry point; the prescribed, systematic version, knows exactly that it is in C Major. Our ears are somewhere in between, or around, these two possibilities.

This flexibility, which allows the same music to be analysed in multiple ways simultaneously (by different ears or different lines of thought), is what allows for flexibility in other musical parameters: dynamics, ornaments, tension.

Thus, two chords or sequences can sound exactly the same, but longer-term memory can always indicate other possibilities for memory and prediction.

It might be called ‘room for interpretation’ – though interpretation a wider field. But above all it indicates an irreducible volatility, full of potential, at the bottom-most level of computational analysis.

Somewhere, far above this analysis, lurks Cannonball Adderley. And the guys playing Autumn Leaves on street corners, giving ears and lives a bit of flow as they go.

digital harmony

midi => harmony

harmonypartition works simply and cleanly with MIDI files.

Using the music21 package, harmonypartition is able to analyze a MIDI file in exactly the same way it analyzes an audio file.

Results from MIDI are cleaner than those from audio – though not necessarily better, since resonance and continuity play such a key role in analysis, performance, and understanding. But MIDI is an important and stable reference in music notation, and provides invaluable information on many levels.

analysing a MIDI file

Below is an analysis of the same progression as on the previous pages: vi-ii-V-I in C Major, from a MIDI file:

bin_a, kpdve_a = pt_analyzeMIDI.analyze_notation_file("vi_ii_V_I.mid", beats_per_slice=2)

This analysis (like all analyses in harmonypartition yields two crucial pieces of information. The first is a binary ‘punch card’ of pitches. The second is an analysis of probable function. These can be seen below. First, in binary (with analysis given below):

Second, coded by proximity and most probable (or, more to the point, laziest possible) harmonic function:

harmonypartition analysis, with chords (large blocks) in C Major (bottom stripe)

This analysis is very close to what derives from the Fourier analysis of a live performance of the same music, here played on a lightly out-of-tune piano:

vi-ii-V-I live on funky piano

further examples: MAESTRO dataset

Then, to take a leap away from the vi-ii-V-I, here is a somewhat more sophisticated examples: Prelude and Fugue in A Minor, WTC II, BWV 889 (Johann Sebastian Bach), from the MAESTRO dataset. First an analysis of the audio recording:

Audio/Harmony analysis: Prelude and Fugue in A Minor, WTC II, BWV 889 (Johann Sebastian Bach) — using librosa stft

And here an analysis of the MIDI version of the same performance:

MIDI/Harmony analysis: Prelude and Fugue in A Minor, WTC II, BWV 889 (Johann Sebastian Bach) — using music21 for chord extraction.

These analyses could use some smoothing and statistical comparison, but all in all, it’s not bad for pure binary logic – which seems like a good, rooted place to have a meeting point.

digital harmony


I’ve built the harmonypartition system to generate harmonic sequences with just a few small numbers. Here is a simple one, built from simple piles of sine waves. Later they can of course expand to MIDI and DAW systems.

harmony as a sequence

So to begin: the following sequence:

base_seq = [4,3,2,1]

…can generate this chord sequence:

vi-ii-V-1 in C Major

This is not such a big deal, but it is already interesting that a harmonic parametrization can be rendered as simply as a melodic contour.

What is interesting is that the system underneath is much more than usually flexible. Change one parameter:

p = 5

… and you get this:

vi-ii-V-1 in C minor

This means, for example, that you could change the entire harmonic atmosphere of music (e.g in a game or film) according to easily structured parameters.

Intuitive relationships among these parameters would allow harmonic atmospheres to change meaningfully based on the motion of a joystick, a changing of levels, or other measurable conditions.

minimal encoding

And the encoding is very small, but non-trivial: it gives not only the notes but the function of each chord, which allows for improvisation. Here, for example is the encoding for the chords of ‘Blue in Green’:


…which sounds like this (with its bass line calculated from the encoding):

blue in green – analyzed chord cycle

…and like this with a field of (mathematically extrapolated, and slightly hysterical) notes above it:

I also use these small generated files (looped, and longer) to help violin students play over chords. Also, the encoding of a chord is small enough (32 bits) that the sharing of harmonic sequences over software (and IoT) could be rigorous on the one hand (for systems), and flexible (for people) on the other.

flexible decompression

What’s curious is that the system employs a type of compression which can be flexibly decompressed – delivering a world of sensible possibilities rather than a single unambiguous solution. This ambiguity derives from the fundamental ternary treatment of the byte.

digital harmony

audio => harmony

harmonypartition can harmonically analyze audio files by dynamically seeking and organizing tonal continuities.

In a great many musical cases, these continuities take the form of familiar keys and chords.

example: vi-ii-V-I

Here, for example, is a vi-ii-V-I progression in C major, generated from fourier analysis of a .wav file:

C Major vi-ii-V-I in sine waves, derived from synthesized audio

Here is the same progression, analyzed from my own lightly out-of-tune piano, with a different voicing:

C Major vi-ii-V-I on piano, live audio

Although the underpinnings and overtones (shown below and atop the central graphs) are shaken by the realities of live sound, the central analysis holds equally.

generating graphs

We can generate graphs with the following python code from the harmonypartition modules:

audio_kpdve_graph.graph_audio_file("C_6251_live.wav", chroma_threshold=0.3)

A firm, unchanging mathematical system undergirds the analysis, working within the idea of number, group, and ternary rather than with statistical operations. In the course of this method, the harmonic system functions as a hybrid, highly efficient form of neural network, using a miniature backpropagation to find meaningful (and maximally lazy) tonal centers.

It is possible in this manner to analyze any audio file for music-harmonic content, exposing large-scale structures in the musical works. Examples of this type of analysis can be seen in the “Audio and Insights” pages at the Charlottesville Chamber Music Festival.

Beethoven: Sonata No. 1 for Violin and Piano in D Major, Op. 12, No. 1, Mvt. 3 with Jennifer Frautschi, violin, and Max Levinson, piano

Schubert: Fantasie for Violin and Piano, D934, excerpts with James Ehnes, violin, and Inon Barnatan, piano

Beethoven: Sonata No. 10 for Violin and Piano in G Major, Op. 96, mvt. 4, with Timothy Summers, violin, and Benjamin Hochman, piano

The same process can be applied to MIDI and to live-streamed audio. The next post will address the process in MIDI.

Please send along any files for analysis — I would be happy to experiment with them in the course of further development, and discuss results.

The fourier analyses use the librosa package for pitch detection.

digital harmony

a new direction

Harmonic analysis of Schubert Fantasie, D. 934, opening bars.

The power of electronic music – even the most dedicated analogue instrumentalist must admit – is beyond question.

Though they do not cover the same musical ground as a finger on a string or breath in a column of air, the filters, echoes, synthesizers, and effects applied to waveforms in electronic music have enormous effect.

Harmony, however, remains insufficiently parametrized. This leaves an important dimension under-explored, and also stifles the integration of digital and acoustic instruments. Tonality guides our own real fingers with memory and prediction, and the methods of electronic music do not well or fully take this into account.

I have been working as a performer of mostly classical music at the Mahler Chamber Orchestra for the past fifteen years, and teaching at the Universität der Künste in Berlin. This work should bridge live performance and musical study in the analog/print style with performance around parameterised digital criteria.

parametrising harmony

I will work to address the process of parametrising harmony over the next weeks and months, exposing a tiny algorithm which back-propagates over a small set of bits, treating the bit as ternary rather than binary. This algorithm, being developed at GitHub as harmony partition, efficiently and flexibly describes a wide range of conventional harmonic usage.

I will document the process of making and using it here, and I hope I can build for you a proper guide, through a series of Jupyter notebooks which outline the theory and applications of this idea. Some of these steps will be pragmatic, some music-theoretical, and some just playful explorations. The main thing is to see if a community and musical language can build around this musical process.

A proper Pypi package will be available in mid-January 2021.


Initial results of the use of this algorithm can be seen at work at the Charlottesville Chamber Music Festival. To begin: Beethoven Sonata, Op. 111, played by Conor Hanick, an analysis of the second movement.

This blog will cover not only the theory, but also the implementation of the algorithm for use in musical practise.

Posture and Composure RightAndLeftHand String Yoga

Feet on the Ground

As you start to play, aim for solidity in left and right hand.

Your left hand should feel full, fat contact with the string; the hairs of the bow should sit thickly and exactly on the string..

Compare the solidity of contact in your hands with the solidity of your feet on the ground.

Try it with double stops as well.