Screenshot 1 (above) and General Description: The software is built on a MIDI display that looks similar to a standard MIDI sequencer or player program in the "Piano Roll" display (center panel of screenshot). The program will play a midi file, or loop through a section of a midi file, and even rapidly switch playing to and from up to 10 user-marked points of the midi file. An additional feature is that the user can guide his mouse-pointer "interactively" through the music (playing a short section of the score, or all notes in one instant of the score, or individual notes, as desired).

To support understanding of the music, the program's has one piano roll that overlays all notes independent of octave (top right panel here), and also highlights the bass note with a hatch pattern. This supports one form of easy chordal/harmony examination.

Additionally, the program has several other forms of visual chord/key identifiers (I call them "perceptograms"), one of which is the colorful thing occupying the larger part of the right panel in the screenshot above. What the version of the perceptogram I have displayed does, is in each of the 24 major and minor keys, is color-code the chord that we would have in that key, and show via height the number of pitch-classes in the chord, as well as use hatches and diagonal slashes to show non-chord pitch classes. (The cross-hatches respresent cases where there is a fit of more notes in another key, and tend to appear more outside of the actual key, while the slashes are the other cases, and are more likely to occur in the actual key of the music.) I use tomato-leaf green for the I chord, and tomato-red for the V chord, and a rich yellow for the IV chord. You get a lot of these, particularly I and V, in the actual key of the music.)

Also, there are little white horizontals in these perceptograms, indicating where the bass note is in the chord.

(In the screenshot, the key for the portion of the music shown is F major throughout, and this is clearly confirmed by all the tomato-coloring in the top F line = F major, of the 24 lines of the perceptogram.)

When the key of the section of music on display is detected and set by the user (using mainly the visual key identifiers described above--this has been done in the screenshot above, shown by the ">>>>" pointing to F major), the grey-white bands on all piano rolls will be in the right places for the key (showing key-scale notes vs. notes outside of the key scale). This feature is, of course, to support development of understanding of the sound of the music in key context (rest tones, active tones, chromatic tones, etc), and also supports determination of key-fit by note-presence and melodic patterns.

Further, the program makes aural key orientation feedback available from simple mouse commands and computer-keyboard presses, to further allow the user to explore and understand the sound of the music in key/tonal context. These sounds include the key reference tones (tonic, cadence, I-IV-V-I progression, chords, scales, individual notes, etc), all from choosable instruments.

Further, the program can detect the pitch from a user singing or humming along (or playing a one-note-at-a-time musical instrument). It can overlay that pitch over the various piano rolls. (In the screenshot, I was playing the MIDI file "manually" by mouse movements, at the point of those vertical red lines, and I was humming along at the C there in the score (an octave below middle C) and am about a 1/6 half step under perfect. (The little windshield-wiperlike thing which can be switched off, but which I have switched on here, runs from half a half-step below to half a half-step above the closest note.)

NOTE1: I have done a good amount of reprogramming, including moving of interface buttons, during April 2016. This newer version of the interface is show in the above screenshot (1) and immediately below in screenshots (1b) and (1c). Most of the other screenshots are from before this modification, so various buttons and checkboxes and listboxes will be in different places in the older screenshots, or may be missing in them.



Screenshot (1b) (below). This is the (new post April 2016) device selection panel. The MIDI output device must be used (otherwise you can't generate the sound of the MIDI notes!) A MIDI keyboard device is optional. And your microphone device is optional (and is for the hum/sing/play along feedback).
(On the right in the white rectangle, when pitch feedback is on, you get at the top the signal from the microphone (use it to adjust level), the autocorrelation function (ignore it if you're not a signal processing person), and a representation of the detected pitch (the one you really want is not here, but overlaid on the piano rolls). In any case, you probably will do fine without having to adjust the default settings.

Screenshot (1c) (below). Showing my hum-along pitch "path" while humming along with a piece of popular music that I downloaded in MIDI file format from the web. (What I first did before humming along is highlight the melody in bright yellow. This highlighting is a feature of my program, and I did it by checking in that second column for channel 4.) That jagged line that often goes through the highlighted notes is my pitch path. (To turn that path on, on the devices panel, I had to turn on "hum-along feedback", and, in addition, on the main panel, I checked "record" and "show prior". ("Record" means record the pitch path sung as the midi file is playing. The actual singing sound you make is not recorded.) Note that the pitch can only be recorded while you are playing a midi file. (Not while you are just swiping with the mouse to listen to a section of the file. When you are swiping with the mouse, you can get pitch feedback with the arrow, etc., but your path through the music will not be recorded or displayed.)

Also shown: the 3 bright pink notes are notes from when I had the program play a cadence. (The pink rectangles show for all of: reference sounds you produce using the mouse--the cadence was one such sound, and notes you play on your MIDI controller, and notes you play using the PC keyboard--recall, I have made the program so you can get 3 octaves of notes identified relative to the tonic of the key you have the program set for by using the PC keyboard.)

Screenshot 2a (above). Showing the harmonic function/harmonic fit "chordal" perceptogram (for a single key only in this case). Note that the perceptograms can be displayed for an identified key, or for all 24 potential keys -- with the latter being for when key is unidentified, or rapidly changing. (The key is C for this section of Haydn's op 17 string quartet mv. 1.)

What the colors represent is chordal function for the key. Green represents tonic function, red dominant function, yellow sub-dominant. The substitute chords are in the same color, but a little more washed out. Orange is III function. The height represents the number of chord notes (i.e. pitch-classes--same pitch different octave counts only once) present. Note red then green represents basically a V-I cadence, which in some musical styles is a reliable sign of intended tonal center.

There are also diagonal criss-crosses and slashes, which represent two forms of lack of fit. Recall the color goes up to a height representing the number of pitch classes of the best-fitting chord for the key. A diagonal criss-cross goes up any additional distance (up to the set plot maximum value MxNts) to the maximum number of pitch classes in the best fitting chord in any key, and further, a one-direction diagnonal takes you any further additional distance (up to MxNts) to the total number of pitch classes in the bit of music being analyzed. The diagonal criss-cross thus tends to indicate the chord is from a different key (either the given key is not the key, or it is a secondary dominant or borrowed or altered chord). The diagonal slash tends to point out non-harmonic tones for a chord in the key. (In this particular case, the two criss-crosses are V/IV and V/V secondary dominants, which is actually easy to determine using other rapid key-test hot-keys that I have in the program.) ( Also, note that, as stated, the settable parameter "MxNts" truncates the plots, so make sure it is set high enough -- depending on the density of the chords for best readability without loss of information.)

I also have an (optional) method of showing inversion within this perceptogram. Screenshot 4, farther below, shows this feature.

Acknowledgement: I am grateful for David Temperley's 2004 book, The Cognition of Musical Structures, many ideas from which have formed the basis for my chord fitting in these chordal perceptograms. (However, any possible imperfections or even idiocy in adapting David's ideas are completely my own contribution!)

Screenshot 2b (above). This is actually the end of a movement in C. The portion of the movement shown is in F, however. The key is set at C by the user, who has not yet responded to the observation that the part of the movement is in F. (When the user sets the key to F, the gray and white lines in the Piano Roll will be in the right place.) Anyway, reading the perceptogram to determine key: it is pretty clear that "C" is the wrong key for this part of the movement. This is because in the "C" line: (1) the abundance of tomato red/tomato-leaf green is too little, especially for a tonally stable part of the movement, and (2) there is a bit too much criss-cross and (3)we have the wrong cadence for this mainstream-period classical music. We see a nice fit at F major, the true key, where we also have that nice V-I cadence at the end. (The chromatic/non-chromatic overlay shown in the piano roll is thus incorrect

Screenshot 3 (below). Showing the tonal-center perceptogram of the "distributional" type plots (the reddish item in the bottom section). The "distributional" perceptogram focusses on what keys the notes fit, in terms of being non-chromatics and also rest-tones. This is as opposed to focussing on chords of the key, which the more colorful "chord" perceptograms address. The chord perceptograms (not these) are generally the more helpful of the two perceptogram types, (consistent with Schoenberg saying key is really about chords), but the "distributional" perceptograms sometimes are helpful. The music in view is the start of Bach's Well-Tempered-Klavier book 1 D major.




Screenshot 4 (below) : Chord position (root, 1st inversion, etc.) is shown in the bottom panel. (On the left is the scale for both inversion and number of pitch classes involved. Inversion number of the chord corresponds to the black horizontal mark within the chord-color, and there is no horizontal mark for root position.) The algorithm, as with all the chordal recognition stuff, is not perfect, but is pretty good. Sometimes it is over-literal, etc., and less standard chords are missed.



Screenshot 5 (below): Showing MIDI plus sung overtones. While a MIDI file is being played through manually (i.e. left-clicking the mouse over sections of music while pointed on the bottom piano roll or lower), the progam is set to give hummed-pitch feedback and show overtone locations for both the hummed pitch and played notes. The lines on the top piano roll to the right of the "position at" in blue represent the first 3 octaves of overtones of all the MIDI notes being sounded, while the ones in black are of the note being sung. Both lines get shorter as the represented overtone gets higher.The purpose of this is to help understanding of the sound of harmony/disharmony. (Disharmony is where any substantial-magnitude overtones differ in pitch non-negligibly but by a small amount--less than the "critical distance".)
Another reason for doing this is it may indicate an explanation where you sing a harmony that sounds best when a bit off note-center: because the overtones match up best.
Technical note: the overtones shown are not actually measured, but are the first 15 overtones by frequency.



Screenshot 6 (below). : To support music with rapidly changing keys, and to improve the ability to explore the music in terms of those changing keys simultaneously in terms of all of: chords, notes, action in the melodic and other voices, and sound, I have made it possible to pull the top Piano Roll (all notes octaves not overlayed) onto a separate window, thus making it possible to view just about everything you want without anything being to scrunched up to see. (Get the separate Piano Roll 1 by checking the new "detxPR1" which means "DETached extra Piano Roll 1". Then hitting the space bar will cycle to the view without Piano Roll 1 on the main window.) As usual, you can alternate trial keys with either the mouse wheel or the Z, X, C, and V computer keyboard keys, and you can also switch from the chordal 24-potential-key perceptogram to the note-presence ones (as in screenshot 3 above) using the B and N keys. (Do note: controlling the vertical range of notes displayed on the detached Piano Roll 1 is done from the attached Piano Roll 1, so you will have to cycle space-bar to put a Piano Roll 1 on the main window when you need to adjust it. The horizontal range = displayed width of notes can be adjusted from Piano Roll 2, however. ) Also note: you can not use the detached Piano Roll 1 to produce sounds of the separate notes as you can with the attached one.
And, oh yes, the stuff on the displays of this image is a segment the end part (second "A" part) of the 2nd movement of Schubert's Piano Sonata D959. I have the relevant part of the Wikpedia article explaining that movement in the shot. (The software is set with the correct key for that section, F sharp minor, which affects determines the gray and white overlays on the Piano Rolls, but the 24-key perceptograms would show all keys the same way regardless of the key my software is set to.)

Screenshot 6b (below).:Showing the chord position detectors (1st inversion, etc.) in the chord perceptogram (as in the bottom panel of screenshots 4 and 5, but in the 24-potential key version).





Screenshot 7 (below) NEW EAR TRAINING FEATURES: These features don't use a midi file. You are listening or singing/humming in a randomly chosen key (that you have some control over, and can change whenever you want). After you adjust parameters of the instruments, key ranges, pitch ranges on the ear-training panel (the right one), you then change keys randomly, change instruments randomly, change quizzed-on pitches, repeat quizzed-on pitches, and get various key-identifying sounds all by using the numeric keypad keys. (Make sure "Num lock" is on!) The aural feedback that is available when there is a MIDI file is also available, such as using your computer keyboard to play notes in the key, and hum-along pitch-feedback.

In the screenshot, the random key is D 0 major (in the scaling of my program, A0 is A220hz = the A just below middle C, which is C0). In the panel on the left, the green line reflects the tonic pitch. There's also the pitch path of someone (O.K., it was me but I was really tired at the time!) attempting a Do Re Mi then Fa So an octave below the tonic, but I was not doing very well, and the Mi is low and the Fa and So are really high. Also there's some undesirable sliding from one pitch to another.

Separately, the current sung pitch (Fa an octave below tonic) is being shown with that red arrow. The pink block is indicating that the last quizzed on tone was Ti just below the tonic. (The quizzed on tone in that pink block is not shown until you hit the numeric keypad ".")