Fig. 22.4 Application to test class RTPServer from Fig. 22.3 (part 6 of 6).

22.5 Java Sound

Many of today’s computer programs capture users’ attention with audio features. Even basic applets and applications can enhance users’ experiences with simple sounds or music clips. With sound programming interfaces, developers can create applications that play sounds in response to user interactions. For example, in many applications, when an error occurs and a dialog box appears on the screen, the dialog is often accompanied by a sound. Users thus receive both audio and visual indications that a problem has occurred. As another example, game programmers use extensive audio capabilities to enhance players’ experiences.

The Java Sound API is a simpler way to incorporate audio media into applications than the Java Media Framework. The Java Sound API is bundled with Java 2 Software Development Kit version 1.3. The API consists of four packages—javax.sound.midi, javax.sound.midi.spi, javax.sound.sampled and javax.sound.sampled.spi. The next two sections focus on packages javax.sound.midi and javax.sound.sampled, which provide classes and interfaces for accessing, manipulating and playing Musical Instrumental Data Interface (MIDI) and sampled audio. The packages ending in .spi provide developers with the tools to add Java Sound support for additional audio formats that are beyond the scope of this book.

The Java Sound API provides access to the Java Sound Engine which creates digitized audio and captures media from the supported sound devices discussed in Section 22.3. Java Sound requires a sound card to play audio. A program using Java Sound will throw an exception if it accesses audio system resources on a computer that does not have a sound card.

22.6 Playing Sampled Audio

This section introduces the features of package javax.sound.sampled for playing sampled audio file formats, which include Sun Audio (.au), Windows Waveform (.wav) and Macintosh Audio Interchange File Format (.aiff). The program in Fig. 22.5 and Fig. 22.6 shows how audio is played using these file formats.

When processing audio data, a line provides the path through which audio flows in a system. One example of a line is a pair of headphones connected to a CD player.

Class ClipPlayer (Fig. 22.5) is an example of how lines can be used. It contains an object that implements interface Clip, which in turn extends interface DataLine. A Clip is a line that processes an entire audio file rather than reading continuously from an audio stream. DataLines enhance Lines by providing additional methods (such as start and stop) for controlling the flow of data, and Clips enhance DataLines by providing methods for opening Clips and methods for precise control over playing and looping the audio.

1// Fig. 22.5: ClipPlayer.java

2 // Plays sound clip files of type WAV, AU, AIFF

3

4 // Java core packages

5 import java.io.*;

6

7// Java extension packages

8 import javax.sound.sampled.*;

9

10 public class ClipPlayer implements LineListener {

11

12// audio input stream

13private AudioInputStream soundStream;

15// audio sample clip line

16private Clip clip;

17

Fig. 22.5 ClipPlayer plays an audio file (part 1 of 4).

122// if replay set, replay forever

123else

132}

133}

134

135// set replay of clip

136public void setReplay( boolean value )

137{

138replay = value;

139}

140

141// stop and close clip, returning system resources

142public void close()

143{

144if ( clip != null ) {

145 clip.stop();

146clip.close();

147}

148}

149

150 } // end class ClipPlayer

Fig. 22.5 ClipPlayer plays an audio file (part 4 of 4).

All Lines generate LineEvents, which can be handled by LineListeners. LineEvents occur when starting, stopping, playing and closing a Line object. Although a Line stops playback automatically when it reaches the end of an audio file, class ClipPlayer implements interface LineListener (line 10) and can close the Clip permanently or replay the Clip (discussed shortly). LineListeners are useful for tasks that must be synchronized with the LineEvent states of a line.

The Clip reads audio data from an AudioInputStream (a subclass of InputStream), which provides access to the stream’s data content. This example loads clips of the audio data before attempting to play it, and therefore is able to determine the length of the clip in frames. Each frame represents data at a specific time interval in the audio file. To play sampled audio files using Java Sound, a program must obtain an AudioInputStream from an audio file, obtain a formatted Clip line, load the AudioInputStream into the Clip line and start the data flow in the Clip line.

To play back sampled audio, the audio stream must be obtained from an audio file. ClipPlayer method openFile (lines 31–54) obtains audio from soundFile (initialized in the ClipPlayer constructor at lines 25–28). Lines 35–36 call AudioSystem static method getAudioInputStream to obtain an AudioInputStream for soundFile. Class AudioSystem facilitates access to many of the resources needed to play and manipulate sound files. Method getAudioInputStream throws an Unsup-

portedAudioFileException if the specified sound file is a non-audio file or if it contains a format that is not supported by Java Sound.

Next the program must provide a line through which audio data can be processed. Line 52 invokes method loadClip (lines 57–106) to open a Clip line and load the audio stream for playback. Line 81 invokes AudioSystem static method getLine to obtain a Clip line for audio playback. Method getLine requires a Line.Info object as an argument, to specify the attributes of the line that the AudioSystem should return. The line must be able to process audio clips of all supported sampled audio formats, so the DataLine.Info object must specify a Clip data line and a general encoding format. A buffer range should also be specified so the program can determine the best buffer size. The DataLine.Info constructor receives four arguments. The first two are the format (of type AudioFormat.Encoding) into which the program should convert the audio data and the AudioFormat of the audio source. The AudioFormat sets the format supported by the line, according to the audio format of the stream. Line 63 obtains the AudioFormat of the AudioInputStream, which contains format specifications that the underlying system uses to translate the data into sounds. Lines 68–69 call AudioSystem method getTargetFormats to obtain an array of the supported AudioFormats. The third argument of the DataLine.Info constructor, which specifies the minimum buffer size, is set to the number of bytes in each frame of the audio stream. Line 70 invokes AudioFormat method getFrameSize to obtain the size of each frame in the audio stream. The maximum buffer size should be equivalent to two frames of the audio stream (line 71). Using the DataLine.Info object, line 74 checks if the underlying audio system supports the specified line. If it does, line 81 obtains the line from the audio system.

When an audio clip starts playing and when it finishes, the program needs to be alerted. Line 84 registers a LineListener for the Clip’s LineEvents. If a LineEvent occurs, the program calls LineListener method update (lines 115–133) to process it. The four LineEvent types, as defined in class LineEvent.Type, are OPEN, CLOSE, START and STOP. When the event type is LineEvent.Type.STOP and variable replay is false, line 120 calls ClipPlayer’s close method (lines 142–148) to stop audio playback and close the Clip. All audio resources obtained previously by the Clip are released when audio playback stops. When the event type is

LineEvent.Type.STOP and variable replay is true, line 131 calls Clip method loop with parameter Clip.LOOP_CONTINUOUSLY, causing the Clip to loop until the user terminates the application. Invoking method stop of interface Clip stops data activity in the Line. Invoking method start resumes data activity.

Once the program finishes validating the Clip, line 87 calls Clip method open with the AudioInputStream soundStream as an argument. The Clip obtains the system resources required for audio playback. AudioSystem method getLine and Clip method open throw LineUnavailableExceptions if another application is using the requested audio resource. Clip method open also throws an IOException if the Clip cannot read the specified AudioInputStream. When the test program (Fig. 22.6) calls ClipPlayer method play (lines 109–112), the Clip method start begins audio playback.

Class ClipPlayerTest (Fig. 22.6) enables users to specify an audio file to play by clicking the Open Audio Clip button. When users click the button, method actionPerformed (lines 37–58) prompts an audio file name and location (line 39) and creates

a ClipPlayer for the specified audio file (line 44). Line 47 invokes ClipPlayer method openFile, which returns true if the ClipPlayer can open the audio file. If so, line 50 calls ClipPlayer method play to play the audio and line 53 calls ClipPlayer method setReplay to indicate that the audio should not loop continuously.

Performance Tip 22.4

Large audio files take a long time to load, depending on the speed of the computer. An alter- native playback form is to buffer the audio by loading a portion of the data to begin playback and continuing to load the remainder as the audio plays. This is similar to the streaming capability provided by JMF.

1 // Fig. 22.6: ClipPlayerTest.java

2 // Test file for ClipPlayer

3

4 // Java core packages

5import java.awt.*;

6 import java.awt.event.*;

7 import java.io.*;

8

9 // Java extension packages

10 import javax.swing.*;

11

12 public class ClipPlayerTest extends JFrame {

13

14// object to play audio clips

15private ClipPlayer clipPlayer;

17// constructor for ClipPlayerTest

18public ClipPlayerTest()

19{

20super( "Clip Player" );

21

22// panel containing buttons

23JPanel buttonPanel = new JPanel();

24getContentPane().add( buttonPanel );

26// open file button

27JButton openFile = new JButton( "Open Audio Clip" );

28buttonPanel.add( openFile, BorderLayout.CENTER );

29

30// register ActionListener for openFile events

31openFile.addActionListener(

Fig. 22.6 ClipPlayerTest enables the user to specify the name and location of the audio to play with ClipPlayer (part 1 of 3).

66// get file from computer

67public File getFile()

68{

69JFileChooser fileChooser = new JFileChooser();

71 fileChooser.setFileSelectionMode(

72JFileChooser.FILES_ONLY );

73int result = fileChooser.showOpenDialog( this );

79return fileChooser.getSelectedFile();

80}

81

82// execute application

83public static void main( String args[] )

84{

85ClipPlayerTest test = new ClipPlayerTest();

87test.setSize( 150, 70 );

88test.setLocation( 300, 300 );

89test.setDefaultCloseOperation( EXIT_ON_CLOSE );

90test.setVisible( true );

91}

92

Fig. 22.6 ClipPlayerTest enables the user to specify the name and location of the audio to play with ClipPlayer (part 2 of 3).

93 } // end class ClipPlayerTest

Fig. 22.6 ClipPlayerTest enables the user to specify the name and location of the audio to play with ClipPlayer (part 3 of 3).

22.7 Musical Instrument Digital Interface (MIDI)

The Musical Instrument Digital Interface (MIDI) is a standard format for electronic music. MIDI music can be created through a digital instrument, such as an electronic keyboard, or through software. The MIDI interface allows musicians to create synthesized digital music that reproduces the actual music. Then they can share their musical creations with music enthusiasts around the world. A MIDI synthesizer is a device that can produce MIDI sounds and music.

Programs can easily manipulate MIDI data. Like other types of audio, MIDI data has a well-defined format that MIDI players can interpret, play and use to create new MIDI data. The Complete Detailed MIDI 1.0 specification provides detailed information on MIDI files. Visit the official MIDI Web site at www.midi.org for information on MIDI and its specification. Java Sound’s MIDI packages (javax.sound.midi and javax.sound.midi.spi) allow developers to access MIDI data.

Interpretation of MIDI data varies between synthesizers, so a file may sound quite different when played on synthesizers other than the one on which it was created. Synthesizers support varying types and numbers of instrumental sounds and different numbers of simultaneous sounds. Usually hardware-based synthesizers are capable of producing higherquality synthesized music than software-based synthesizers.

Many Web sites and games use MIDI for music playback, as it enables developers to entertain users with lengthy, digitized music files that do not require a lot of memory. In comparison, sampled audio files can grow to be quite large. Package javax.sound.midi enables programs to manipulate, play and synthesize MIDI. Java Sound supports MIDI files with mid and rmf (Rich Music Format or RMF) extensions.

The example presented in Sections 22.7.1 through 22.7.4 covers MIDI synthesis, playback, recording and saving. Class MidiDemo (Fig. 22.10) is the main application class that utilizes classes MidiData (Fig. 22.7), MidiRecord (Fig. 22.8) and MidiSynthesizer (Fig. 22.9). Class MidiSynthesizer provides resources for generating sounds

and transmitting them to other MIDI devices, such as recorders. Class MidiData handles MIDI playback, track initialization and event information. Class MidiRecord provides MIDI recording capabilities. Class MidiDemo ties the other classes together with an interactive GUI that includes a simulated piano keyboard, play and record buttons, and a control panel for configuring MIDI options. Class MidiDemo also uses MIDI event-synchroniza- tion to play a MIDI file and highlight the appropriate piano keys, simulating someone playing the keyboard.

An integral part of this MIDI example is its GUI, which allows users to play musical notes on a simulated piano keyboard (see screen capture in Fig. 22.10). When the mouse hovers over a piano key, the program plays the corresponding note. In this section, we refer to this as user synthesis. The Play MIDI button in the GUI allows the user to select a MIDI file to play. The Record button records the notes played on the piano (user synthesis). Users can save the recorded MIDI to a file using the Save MIDI button and play back the recorded MIDI using the PlayBack button. Users can click the Piano Player button to open a MIDI file, then play that file back through a synthesizer. The program signifies synchronization of notes and piano keys by highlighting the key that corresponds to the note number. This playback and synchronization ability is called the “piano player.” While the “piano player” is running, users can synthesize additional notes, and record both the old audio material and the new user-synthesized notes by clicking the Record button. The JComboBox in the upper-left corner of the GUI enables users to select an instrument for synthesis. Additional GUI components include a volume control for user-synthesized notes and a tempo control for controlling the speed of the “piano player.”

Testing and Debugging Tip 22.1

Testing the MIDI file playback functions requires a sound card and an audio file in MIDI format.

22.7.1 MIDI Playback

This section discusses how to play MIDI files and how to access and interpret MIDI file contents. Class MidiData (Fig. 22.7) contains methods that load a MIDI file for playback. The class also provides the MIDI track information required by the “piano player” feature. A MIDI sequencer is used to play and manipulate the audio data. Often, MIDI data is referred to as a sequence, because the musical data in a MIDI file is composed of a sequence of events. The steps performed in MIDI playback are accessing a sequencer, loading a MIDI sequence or file into that sequencer and starting the sequencer.

1// Fig. 22.7: MidiData.java

2// Contains MIDI sequence information

3 // with accessor methods and MIDI playback methods

4

5 // Java core package

6 import java.io.*;

7

8 // Java extension package

9 import javax.sound.midi.*;

10

Fig. 22.7 MidiData loads MIDI files for playback (part 1 of 5).

11 public class MidiData {

12

13// MIDI track data

14private Track track;

16// player for MIDI sequences

17private Sequencer sequencer;

19// MIDI sequence

20private Sequence sequence;

22// MIDI events containing time and MidiMessages

23private MidiEvent currentEvent, nextEvent;

24

25// MIDI message usually containing sounding messages

26private ShortMessage noteMessage;

27

28// short, meta, or sysex MIDI messages

29private MidiMessage message;

30

31// index of MIDI event in track, command in MIDI message

32private int eventIndex = 0, command;

33

34// method to play MIDI sequence via sequencer

35public void play()

36{

37// initiate default sequencer

38try {

50sequencer.start();

51}

52

53// MIDI resource availability error

54catch ( MidiUnavailableException noMidiException ) {

55noMidiException.printStackTrace();

56}

57

58// corrupted MIDI or invalid MIDI file encountered

59catch ( InvalidMidiDataException badMidiException ) {

Fig. 22.7 MidiData loads MIDI files for playback (part 2 of 5).

116// set current MIDI event to first event in track

117currentEvent = track.get( eventIndex );

118

119// get MIDI message from event

120message = currentEvent.getMessage();

122// track initialization successful

123return true;

124

125 } // end method initializeTrack

126

127// move to next event in track

128public void goNextEvent()

129{

130eventIndex++;

131currentEvent = track.get( eventIndex );

132message = currentEvent.getMessage();

133}

134

135// get time interval between events

136public int getEventDelay()

137{

138// first event's time interval is its duration

139if ( eventIndex == 0 )

140 return ( int ) currentEvent.getTick();

141

142// time difference between current and next event

143return ( int ) ( track.get( eventIndex + 1 ).getTick() -

144currentEvent.getTick() );

147// return if track has ended

148public boolean isTrackEnd()

149{

150// if eventIndex is less than track's number of events

151if ( eventIndex + 1 < track.size() )

154return true;

155}

156

157// get current ShortMessage command from event

158public int getEventCommand()

159{

160if ( message instanceof ShortMessage ) {

162 // obtain MidiMessage for accessing purposes

163 noteMessage = ( ShortMessage ) message;

164return noteMessage.getCommand();

165}

166

167return -1;

168}

Fig. 22.7 MidiData loads MIDI files for playback (part 4 of 5).

169

170// get note number of current event

171public int getNote()

172{

173if ( noteMessage != null )

174 return noteMessage.getData1();

175

176return -1;

177}

178

179// get volume of current event

180public int getVolume()

181{

182return noteMessage.getData2();

183}

184

185 } // end class MidiData

Fig. 22.7 MidiData loads MIDI files for playback (part 5 of 5).

To play a MIDI file with a sequence, a program must obtain the MIDI sequence and check for compatibility issues. MidiData method initialize (lines 73–94) obtains a Sequence of MIDI data from a file with MidiSystem method getSequence (line 77). A Sequence contains MIDI tracks, which, in turn, contain MIDI events. Each event encapsulates a MIDI message of instructions for the MIDI devices. Individual tracks of a MIDI sequence are analogous to tracks on a CD. However, while CD tracks are played in order, MIDI tracks are played in parallel. A MIDI track is a recorded sequence of data. MIDIs usually contain multiple tracks. Method getSequence can also obtain a MIDI sequence from a URL or an InputStream. Method getSequence throws an InvalidMidiDataException if the MIDI system detects an incompatible MIDI file.

Portability Tip 22.2

Because of incompatible file parsers in different operating systems, sequencers may not be able to play RMF files.

After obtaining a valid MIDI sequence, the program must obtain a sequencer and load the sequence into the sequencer. Method play (lines 35–64) in class MidiData calls MidiSystem method getSequencer (line 41) to obtain a Sequencer to play the Sequence. Interface Sequencer, which extends interface MidiDevice (the superinterface for all MIDI devices), provides the standard sequencer device to play MIDI data. If another program is using the same Sequencer object, method getSequencer throws a MidiUnavailableException. Line 44 calls Sequencer method open to prepare to play a Sequence. Sequencer method setSequence (line 47) loads a MIDI Sequence into the Sequencer and throws an InvalidMidiException if the Sequencer detects an unrecognizable MIDI sequence. Line 50 begins playing the MIDI sequence by calling the Sequencer’s start method.

In addition to MIDI playback methods, class MidiData also provides methods that enable a program to access the events and messages of a MIDI sequence. As we shall see, class MidiDemo (Figure 22.10) uses class MidiData to access the data in a MIDI file for synchronizing the highlighting of piano keys. The MIDI events are stored in the MIDI’s

tracks, which are instances of class Track (package javax.sound.midi). MIDI events in MIDI tracks are represented by class MidiEvent (package javax.sound.midi). Each MIDI event contains an instruction and the time at which it should occur. The individual events in a track contain messages of type MidiMessage that specify the MIDI instructions for a MidiDevice. There are three types of MIDI messages—ShortMessage, SysexMessage and MetaMessage. ShortMessages are explicit musical instructions, such as the specific notes to play and pitch changes. The other two less-used messages are SysexMessages, system-exclusive messages for MIDI devices, and MetaMessages, which may indicate to a MIDI device that the MIDI has reached the end of a track. This section deals exclusively with ShortMessages that play specific notes.

Next, the program must obtain the tracks and read their events. MidiData method initializeTrack (lines 97–125) invokes Sequence’s getTracks method (line 100) to obtain all of the tracks in the MIDI sequence. Lines 108–114 determine the longest track in the MIDI and set it as the one to play. Line 117 obtains the first MIDI event in the Track by invoking its get method with the index of the event in the track as the parameter. At this point eventIndex is set to 0 (line 32). Line 120 obtains the MIDI message from the MIDI event using method getMessage of class MidiEvent. To help a program step through each event in the tracks, the program can call MidiData method goNextEvent (lines 128–133) to load the next event and message. Method goNextEvent increments eventIndex in the loaded MIDI Track and finds the next event’s MidiMessage.

In addition to reading the events, the program must also determine how long each event lasts and the spacing between events. Method getEventDelay (lines 136–145) returns the duration of a MidiEvent as the time difference between two events in the MIDI sequence (lines 143–144). The MidiEvent’s getTick method provides the specific time when the event takes place (also called a time stamp). Lines 139–140 return the first MidiEvent’s time stamp as the event’s duration.

Class MidiData provides other methods to return the commands, the note numbers and the volume of note-related ShortMessages. Method getEventCommand (lines 158–168) determines the command number representing the command instruction. Line 160 of method getEventCommand indicates whether the currently loaded MidiMessage is a ShortMessage. If so, line 163 assigns the ShortMessage to object noteMessage and line 164 returns the ShortMessage’s command status byte by invoking

ShortMessage’s getCommand method. Method getEventCommand returns -1 if the event does not contain a ShortMessage. MidiData method getNote (lines 171– 177) invokes ShortMessage method getData1 (line 174) to return the note number. Method getVolume (lines 180–183) invokes ShortMessage method getData2 to return the volume. Class MidiData also provides an indication of the end of a track in method isTrackEnd (lines 148–155), which determines whether the event index has surpassed the number of events in the track (line 151).

22.7.2 MIDI Recording

A program can record MIDI using a sequencer. Class MidiRecord (Fig. 22.8) handles the recording functions of this MIDI demo using an object that implements interface Sequencer as a MIDI recorder. As long as the MIDI devices are set up correctly, interface Sequencer provides simple methods for recording. Class MidiRecord has a constructor (lines 29–32) that receives an object that implements interface Transmitter as an

argument. A Transmitter sends MIDI messages to a MIDI device that implements interface Receiver. Think of Transmitters and Receivers as output and input ports respectively for MIDI devices.

1// Fig. 22.8: MidiRecord.java

2 // Allows for recording and playback

3 // of synthesized MIDI

4

5 // Java core packages

6 import java.io.*;

7

8 // Java extension package

9 import javax.sound.midi.*;

10

11 public class MidiRecord {

12

13// MIDI track

14private Track track;

16// MIDI sequencer to play and access music

17private Sequencer sequencer;

18

19// MIDI sequence

20private Sequence sequence;

22// receiver of MIDI events

23private Receiver receiver;

25// transmitter for transmitting MIDI messages

26private Transmitter transmitter;

27

28// constructor for MidiRecord

29public MidiRecord( Transmitter transmit )

30{

31transmitter = transmit;

32}

33

34// initialize recording sequencer, set up recording sequence

35public boolean initialize()

36{

37// create empty MIDI sequence and set up sequencer wiring

38try {

Fig. 22.8 MidiRecord enables a program to record a MIDI sequence (part 1 of 3).

59makeTrack();

60}

61

62// invalid timing division specification for new sequence

63catch ( InvalidMidiDataException invalidMidiException ) {

64 invalidMidiException.printStackTrace();

65return false;

66}

67

68// sequencer or receiver unavailable

69catch ( MidiUnavailableException noMidiException ) {

70 noMidiException.printStackTrace();

71return false;

72}

73

74// MIDI recorder initialization successful

75return true;

76

77 } // end method initialize

78

79// make new empty track for sequence

80public void makeTrack()

81{

82// if previous track exists, delete it first

83if ( track != null )

86// create track in sequence

87track = sequence.createTrack();

88}

89

90// start playback of loaded sequence

91public void play()

92{

93sequencer.start();

94}

95

96// start recording into sequence

97public void startRecord()

98{

99// load sequence into recorder and start recording

Fig. 22.8 MidiRecord enables a program to record a MIDI sequence (part 2 of 3).

102

103 // set track to recording-enabled and default channel 104 sequencer.recordEnable( track, 0 );

105

106sequencer.startRecording();

107}

108

109// sequence contains bad MIDI data

110catch ( InvalidMidiDataException badMidiException ) {

117// stop MIDI recording

118public void stopRecord()

119{

120sequencer.stopRecording();

121}

122

123// save MIDI sequence to file

124public void saveSequence( File file )

125{

126// get all MIDI supported file types

127int[] fileTypes = MidiSystem.getMidiFileTypes( sequence );

129if ( fileTypes.length == 0 ) {

130 System.err.println( "No supported MIDI file format!" );

131return;

132}

133

134// write recorded sequence into MIDI file

135try {

136MidiSystem.write( sequence, fileTypes[ 0 ], file );

137}

138

139// error writing to file

140catch ( IOException ioException ) {

141ioException.printStackTrace();

142}

143

144 } // end method saveSequence

145

146 } // end class MidiRecord

Fig. 22.8 MidiRecord enables a program to record a MIDI sequence (part 3 of 3).

The first step of recording MIDI data is similar to the playback mechanism in class MidiData. In addition to obtaining an empty sequence and a sequencer, a MIDI recording program needs to connect the transmitters and receivers. After successfully “wiring” the sequencer’s receiver as the “IN PORT,” the recorder loads the empty sequence into the sequencer to start recording to a new track in the sequence. The following discussion covers these steps.

Method initialize (lines 35–77) of class MidiRecord sets up the sequencer for recording. Line 41 of method initialize instantiates an empty sequence. MidiRecord will record data to the empty sequence once the transmitter is connected to the receiver. Line 48 obtains the recording sequencer’s receiver and line 57 specifies that transmitter will send its messages to receiver.

MIDI messages must be placed in a track, so method initialize invokes method makeTrack (lines 80–88) to delete the previous existing track (line 84) and to create an empty Track (line 87). Method makeTrack can also be called from an external class to record a new sequence without instantiating new sequencers and a new sequence.

After setting up a sequencer and an empty sequence, calling MidiRecord method startRecord (lines 97–115) starts the recording process. Line 101 loads an empty sequence into the sequencer. Sequencer method recordEnable is called and passed the track object and a channel number as arguments (line 104), which enables recording on that track. Line 106 invokes Sequencer’s startRecording method to start the recording of MIDI events sent from the transmitter. Sequencer’s stopRecording method stops recording and is called in MidiRecord’s stopRecord method (lines 118–121).

Class MidiRecord can also supports save a recorded sequence to a MIDI file using its saveSequence method (lines 124–144). Although most MIDI sequences can support MIDI type 0 files (the most common type of MIDI file), the sequence should be checked for other supported file types. Line 127 obtains an array of MIDI file types supported by the system for writing a sequence to a file. The MIDI file types are represented by integer values of 0, 1 or 2. Using the first supported file type, the MidiSystem writes the sequence to a specified File (line 136) passed into method saveSequence as an argument. MidiRecord’s play method (lines 91–94) enables the program to play back the newly recorded sequence.

22.7.3 MIDI Synthesis

This MidiDemo program provides an interactive piano that generates notes according to the keys pressed by the user. Class MidiSynthesizer (Fig. 22.9) generates these notes directly, and sends them to another device. Specifically, it sends the notes to a sequencer’s receiver through a transmitter to record the MIDI sequence. Class MidiSynthesizer uses an object that implements interface Synthesizer (a sub-interface of MidiDevice) to access the default synthesizer’s sound generation, instruments, channel resources and sound banks. A SoundBank is the container for various Instruments, which instructs the computer on how to make the sound of a specific note. Different notes made by various instruments are played through a MidiChannel on different tracks simultaneously to produce symphonic melodies.

1 // Fig. 22.9: MidiSynthesizer.java

2 // Accessing synthesizer resources

3

4 // Java extension package

5 import javax.sound.midi.*;

6

Fig. 22.9 MidiSynthesizer can generate notes and send them to another MIDI device (part 1 of 4).

7 public class MidiSynthesizer {

8

9 // main synthesizer accesses resources

10 private Synthesizer synthesizer;

11

12// available instruments for synthesis use

13private Instrument instruments[];

14

15// channels through which notes sound

16private MidiChannel channels[];

19// transmitter for transmitting messages

20private Transmitter transmitter;

21

22// receiver end of messages

23private Receiver receiver;

25// short message containing sound commands, note, volume

26private ShortMessage message;

27

28// constructor for MidiSynthesizer

29public MidiSynthesizer()

30{

31// open synthesizer, set receiver,

32// obtain channels and instruments

33try {

Fig. 22.9 MidiSynthesizer can generate notes and send them to another MIDI device (part 2 of 4).

67// synthesizer, receiver or transmitter unavailable

68catch ( MidiUnavailableException noMidiException ) {

69noMidiException.printStackTrace();

70}

71

72 } // end constructor

73

74// return available instruments

75public Instrument[] getInstruments()

76{

77return instruments;

78}

79

80// return synthesizer's transmitter

81public Transmitter getTransmitter()

82{

83return transmitter;

84}

85

86// sound note on through channel

87public void midiNoteOn( int note, int volume )

88{

89channel.noteOn( note, volume );

90}

91

92// sound note off through channel

93public void midiNoteOff( int note )

94{

95channel.noteOff( note );

96}

97

98// change to selected instrument

99public void changeInstrument( int index )

100{

101Patch patch = instruments[ index ].getPatch();

103 channel.programChange( patch.getBank(),

104patch.getProgram() );

105}

106

107// send custom MIDI messages through transmitter

108public void sendMessage( int command, int note, int volume )

109{

Fig. 22.9 MidiSynthesizer can generate notes and send them to another MIDI device (part 3 of 4).

110// send a MIDI ShortMessage using this method's parameters

111try {

112 message = new ShortMessage();

113

114 // set new message of command (NOTE_ON, NOTE_OFF), 115 // note number, volume

116 message.setMessage( command, note, volume );

117

118 // send message through receiver

119receiver.send( message, -1 );

120}

121

122// invalid message values set

123catch ( InvalidMidiDataException badMidiException ) {

124badMidiException.printStackTrace();

125}

126

127 } // end method sendMessage

128

129 } // end class MidiSynthesizer

Fig. 22.9 MidiSynthesizer can generate notes and send them to another MIDI device (part 4 of 4).

MidiSynthesizer’s constructor (lines 29–72) acquires the synthesizer and initializes related resources. Line 34 obtains a Synthesizer object from the MidiSystem and line 38 opens the Synthesizer. To enable sounds to be played and recorded at the same time, lines 41–47 obtain the Transmitter and Receiver of the Synthesizer. When a MIDI message is sent to the synthesizer’s receiver, the synthesizer executes the message’s instruction, generating notes, and the transmitter sends that message to designated Receivers of other MidiDevices.

Common Programming Error 22.3

A MidiUnavailableException occurs when a program attempts to acquire unavail- able MidiDevice resources such as synthesizers and transmitters.

MIDI messages are sent to the MidiSynthesizer from MidiDemo as a result of either pressing a piano key or a MidiEvent in the preloaded track of MidiData. A note can be generated by accessing the channels of the synthesizer directly. For simplicity, MidiSynthesizer uses only the first channel (out of a possible 16) to sound notes. Line 57 invokes Synthesizer method getChannels to obtain all 16 channels from synthesizer, and line 60 sets the default channel to the first channel. A MidiChannel sounds a note by calling its noteOn method with the note number (0–127) and a volume number as arguments. MidiChannel’s noteOff method turns off a note with just the note number as an argument. MidiSynthesizer accesses these MidiChannel methods through method midiNoteOn (lines 87–90) and method midiNoteOff (lines 93–96), respectively.

A synthesizer can use its default instruments to sound notes. Line 54 obtains the default instrument available through the synthesizer or through a default sound bank by invoking Synthesizer method getAvailableInstruments. A sound bank usually has 128 instruments. The instrument in use can be changed by invoking MidiSynthesizer

method changeInstrument (lines 99–105). Lines 103–104 invoke MidiChannel’s programChange method to load the desired instrument program with the bank and program number obtained from patch (line 104) as the parameters. A Patch is the location of a loaded instrument.

Performance Tip 22.5

A program can import more instruments by loading a customized sound bank through Syn- thesizer method loadAllInstruments with a SoundBank object.

By sending MidiMessages to a Synthesizer’s Receiver, a program can invoke the synthesizer to sound notes without using its channels. Sending MidiMessages to a MidiDevice’s Receiver also allows the device’s Transmitters to send these messages to another MidiDevice’s Receiver.

In MidiSynthesizer’s sendMessage method (lines 108–127), lines 112–116 create a new ShortMessage from the parameters of method sendMessage and send the message to the synthesizer’s receiver (line 119). Line 116 of method sendMessage invokes ShortMessage method setMessage to set the contents of the message’s instructions using three int arguments: a command, the note to play and the volume of the note. Method setMessage throws an InvalidMidiDataException if the designated command and parameter values are invalid.

When creating a new ShortMessage using method setMessage, the meaning of the second and third arguments vary depending on the command. Command ShortMessage.NOTE_ON designates the second parameter to be the note number and third argument to be the velocity (i.e. volume) of the note. The ShortMessage.PROGRAM_CHANGE command designates the second argument as the instrument program to use and ignores the third argument.

Line 119 sends the created ShortMessage to the synthesizer’s receiver by calling Receiver method send with the MidiMessage and a time stamp as its arguments. MidiSynthesizer does not deal with the complexity of MIDI synthesis timing. The receiver sends a value of -1 for the time stamp parameter to designate that the time stamp should be ignored. The sequence recorder in class MidiRecord takes care of timing issues when it receives the messages.

Up to this point, we have discussed the tools needed to create our MIDI piano. In brief synopsis, class MidiDemo (Fig. 22.10) uses class MidiSynthesizer to generate sounds and to access channels and instruments. MidiDemo uses MidiData to playback MIDI files and access MIDI track information. MidiRecord provides the recording function for MidiDemo, which receives messages from MidiSynthesizer.

22.7.4 Class MidiDemo

We now present class MidiDemo (Fig. 22.10), which provides the GUI for our piano as well as other GUI components to control the capabilities of this example.

Using a for loop, utility method makeKeys (lines 86–155) in class MidiDemo creates 64 buttons that represent 64 different piano keys. Whenever the mouse hovers over a key, the program sounds the designated note. Method makeKeys arranges the keys at the bottom of the frame using each button’s setBounds method (line 106) to designate the location and size of the buttons. The program arranges the buttons horizontally according to their index in array noteButton.

53

54// constructor for MidiDemo

55public MidiDemo()

56{

57super( "MIDI Demo" );

58

59container = getContentPane();

60container.setLayout( new BorderLayout() );

62// synthesizer must be instantiated to enable synthesis

63midiSynthesizer = new MidiSynthesizer();

64

65// make piano keys

66makeKeys();

67

68// add control panel to frame

69controlPanel = new JPanel( new BorderLayout() );

70container.add( controlPanel, BorderLayout.NORTH );

74// add button panel to frame

75buttonPanel = new JPanel( new GridLayout( 5, 1 ) );

76controlPanel.add( buttonPanel, BorderLayout.EAST );

78// make GUI

79makePlaySaveButtons();

80makeRecordButton();

81makePianoPlayerButton();

83 } // end constructor

84

85// utility method making piano keys

86private void makeKeys()

87{

88// panel containing keys

89JPanel keyPanel = new JPanel( null );

90container.add( keyPanel, BorderLayout.CENTER );

92// piano keys

93noteButton = new JButton[ MAX_KEYS ];

95// add MAX_KEYS buttons and what note they sound

96for ( int i = 0; i < MAX_KEYS; i++ ) {

97

98 final int note = i;

99

100 noteButton[ i ] = new JButton();

101

102 // setting white keys

103 noteButton[ i ].setBackground( Color.white );

104

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 2 of 14).

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 3 of 14).

157// set up configuration controls

158private void makeConfigureControls()

159{

160JPanel configurePanel =

161 new JPanel( new GridLayout( 5, 1 ) );

162

163 controlPanel.add( configurePanel, BorderLayout.WEST );

164

165 instrumentBox = new JComboBox(

166 midiSynthesizer.getInstruments() );

167

168 configurePanel.add( instrumentBox );

169

170// register an ActionListener for instrumentBox events

171instrumentBox.addActionListener(

188JLabel volumeLabel = new JLabel( "volume" );

189configurePanel.add( volumeLabel );

190

191 volumeSlider = new JSlider(

192 SwingConstants.HORIZONTAL, 5, 80, 30 );

193

194// register a ChangeListener for slider change events

195volumeSlider.addChangeListener(

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 4 of 14).

209

210 configurePanel.add( volumeSlider );

211

212JLabel tempLabel = new JLabel( "tempo" );

213configurePanel.add( tempLabel );

214

215 resolutionSlider = new JSlider(

216 SwingConstants.HORIZONTAL, 1, 10, 1 );

217

218// register a ChangeListener slider for change events

219resolutionSlider.addChangeListener(

234resolutionSlider.setEnabled( false );

235configurePanel.add( resolutionSlider );

237 } // end method makeConfigureControls

238

239// set up play and save buttons

240private void makePlaySaveButtons()

241{

242playButton = new JButton( "Playback" );

244// register an ActionListener for playButton events

245playButton.addActionListener(

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 5 of 14).

261buttonPanel.add( playButton );

262playButton.setEnabled( false );

264 listenButton = new JButton( "Play MIDI" );

265

266// register an ActionListener for listenButton events

267listenButton.addActionListener(

298// register an ActionListener for saveButton events

299saveButton.addActionListener(

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 6 of 14).

313 } // end ActionListener

314

315 ); // end call to method addActionListener

316

317buttonPanel.add( saveButton );

318saveButton.setEnabled( false );

320 } // end method makePlaySaveButtons

321

322// make recording button

323private void makeRecordButton()

324{

325recordButton = new JButton( "Record" );

327// register an ActionListener for recordButton events

328recordButton.addActionListener(

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 7 of 14).

385// create Piano Player button and functionality

386private void makePianoPlayerButton()

387{

388pianoPlayerButton = new JButton( "Piano Player" );

390// register an ActionListener for pianoPlayerButton events

391pianoPlayerButton.addActionListener(

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 8 of 14).

438// inner class handles MIDI timed events

439private class TimerHandler implements ActionListener {

441// simulate key note of event if present, jump to next

442// event in track and set next delay interval of timer

443// method invoked when timer reaches next event time

444public void actionPerformed( ActionEvent actionEvent )

445{

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 9 of 14).

479// determine which note to sound

480// according to MIDI messages

481private void noteAction()

482{

483// during Note On message, sound note and press key

484if ( midiData.getEventCommand() ==

508// receiving Note Off message will sound off note

509// and change key color back to white

510else

Fig. 22.10 MidiDemo provides the GUI than enables users to interact with the application (part 10 of 14).

Look-and-Feel Observation 22.3

To arrange GUI components at specific locations without the help of Layout Managers, set the layout of the panel containing the components to null. By default, a JPanel sets a FlowLayout LayoutManager when the panel is instantiated with no arguments.

Lines 110–151 register MouseListeners for each piano-key button. The program calls method mouseEntered (lines 116–132) when the mouse hovers over that button. If the program is not in recording mode, method mouseEntered directly accesses the channels in class MidiSynthesizer to sound the note (lines 125–127). Otherwise, method mouseEntered invokes MidiSynthesizer’s sendMessage method to send a note message to the synthesizer and to the recording device (lines 119–122). Lines 130–131 set the button’s background color to blue to indicate that the note is being played. When the mouse is no longer hovering over the button, the program calls method mouseExited (lines 135– 147) to turn off the note and change the button’s background to its original color.

Out of the possible 128 notes, only the middle 64 notes are accessible via the piano in the example. The range of notes can be changed by modifying the constants in line 52. Constant FIRST_NOTE is the value of the first key and the sum of FIRST_NOTE and MAX_KEYS is the value of the last key. Constant MAX_KEYS specifies the number of piano keys to create.

Class MidiDemo invokes method makeConfigureControls (lines 158–237) to setup the program’s MIDI controls, which consist of an instrument selector JComboBox, a user-synthesis volume changer JSlider and a “piano player” tempo changer JSlider. When users select an instrument, the program calls instrumentBox method actionPerformed (lines 177–182) to change to the selected instrument program by invoking MidiSynthesizer method changeInstrument with the selected instrument index as the parameter.

When users drag the volume slider, the program calls volumeSlider method stateChanged (lines 201-204) to change the volume. Note that changing the volume affects only the volume of user-synthesized MIDI notes. When users drag the tempo slider, the program calls resolutionSlider’s stateChanged method (lines 225-228) to set the tempo.

Invoking method makePlaySaveButtons (lines 240–320) sets up the Play MIDI, Playback and Save buttons. Clicking Play MIDI invokes the actionPerformed method of the listenButton (lines 273–288) to playback an opened MIDI file in its entirety using class MidiData. Line 275 obtains a file from a file-chooser dialog using MidiDemo method getFile (lines 549–562). Lines 280–284 initialize and play the MIDI file using the instantiated midiData object. When the user clicks Playback, line 254 plays the recorded MIDI. This button is enabled only if some recording took place. Clicking the Save button allows the user to save the recorded sequence to a file (lines 307–310).

Method makeRecordButton (lines 323–383) creates the Record button and a listener for it. Clicking the button when it is set to recording mode (line 337) creates a new recorder using class MidiRecord (lines 339–348). If a recorder has already been created, line 351 invokes MidiRecord’s makeTrack method to make a new track for object midiRecord. When recording starts (line 353), lines 356–360 turn the recording button into a stop button and disable the playButton temporarily. When users stop the recording by clicking the recording button again, the GUI returns to its state prior to recording and the user can playback and save the MIDI sequence (lines 365–373).

Chapter 22 Java Media Framework and Java Sound (on CD) 1315

“Piano Player”

The “piano player” feature of this example synchronizes the playing of a note with the highlighting of the corresponding key. The program first obtains a MIDI track from a user-spec- ified MIDI file using class MidiData. A Timer synchronizes the MIDI events in the MIDI sequence. When the Timer reaches a MIDI event’s time stamp, it resets its delay to the time stamp of the next event to synchronize. The delay of a timer is the period of time that it waits before it causes an event.

The “piano player driver” responsible for synchronization is located in main class MidiDemo. Method makePianoPlayerButton (lines 386–436) loads the MIDI file and initializes the Timer that handles the timing of musical notes. As with the listenButton, method actionPerformed (lines 397–428) of makePlayerPianoButton uses class MidiData to load MIDI data. Lines 399–408 open a file from a file dialog box and load the MIDI data from the file. Line 410 invokes MidiData’s initializeTrack method to obtain the longest track from the loaded MIDI and to obtain the first MIDI event message from the track.

Line 414 invokes MidiData’s getResolution method (lines 67–70 in Fig. 22.7) to obtain the default tempo of the “piano player”. Lines 418–420 instantiate a new Timer object that drives the playing of notes at the time of each MIDI event. Line 419 sets the delay of the timer to be the duration of the first MidiEvent in the track by invoking MidiData’s getEventDelay method. To allow user-specified tempo changes to the “piano player,” line 424 enables the tempo slider and line 419 sets the delay of the timer to be multiplied by the value of resolution. The resolution variable, which specifies the tempo of “piano player,” is multiplied by the event interval time to obtain a new interval time used by pianoTimer. PianoTimer’s delay is set only for the first event’s duration to start the timer. Later, inner class TimerHandler (lines 439–477) method actionPerformed (lines 444–475) resets the timer’s delay to the next MidiEvent’s duration (lines 472–473). Line 426 starts the pianoTimer.

At the time of the next MIDI event, class TimerHandler’s actionPerformed method synthesizes a note from the track and “presses” a key on the piano. Using pianoTimer and its event handler, the program iterates through the track, simulating note events when it encounters appropriate MidiEvents. Inner class TimerHandler drives the synchronization by iterating through all the MidiEvents on a given track and calling method actionPerformed to play the piano. Line 451 of method actionPerformed invokes utility method noteAction (lines 481–530) to sound the note and to change the color of the specific piano key, given that the event contains a note message and that the note is within the range designator for the piano keys in this example.

Lines 484–485 of method noteAction determine whether the current MidiEvent contains a note command by invoking method getEventCommand of class MidiData. Lines 488–489 invoke MidiData method getNote to determine whether the note number specified in the noteMessage is a valid note within the range of possible piano keys, specified in constants FIRST_NOTE and MAX_KEYS (line 52).

If the command is a ShortMessage.NOTE_ON command and is within range of the piano keys, the synthesizer receives the specified note message (lines 497–498) and the corresponding piano key’s color becomes red for the duration of the event (line 494). Line 491 obtains the corresponding piano button number (lastKeyOn) that should become red.