 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
Java Forum / General / July 2006Oscilloscope trace of sound signal
How fragile is this AudioSystem code? 'AudioTrace' aims to draw an oscilloscope style trace of the sounds currently being played through the PC's audio system, but it is based on the assumption that it is getting a stereo, 16 bit signal. E.G. the command line output on my system is.. Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian You can browse the files here.. http://www.physci.org/test/oscilloscope/ It contains the source code (an SSCCE, of course), as well as a launchable jar file of the same (for those even more pushed for time!). In subdirectory 'screenshots' you can find typical output of bird song and Veruca Salt's 'Only You Know', as well as a JPEG of the intro dialog. In 'sounds' I have included the two samples from which the birdsong screenshots were grabbed. (I do not have any distribution rights for the music.) Do I need to worry about other formats of sound? What output do you get in the JOptionPane/command line? Does the screen output look like an oscilloscope trace from an harmonic analog signal? ( Note, the code will become an open source screensaver as part of the saverbeans project, and can currently be downloaded from URL above. BTW - the 'meaty part' is in the painComponent() method of AudioPlotPanel, a default access class included in AudioTrace.java ) AndrewT. > 'AudioTrace' aims to draw an oscilloscope style trace > of the sounds currently being played through the PC's [quoted text clipped - 4 lines] > Format: PCM_SIGNED 44100.0 Hz, 16 bit, > stereo, 4 bytes/frame, little-endian Bad news from here, I'm afraid. I've tried it on two machines, one a WinXP laptop and the other a Win2K deskside box. In both cases the printed output is just as in your example, but the display doesn't reflect the sounds that are playing. On one machine the display is just a flat line, on the other there is a certain amount of jitter which I /guess/ is quiet noise from whatever device it is actually listening to. I tried to find something that would send sound to whatever that device is (or might be), but all the sources on this box (CDs DVDs mp3s midi files) all seem to end up being "rendered" via what Windows calls "Wave". I have only one physical sound card in either machine. Let me know if you want me to run any more diagnostics. -- chris I wrote: > I tried to find something that would send sound to whatever that device > is Ah, found it. It's listening to the microphone input. Maybe I misunderstood what the program was intended to do. -- chris > I wrote: > [quoted text clipped - 3 lines] > Ah, found it. It's listening to the microphone input. Maybe I misunderstood > what the program was intended to do. You did not misunderstand, I stuffed up. I had been lulled into a false sense of security by my own system, in which both lines found seemed to be streaming the exact same data. If you would be so kind, try compiling the source* with.. static boolean allLines = true; ..to get a trace of every line. My original idea was to display all lines to the user and ask them to 'choose the interesting looking one', but the seemingly identical traces on my own system caused me to recompile it false and forget about it. * I will upload a new version of the jar that displays all lines, but I will not be able to upload it till tomorrow. ( Thanks for your attention thus far. :) Andrew T. > > I wrote: > > [quoted text clipped - 3 lines] > > Ah, found it. It's listening to the microphone input. Maybe I misunderstood > > what the program was intended to do. ... > * I will upload a new version of the jar that displays all > lines, but I will not be able to upload it till tomorrow. Or maybe tonight.. http://www.physci.org/test/oscilloscope/audiotraces.jar The new jar is user configurable to show all lines found (the default) and change the no. of frames sampled (default 256, like before). I also added a green border (shrugs). The new source has replaced the old, available here.. http://www.physci.org/test/oscilloscope/ Andrew T. > If you would be so kind, try compiling the source* with.. > static boolean allLines = true; > ..to get a trace of every line. Done that. The output now is: Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian I added some extra tracing too, and it appears that the first line corresponds to some sort of virtual sound device maintained by Windows which is an alias for the second. Thge second is the real sound device in this laptop. I remembered that I had a USB headset and plugged that in too (which automatically adds a new hardware sound device to Windows and makes the virtual device point to it), and the output changed to: Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian Format: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian In both cases the top two lines produced similar (but not actually identical) traces -- clearly showing the same sounds, but possibly subject to some sort of timing differences. Which may in part be because the program pushes the machine to 100% CPU so maybe the software in the virtual device can't quite keep up. In both cases the only lines displayed were connected to microphones (two different mics in the second test). One more test. Unplugging the headset (and so reverting to the original 1 virtual + 1 real device), and activated the recording control aspect of the Windows (ultra confusing) volume control applet. It seems that the recording control suggests 4 possible sources of input: Mono Mic Sterio Out Microphone Telphony I tried all of them, and in each case ended up with the same format as above (even the "mono" mic was stereo). Selecting the "Stereo out" option seems to connect the input to the current output (why?) and -- at last -- your program showed the current sound output. Unfortunately this feature seems to be in the device itself rather than software, because the headset driver only offers the "Microphone" option. I'd try this on a Linux box too, but my Linux installation runs in VMWare over a Windows host OS. And fathoming what's going on when we have Java's sound system sitting on top of the Linux sound system (whatever /that/ is), on top of VMWare's peculiar hackery, on top of Windows wierdness, would undoubtedly make my brain explode... -- chris > Selecting the "Stereo out" option seems to > connect the input to the current output (why?) and -- at last -- your > program > showed the current sound output. This is sort of soundcard specific, but most cards have a similar option. They are alternatively called "Stereo Mixer", "Wave Mixer", and I've even seen "What U Hear (tm)" on a SounbBlaster Live!(tm). Sometimes you do indeed want to record "What U Hear(tm)", e.g. to record a voice chat with someone. It's better than holding the microphone up close to your speakers. Other times, you want to record what you speak into a microphone, and not record any of the sound that may be coming from your computer. So you'd pick "Microphone". - Oliver > > Selecting the "Stereo out" option seems to > > connect the input to the current output (why?) and -- at last -- your > > program > > showed the current sound output. Thanks for the comprehensive reply. I'll have a closer look off-line. > This is sort of soundcard specific, but most cards have a similar > option. ... Thanks for the extra info. Still hoping to get test results from more boxes. The most important questions are.. >>> What output do you get in the JOptionPane/command line? >>> Does the screen output look like an oscilloscope trace >>> from an harmonic analog signal? That URL for the quick launch jar is.. >>> http://www.physci.org/test/oscilloscope/audiotraces.jar Andrew T. <snip testing and configuration saga> .... > I tried all of them, and in each case ended up with the same format as above > (even the "mono" mic was stereo). Selecting the "Stereo out" option seems to > connect the input to the current output (why?) and -- at last -- your program > showed the current sound output. I missed that bit in my original reading, and have been working under the assumption that the problem was signal levels.. (shrugs) in any case, I made some significant changes based on the results of a second PC that had 'amplification issues' ..perhaps this version might work better for you. (It is a worry you had to go that deeply into system config. to see anything though..) My message (prepared off-line) is.. Current traces calculate the audio level by looking at the large order byte, ignoring the small order byte completely. * There are some significant changes to the audio trace code, mostly based on the fact that a third machine displayed results very similar to what you described. Most significant in the changes is that the visual 'gain' of the signal can be configured, http://www.physci.org/test/oscilloscope/screenshots/sinetone.png the screnshot shows a gain of 1 as it displays the signal generated by the 'Audio Tone' jar to it's left. Audio Tone is a jar here, http://www.physci.org/test/oscilloscope/tone.jar it generates a more predictable signal - a simple sine tone with a harmonic (if you want). The default volume is 20% (for good reason), but for these screenshots I ramped it up to 90% volume. I suppose you, Chris can especially appreciate how different this will appear at gain 1 on your system. (Having seen it on the 3rd PC, I can vouch it is - quite oddly boring). I am guessing you'll need to push the gain beyond 7 (the default is 5) before you see a trace ranging this big, and then it will be very blocky. * Second, now there is an option to DisplayEachByte** of the four bytes in each audio frame. http://www.physci.org/test/oscilloscope/screenshots/sinetoneallbytes-a.png The size of the small order bytes has been knocked down to 1/8 the display height at 'gain 1', by gain 8 it should be clipping. http://www.physci.org/test/oscilloscope/screenshots/sinetoneallbytes-b.png Here's a screenshot of 'all lines' showing a gain of 1 and seven in the separate trace panels. ** The reason I added this option is because I cannot for the life of me figure how to combine these bytes with the larger order bytes in any sensible way. * Which has become especially important since most of the audio trace detail of two of the three PC's tested, lies in those small order bytes. Increasing the gain can amplify the visual trace to the point of doing something interesting, but I need the small bytes to give the trace any detail. BTW - You can click the trace(s) to toggle the gain through 15 and back to 1. - All the screenshots were done with a fade rate of 100% (clear old traces) in order to simplify the image, the default is a slowish fade for each trace - much prettier. - the byte conversion has been shifted to the 'frameToSignedDoubles' method (bottom of AudioTrace source) - Tooltips galore - most notably on the entire Visual Gain and All Bytes panels. - The default settings are visually complex. To simplify the trace, flip the default values of the check boxes and increase fade rate to 100%. - All files available from http://www.physci.org/test/oscilloscope/ Andrew T. .... > Second, now there is an option to DisplayEachByte** of the four > bytes in each audio frame. > http://www.physci.org/test/oscilloscope/screenshots/sinetoneallbytes-a.png Oops! Visitors cannot directly access the screenshots. Go through http://www.physci.org/test/oscilloscope/screenshots/ and they should be viewable by clicking the links of the directory listing. Andrew T. > I missed that bit in my original reading, and have been working > under the assumption that the problem was signal levels.. > (shrugs) in any case, I made some significant changes based on > the results of a second PC that had 'amplification issues' ..perhaps > this version might work better for you. It's pretty and rather fun, but I'm afraid the basic problem is still there. On my the machine program doesn't read output audio, only input (mic, etc). I can still tell the audio device driver to connect the audio input to its audio output, but that's the only way to capture audio output. If I wind the gain up then I just see random noise. > Second, now there is an option to DisplayEachByte** of the four > bytes in each audio frame. > ** The reason I added this option is because I cannot for the life > of me figure how to combine these bytes with the larger order > bytes in any sensible way. I modified the last part of frameToSignedDoubles(), the bit after the diagram, to read: ================= if (displayEachByte) { // transform this 'small order' bytes to doubles // ranging from -1/8 to 1/8 d[0] = b[1]/256.0; d[1] = b[3]/256.0; d[2] = b[0]/(256.0*smallByteDampingFactor); d[3] = b[2]/(256.0*smallByteDampingFactor); } else { for (int cc = 0; cc < channels; cc++) { d[cc] = (b[cc*2+1]*256 + b[cc*2])/32678.0; } } return d; ================= and it seems to work correctly. Of course that's stll hard-wiring the assumption of 16-bit little-endian audio. -- chris .... > I modified the last part of frameToSignedDoubles(), the bit after the diagram, > to read: ... > else { > for (int cc = 0; cc < channels; cc++) { > d[cc] = (b[cc*2+1]*256 + b[cc*2])/32678.0; > } ... > and it seems to work correctly. That's not the way it looks from here. If I use the tone.jar to make a sine tone, the traces contain a distinct 'sawtooth' effect from the small bytes. This made me think that maybe we should /subtract/ them, but while that reduced the number of 'sawteeth' per cycle - it did not remove them. ..Aaaargh! .... (moved from above) .. > ...I'm afraid the basic problem is still there. > On my the machine program doesn't read output audio, only input (mic, etc). I > can still tell the audio device driver to connect the audio input to its audio > output, but that's the only way to capture audio output. That is a pity, and a major detriment to the saver. (I cannot see a way around it, unless the end user is both prepared and able to reconfigure their sound for the trace to work - and that is not somethiong -I'd- do for a sound visualisation.) Still - it will work 'out of the box' for some people, and potential users will be able to check the PC quickly with a WebStarted preview. I will mention it in the program documentation, as well. Andrew T. > > for (int cc = 0; cc < channels; cc++) { > > d[cc] = (b[cc*2+1]*256 + b[cc*2])/32678.0; [quoted text clipped - 6 lines] > If I use the tone.jar to make a sine tone, the traces contain > a distinct 'sawtooth' effect from the small bytes. Ah. My mistake, sorry. Try: d[cc] = (b[cc*2+1]*256 + (b[cc*2] & 0xFF))/32678.0; the point is that we have two /unsigned/ values which we then want to normalise to the range +/- 2**15 (roughly). Just treating the high byte as signed will do most of the work for us, but we have to convert the second byte to unsigned before adding it in. I had seen the "jaggies" but assumed it was just noise -- funny how fixing the code makes the sound-card work better ;-) > That is a pity, and a major detriment to the saver. It is unfortunate. I think it's probably all part-and-parcel with the DRM thing. I now seem to recall that Microsoft don't provide an API for retrieving sounds as they are played at all (There is a way of getting a deliberately low-resolution approximation for "visualisations" and the like). It doubly unfortunate that in the cases where the device driver provides a loopback, the MS interface to turning it on is about as non-intuitive as can easily be imagined. Blame MS, and -- after all, this /is/ Java -- hope that it works better on more open systems. -- chris ... > > If I use the tone.jar to make a sine tone, the traces contain > > a distinct 'sawtooth' effect from the small bytes. > > ..Try: > > d[cc] = (b[cc*2+1]*256 + (b[cc*2] & 0xFF))/32678.0; Yes! You are a champion, Chris. Thank you. .... > I had seen the "jaggies" but assumed it was just noise -- funny how fixing the > code makes the sound-card work better ;-) LOL! > > That is a pity, and a major detriment to the saver. > > It is unfortunate. I think it's probably all part-and-parcel with the DRM > thing. Huh? I had not had much internet time recently and was tuning out any references to MS, so I was unaware of DRM till now. Looks horrid - yet another reason to stick with my older XP until I can find the need/motivation to switch to *nix. >..I now seem to recall that Microsoft don't provide an API for retrieving > sounds as they are played at all (There is a way of getting a deliberately [quoted text clipped - 6 lines] > Blame MS, and -- after all, this /is/ Java -- hope that it works better on more > open systems. If you're right, this saver will become less useful on Win. systems as time goes by and people upgrade. That would be a distinct pity. :-( Andrew T. (no sound signal detected in Java based saver) > It is unfortunate. I think it's probably all part-and-parcel with the DRM > thing. I now seem to recall that Microsoft don't provide an API for retrieving > sounds as they are played at all (There is a way of getting a deliberately > low-resolution approximation for "visualisations" and the like). I am writing the documentation for the saver, and want to inform the user why it fails in as few words as possible*. Would you check if 'DRM' is found in your Windows help? "* If you only see a flat line or dot for each channel, it might be due to running a Unix/Linux system with no audio loop-back installed, or a Windows system with DRM." Andrew T. > (no sound signal detected in Java based saver) > [...] > I am writing the documentation for the saver, and want to > inform the user why it fails in as few words as possible*. > Would you check if 'DRM' is found in your Windows help? Nothing. (What would you expect ;-) > "* If you only see a flat line or dot for each channel, > it might be due to running a Unix/Linux system > with no audio loop-back installed, > or a Windows system with DRM." I'm not sure it's a good idea to mention DRM -- it isn't that some DRM system is actively /preventing/ audio monitoring, it's just that (as I understand it) MS don't see a valid need to insist that audio kit /provides/ the feature (or, unfortunately, a convenient way to turn it on when available). -- chris > > (no sound signal detected in Java based saver) > > [...] [quoted text clipped - 11 lines] > I'm not sure it's a good idea to mention DRM -- it isn't that some DRM system > is actively /preventing/ audio monitoring, My research at microsoft suggested they provided sound only to 'registered and correct' plug-ins that themselves fed the sound only to other such 'r&c' plug-ins. Failure of a plug-in supplier to adequately protect the audio bytes could result in their license (to access the sound system) being revoked. I very much doubt that Java in general would be recognised a trusted sink for sound-bytes given that philosophy. >..it's just that (as I understand it) > MS don't see a valid need to insist that audio kit /provides/ the feature (or, > unfortunately, a convenient way to turn it on when available). But you did find a way to ..re-route the sound from within the (DRM) OS? No external software or hardware required? ( Note that adding software or hardware to circumvent the effects of a DRM system might be considered illegal, in some places. ) Andrew T. [me:] > > I'm not sure it's a good idea to mention DRM -- it isn't that some DRM > > system is actively /preventing/ audio monitoring, [quoted text clipped - 4 lines] > supplier to adequately protect the audio bytes could result in > their license (to access the sound system) being revoked. Hmm, then it's gone further than I realised :-( > But you did find a way to ..re-route the sound from within > the (DRM) OS? No external software or hardware required? Not exactly. The OS and Windows GUI provide the configuration screens to allow me to tell the hardware (which came with the machine) to set up a loopback. Presumably the OS asks the device what inputs it has available, and it configures the "select an input" window from that. If the device includes a loopback then that is listed along with all the rest. -- chris ... > > But you did find a way to ..re-route the sound from within > > the (DRM) OS? No external software or hardware required? > > Not exactly. The OS and Windows GUI .. Isn't the 'Windows GUI' simply a GUI for configuring 'The OS'? I do not understand the distinction. >...provide the configuration screens to allow > me to tell the hardware (which came with the machine) to set up a loopback. > Presumably the OS asks the device what inputs it has available, and it > configures the "select an input" window from that. If the device includes a > loopback then that is listed along with all the rest. OK - I might revise the instructions based on that. ..but did the help find 'DRM'? How does a Win. user find out if DRM is enabled on their PC? Andrew T. > ... >> > But you did find a way to ..re-route the sound from within [quoted text clipped - 4 lines] > Isn't the 'Windows GUI' simply a GUI for configuring 'The OS'? > I do not understand the distinction. I'm guessing the tool that Chris was using is called "Volume Control". It comes with every installation of Windows (AFAIK), but I wouldn't say it's "core" to the OS. You could probably delete that program and no harm would come to your system (except that you might have difficulty controlling the volume from then on, unless you find an alternative, equivalent program). >>...provide the configuration screens to allow >> me to tell the hardware (which came with the machine) to set up a [quoted text clipped - 8 lines] > ..but did the help find 'DRM'? How does a Win. user > find out if DRM is enabled on their PC? Windows XP SP2 help searches locally, and in the Mirosoft Knowledge Base. Locally, there are zero results, and in the MSKB, there are 15 results at the time of writing. The topics in the MSKB mostly deal with Windows Media Player, which makes sense, because that's the only Microsoft application on WinXP SP2 which actually uses DRM, AFAIK. DRM is a technology, like XML. It isn't "enabled on a PC" [*]. Applications can be written to be aware of DRM, or they might not be. If they weren't written to be DRM aware, then they won't be able to decode DRM protected files (just like a program which doesn't know what XML is probably won't be able to parse them). Windows XP itself is not DRM aware, but Vista is planned to be. Media Player *IS* DRM aware. So that means it's possible to construct media files that can only be played for a period of 24 hours after it is first downloaded, for example (to simulate some sort of online movie rental store). Again WinXP isn't DRM aware, so it won't automagically delete the file after 24 hours. It's just that Media Player will refuse to play the file after that period. - Oliver [*] You might be thinking of Trusted Computing or TC, which is sort of to hardware what DRM is to software. Intel is considering making a line of Trusted CPUs (tm), but they seem to be holding off, since the public opinion on TC has been mostly negative so far. TC is present in the XBox 360, and is planned to be in the Playstation 3. .. > Windows XP itself is not DRM aware, but Vista is planned to be. Media > Player *IS* DRM aware. <snip lots more> Thanks for the comprehensive explanation - that has cleared up a lot of misconceptions I had. Andrew T. Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.