Sampling Short Sequences From Long MP3 Recordings? 278
mehl writes "I am a professor for social psychology at the University of Arizona and I am looking for help with finding / developing a special program. In my research, I ask participants to carry around a digital voice recorder while they go about their normal lives. The voice recorder then tracks the ambient sounds in their environments and produces an 'acoustic log' of a person's day. We then use these ambient sound recordings as source data for various person perception studies. For privacy reasons, we are required to sample brief snippets of ambient sounds instead of recording an entire day continuously ('Big Brother is listening to you...'). So far, we have achieved this by modifying the hardware of a digital voice recorder (triggering it with an external microchip). With the high turn-over in player models, however, this strategy has turned out to be short-sighted (every half a year we have to build a new chip). I am thinking about switching strategy, recording continuously in the first place (no problem with the current generation of flash memory) and then sampling (random) snippets after the fact from the continous recordings. Does anybody know of an existing program that can randomly (or pseudo-randomly; e.g., 30 sec every 10 min) and automatically sample short sequences from a day-long (18 hours) mp3 recording? What would it entail to develop such a program (for Windows)?."
Mp3 splitter (Score:3, Informative)
mpgedit is what you want (Score:4, Informative)
mpgedit has a -e flag so you could do
mpgedit -e 1:20-1:30 -e 600:20-601:10 -f file.mp3
Would grab from 1 minute 20 seconds to 1 minute 30 seconds, and from minute 600, 20 seconds to minute 601, 10 seconds of file.mp3 into file_1.mp3 and file_2.mp3
I use it to cut the 2 hour mp3 of the bbc news hours into 5 minute chunks so my mp3 player will let me skip segments.
With a bit of scripting you could do random cuts of any size mp3.
It claims to work under windows, but I have not tested it there.
Audacity? (Score:4, Informative)
M
Re:Audacity? (Score:5, Funny)
Re:Audacity? (Score:4, Insightful)
Re:Audacity? (Score:3, Insightful)
dd if=/dev/sound bs=[size of 10 second sample] count=1 | gzip -c >> output.raw
sleep [time between samples]
done
Whaddya mean, you must use windows at any cost?
Re:Audacity? (Score:5, Insightful)
What amazes me is that this precisely the sort of thing to do in software, yet the submitter contends that up until now they've done it with a custom designed and custom built chip.
I mean, the guys at http://rockbox.haxx.se could do it in about 15 minutes, and provide a GUI on the MP3 recorder.
Naturally, this is easier to do when MP3 recorders are running open source software, so to an extent this is a rant at all the manufacturers of computing appliances who see a competive advantage in closed source: you lost my purchase, and who knows how many purchases from this professor and while requiring him to shell out even more for a custom chip to do in hardware what is ridiculously easy to do in software (as the parent poster shows by using that complex and little-known software technique, calling sleep()).
I'm not criticizing the parent: his code does the right and the simplest thing; the problem is with manufacturers whose use of closed source precludes such elegantly simple solutions.
A big part of the reason that the IBM PC took off in such a big way in the early 1980s was becuase it was open source: not the code to MS-DOS, but the architechtural specification of the hardware itself. This cost IBM money when "clone"-makers were able to produce "monkey copies" of the PC, but that very competetion made PCs much more popular, and soon dominant over other (possivly techically superior) brands like Ape or Amiga. The open specs also allowed a myriad of other companies to offer add-on hardware from co-processors to graphics cards, and even allowed software authors to optimize for the IBM-PC (who here is old enough to remember what peripheral's memory was mapped to address B800, and the advantages to manipulating the data there directly?)
Did opening the IBM-PCs architecture finally cost or benefit IBM. We'll never know. But it is likely that the PC market, and thus the market for home PCs and peripherals, and software for home PCs and the World Wide Web, would never have been near as big without IBM's decision to open the PC even to competitors.
Manufacturers of closed-source appliances would do well to consider what they gain, and what they lose by preventing customers from fully using their products. (The careful reader will see that the antecedent of "their" in the previous sentence is ambiguous; the astute reader will understand why: just whose product is it, the company that makes it or the customer who buys it?)
Yeah I know a program (Score:4, Informative)
Some tools (Score:3, Interesting)
Might want to check with an acoustics lab.
Try http://www.ee.sunysb.edu/~cspv/CSPV.html or something similar.
Simple... (Score:5, Funny)
Pick up Microsoft Visual C++ then look at their time and sound librar.. uh.. oh.. IT colour scheme hurt brain... cannot continue.. blarerhfdsl jjjjjjjjjjjjjj fjwkef
Re:Simple... (Score:2)
Easy... (Score:5, Funny)
He frequently samples other artists' work and then makes millions. Reminds me of an archived Onion article, which you now must pay for
Re:Easy... (Score:5, Funny)
Re:Easy... (Score:5, Funny)
Re:Easy... (Score:2)
Use mp3split (Score:5, Informative)
Re:Use mp3split (Score:3, Interesting)
Re:flame bait (Score:5, Funny)
Privacy concerns... (Score:4, Insightful)
Re:Privacy concerns... (Score:2, Interesting)
Re:Privacy concerns... (Score:4, Funny)
What's the difference? Either way, all you'll end up with is a bunch of MP3s of people saying "Fuck!" and "Goddamnit!" and "Aaw, shit!"...
Re:Privacy concerns... (Score:3, Informative)
Re:Privacy concerns... (Score:5, Funny)
Perl can (Score:2, Insightful)
This is really simple (Score:5, Informative)
Re:This is really simple (Score:2)
Expect that mpeg layer 3 audio uses a bit reservoir [wlv.ac.uk], even in constant bitrate encoding (the bits allocated to any one frame of 1152 samples changes).
The IMDCT used by layer 3 also means that the samples out depend on each frame and the previous frame, even if the bit reservoir feature is not in use.
Java? Python? PERL? (Score:3, Insightful)
Does anybody know of an existing program that can randomly (or pseudo-randomly; e.g., 30 sec every 10 min) and automatically sample short sequences from a day-long (18 hours) mp3 recording? What would it entail to develop such a program (for Windows)
10 a = rand number
20 b = a + 30 seconds
30 open MP3 with appropriate sound API, get sound between a and b
40 save a and b to a table so you don't use those values again
50 goto 10
Why does this seem absurdly simple to me? I think just about any modern language with a decent set of libraries will be able to handle this, if there isn't a shareware app out there already. In any case I couldn't imagine that it'd take more than half a day or so to do this in Java or Python.
really random (Score:5, Funny)
Re:Java? Python? PERL? (Score:2)
I would personally be interested in hearing about the results of these studies, and am glad that it was on the front page.
While it may be simple to you... (Score:3, Funny)
Re:Java? Python? PERL? (Score:5, Insightful)
It probably probably sounds simple to you because it is. However, this professor said that pyschology was his field, not computer science or programming. I personally think it's great that someone that far out of a tech field has found an useful application for modern technology AND that he decided to check with those most likely to understand the finer details of the techonology. Maybe
Re:Java? Python? PERL? (Score:3, Informative)
Oddly enough, I speak from experience, being both an okay programmer, and having done almost exactly this in the past using perl. We were gathering 30 to 60 second samples of tracks off of audio CDs at different offsets (dependant upon the type of music) for online streaming. CDs came in one side of the process, were ripped, sampled, and c
Re:Java? Python? PERL? (Score:5, Informative)
Agreed..
In any case I couldn't imagine that it'd take more than half a day or so to do this in Java or Python.
Or five minutes in Perl with MP3::Splitter [cpan.org]:
Re:Java? Python? PERL? (Score:5, Funny)
Me: In any case I couldn't imagine that it'd take more than half a day or so to do this in Java or Python.
You: perl -MMP3::Splitter -e 'mp3_split($_,{},[ rand(64800), 30 ], ...) for @ARGV' filename.mp3
I would like to take this opportunity to say that Perl guys are smartasses.
Re:Java? Python? PERL? (Score:3, Funny)
Re:Java? Python? PERL? (Score:4, Funny)
Re:Java? Python? PERL? (Score:3, Funny)
10 a = number of submissions
20 b = INT rand number from 0->1 * a
30 publish submission number b
Re:Java? Python? PERL? (Score:2)
Re:Java? Python? PERL? (Score:2)
Re:Java? Python? PERL? (Score:2)
10 a = rand number
20 b = a + 30 seconds
30 open MP3 with appropriate sound API, get sound between a and b
40 save a and b to a table so you don't use those values again
50 goto 10
**
eh? he goes back to the '10' where he makes a totally new random number(not increment the old). the pseudo-program choosing random points and recording 30 seconds from them, in random order.
besides, the whole point was to show that doing a script for doing this is a LOT simpler than hacking some microprocessor to make the mp3 r
...recording continuously in the first place... (Score:2, Insightful)
I see now.... (Score:2, Informative)
Re:...recording continuously in the first place... (Score:3, Insightful)
For those of you with privacy concerns and questioning the use of Windows for the target platform, I'd like to submit that maybe the intent is to let those participating in the study run the program to extract the desired snippets and send them in, which mitigates somewhat the privacy concerns and probably *would* require that the app run on Windows.
Not trying to point the finger at FerretFrottage...just a convenient reply point...
Why the artificially imposed limit of Windows only (Score:2, Insightful)
Maybe it's because my background is in the *nix world, but it'd be a half-day project to get this up and running done on a *nix box with something free from sourceforge.
Usually one wants to design the solution to fit the problem, not to introduce more compl
Modified PyMP3Cut? (Score:5, Informative)
Just cut it (Score:2)
Does it have to be windows? (Score:2)
Just script it to open the file, play back the predetermined segments and capture the audio to sequentially numbered sound files.
Easy!
DON'T DECODE (Score:5, Insightful)
If you set things up properly (namely limiting the use of the interframe bit reservoir), then there are many utilities which will allow you to pull out specific frames from within an MP3 file. This should both be much faster from a processing standpoint, and not incur more data loss from two encodings.
Re:DON'T DECODE (Score:2, Interesting)
dd if=/home/britney/oops.mp3 of=/home/kazaa/sample.mp3 bs=(not sure?) count=(duration)
Meh, whatever, you get my point. You specify the frame size in the bs= parameter, the id3,etc, offset in the "where to start" parameter, and go from there.
Or just use one of the metric assload of utilities out there that already do this.
Re:DON'T DECODE (Score:4, Informative)
Regards,
Steve
Pick a recorder and stick with it (Score:2)
Not difficult (Score:5, Informative)
http://www.puchalla-online.de/cutmp3.html
There's lots of pure windows code to do this too:
http://www.programurl.com/software/cutter.htm
But if you want to code this yourself, there's some excellent Perl libraries for managing MP3:
http://search.cpan.org/~nuffin/MPEG-Audio-Frame
(and most directly speaking to what you're working on)
http://search.cpan.org/~ilyaz/MP3-Splitter-0.02
It's not too bad to use Perl either, especially with the Perl Packager. Given only one host with the full Cygwin Perl install, you can create compiled executables that encapsulate everything you need down to a single file. It rocks!
http://search.cpan.org/~autrijus/PAR-0.85/scrip
I imagine though that you'd eventually want to only analyze random chunks that contain speech, or at least speech like frequency distributions. This is trickier, and I don't know if there's Perl code to do it. Maybe you could investigate Praat's internal scripting language?
http://www.fon.hum.uva.nl/praat/
Praat is pretty mind-bogglingly cool -- it's worth checking out no matter what.
--Dan
P.S. Yes, I've been working on some mildly related stuff. How could you tell?
Um... legal issue??? (Score:2)
Re:Um... legal issue??? (Score:2)
Re:Um... legal issue??? (Score:2)
what batteries do you use? (Score:5, Interesting)
Re:what batteries do you use? (Score:2)
DUH!
actually, there are mp3 players that boast 14 hours of play, im sure there are a few out there than can do well more than that for recording.
Re:what batteries do you use? (Score:2)
I don't think this meets your privacy standards... (Score:4, Insightful)
Would you submit them under subpeona? Will they be destroyed? When? How?
I think anybody in the CS dept could write the program you require. But I am pretty sure you are going to have to keep coming up with a way to do this "pre-record". And exactly why is it so hard to just buy *enough* recorders?
Re:I don't think this meets your privacy standards (Score:3, Informative)
To see what I'm talking about, check out
Re:I don't think this meets your privacy standards (Score:2, Insightful)
I don't know about MP3... (Score:2, Informative)
Visit my website to contact me if you need further help/code assistance.
~D
Sounds very important (Score:2, Funny)
Just ask the RIAA (Score:5, Funny)
Re:Just ask the RIAA (Score:4, Funny)
Why change hardware? (Score:3, Insightful)
And the idea of recording an entire day and sampling seems terrible in terms of both privacy and efficiency. If you can record a day's samples in a $75 MP3 flash-based player plus a tiny circuit that randomly presses the record button twice every so often, why bother using a $400 hard-drive based MP3 player, recording an entire day, then copying that day to a desktop computer, then sampling out random tiny bits of the day.
easiest solution (Score:2)
get a microphoned pocketpc with 128mb ram. price = $200 on ebay.
do a little programming, taking care to make note of the fact that you will most likely need to "wake up" the machine from a sleep state, start recording, and shut down. time to program for an experienced pocket pc programmer: 4 hours.
done.
A use for undergrads (Score:2, Insightful)
How about for mac? (Score:2)
Import the song into iTunes
repeat until end-of-song
select random song chunk
paste into quicktime player
end repeat
save quicktime player document
import back into iTunes
From what I understand, you can sort of do the same thing with that iTunes COM library?
A few things. (Score:4, Interesting)
You are at University.
Is there a music lab?
Is there a Computer Science dept?
Is there an electronics dept?
If you can answer yes to 3 of these question most of your problems can solve themselves.
Talk to the Deans of those departments, explain your needs and suggest that the students in these depts may participate in the construction of what you need for their labs or projects.
I'm sure you would have students banging down the door to work on a project.
My point is, use what you have available.
I read a few posts down that someone "couldn't handle, dropped out of college because of professors like this". Well to me this sounds like a lesson with a deeper meaning than just some sort of useless project.
Perhaps it is an effort to show how even the simplest of experiments present difficult logistical problems.
What would the RIAA think? (Score:2)
flabbergasted (Score:2)
Re:flabbergasted (Score:2)
On linux... (Score:2)
head -c*random byte offset* *file.mp3* | tail -c *small number of bytes*
UNIX solution (Score:3, Informative)
#!/bin/bash
for i in $(seq n)000
do
- mpg123 -w out$i.wav -k $i -n f source_filename.mp3
doneWhere n is the number of MP3 frames to skip, in thousands, f is the number of frames to extract per iteration, and source_filename.mp3 is your MP3 file. For a 128kbps MP3, if you wish to extract 30s of audio for roughly every 10 minutes, you would use n=22 and f=1100. Output would be in files named out1000.wav, out2000.wav, and so on. Experimentation with the numbers is encouraged when using different bitrates. Please feel free to critique my bash code -- I am a little rusty.
Don't be so hard on this guy... (Score:5, Insightful)
Re:Don't be so hard on this guy... (Score:2)
if there isn't a simple solution... (Score:2)
Not Windows But (Score:2)
$percent_chance = 25
while (read( STDIN, $_
print $_
} }' >|
Breaks it up into 2 second blocks with a 25 percent chance of choosing each block ( which works better with your average mp3 you might have lying around than 30 seconds blocks and 5%)
Output is raw 16 bit signed PCM
sox -s -t raw -c 2 -w -r 44100
will play it
Adding a PCM header, re-mpeging the data
You Want Snack (Score:2)
The Snack library (homepage here [speech.kth.se]) provides high-level manipulation of a wide variety of audio formats (including MP3) to Python, Tcl and Ruby. It's already been used to write several applications including a sound editor [speech.kth.se] and an MP3 player [sourceforge.net].
All of the scripting languages I listed also support the Tk widget set so you can get a GUI up and running quickly. Plus, since it's all cross-platform, you'll easily be able to port your work to *nix or MacOS.
I refuse to help this dude (Score:5, Funny)
James Pennebaker and I developed the method at the University of Texas at Austin at the end of the last century.
Oh you mean FOUR YEARS AGO? Bunghole.
Here's a nice elegant hack. (Score:5, Funny)
On the other hand, you could do it with an embedded linux device too; the frequent battery changes will have the desirable effect.
Okay I confess I wrote this post to confuse the moderators into inaction; he's bashing windows -- no, he's bashing linux -- oh FUCK what to do...
The correct moderation, gentle mod point merchant, is `funny'.
use the pipe, luke (Score:2)
sox: for all your automated audio-editing needs (Score:2, Informative)
customize the mic, not the player.. (Score:3, Insightful)
>turn-over in player models, however, this strategy has
>turned out to be short-sighted (every half a year we have
>to build a new chip).
Almost every recorder has a noise/voice activation mode and most of the ones I've used had a mic or line-in as well. So just have an external mic -> custom chip that cuts it off when needed -> recorder. I suppose you could crack open the player and get between the internal mic if you wanted to as well. Either way the custom bit will never go obsolete. I record lectures I go to and found quality is much better with a good external mic anyway, even if it's a pocket size one like the small Sony conference mics.
how about a conciseness algorithm? (Score:4, Funny)
Use mpgedit... (Score:3, Informative)
A brief command line example:
mpgedit -e39-44 -e111-137 -e222-244 huge.mp3
Will create huge_1.mp3 huge_2.mp3 and huge_3.mp3, 5, 26 and 22 seconds in length respectively.
The script you need to write will generate a command line similar to this example, generating random time offsets and segment lengths. You can determine the total input file play time by running "mpgedit huge.mp3" and parse the time from the "Track length:" field.
Side question ... (Score:3, Insightful)
Is that legal? (Score:5, Insightful)
It'd be like having the FBI record your calls 24/7, but only listen to them if something came up, or having police be able to raid your house the moment they suspected you, but not look at what they gathered without a warrant. Even if they were completely honest in carrying it out, it's still too Big Brother-ish.
I think that, if the requirements say you can only record brief snippets, then you can only record brief snippets; not record everything and only listen to brief snippets. While it's the same from your perspective, it's not the same in terms of what's actually happening, and it sounds like it's completely bypassing your privacy concerns.
If you're already designing an external microchip, why not design a whole little recorder?
Ask Greg Abowd (Score:3, Interesting)
I think his goals are more along the lines of automating segmentation and indexing of the audio for easy searching of your entire last day/week/year/decade of conversations with people.
Anyway, you might be interested in the kinds of things he's doing. But actually picking out random snippets of mp3 audio should be a trivial coding task. I'm sure there have already been a dozen libraries/scripting tools/command-line solutions proposed already in previous posts.
Comment removed (Score:4, Funny)
Re:Still big brother (Score:4, Interesting)
the OP said as much
compare, for example, to the latest (federal) medical privacy rules, www.hhs.gov/ocr/hipaa/ [hhs.gov]
Re:Still big brother (Score:2, Insightful)
LK
Re:Still big brother (Score:3, Interesting)
It's a sort of 'if there's nobody in the forest, the sound was never heard' type of solution.
Re:Still big brother (Score:3, Interesting)
I've built a simple Java app that parses through an mp3 file and can do exactly this. You feed it three parameters... the begin time, the duration, and a filename and it outputs the request.
I originally did this for a *large* organization which had a huge number of sound recordings and they wanted specific cuts and already had the offsets and durations.
Drop me an email for details.
Re:MP3 chunks? (Score:2, Insightful)
There you go. Write some script to make up the values in parentheses.
On to tackle the next great engineering mystery of computer science. Maybe I'll solve some of those NP-complete problems, or install one of those really sweet case fans that light up when the music plays!
Re:MP3 chunks? (Score:2, Insightful)
Problem solved. Next!
Re:Maybe...... (Score:2)
Re:develop... (Score:4, Informative)
A flip answer but correct in the sense that this is simply a problem of calculation. You don't need any GUI or any fancy interface. I suggest, since you are at a school anyway, that you swing by the computer science department and get some senior to do this for his independent study project. All it needs to do is take the input sound file and put out the random samples. Requirements: 1. the input file, 2. parameters for how often and for how long to randomly sample (could be in a text file) and 3. the output file. No Windows, (MS, X, or other), required. Heck, it could be a DOS program (depending on the input file sizes).
Re:Exactly... (Score:2)
Boy, does this take me back. Right back to seventh grade math, when we were learning to do stuff like factor polynomials and kids would ask in their whiniest voices: "But why do we have to learn this? What good is it for? I'm never going to have to do this in real life..."
Of course, there was never a satisfying answer from the perspective of a seventh-grader. The more perceptive among us knew that this was just stuff you had to learn, sorta like eating your vegetables. It