Min Hash - SIPL
Min Hash - SIPL
Min Hash - SIPL
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
People Metering Using<br />
Mobile Devices<br />
Yehoraz Kasher Annual EE Projects Contest<br />
Students:<br />
Supervisor:<br />
Oded Yeruhami<br />
Yuval Bahat<br />
Rafi Steinberg<br />
May 23 rd , 2010
Outline<br />
• People metering<br />
• People metering using mobile devices<br />
• Algorithm description<br />
• Our innovations<br />
• Conclusion<br />
2/35
3/35<br />
People Metering
The People Meter<br />
Rating measurement method today<br />
Drawbacks:<br />
– Designated hardware<br />
– Very small control group<br />
– Monitors specific TV set<br />
– Confidentiality of control group<br />
– Impersonal<br />
4/35
People Metering Using Mobile Devices<br />
Query<br />
Fingerprint<br />
Creation<br />
Reference<br />
Fingerprint<br />
Creation<br />
Matching<br />
Matched Channel / No Match<br />
5/35
People Metering Using Mobile Devices<br />
• As suggested by MobileRL<br />
– No special hardware required<br />
– Personal<br />
– Carried everywhere<br />
– Can also be used to monitor<br />
radio, video, music etc.<br />
6/35
System Requirements<br />
• High accuracy<br />
• Robustness to noisy environment<br />
• Real time results<br />
• Cellphone constraints:<br />
»Privacy<br />
»Computational effort<br />
»Power consumption<br />
»Sent data size<br />
7/35
Literature Survey<br />
• ASF – Audio Spectrum Flatness (Hellmuth et al., 2001)<br />
• An algorithm by (Haitsma & Kalker, 2003)<br />
• Spectral Similarity (Yang, 2001)<br />
• Normalized Spectral Subbands Centroids (Seo et al., 2005)<br />
• Waveprint - Content Fingerprinting Using Wavelets,<br />
by (Baluja & Covell, 2006)<br />
8/35
System Layout<br />
Feature Extraction<br />
Extracting significant data<br />
Fingerprints<br />
Matching<br />
Matched channel /<br />
9/35<br />
No match
Fingerprint Creation<br />
Feature Extraction<br />
Spectrogram<br />
creation<br />
Matching<br />
2-D Haar<br />
Wavelet<br />
Taking<br />
Strongest<br />
Coefficients<br />
<strong>Min</strong><br />
<strong>Hash</strong><br />
10/35<br />
Fingerprint<br />
<strong>Min</strong>-<strong>Hash</strong> vector #1<br />
<strong>Min</strong>-<strong>Hash</strong> vector #2<br />
<strong>Min</strong>-<strong>Hash</strong> vector #3<br />
<strong>Min</strong>-<strong>Hash</strong> vector #4<br />
Sub-Fingerprints
Frequency<br />
Spectrogram<br />
Represents audio visually<br />
stride<br />
Spectrogram creation<br />
2-D Haar Wavelet<br />
Taking strongest coefficients<br />
<strong>Min</strong>-<strong>Hash</strong><br />
Time<br />
Divide to overlapping sub-spectrograms<br />
Fine-grained temporal resolution<br />
11/35
Wavelet Transform<br />
• Good for pointing out local data in images<br />
• Keeping only the strongest coefficients<br />
Spectrogram creation<br />
2-D Haar Wavelet<br />
Taking strongest coefficients<br />
<strong>Min</strong>-<strong>Hash</strong><br />
Wavelet<br />
Transform<br />
Keeping<br />
Strongest<br />
Coefficients<br />
12/35<br />
Maintains “interesting” data in noisy images<br />
Result – sparse binary vector
<strong>Min</strong>-<strong>Hash</strong><br />
How can we compare 2 sparse vectors<br />
efficiently<br />
<strong>Min</strong>-<strong>Hash</strong> (Cohen et al., 2001)<br />
Spectrogram creation<br />
2-D Haar Wavelet<br />
Taking strongest coefficients<br />
<strong>Min</strong>-<strong>Hash</strong><br />
Sparse vector compact representation :<br />
• p permanent vector permutations<br />
• Keeping the index of the first “1”<br />
01 1 0 0 1 10 1 0 0 1 0 1 1 0 0 1 0 1 1<br />
0<br />
13/35<br />
• Result – A vector with p elements (here, p=4):<br />
1 3 6 2<br />
Sub-fingerprint<br />
• Compactly representing the sparse vector<br />
• Similar sparse vectors yield similar <strong>Min</strong>-hash vectors
Feature Extraction – Summary<br />
Spectrogram<br />
creation<br />
Haar Wavelet Transform<br />
Keeping strongest coefficients<br />
Sub-Fingerprint Sparse Binary #1 Vector<br />
Fingerprint<br />
Sub-Fingerprint #2<br />
<strong>Min</strong> <strong>Hash</strong> Vector (p elements)<br />
Sub-Fingerprint #3<br />
14/35
Fingerprint Matching<br />
Feature Extraction<br />
Matching<br />
Sub-Fingerprint<br />
Sub-Fingerprints<br />
Candidate Sub-Fingerprint Selection<br />
Fingerprint Temporal Alignment<br />
(LSH)<br />
Candidates<br />
Query<br />
References<br />
Matching<br />
15/35<br />
Best Match
Locality Sensitive <strong>Hash</strong>ing<br />
(Gionis et al., 1999)<br />
LSH<br />
Alignment<br />
Matching<br />
Query <strong>Min</strong>-hash<br />
References <strong>Min</strong>-hash<br />
LSH<br />
Match<br />
Candidates<br />
• Reduces matching problem dimensions<br />
• Low computational complexity<br />
• Efficiently narrows down matching candidates<br />
16/35
LSH<br />
Fingerprints Temporal<br />
Distance Calculation<br />
Alignment<br />
Alignment<br />
Matching<br />
query stride<br />
reference stride<br />
DistanceDistance<br />
Distances<br />
17/35<br />
Grade Calculation
Verifying Our Implementation<br />
• Forced matching scenario<br />
• Tested with various digitally added noises<br />
White Gaussian noise, echo…<br />
Our system produced good results<br />
However our problem is more difficult…<br />
• No forced matching scenario<br />
• Recordings in a noisy environment<br />
18/35
References<br />
• 120 samples<br />
• 1 minute long<br />
Queries<br />
• 430 samples<br />
• 8 seconds long<br />
Datasets by<br />
Two query types:<br />
• “Good” recordings<br />
19/35<br />
• “Bad” recordings<br />
69% Match<br />
19% Match
Feature Extraction<br />
Matching Criterion<br />
Distance threshold<br />
Matching<br />
– Requires threshold adjusting (varies with<br />
recording quality)<br />
Metrics<br />
Precision & Recall (per threshold)<br />
20/35<br />
Precision<br />
Recall<br />
True identification<br />
All identified<br />
True identification<br />
All queries
Original Algorithm Results<br />
Recall=13%<br />
Precision=96%<br />
Recall=65%<br />
Precision=78%<br />
21/35
Problem:<br />
Bad recordings - very low success rate<br />
Let's have a closer look…<br />
22/35
Frequency<br />
Frequency<br />
Success Rates Problem<br />
• Main problem appears in “bad recordings”<br />
Reference<br />
Query – “bad recording”<br />
Time<br />
23/35<br />
Time
Proposed Solution<br />
Biasing the<br />
wavelet picking<br />
Strongest wavelets picking histogram<br />
Frequency<br />
dimension<br />
Time<br />
dimension<br />
24/35<br />
DC Freq. Time Time/Freq.
After Weighted Wavelet Picking<br />
Good Recordings<br />
Recall=90%<br />
Precision=97%<br />
Recall=65%<br />
Precision=78%<br />
25/35
After Weighted Wavelet Picking<br />
Bad Recordings<br />
Recall=49%<br />
Recall=13%<br />
Precision=99%<br />
Precision=96%<br />
(Th.=0) Recall =<br />
Precision = 58%<br />
26/35
Feature Extraction<br />
Matching Criterion<br />
Matching<br />
Different criterion:<br />
Recurrence check<br />
– Sending up to N queries sequentially<br />
– Looking for M recurring matches<br />
Advantages<br />
27/35<br />
– Increases success rates<br />
– No threshold needed<br />
– Independent of environment<br />
– Overcomes sporadic noise
Recurrence Check<br />
Recall=58%, N 10, M 4, Channels # 30<br />
P true =93% P false =0.9%<br />
For bad recordings!<br />
But…<br />
Increases size of sent data<br />
28/35
Reducing Signature Size – 1 st Solution<br />
Size<br />
<br />
<br />
Byte <br />
Depends on stride size<br />
Query stride<br />
#sub-fingerprints min-hash vector length<br />
(13.24KB for 8 sec. query)<br />
Reference stride<br />
Solution:<br />
- Switch between strides<br />
29/35<br />
- Change number of permutations
Reducing Signature Size – 1 st Solution<br />
Reference sub fingerprints20 Query sub fingerprints0.2<br />
<strong>Min</strong>-hash Permutation 0.5<br />
Recall<br />
55%, Querysize 1.33KB<br />
N 10, M 4<br />
P true =93%, P false =0.9%, E[sent size] = ~9KB<br />
For bad recordings!<br />
30/35
Reducing Signature Size – 2 nd<br />
• Golomb-Rice coding (Golomb & Solomon, 1966)<br />
• Near-entropy coding for an infinite geometrically<br />
distributed input<br />
• Utilizing <strong>Min</strong>-hash distribution - close to geometric :<br />
Cumulative Distribution Function<br />
Solution<br />
~20%<br />
Compression<br />
31/35
Conclusion<br />
Implemented a people metering<br />
system using mobile devices<br />
– Personal<br />
– Carried everywhere<br />
– Not only TV<br />
32/35
Conclusion<br />
Based on Waveprint algorithm by<br />
Innovation #1<br />
Biasing the wavelet picking<br />
- Match rates<br />
Innovation #2<br />
Recurrence check<br />
- Match rates<br />
33/35<br />
- Environment independent
Conclusion<br />
Innovation #3<br />
Reducing sent fingerprint size<br />
Innovation #4<br />
Compressing sent data<br />
34/35<br />
Sent data size
Conclusion<br />
• System is suitable for commercial use<br />
For example:<br />
P true =93%, P false =0.9%, E[sent size] = ~9KB<br />
• Supplied to MobileRL<br />
• A paper in the writing<br />
35/35
Acknowledgments<br />
• Rafi Steinberg<br />
• <strong>SIPL</strong> staff<br />
– Yair Moshe<br />
– Nimrod Peleg<br />
• MobileRL<br />
– Aron Weiss, CTO
37<br />
Backup
Reference<br />
[1] O.Hellmuth et al., Advanced Audio identification using MPEG-7<br />
content description. Fraunhofer institute for integrated circuits,<br />
Convention paper 5463, 111th convention USA, September 2001.<br />
[2] J.Haitsma, T.Kalker, A highly robust fingerprinting system. Philips<br />
Research, Journal of new music research, Volume 32, Number 2,<br />
pp. 211-221(11), June 2003.<br />
[3] C.Yang, Music database retrieval based on spectral similarity.<br />
Stanford university database group technical report 2001-14, IEEE<br />
workshop on applications of signal processing, 2001.<br />
[4] Jin S Seo et al., Audio Fingerprinting based on normalized spectral<br />
subband centroids. IEEE International Conference, pages 213-216<br />
Vol. 3, March 2005.<br />
38
Reference – Cont.<br />
[5] Shumeet Baluja, Michele Covell (2006) Content Fingerprinting Using<br />
Wavelets. CVMP, 198-207<br />
[6] E. Cohen, et al.. (2001) Finding interesting associations without<br />
support pruning. Knowledge and Data Engineering, 13(1)<br />
[7] A. Gionis, P. Indyk, R. Motwani (1999), Similarity search in high<br />
dimensions via hashing. Proc. InternationalConference on Very<br />
Large Data Bases,.<br />
[8] Golomb, Solomon W. (1966) Run-length encodings. IEEE Trans<br />
Info Theory 12(3):399<br />
[9] A students project guided by Mr. Yair Moshe – Initial Code.<br />
39
2nd Solution – Golomb Rice (Cont.)<br />
•Uses run-length encoding –<br />
Code-length rises with value<br />
#<br />
1<br />
95<br />
96<br />
255<br />
coded<br />
000001<br />
11011111<br />
111000000<br />
1111111011111<br />
t=200, M=32 <br />
Compressed to 85%<br />
t=300, M=19 <br />
Compressed to 78%<br />
(Added to the query size decreasing)<br />
40
Spectrogram creation<br />
How can we compare two sparse<br />
vectors efficiently<br />
Jaccard Coefficient:<br />
2-D Haar Wavelet<br />
Taking strongest coefficients<br />
<strong>Min</strong>-<strong>Hash</strong><br />
J M , N<br />
<br />
<br />
M<br />
M<br />
N<br />
N<br />
Problem:<br />
•Long sparse binary vectors<br />
41<br />
•Similarity calculation is complicated
Change language