ReSpeaker 4-Mic 声源定位

Sound Source Tracking

0. 准备工作

硬件为ReSpeaker 4-Mic，将其与树莓片连接。

将SoundSourceTracking.zip拷贝至home目录。运行如下指令。请全部使用我们修改后的代码，不要使用github上官方源码，会有各种问题。

cd ~
unzip SoundSourceTracking.zip
cd ~/SoundSourceTracking
unzip 4mics_hat-master.zip
unzip odas.zip
unzip seeed-voicecard-5.9.zip
1
2
3
4
5
6

1. 安装ReSpeaker 4-Mic驱动

Step 1. 安装驱动并重启。

cd ~/SoundSourceTracking/seeed-voicecard-5.9
sudo ./install.sh
sudo reboot now
1
2
3

打印如下内容则为安装成功：

Step 2. 选择声卡

sudo raspi-config
# Select 1 System options
# Select S2 Audio
# Select MAI PCM i2s-hifi-0
# Select Finish
1
2
3
4
5

Step3. 检查声卡是否安装成功，打印出四个seeed4micvoicec即为成功

(base) mman@raspberrypi:~ $ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
lavrate
    Rate Converter Plugin Using Libav/FFmpeg Library
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pulse
    PulseAudio Sound Server
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
default
    Playback/recording through the PulseAudio sound server
playback
ac108
usbstream:CARD=Headphones
    bcm2835 Headphones
    USB Stream Output
usbstream:CARD=vc4hdmi0
    vc4-hdmi-0
    USB Stream Output
usbstream:CARD=vc4hdmi1
    vc4-hdmi-1
    USB Stream Output
hw:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Direct hardware device without any conversions
plughw:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Hardware device with all software conversions
sysdefault:CARD=seeed4micvoicec
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Default Audio Device
dsnoop:CARD=seeed4micvoicec,DEV=0
    seeed-4mic-voicecard, bcm2835-i2s-ac10x-codec0 ac10x-codec0-0
    Direct sample snooping device
usbstream:CARD=seeed4micvoicec
    seeed-4mic-voicecard
    USB Stream Output
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

Step 4. 通过audacity测试麦克风是否正常

sudo apt install audacity
audacity                      // run audacity
1
2

Step 5. 灯光测试，运行下列程序观察灯光是否闪烁

cd ~/SoundSourceTracking/4mics_hat-master
sudo apt-get install portaudio19-dev libatlas-base-dev
pip3 install -r requirements.txt
python3 interfaces/pixels.py
1
2
3
4

2. 声源定位与跟踪

Step 1. 安装依赖项

sudo apt-get install libfftw3-dev libconfig-dev libasound2-dev libgconf-2-4
sudo apt-get install cmake
sudo apt-get install libfftw3-dev
sudo apt-get install libconfig-dev
sudo apt-get install libasound2-dev
sudo apt-get install libpulse-dev
1
2
3
4
5
6

Step 2. 编译我们改版的ODAS程序

cd ~/SoundSourceTracking/odas/build
cmake ..
make
1
2
3

Step3. 运行声源定位程序

cd ~/SoundSourceTracking/odas/build/bin
./odaslive -c ~/SoundSourceTracking/odas/config/odaslive/respeaker_4_mic_array.cfg
1
2

正常会打印如下数据：(报错请看后面)

{
    "timeStamp": 334,
    "src": [
        { "x": 0.082, "y": -0.678, "z": 0.731, "E": 0.167 },
        { "x": -0.123, "y": -0.621, "z": 0.774, "E": 0.079 },
        { "x": -0.375, "y": -0.516, "z": 0.770, "E": 0.036 },
        { "x": -0.434, "y": -0.597, "z": 0.675, "E": 0.027 }
    ]
}
{
    "timeStamp": 334,
    "src": [
        { "id": 4, "tag": "dynamic", "x": -0.126, "y": -0.967, "z": 0.222, "activity": 0.005, "sita": -97.450, "h": 1.347 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan }
    ]
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

重点查看第二个大括号内的数据，sita为音源在xOy平面(麦克风所在平面)的角度，h为音源的高度，注意h与现实的极性相反。

无报错跳过此部分，若遇到报错：Source hops: Cannot open audio device hw:x,0: No such file or directory

运行

python ~/SoundSourceTracking/odas/audio_dev.py
1

打印如下

ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_headpho.pcm.hdmi.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4745:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5233:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM hdmi
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.bcm2835_headpho.pcm.hdmi.0:CARD=0,AES0=4,AES1=130,AES2=0,AES3=2'
ALSA lib conf.c:4745:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5233:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM hdmi
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port
ALSA lib pcm_a52.c:823:(_snd_pcm_a52_open) a52 is only for playback
ALSA lib conf.c:5200:(snd_config_expand) Unknown parameters {AES0 0x6 AES1 0x82 AES2 0x0 AES3 0x2  CARD 0}
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM iec958:{AES0 0x6 AES1 0x82 AES2 0x0 AES3 0x2  CARD 0}
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card
ALSA lib pcm_hw.c:1829:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1829:(_snd_pcm_hw_open) Invalid value for card
Input Device id  2  -  seeed-4mic-voicecard: bcm2835-i2s-ac10x-codec0 ac10x-codec0-0 (hw:5,0)
Input Device id  7  -  pulse
Input Device id  10  -  ac108
Input Device id  12  -  default
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

定位这一行

Input Device id  2  -  seeed-4mic-voicecard: bcm2835-i2s-ac10x-codec0 ac10x-codec0-0 (hw:5,0)
1

看最后hw:5,0，请记住这个5

修改文件~/SoundSourceTracking/odas/config/odaslive/respeaker_4_mic_array.cfg的第18行，将card = 3;中的3修改为前面记住的数字5即可

Step 4. 获取音源跟踪的数据

音源跟踪的数据位于~/SoundSourceTracking/odas/build/bin/sst.txt中，重新启动后改文本会同步更新，只需不断读取该文本的数据即可。以下为文本中的数据的事例。

{
    "timeStamp": 303,
    "src": [
        { "id": 4, "tag": "dynamic", "x": -0.125, "y": -0.967, "z": 0.222, "activity": 0.094, "sita": -97.346, "h": 1.347 },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan },
        { "id": 0, "tag": "", "x": 0.000, "y": 0.000, "z": 0.000, "activity": 0.000, "sita": 0.000, "h": nan }
    ]
}
1
2
3
4
5
6
7
8
9

链接: https://pan.baidu.com/s/1YX3aOW3js9WRyQI7XX4I2Q?pwd=whsr 提取码: whsr
–来自百度网盘超级会员v6的分享