close

Вход

Забыли?

вход по аккаунту

?

JP2013183286

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2013183286
Abstract: The present invention makes it possible to notify the direction of generation of sounds
generated in the surroundings even in situations where the surrounding sounds can not be
recognized. A mobile terminal device 300 obtains arrival time differences of a plurality of
microphones M1 to Mn arranged around a casing and detecting a sound emitted by a
surrounding sound source and a plurality of sounds detected by the plurality of microphones M1
to Mn. The sound source direction calculation unit 302 that calculates the direction of the sound
source based on the sound processing unit 301 that performs the sound processing and the
arrival time difference of each sound acquired by the sound processing unit 301, and the sound
source direction calculated by the sound source direction calculation unit 302 And a notification
unit 303 for notifying. [Selected figure] Figure 9
PORTABLE TERMINAL DEVICE, SOUND SOURCE DIRECTION DETECTING METHOD, AND
PROGRAM
[0001]
The present invention relates to a portable terminal device, a sound source direction detection
method, and a program.
[0002]
In a portable terminal device such as a portable audio player or a portable telephone, when
listening to music or making a call using an earphone, the user can not hear surrounding sounds.
03-05-2019
1
For example, there may be a case that the user does not notice the approach of a car by not
hearing a sound while walking, or a case where he does not notice a person walking nearby or
riding a bicycle. Therefore, even in a situation where the user can not recognize surrounding
sounds, such as using earphones, a technique for detecting the surrounding sounds and notifying
the user of the sound generation direction is desired.
[0003]
Patent Document 1 relates to a notification device for a vehicle, detects the generation of an
alarm sound (a horn sound, a siren sound, etc.) outside the vehicle, identifies the type of the
alarm sound, and sends an alarm sound to the deaf driver. There has been proposed a technique
for notifying so that the type can be identified.
[0004]
In Patent Document 2, a belt incorporating four sound detection and transfer blocks including a
microphone and a vibration motor is attached to the body to arrange it on the horizontal plane of
the body, and the microprocessor first changes the signal. There has been proposed a technology
for transmitting the sound source direction to the wearer by determining the detected position of
the sound detection / transmission block as the direction of the sound source and vibrating only
the vibration motor of the block.
[0005]
Further, in Patent Document 3, an object detection sensor (for example, a photoelectric sensor) is
disposed on a side surface (left side surface) of a housing of a mobile telephone device, and in the
back direction of the user of the mobile telephone device There has been proposed a technique
for notifying, when detecting an existing object or an object approaching from the rear direction,
using a vibrator for vibrating the case or a speaker.
[0006]
Further, in Patent Document 4, at least three microphones are evenly arranged on the same
plane at an interval of 120 degrees, and the selected two of these microphones are regarded as
one set, and the output of the microphone is The correlation coefficients are obtained by
inputting the time difference which is a function of the sweep angle to the plurality of correlators
as input to the corresponding correlators respectively, and the obtained plurality of correlation
coefficients are added to obtain the polar coordinates. A technique has been proposed for
displaying the direction of arrival of sound by plotting at the sweep angle position.
03-05-2019
2
[0007]
Further, in Patent Document 5, for example, a portable terminal module detects a sound such as
crying, crying, breaking glass, etc., recognizes the type of the sound, and vibrates a headset worn
by a parent etc. For example, a technology for warning is proposed.
[0008]
JP-A-2002-240660 JP-A-2005-349155 JP-A-2010-171673 JP-A-11-083982 JP-A-2008529126
[0009]
However, Patent Documents 1 to 5 described above are devices aimed at detecting and notifying
the user of the generation of surrounding sound and the direction of the generation of the sound,
and the user is required to In a situation where the user can not recognize the sound, the user is
unable to notify the user of the generation of surrounding sounds or the direction of the sound
generation with the device in use.
[0010]
In addition, although patent document 3 is a technique regarding a mobile telephone apparatus,
the object detection sensor (for example, photoelectric sensor) detects the object which exists in
the back direction of a user, or the object which approaches from a back direction. There is a
problem that it is impossible to detect the generation of surrounding sounds and the direction of
the generation of sounds.
[0011]
Therefore, an object of the present invention is to provide a portable terminal device capable of
notifying a generation direction of a sound generated in the surrounding even in a situation
where the surrounding sound can not be recognized, a sound source direction detecting method,
and a program.
[0012]
A portable terminal device according to the present invention includes a plurality of microphones
disposed around a housing and detecting sounds emitted by a surrounding sound source, an
audio processing unit acquiring an arrival time difference of each sound detected by the plurality
of microphones, and A sound source direction calculation unit that calculates the direction of the
03-05-2019
3
sound source based on the arrival time difference of each sound acquired by the sound
processing unit; and a notification unit that notifies the direction of the sound source calculated
by the sound source direction calculation unit It is a portable terminal device characterized by
having.
[0013]
The sound source direction notifying method according to the present invention comprises the
steps of: detecting a sound emitted by a surrounding sound source with a plurality of
microphones arranged around a casing; acquiring an arrival time difference of each sound
detected with the plurality of microphones; A sound source direction notifying method
comprising: calculating the direction of the sound source based on the arrival time difference of
each sound; and notifying the calculated direction of the sound source.
[0014]
A program according to the present invention detects a sound emitted by a surrounding sound
source with a plurality of microphones arranged around a housing on a computer of a portable
terminal device, and obtains an arrival time difference of each sound detected with the plurality
of microphones And calculating the direction of the sound source based on the arrival time
difference of each sound, and notifying the calculated direction of the sound source.
[0015]
According to the present invention, it is possible to notify the generation direction of the sound
generated around, even in a situation where the surrounding sound can not be recognized.
[0016]
It is an external view which shows the external appearance of the portable terminal device 101
by 1st Embodiment of this invention.
It is a block diagram which shows the structure of the portable terminal device 101 by the 1st
Embodiment.
It is a conceptual diagram for demonstrating the sound source direction detection method by the
1st Embodiment.
03-05-2019
4
It is a conceptual diagram for demonstrating the sound source direction detection method by the
1st Embodiment.
It is a flowchart for demonstrating the operation | movement (main routine) of the portable
terminal device 101 by 1st Embodiment of this invention.
It is a flowchart for demonstrating the operation | movement (sound source direction calculation
method) of the portable terminal device 101 by the 1st Embodiment of this invention.
It is a conceptual diagram which shows the example of notification of the sound source direction
in the portable terminal device 101 by the 1st Embodiment.
It is a flowchart for demonstrating the operation | movement of the portable terminal device 101
by the 2nd Embodiment of this invention.
FIG. 6 is a block diagram showing the configuration of Supplementary Note 1;
[0017]
Hereinafter, embodiments of the present invention will be described with reference to the
drawings.
[0018]
A.
First Embodiment First, a first embodiment of the present invention will be described.
FIG. 1 is an external view showing the appearance of a portable terminal device 101 according to
a first embodiment of the present invention.
03-05-2019
5
In FIG. 1, the main body of the portable terminal device 101 includes a display 102, operation
buttons 103, and four microphones (hereinafter simply referred to as microphones) M1, M2, M3,
and M4.
Four microphones M 1, M 2, M 3 and M 4 are provided at diagonal corners of the case of the
portable terminal device 101, and detect sounds around the portable terminal device 101.
Further, inside the outer periphery of the housing of the portable terminal device 101, vibration
parts V1 to V8 are incorporated so that only a specific part vibrates.
[0019]
FIG. 2 is a block diagram showing the configuration of the portable terminal device 101
according to the first embodiment.
The parts corresponding to FIG. 1 are assigned the same reference numerals and explanation
thereof is omitted. In FIG. 2, the four microphones M 1, M 2, M 3 and M 4 are connected to the
sound source processing unit 202. The sound source processing unit 202 acquires the arrival
time difference of the sound detected by each of the microphones M1 to M4 from the sound
input from each of the microphones M1 to M4, and transmits information to the CPU 201. The
ROM 203 stores an instruction code for the CPU 201 to operate. The RAM 204 is also used as a
working area for temporarily recording data necessary for the CPU 201 to operate.
[0020]
The CPU 201 executes the instruction code stored in the ROM 203. In particular, in the first
embodiment, the CPU 201 calculates the direction of the sound source, that is, the sound
generation direction, based on the arrival time difference at each of the microphones M1 to M4
supplied from the sound source processing unit 202. The CPU 201 also controls the vibration
control unit 205 to notify the user of the direction of sound generation. The vibration control
unit 205 drives the vibration unit Vi (i = 1 to 8) disposed at a position corresponding to the
sound generation direction under the control of the CPU 201.
03-05-2019
6
[0021]
Further, the CPU 201 controls the display control unit 206 in accordance with the direction in
which the sound is generated. Under the control of the CPU 201, the display control unit 206
displays the sound generation direction on the display 207 in a display form that can be viewed
at first glance.
[0022]
As described above, in the first embodiment, it is possible to notify the user of the detected sound
generation direction also by notifying the generation direction of the sound by vibrating the
vibration part Vi and displaying it on the display 207. It has become. Note that either one of them
may be used, and it may be set by the user whether to vibrate, to display, or to execute both.
[0023]
FIG. 3 and FIG. 4 are conceptual diagrams for explaining the sound source direction detection
method according to the first embodiment. The case where a sound is generated from the sound
source A located in the oblique direction of the portable terminal device 101 in FIG. 3 will be
described as an example. The sound generated by the sound source A first reaches the
microphone M1 of the portable terminal device 101, and then reaches the microphones M2, M3,
and M4 in order. The sound generation direction can be calculated from the two microphones as
follows.
[0024]
First, it is assumed that the difference in arrival time of sound between the microphone M1 and
the microphone M2 is ΔtM1-M2 (s). Since the distance between the microphone M1 and the
microphone M2 is a fixed value, it is assumed that dM1-M2 (m), and the speed of sound is c (m /
s). At this time, since ΔdM1-M2 can be represented by ΔdM1-M2 = dM1-M2 · sinθ M1-M2 and
ΔdM1-M2 = c · ΔtM1-M2, the direction θ M1-M2 of the sound source A is sin θ M1-M2 It can
be calculated that c = Δt M 1 −M 2 / d M 1 −M 2.
03-05-2019
7
[0025]
However, as shown in FIG. 4, the sound arriving at ΔtM1-M2 (s) has the same numerical value
even if it is generated from the sound source B. Therefore, since the same value is obtained for
ΔdM1-M2, the direction from the portable terminal device 101 becomes unclear whether it is
generated from the sound source A or generated from the sound source B. That is, assuming that
the direction of the sound source based on the short side of the portable terminal device 101 is
θ, θ = θ1 when generated from the sound source A, but θ = θ2 when generated from the
sound source B. .
[0026]
Therefore, as shown in FIG. 3, θM1-M3 is calculated from the difference in arrival time of sound
to the microphones M1 and M3. If the direction based on the short side of the portable terminal
device 101 is calculated, the same angle as θ1 can be derived from θM1-M3, and θ2 can be
derived as in the case of calculation for the microphones M1 and M2. (A triangle having an angle
θM1-M3 on the side connecting the microphones M1 and M3 is similar to a triangle having an
angle θM1-M2 on the side connecting the microphones M1 and M2). Therefore, the sound
source A located in the direction of θ1 can be identified as the correct sound source.
[0027]
Next, the operation of the above-described first embodiment will be described. FIG. 5 is a
flowchart for explaining the operation (main routine) of the mobile terminal device 101
according to the first embodiment. First, in the portable terminal device 101, the microphones
M1, M2, M3 and M4 detect the sound from the sound source (step S10). At this time, the sound
source processing unit 202 acquires the arrival time difference of the sound in the microphones
M1, M2, M3, and M4. Next, the CPU 201 calculates the sound generation direction from the
arrival time difference of the sound (step S12). Details of the method of calculating the direction
of occurrence will be described later.
[0028]
Next, the CPU 201 controls the vibration control unit 205 to vibrate the vibrating unit Vi
03-05-2019
8
disposed at the position corresponding to the sound generation direction, and controls the
display control unit 206 to generate the sound generation direction. Is displayed on the display
207 to notify the user of the direction of sound generation (step S14).
[0029]
FIG. 6 is a flowchart for explaining the operation (sound source direction calculation method) of
the portable terminal device 101 according to the first embodiment.
The method of calculating the sound generation direction in step S12 described above will be
described in detail with reference to FIG. First, the sound source processing unit 202 notifies the
CPU 201 of the arrival time difference of sound in the microphones M1, M2, M3, and M4 (step
S20). Next, the CPU 101 determines the first direction based on the time difference between the
detection sound of the first detection microphone Ml and the detection sound of the second
detection microphone Mm among the four microphones M1, M2, M3, and M4. Is calculated (step
S22). For details, refer to the description of FIG.
[0030]
Next, the CPU 101 calculates the second direction from the difference in arrival time between the
detection sound of the microphone M1 detected first and the detection sound of the microphone
Mn detected third (step S24). For details, refer to the description of FIG. Next, the CPU 101
calculates an angle at which the first direction and the second direction are satisfied (see FIG. 3),
and determines the sound generation direction, that is, the direction of the sound source from
the angle (step S26).
[0031]
FIGS. 7A and 7B are conceptual diagrams showing an example of sound source direction
notification in the portable terminal device 101 according to the first embodiment. When the
sound generation direction, that is, the direction of the sound source is determined in the process
described above, as shown in FIGS. 7A and 7B, the CPU 101 determines the vibration unit Vi
disposed at a position corresponding to the sound generation direction. And causes the display
102 to display its own position and the position of the sound source.
03-05-2019
9
[0032]
In FIG. 7A, when a sound is detected in front of the left of the user, the vibrating portion V1
disposed on the upper left of the mobile terminal device 101 vibrates, and the display 102
receives the sound. A sound source (○) is displayed on the left front from the position (().
Further, in FIG. 7B, when a sound is generated on the left side of the user, the vibrating portion
V8 disposed on the left side of the portable terminal device 101 vibrates, and the display 102
displays the position of the user. The sound source (○) is displayed on the left from (Δ).
[0033]
In the first embodiment described above, when the sound generation direction, that is, the
direction of the sound source is determined, the vibrating portion Vi disposed at a position
corresponding to the sound generation direction is vibrated and graphically displayed on the
display 102. However, it may be one or the other. Also, whether to vibrate, display, or execute
both may be set by the user.
[0034]
In the first embodiment described above, the vibrating portions V1 to V8 are disposed around
the inside (four corners, four sides) of the inside of the casing of the portable terminal device
101. However, the present invention is not limited to this. It may be made to vibrate so that it
may respond to all directions by combination of -V4. That is, the configuration and structure of
the vibrating portion are not limited as long as at least eight directions of top, bottom, left, right,
upper left, upper right, lower right, and lower left can be identified and vibrated.
[0035]
According to the first embodiment described above, the user of the portable terminal device 101
recognizes the direction of generation of the sound (the direction of the sound source) occurring
around the user even in a situation where no sound can be heard. Can. Further, since the
direction can be recognized from the vibration location of the portable terminal device 101, the
03-05-2019
10
user can recognize the direction in which the sound is generated even if the user looks away
from the display 102.
[0036]
B. Second Embodiment Next, a second embodiment of the present invention will be described.
The second embodiment of the present invention is characterized in that the user is notified
when the sound source approaches the user. When the sound source is approaching due to the
Doppler effect, the frequency becomes high. Then, the sound of the same sound source is
acquired at least twice, and the frequency of each sound is compared. If the frequency is high, it
is determined that the sound source is approaching and the user is notified. If the frequency does
not increase, it is determined that the user is at the same distance or away from the user, and the
user is not notified. Acquisition of the sound for performing comparison may be any one of the
four microphones M1 to M4. In addition, since the structure of the portable terminal device 101
is the same as that of FIG. 2, description is abbreviate | omitted.
[0037]
FIG. 8 is a flowchart for explaining the operation of the mobile terminal device 101 according to
the second embodiment. First, in the portable terminal device 101, sounds are detected by the
microphones M1, M2, M3 and M4 (step S30). This is the first sound detection. At this time, the
sound source processing unit 202 acquires the frequency of the first sound. Next, as in the first
embodiment described above, the sound source processing unit 202 calculates the arrival time
difference of sound at the microphones M1, M2, M3, and M4, and the CPU 201 generates the
sound from the arrival time difference of the sound. Is calculated (step S32).
[0038]
Next, the CPU 201 records the frequency of the first sound in the RAM 204 (step S34), and
determines whether the frequency of the sound that can be compared is recorded (step S36). In
this case, since it is the first sound acquisition, there is no frequency to compare (NO in step
S36). Therefore, in this case, the process returns to step S30, and the sound is detected again by
the microphones M1, M2, M3 and M4 (701). This is the second sound detection. At this time, the
sound source processing unit 202 acquires the frequency of the second sound as in the case of
the first sound detection. Then, in step S32, the generation direction of the acquired sound is
03-05-2019
11
calculated, and in step S34, the frequency of the second sound is recorded in the RAM 204.
[0039]
Next, in step S36, it is determined whether there is a frequency to be compared. In this case,
since the frequency of the first sound is recorded (YES in step S36), the CPU 201 determines
whether or not the frequency of the second sound is higher than the frequency of the first sound.
(Step S38). Then, if the frequency of the second sound is not higher (NO in step S38), it is
determined that the sound source is keeping the same distance as the user or is distant, and the
notification to the user is given. Is not performed, and the process ends.
[0040]
On the other hand, if the frequency of the second sound is higher than the frequency of the first
sound (YES in step S38), it is determined that the sound source is approaching the user, and the
user is The direction of occurrence is notified (step S40). In the notification method of the sound
generation direction, as in the first embodiment described above, the vibration unit Vi disposed
at the position corresponding to the sound generation direction is vibrated and graphically
displayed on the display 102, or Alternatively, notification may be made.
[0041]
According to the second embodiment described above, for example, the user does not respond to
the sound emitted by the user himself, such as his footsteps, so that erroneous detection can be
prevented. In addition, since it does not respond to a sound source that does not approach the
user, unnecessary operations can be prevented, leading to a reduction in power consumption.
[0042]
In the second embodiment described above, the second sound needs to be from the same
direction as the first sound. For example, it is determined that the sound comes from the same
direction within a predetermined tolerance range such as ± 2 degrees in angle. That is, the
condition is that they are the same sound source.
03-05-2019
12
[0043]
Further, in the above-described second embodiment, for the second detection of the sound, in
order to take a fixed period of time from the detection of the first sound, for example, an interval
may be obtained with a timer or the like.
[0044]
In the first and second embodiments described above, the sound source direction is notified by
vibration or display on the display 102. However, the present invention is not limited to this, and
a synthetic voice is notified (when using an earphone, The voice direction of the sound source
may be forcibly output to the earphone.
[0045]
Further, in the first and second embodiments described above, the position (Δ) of the user and
the sound source (表示) are displayed on the display 102, but the present invention is not limited
to this, and the sound source direction can be known. For example, any display form may be
used.
For example, an arrow indicating the sound source direction may be displayed.
[0046]
Also, although the first and second embodiments are examples applied to a mobile phone as a
mobile terminal device, the present invention is not limited to this, and a smartphone, a mobile
game machine, a mobile PC (personal computer (personal computer) ), Portable communication
devices, and other electronic devices.
[0047]
The features of the present invention will be additionally described below.
Some or all of the above embodiments may be described as in the following appendices, but is
03-05-2019
13
not limited to the following.
(Supplementary Note 1) FIG. 9 is a block diagram of supplementary note 1. The correspondence
between FIG. 9 and FIG. 2 will be described. The microphones M1 to Mn illustrated in FIG. 9
correspond to the microphones M1 to M4 in FIG. 2, and the sound source processing unit 301
corresponds to the sound source processing unit 202 in FIG. 2. Further, the sound source
direction calculation unit 302 in FIG. 9 corresponds to the CPU 201 in FIG. 2, and the notification
unit 303 in FIG. 9 corresponds to the display control unit 206, the display 102 or the vibration
control unit 205 in FIG. It corresponds to
[0048]
As shown in this figure, the invention described in Appendix 1 includes: a plurality of
microphones M1 to Mn which are disposed around a casing and detect sounds emitted from a
surrounding sound source; each sound detected by the plurality of microphones M1 to Mn A
sound processing unit 301 for acquiring an arrival time difference of the sound source; a sound
source direction calculating unit 302 for calculating the direction of the sound source based on
the arrival time difference of each sound acquired by the sound processing unit 301; And a
notification unit 303 for notifying of the direction of the sound source calculated by 302. A
mobile terminal device 300 characterized by:
[0049]
(Supplementary Note 2) The sound source direction calculation unit calculates the first direction
from the difference in arrival time between the detection sound of the first detected microphone
and the detection sound of the second detected microphone among the microphones, A second
direction is calculated from the difference in arrival time between the detection sound of the first
detected microphone and the detection sound of the third detected microphone, and both the
first direction and the second direction are satisfied. The mobile terminal device according to
claim 1, wherein the angle is determined to be the direction of the sound source.
[0050]
(Supplementary Note 3) The mobile terminal device according to Supplementary note 1 or 2,
wherein the notification unit notifies the direction of the sound source by vibrating a part of a
housing corresponding to the direction of the sound source. .
[0051]
(Supplementary Note 4) A plurality of vibration units disposed around the casing is provided, and
03-05-2019
14
the notification unit vibrates the vibration units disposed at a position corresponding to the
direction of the sound source among the plurality of vibration units. 5. The portable terminal
device according to claim 3, wherein the direction of the sound source is notified.
[0052]
(Supplementary Note 5) The mobile terminal device according to any one of Supplementary
notes 1 to 4, wherein the notification unit notifies the direction of the sound source by displaying
the direction of the sound source with visible graphics. is there.
[0053]
(Supplementary Note 6) The notification unit displays a marker indicating the position of the
user and a marker indicating the sound source in a direction corresponding to the direction of
the sound source with respect to the position of the user. 7 is the portable terminal device
according to 5.
[0054]
(Supplementary Note 7) A sound emitted by the same sound source is detected at least twice by
at least one of the plurality of microphones, and it is determined whether the sound source is
approaching based on a change in frequency of each sound. The sound source proximity
determination unit is further provided, and the notification unit notifies the direction of the
sound source calculated by the sound source direction calculation unit when the sound source
proximity determination unit determines that the sound source is approaching. It is a portable
terminal device according to any one of supplementary notes 1 to 6 characterized by the above.
[0055]
(Supplementary Note 8) A step of detecting a sound emitted by a surrounding sound source by a
plurality of microphones arranged around a casing, a step of acquiring an arrival time difference
of each sound detected by the plurality of microphones, and an arrival of each sound A sound
source direction notifying method comprising: calculating a direction of the sound source based
on a time difference; and notifying the calculated direction of the sound source.
[0056]
(Supplementary Note 9) A step of detecting sounds emitted by a surrounding sound source with
a plurality of microphones arranged around a housing in a computer of a portable terminal
device, and a step of acquiring an arrival time difference of each sound detected with the
plurality of microphones And a step of calculating the direction of the sound source based on the
03-05-2019
15
arrival time difference of each sound, and notifying the calculated direction of the sound source.
[0057]
(Supplementary Note 10) When a sound emitted by a sound source is detected at least twice by
at least one of the plurality of microphones, a predetermined time elapsed from the detection of
the first sound to the detection of the second sound The mobile terminal device according to
Appendix 7, further comprising: a clock unit configured to clock the clock.
[0058]
(Supplementary Note 11) The mobile terminal device according to any one of Supplementary
notes 1 to 7, wherein the notification unit notifies the direction of the sound source by synthetic
speech.
[0059]
101, 300 Mobile terminal device 102 Display 103 Operation button 201 CPU 202, 301 Sound
source processing unit 203 ROM 204 RAM 205 Vibration control unit 206 Display control unit
302 Sound source direction calculation unit 303 Notification unit M1 to M4 Microphone V1 to
V8 Vibration unit
03-05-2019
16
Документ
Категория
Без категории
Просмотров
0
Размер файла
27 Кб
Теги
jp2013183286
1/--страниц
Пожаловаться на содержимое документа