close

Вход

Забыли?

вход по аккаунту

?

JP2010199739

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2010199739
An object of the present invention is to make it possible for an observer to clearly understand
where in a three-dimensional image a part that appears to pop out is. A display device is provided
based on a data acquisition unit for acquiring a stereoscopic image and sound data, a parallax
amount detection unit for detecting a parallax amount of each region of the stereoscopic image,
and a parallax amount of each region of the stereoscopic image. A sound source position
determination unit 43 for determining the sound source position of the reproduction sound field
and a stereoscopic image are stereoscopically displayed by the display device 24 and the
reproduced sound at the sound source position on the display device 24 determined by the
sound source position determination unit 43 An image reproduction control unit 44 and a sound
reproduction control unit 45 are provided which set the real or virtual sound source position of
the field and reproduce sound data as sound by the sound reproduction device 25. [Selected
figure] Figure 1
Three-dimensional display control device, three-dimensional display system, and threedimensional display control method
[0001]
The present invention relates to a three-dimensional display control device, a three-dimensional
display system, and a three-dimensional display control method that allow an observer to clearly
know where a part of a three-dimensional image appears to be popping out.
[0002]
In Patent Document 1, when switching from planar display to stereoscopic display, when
switching from stereoscopic display to planar display, etc., via a stereoscopic display mode image
04-05-2019
1
created from a planar image or a planar display mode image created from a stereoscopic image,
A configuration for performing transition processing is described.
[0003]
According to Patent Document 2, when switching the display from the planar video to the
stereoscopic video, when switching the display from the stereoscopic video to the planar video,
etc., the pixel value ratio of the video data of the current display is gradually decreased over a
predetermined time. A configuration to make it 0% is disclosed.
[0004]
Patent Document 3 is directed to a moving image having sound source position information of an
original sound field, is composed of a flat display and a group of transparent flat speakers
arranged on the front surface, and specifies the speaker based on the sound source position
information of the original sound field. An arrangement for performing sound reproduction is
disclosed.
[0005]
In Patent Document 4, in a two-way communication system, the display, camera, speaker,
microphone, etc. are controlled to display the size and position of the person displayed on the
screen, the illuminance in the room, and the position at which the sound is output. A
configuration is disclosed that matches and reproduces between a place and a reproduction
place.
[0006]
JP 2005-136541 JP JP 2005-109568 JP JP 2007-27846 JP JP 2008-5122 JP
[0007]
The configurations of Patent Documents 1 and 2 can suppress the sense of discomfort of the
observer who feels when being stereoscopically displayed suddenly by gradually switching from
two-dimensional display to three-dimensional display by taking time due to transition processing.
As a result of the transition process, there is a problem that the observer can not grasp which
part of the stereoscopic image appears to be popping out.
04-05-2019
2
That is, at the time of three-dimensional display, the observer may not be able to clearly grasp
the front side portion of the three-dimensional image, which may cause a problem of lack of
three-dimensional effect.
In the worst case, the stereoscopic image is not recognized as a stereoscopic image.
[0008]
Further, in the configuration of Patent Document 1, it is necessary to create a stereoscopic
display mode image or a planar display mode image.
[0009]
Further, the configurations of Patent Documents 3 and 4 are for moving images with sound
source position information of the original sound field, and reproduce the sense of presence in
the original sound field by reproducing using the sound source position information of the
original sound field as it is. In the case of switching from two-dimensional display to threedimensional display, it is not possible to expect the effect of making it possible for the observer
to clearly understand which part of the three-dimensional image appears to be popping out.
[0010]
The present invention has been made in view of the above circumstances, and a stereoscopic
display control device, a stereoscopic display system, and a stereoscopic display control that
allow an observer to clearly know where a part of a stereoscopic image appears to be popping
out. Intended to provide a method.
[0011]
In order to achieve the above object, the present invention is a stereoscopic display control
device that performs stereoscopic display with sound reproduction using a display device and a
plurality of sound reproduction devices, and is a data acquisition unit that acquires a
stereoscopic image and sound data A parallax amount detection unit for detecting the parallax
amount of each area of the stereoscopic image; and a sound source position for determining the
sound source position of the reproduction sound field on the display device based on the parallax
amount of each area of the stereoscopic image While displaying the three-dimensional image
stereoscopically by the determination means and the display device, the real or virtual sound
source position of the reproduction sound field is set at the sound source position on the display
device determined by the sound source position determination means; And a reproduction
control means for reproducing the sound data as a sound by the sound reproduction device.
04-05-2019
3
[0012]
According to the present invention, since the sound source position of the reproduction sound
field is set on the display device based on the parallax amount of each area of the stereoscopic
image, it is clear to the observer where the part appearing in the stereoscopic image is visible. It
becomes possible to make it grasp.
[0013]
Further, in one aspect of the present invention, the sound source position determination unit
divides the stereoscopic image into an area having a large parallax amount and an area having a
small parallax amount, and corresponds to an area having a large parallax amount of the
stereoscopic image. The position on the display device to be determined is determined as the
sound source position.
[0014]
According to this configuration, the observer can recognize the area where the stereoscopic
image appears to be popped out.
[0015]
Further, in one aspect of the present invention, the reproduction control means moves the sound
source position on the display device when there are a plurality of areas where the amount of
parallax is large in the three-dimensional image.
[0016]
According to this configuration, the movement of the sound source position makes it easy to
recognize an area having a large amount of parallax.
[0017]
Further, in one aspect of the present invention, the sound source position determining unit
determines an area having the largest parallax amount among areas obtained by dividing the
stereoscopic image into a plurality of areas, and sets the area having the largest parallax amount
of the stereoscopic image. The corresponding position on the display device is determined as the
sound source position.
04-05-2019
4
[0018]
According to the present invention, the observer can recognize the most visible area of the
stereoscopic image.
[0019]
In one aspect of the present invention, the reproduction control means continuously reproduces
the sound before and after switching when switching from planar display to stereoscopic display
and switching a stereoscopic image to be stereoscopically displayed, for a predetermined time,
Sound reproduction is performed based on the sound source position determined based on the
parallax amount of each region of the stereoscopic image.
[0020]
According to this configuration, without stopping the sound output, it is possible to allow the
observer to recognize a part that appears to be popping out of the stereoscopic image.
[0021]
Further, in one aspect of the present invention, the reproduction control means sets a volume of
a sound reproduction device corresponding to the sound source position on the display device
among the plurality of sound reproduction devices more than a volume of another sound
reproduction device. Enlarge.
[0022]
In one aspect of the present invention, the plurality of sound reproducing devices are configured
by a sound reproducing device group arranged in a two-dimensional manner on the front surface
or the back surface of the display device.
[0023]
In one aspect of the present invention, the sound reproduction device is a transparent flat sound
reproduction device, and is disposed on the front surface of the display device.
[0024]
According to this configuration, it is easier to specify the sound source position of the
reproduction sound field by the observer than when the sound reproduction device is arranged
04-05-2019
5
on the back of the display device.
Moreover, since it is planar, the display device and the sound reproducing device can be
integrated and manufactured thin.
[0025]
In one aspect of the present invention, the plurality of sound reproducing devices are arranged
around the display device.
[0026]
Further, in one aspect of the present invention, the plurality of sound reproduction devices are
disposed apart from the display device, and the reproduction control means performs sound
image localization with the sound source position as a virtual sound source position of the
reproduction sound field. .
[0027]
Further, in one aspect of the present invention, the plurality of sound reproduction devices are
constituted by headphones or earphones worn on the head of a person, and the reproduction
control means sets the sound source position as a virtual sound source position of the
reproduction sound field. Perform sound image localization.
[0028]
Further, the present invention provides a stereoscopic display system comprising the
stereoscopic display control device, the display device, and the sound reproduction device.
[0029]
Further, the present invention is a stereoscopic display control method for performing
stereoscopic display with sound reproduction using a display device and a plurality of sound
reproduction devices, comprising: a data acquisition step of acquiring a stereoscopic image and
sound data; A sound source position determining step of determining a sound source position of
a reproduction sound field on the display device based on a parallax amount detection step of
detecting a parallax amount of each region; a sound source position of the reproduction sound
field based on the parallax amount of each region of the three-dimensional image; The threedimensional display of the three-dimensional image is performed by the device, and the real or
virtual sound source position of the reproduction sound field is set at the sound source position
04-05-2019
6
on the display device determined by the sound source position determination unit. And a
reproduction step of reproducing sound data as a sound.
[0030]
According to the present invention, it is possible to make the observer clearly grasp where the
part that appears to be popped out of the three-dimensional image is.
[0031]
The block diagram which shows an example of the three-dimensional display system to which the
present invention is applied The explanatory view which shows the structural example of the
display which enables three-dimensional display The explanatory view (A) used for the
explanation of the relation between subject distance and parallax amount (B) is an explanatory
view when there is a large difference in the positions of corresponding points between the right
eye image and the left eye image, (C) is a parallax Explanatory drawing which shows quantity The
explanatory image which shows the right eye image and left eye image which were obtained by
imaging the cup The explanatory image (A) and (B) which shows the captured image and distance
image of the cup are two dimensional array on the back of the display The front view and the
side view (A) and (B) of the display device and the sound reproduction device in the case where
the sound reproduction device group is arranged and the region having a large amount of
parallax is the sound source position A display device in the case where a sound reproduction
device group is disposed and an area with the largest parallax amount is used as a sound source
position A front view and a side view (A) and a side view (B) of the noise reproduction device are
display devices in which a sound reproduction device group of a two-dimensional array is
arranged on the front of the display device Front and side views of the sound reproducing
apparatus. Flowcharts (A) and (B) showing the flow of three-dimensional display control
processing to which the present invention is applied are a front view and a side view of the
display device and the sound reproducing apparatus (A) is a front view showing an example of a
single display, (B) is a front view showing an example of a multi-display.
[0032]
Hereinafter, embodiments of the present invention will be described in detail with reference to
the attached drawings.
[0033]
FIG. 1 is a block diagram showing an example of a stereoscopic display system to which the
present invention is applied.
04-05-2019
7
[0034]
In FIG. 1, the stereoscopic display system 100 includes a data storage unit 22, a display device
24, a sound reproduction unit 25 (speaker group), and a stereoscopic display control device 30.
[0035]
The data storage unit 22 is a storage unit that stores data such as an image (image data) and
sound data.
For example, a non-volatile storage device such as a memory card, a hard disk, an optical disk,
etc. is used.
[0036]
The display device 24 is configured of a display device capable of stereoscopically displaying a
stereoscopic image obtained by imaging at a plurality of viewpoints.
For example, display devices such as an LCD (liquid crystal display device), a plasma display
device, and an organic EL display device are used.
The mechanism of stereoscopic display will be described later.
[0037]
The sound reproduction device 25 is composed of a plurality of sound reproduction devices.
A specific example and arrangement of the sound reproduction device 25 will be described later.
[0038]
04-05-2019
8
The stereoscopic display control device 30 according to the present embodiment includes an
operation unit 31, a data acquisition unit 32, a display image memory 33, a display driver 34, a
sound reproduction driver 35, and a CPU 40.
[0039]
The operation unit 31 is configured by a known input device such as a keyboard and a mouse.
[0040]
The data acquisition unit 32 acquires image and sound data from the data storage unit 22 in this
example.
In this example, although acquisition is performed from the data storage unit 22, image and
sound data may be acquired through a communication network such as the Internet.
Also, image and sound data attachments may be obtained directly by a known camera and
microphone.
[0041]
The display image memory 33 is a storage unit dedicated to a display image that temporarily
stores image data to be displayed by the display device 24.
For example, a high-speed accessible storage device such as a RAM is used.
[0042]
The display driver 34 is configured by a circuit that drives the display device 24.
[0043]
The sound reproduction driver 35 is configured by a circuit that drives the sound reproduction
device 25.
04-05-2019
9
[0044]
A CPU (Central Processing Unit) 40 controls each part of the three-dimensional display control
device 30, and performs three-dimensional display with sound reproduction using the display
device 24 and the sound reproduction device 25.
Details of the functions of the CPU 40 will be described later.
FIG. 2 is an explanatory view showing an example of a structure which enables stereoscopic
display on the display device 24. As shown in FIG.
The present example is a lenticular method, and uses a lenticular lens 24 a having a
semicylindrical lens group.
The lenticular lens 24a is disposed on the front surface of the display device 24 (viewer's point of
view, that is, z-axis direction in which the left eye EL and the right eye ER exist), and a plurality of
cylindrical convex lenses are connected in the x-axis direction in FIG. It consists of
The display area of the stereoscopic image displayed on the display device 24 is composed of a
right-eye strip image display area 24R and a left-eye strip image display area 24L.
The right-eye strip image display area 24R and the left-eye strip image display area 24L each
have a strip shape elongated in the y-axis direction in FIG. 2 of the screen, and are alternately
arranged in the x-axis direction in FIG.
Each convex lens constituting the lenticular lens 24 a displays each set of rectangular images
including a set of right-eye rectangular image display areas 24 R and a left-eye rectangular image
display area 24 L based on a given observation point of the observer It is formed in the position
corresponding to field 24c.
04-05-2019
10
In FIG. 2, the right-eye strip image displayed in the right-eye strip image display area 24R of the
display device 24 is incident on the right eye ER of the observer by the light refracting action of
the lenticular lens 24a.
Further, the left-eye strip image displayed in the left-eye strip image display area 24L of the
display 24 is incident on the viewer's left eye EL by the light refracting action of the lenticular
lens 24a.
Therefore, the right eye of the observer sees only the right eye strip image and the left eye of the
observer sees only the left eye strip image, and the right eye image and the left eye, which are a
collection of these right eye strip images Stereo vision can be achieved by the left-right parallax
of the left-eye image, which is a set of eye strip images.
[0045]
In addition, although the case where a lenticular system was used was demonstrated to an
example as a structure of the display apparatus 24 for three-dimensional display using FIG. 2,
this invention is not specifically limited to a lenticular system.
[0046]
For example, a light direction control method (time-division light direction control back) in which
the direction of the backlight that illuminates the back surface of the LCD (liquid crystal display
device) constituting the display device 24 is controlled in time division to the right and left eyes
of the observer Stereoscopic display may be performed using a light method).
For the light direction control method, Kentaro Toyooka, Tetsuya Miyashita, Tatsuo Uchida,
“Three-dimensional display using time-division light direction control backlight,” Proceedings
of the 2000 Japan Liquid Crystal Society Symposium, pp. 137-138 (2000 And JP-A-2004-20684.
[0047]
It may be self-luminous or another light source may be used to control the amount of light.
04-05-2019
11
In addition, there is no limitation on the polarization type, anaglyph, naked eye type, etc.
In addition, a method in which liquid crystals or organic ELs are stacked in multiple layers may
be used.
[0048]
In the present embodiment, the display device 24 may be any display device capable of both flat
display and stereoscopic display.
[0049]
Next, details of the functions of the CPU 40 will be described.
[0050]
In FIG. 1, the CPU 40 includes a main control unit 41, a parallax amount detection unit 42, a
sound source position determination unit 43, an image reproduction control unit 44, and a sound
reproduction control unit 45.
[0051]
The main control unit 41 controls all units inside and outside the CPU 40 in a centralized
manner.
[0052]
The parallax amount detection unit 42 detects the parallax amount for each divided area
obtained by dividing the stereoscopic image.
In this example, assuming that the superimposed image of the right eye image and the left eye
image is a stereoscopic image, the amount of parallax to be represented is calculated in each of
the divided regions (for example, 7 × 7 divided regions) obtained by dividing the stereoscopic
image. .
04-05-2019
12
[0053]
First, the amount of parallax will be described using FIGS. 3 and 4.
[0054]
In FIG. 3, the distance (base line length) between the optical axes and the light in the
photographing optical system 14R for the right eye and the photographing optical system 14L
for the left eye in the camera 10 (image pickup apparatus) not shown in FIG. The angle between
the axes (convergence angle) is assumed to be fixed.
A subject image is formed on the right-eye imaging device 134R by the right-eye imaging optical
system 14R, and the right-eye image is generated by the imaging device 134R.
The subject image is formed on the left-eye imaging device 134L by the left-eye imaging optical
system 14L, and the left-eye image is generated by the imaging device 134L.
Further, in FIGS. 4A to 4C, it is assumed that the magnification is equal between the right-eye
image and the left-eye image.
[0055]
In FIG. 4A, the subject is farther from the plurality of imaging optical systems 14R and 14L in
FIG. 3 (in fact, the object is closer to the intersection of the optical axes of both imaging optical
systems 14R and 14L). As shown, the position of the corresponding point 200R of the subject in
the right eye image matches the position of the corresponding point 200L of the subject in the
left eye image (which is the point corresponding to the corresponding point 200R of the right
eye image) Come.
In practice, when the subject is located at the intersection of the optical axes of the plurality of
photographing optical systems 14R and 14L, the positions of the corresponding points 200R and
200L coincide between the right eye image and the left eye image.
04-05-2019
13
On the other hand, as shown in FIG. 4B, the position of the corresponding point 200R of the
subject in the right-eye image as the subject is closer to the plurality of photographing optical
systems 14R and 14L in FIG. The position of the corresponding point 200L of the subject in the
left-eye image is separated.
[0056]
As shown in FIG. 4C, the difference d between the positions of the corresponding points 200R
and 200L between the right-eye image and the left-eye image corresponds to the “disparity
amount” indicating the magnitude of the parallax of the two eyes in the stereoscopic image. .
That is, as described above, when the base line length and the convergence angle are fixed and
the magnification of the image is equal, the “disparity amount” is determined by the subject
distance.
The “parallax amount” corresponds to an amount (projection amount) seen by the observer
when the stereoscopic display is performed.
[0057]
The parallax amount detection unit 42 of this example first calculates the correspondence
between the positions (coordinates) of the corresponding points 200R and 200L in the captured
image (right eye image and left eye image) of each viewpoint and the parallax amount d of both
eyes. Ask.
Next, the coordinates (intermediate coordinates) of the intermediate position between the
corresponding points 200R and 200L are determined and associated with the parallax amount d.
[0058]
For example, when the cup 210 is imaged as shown in FIG. 5, the coordinates of corresponding
04-05-2019
14
points (221R, 222R, etc.) on the right-eye image 220R and the left-eye image 220L for at least
the entire main subject 210 The differences d1 and d2 with the coordinates of the corresponding
points (221L, 222L, etc.) are calculated.
Next, coordinates of an intermediate position (an intermediate position between 221R and 221L,
an intermediate position between 222R and 222L) at each combination of corresponding points
(221R and 221L, 222R and 222L) are determined, and the coordinates and parallax of each
intermediate position The quantities d1 and d2 are associated with each other.
Then, based on the coordinates of the intermediate position, representative of the parallax
amount for each divided area (for example, 7 × 7 divided areas) on the superimposed image
(stereoscopic image) in which the right eye image and the left eye image are superimposed.
Determine the value (eg maximum value).
[0059]
Although only two sets of corresponding points 221R and 222R and corresponding points 221L
and 222L are shown in FIG. 5 for convenience of illustration, in actuality, the amount of parallax
is determined at least over the entire main subject, ie, the cup 210.
[0060]
As a first aspect of the parallax amount detection, there is an aspect of calculating the parallax
amount from a stereoscopic image.
For example, in the right eye image and the left eye image constituting a stereoscopic image, an
edge whose color and / or luminance change is detected, respectively, and is detected based on
the edge detected in the right eye image and the edge detected in the left eye image Then, the
pixels on the right eye image and the pixels on the left eye image are associated with each other,
and the difference d in the position between the associated pixels is calculated.
Moreover, there exists an aspect which acquires a distance image as a 2nd aspect of parallax
amount detection.
04-05-2019
15
Specifically, at the time of imaging, a distance image is acquired using a known distance image
sensor, and a stereoscopic image and a distance image are associated with each other and stored
in the data storage unit 22.
And at the time of reproduction | regeneration of a stereo image, a distance image is acquired
from the data storage part 22, and the distance image is used as parallax amount information.
For example, an infrared light pulse is irradiated to the imaging target range, and the delay time
from the irradiation of the infrared light pulse (infrared echo) reflected in the imaging target
range corresponds to each pixel of the right eye image and the left eye image The subject
distance corresponding to each pixel of the stereoscopic image is calculated by adding and
measuring and associating the measured delay time with the subject distance.
That is, a distance image sensor generates a so-called distance image.
FIG. 6 shows an example of a captured image 220 (right-eye image or left-eye image) of a cup
with a flower pattern and a distance image 230 corresponding to the captured image 220.
The captured image 220 obtained by the imaging elements 134R and 134L is an image
configured by pixels having pixel values (pixel information) indicating the color and luminance of
each point on the subject, and corresponds to an image visually recognized by a person.
On the other hand, the distance image 230 is an image constituted by pixels having pixel values
(distance information) indicating the distance to each point on the subject. Here, the pixel value
(distance information) of the distance image is treated as the amount of parallax. In other words,
the distance image is used as an image (disparity amount image) configured by pixels having
pixel values indicating the disparity amount. In the distance image of FIG. 6, the object distance
(disparity amount) is divided into three levels for convenience, but it goes without saying that the
area may be divided more finely. In addition, since the subject image differs between the righteye image and the left-eye image due to the different viewpoints, the distance image sensor
corresponding to the photographing optical system 14R for the right eye and the photographing
optical system 14L for the left eye are different. By providing a corresponding distance image
sensor in the camera 10, distance images respectively corresponding to the respective captured
images (right-eye image and left-eye image) are accurately obtained. Note that it is not necessary
to correspond the pixels of the distance image and the pixels of the stereoscopic image on a
04-05-2019
16
pixel-by-pixel basis. The subject distance may be determined as the parallax amount for each
two-dimensional block consisting of a plurality of pixels (for example, 3 pixels × 3 pixels) of the
stereoscopic image. Also, instead of infrared light, electromagnetic waves of other wavelengths
(for example, ultrasonic waves) may be used to acquire a distance image. When stereoscopic
display is performed, the parallax amount detection unit 42 acquires the distance image
associated with the stereoscopic image (right-eye image and left-eye image) from the data
storage unit 22 and refers to the pixel values of the distance image. The amount of parallax of
each pixel of the stereoscopic image is detected.
[0061]
In the first aspect described above, there is an advantage that cost reduction of camera hardware
can be achieved by acquiring the parallax amount based on the stereoscopic image itself, and in
the second aspect, the distance image sensor is used. By creating a distance image, there is an
advantage that the amount of parallax can be easily obtained. The sound source position
determination unit 43 determines the sound source position of the reproduction sound field on
the display device 24 based on the amount of parallax for each divided area of the stereoscopic
image. There are various ways of determining the sound source position, which will be described
later.
[0062]
The image reproduction control unit 44 causes the display device 24 to display an image through
the display driver 34. The image reproduction control unit 44 of this example can perform twodimensional display of a two-dimensional image and three-dimensional display of a threedimensional image. It may be configured to be switchable between stereoscopic display of a
stereoscopic image and planar display (displaying only an image of a single viewpoint).
[0063]
The sound reproduction control unit 45 causes the sound reproduction device 25 to reproduce
the sound via the sound reproduction driver 35. When the sound reproduction device 25 is an
analog signal input, the sound data is converted from the digital signal into an analog signal of
the input format of the sound reproduction device 25 and output to the sound reproduction
device 25. When the sound reproduction device 25 is a digital signal input, sound data is
04-05-2019
17
converted into a digital signal of the input format of the sound reproduction device 25.
[0064]
The sound reproduction control unit 45 of the present embodiment sets the real or virtual sound
source position of the reproduction sound field to the sound source position on the display
device 24 determined by the sound source position determination unit 43 and Play data as
sound. Hereinafter, setting of a real or virtual sound source position in a reproduction sound field
may be referred to as “sound image localization”. There are various arrangements of the sound
reproduction apparatus 25 and sound image localization suitable for the arrangement, which will
be described in detail later.
[0065]
In addition, when switching from two-dimensional display to three-dimensional display or when
switching a three-dimensional image to be three-dimensionally displayed, the sound reproduction
control unit 45 of the present embodiment continuously reproduces sound before and after
switching and For example, for 3 seconds, the sound source position on the display device 24
determined based on the parallax amount of each divided area of the stereoscopic image by the
sound source position determination unit 43 is set as the real or virtual sound source position of
the reproduction sound field Play.
[0066]
In the actual sound source position setting, when the sound reproducing device 25 is disposed on
the front or back or the periphery of the display device 24, the position on or near the sound
source position on the display 24 determined by the sound source position determination unit 43
The sound reproduction device 25 actually reproduces the sound.
In the virtual sound source position setting, for example, when the sound reproduction device 25
is disposed apart from the display device 24, if a sound source exists at the sound source
position on the display device 24 determined by the sound source position determination unit
43, a stereoscopic image The sound is reproduced so that the observer (who is the listener of the
sound) feels. In the present specification, both the real sound source position setting and the
virtual sound source position setting in the reproduction sound field are referred to as “sound
image localization”.
04-05-2019
18
[0067]
There are various arrangements of the sound reproduction device 25. First, there is a mode in
which sound reproduction device groups are two-dimensionally arranged on the front or back of
the display device 24 to perform sound image localization. Second, there is a mode in which a
plurality of sound reproducing devices are arranged in a one-dimensional or two-dimensional
manner around the display device 24 to perform sound image localization. Third, there is a mode
in which a plurality of sound reproducing devices are disposed apart from the display device 24
to perform sound image localization. Fourth, there is a mode in which headphones or earphones
are attached to the head of a person to perform sound image localization.
[0068]
In the front view of FIG. 7A and the side view of FIG. 7B, the display device 24 and the sound
reproduction device 25 group in the case where the plurality of sound reproduction devices 25
in a two-dimensional array are arranged on the back side of the display device 24 Indicates The
sound source position determination unit 43 divides the stereoscopic image (right-eye image and
left-eye image) into divided areas in which the amount of parallax is larger than the threshold
and divided areas in which the amount of parallax is equal to or smaller than the threshold. The
position on the display unit 24 corresponding to the divided area (the hatched area in 7 (A)) is
determined as the sound source position of the reproduction sound field. The sound reproduction
control unit 45 is a sound reproduction device 25 (indicated by hatching in the sound
reproduction device 25 (FIG. 7B) positioned at or near the sound source position among the
sound reproduction devices 25 two-dimensionally arrayed on the back of the display device 24.
Output sound from the playback device).
[0069]
In the present example, the case where the stereoscopic image is divided into 7 × 7 is shown as
an example, but the number of divided areas is arbitrary. For example, the maximum value of the
amount of parallax in the divided area is compared with a threshold.
[0070]
04-05-2019
19
In the present example, the display device 24 is provided with a large number of holes. For
example, in the display device 24 shown in FIG. 2, holes are provided between the strip image
areas 24R and 24L. The sound generated by the sound reproducing device 25 disposed on the
back of the display 24 is transmitted to the front of the display 24 through the holes.
[0071]
In the front view of FIG. 8A and the side view (B) of FIG. 8, among the divided areas of the
stereoscopic image, the divided area having the largest parallax amount is determined. In this
example, the 3D image of the eyebrow is divided into 7 × 7, and it is determined that the divided
area (the divided area drawn with oblique lines in FIG. 8A) in which the scissors of the eyelid are
present is the parallax amount maximum. The position on the display device 24 corresponding to
the divided area is determined as the sound source position, and the sound reproducing device
25 located at or near the sound source position among the sound reproducing devices 25 twodimensionally arrayed on the back of the display device 24. The sound is output from (the sound
reproduction device which has been hatched in FIG. 8B).
[0072]
The front view of FIG. 9 (A) and the side view of FIG. 9 (B) show the display device and the sound
reproduction device group when the sound reproduction device group of the two-dimensional
array is arranged on the front side of the display device.
[0073]
In this example, the sound reproducing devices 25 are transparent and planar sound reproducing
devices 25 so that the image on the display 24 can be seen through.
For example, a transparent diaphragm is provided on the front side of the display device 24, and
the diaphragm is vibrated by an electric signal to output a sound. Here, even if not all the sound
reproduction devices 25 are transparent, at least a portion (for example, half or more) may be
transparent. A known transparent speaker can be used as such a sound reproduction device.
04-05-2019
20
[0074]
FIG. 10 is a flowchart showing a flow of an example of stereoscopic display control processing to
which the present invention is applied. This process is executed by the CPU 40 of FIG. 1
according to a program.
[0075]
In step S1, the main control unit 41 detects an operation on the operation unit 31.
[0076]
In step S2, the main control unit 41 determines whether to perform reproduction based on the
content of the operation.
When the reproduction is not performed, the process returns to step S1. When the reproduction
is performed, the process proceeds to step S3.
[0077]
In step S3, the data acquisition unit 32 acquires image and sound data.
[0078]
In step S4, the main control unit 41 determines whether the acquired image is a stereoscopic
image or a planar image.
If it is a stereoscopic image, the process proceeds to step S6. If it is a planar image (if it is not a
stereoscopic image), the process proceeds to step S5.
[0079]
04-05-2019
21
If the image is a flat image, the image reproduction control unit 44 performs image reproduction
and the sound reproduction control unit 45 performs sound reproduction in step S9. In this
example, all sound reproducing devices are selected. That is, as shown in the plan view of FIG.
11A and the side view of FIG. 11B, a plane image is displayed on the display unit 24 in plan view,
and sound (for example, background sound) is reproduced from all the sound reproduction
devices 25. . In this example, the sound reproduction devices 25 are two-dimensionally arranged
on the back surface of the display device 24, and the sound reproduction device 25 on the back
surface of the area (entire area) hatched on the display device 24 in FIG. Sound is output.
[0080]
If the image is a stereoscopic image, the parallax amount detection unit 42 detects the parallax
amount of each divided area of the stereoscopic image in step S6. For example, the distance
image is acquired from the data storage unit 22 and the amount of parallax is detected.
Alternatively, corresponding points are detected corresponding to each other in the captured
images (right-eye image and left-eye image) of each viewpoint constituting the stereoscopic
image, and the difference in position on the stereoscopic image between the detected
corresponding points is calculated as the parallax amount Do. Then, a representative value of the
amount of parallax is determined for each divided area.
[0081]
In step S7, the sound source position determination unit 43 determines the pop-out portion of
the stereoscopic image. For example, the divided area where the representative value of the
amount of parallax is the largest is determined, and the position on the display device 24
corresponding to the divided area is determined as the sound source position. A divided area in
which the representative value of the amount of parallax is larger than a threshold may be
determined, and the position on the display device 24 corresponding to the divided area may be
determined as the sound source position.
[0082]
In step S8, the sound reproduction control unit 45 sets a real or virtual sound source position in
the reproduction sound field. In this example, the sound reproduction device 25 located at the
sound source position (or the sound reproduction device 25 located near the sound source
04-05-2019
22
position) is selected.
[0083]
Then, in step S9, the image reproduction control unit 44 stereoscopically displays a stereoscopic
image on the display device 24, and the sound reproduction control unit 45 reproduces a sound
from the sound reproduction device 25. Here, the sound output is performed only from the
sound reproducing device 25 selected in step S8 for a certain period of time after the
stereoscopic image is displayed.
[0084]
Just before the switching of the image, as shown in FIGS. 11A and 11B, the sound is output from
the all-tone reproducing device 25 of the sound reproducing device group arranged in a twodimensional shape, but immediately after the switching of the image. As shown in FIGS. 8A and
8B, the sound is output only from the sound reproduction device 25 disposed at the location
where the amount of parallax is large. FIGS. 8A and 8B show the case where the sound is output
only from the sound reproducing device 25 disposed at the position where the parallax amount
is the largest, but FIGS. 7A and 7B show the case. As described above, the sound may be output
from a plurality of sound reproducing devices 25 arranged at locations where the amount of
parallax is larger than the threshold.
[0085]
Thus, the observer can recognize the stereoscopic effect of the switched stereoscopic image
while flowing the background sound (so-called BGM).
[0086]
The flow of the reproduction control process has been described by taking the case where the
sound reproduction device 25 is disposed on the back of the display device 24 as an example
using FIGS. 8 and 11, but as shown in FIGS. 9A and 9B. Even when the transparent planar sound
reproducing device 25 is disposed on the front surface of the display device 24, the reproduction
control processing may be performed in the same manner.
[0087]
04-05-2019
23
As described above, when the sound reproduction device group is disposed on the back or front
of the display device 24, the sound reproduction control unit 45 determines the sound source
position of the display device 24 (or the position near it) determined by the sound source
position determination unit 43. Sound reproduction is performed from the sound reproduction
device 25 disposed in
When controlling the actual sound source position on the display device 24 in this manner, it is
preferable to be able to directly show the observer of the stereoscopic image a region where the
amount of parallax of the stereoscopic image is large.
The sound output level (volume) of the sound reproduction device 25 corresponding to the
sound source position may be made larger than that of the other sound reproduction devices.
[0088]
In addition, by integrating the display device 24 and the sound reproduction device 25, it is
possible to effectively use the indoor space.
[0089]
As shown in FIGS. 12A to 12C, a plurality of sound reproduction devices 25 may be arranged
around the display device 24.
The arrangement may be one-dimensional or two-dimensional. In this case, the sound
reproduction control unit 45 performs sound reproduction from the sound reproduction device
closest to the sound source position of the display device 24 determined by the sound source
position determination unit 43.
[0090]
Note that there are various modes of sound image localization, and even if a plurality of sound
reproducing devices 25 are not arranged on the front or back or the periphery of the display 24,
04-05-2019
24
they may be arranged at a position separated from the display 24. The sound reproduction
control unit 45 performs sound image localization using the sound source position on the display
device 24 determined by the sound source position determination unit 43 as a virtual sound
source position. For such sound image localization, a known sound image localization technique
used in a known sound field reproduction system can be used.
[0091]
For example, a person is listening to two-channel sound with both ears, and in the corresponding
two-channel (stereo) reproduction sound field, sound image localization can be performed using
two sound reproduction devices. The reproduction sound field may be limited to the vicinity of
the listener's ear, and sound image localization may be performed using headphones or
earphones worn on the head of a person. Of course, a speaker may be used.
[0092]
In a multi-channel playback sound field, sound image localization can be easily performed to the
side or back of the listener. For example, in a 5.1 channel surround system, five sound
reproducing devices and one sound reproducing device dedicated to bass are disposed at the
front right, front center, front left, rear right and rear left so as to surround the listener. The
sound image can be well localized from the side of the listener to the rear. It is also possible to
localize the sound image behind the listener without using the sound reproduction device behind
the listener by using the indoor transmission path (for example, echo).
[0093]
Specifically, sound image localization is performed by controlling the level difference, time
difference, phase difference and the like of the sound.
[0094]
As described above, according to the present invention, when switching from two-dimensional
display to three-dimensional display and when switching a three-dimensional image to be threedimensionally displayed, by using sound image localization, where in the three-dimensional
image appears to pop out It is possible to make the observer know if there is any.
04-05-2019
25
[0095]
However, along with the sound image localization, the stereoscopic image displayed on the
display device 24 may be gradually completed.
Specifically, when switching from two-dimensional display to three-dimensional display, and
when switching a three-dimensional image to be three-dimensionally displayed, from a region
with a large amount of parallax to a region with a small amount of parallax Information of a
stereoscopic image to be given to the display device 24 is added in order from the area having a
small amount of parallax to the area having a large amount of parallax among them.
[0096]
For example, the sound reproduction control unit 45 causes the display device 24 to sequentially
display the missing portion of the stereoscopic image from the state in which all the stereoscopic
images are not displayed or the state in which the stereoscopic image is partially displayed on
the display device 24. Go.
[0097]
Also, for example, after the sound reproduction control unit 45 causes the display device 24 to
stereoscopically display a stereoscopic image in a single color, the sound reproduction control
unit 45 sequentially adds color information of the stereoscopic image to be provided to the
display device 24 to sequentially display stereoscopic images. Gradually change from single color
(eg, monochrome) stereoscopic display to color stereoscopic display.
[0098]
Further, for example, after the sound reproduction control unit 45 causes the display device 24
to stereoscopically display a stereoscopic image with a single luminance, the sound reproduction
control unit 45 sequentially adds luminance information of the stereoscopic image to be
provided to the display device 24 Is gradually changed from single-brightness stereoscopic
display to multi-tone brightness stereoscopic display.
[0099]
Also, for example, after the sound reproduction control unit 45 causes the display device 24 to
display only a single viewpoint image forming a stereoscopic image, the sound reproduction
control unit 45 sequentially adds other viewpoint images of the stereoscopic image to be
04-05-2019
26
provided to the display device 24. Thus, the three-dimensional image is gradually changed from
the two-dimensional display to the three-dimensional display.
[0100]
Also, for example, after causing the display device 24 to roughly display the stereoscopic image,
the sound reproduction control unit 45 changes each region of the stereoscopic image from
rough display to precise display.
[0101]
The sound reproducing apparatus is not particularly limited to the above-described example.
For example, in the case of being disposed on the front or back of the display device 24, the
shape of the sound reproduction device 25 is not particularly limited to a plane.
Further, in the case of arranging around the display device 24 and in the case of arranging away
from the display device 24, the shape and the array shape of the sound reproducing device are
not particularly limited.
Also, for example, the headphones are not particularly limited to those worn on the ear, and for
example, bone-conduction headphones may be used.
[0102]
Also, the sound source position may be moved.
For example, when there are a plurality of divided areas in which the amount of parallax is larger
than the threshold in the stereoscopic image, the sound source position is moved in the order of
small amount of parallax or in order of large amount of parallax.
That is, the sound source position determination unit 43 moves the sound source position from
04-05-2019
27
the divided area with a small amount of parallax toward the divided area with a large amount of
parallax using the auditory characteristics of a person who easily recognizes the sound source
position by movement of the sound source position. Alternatively, the sound source position may
be moved from a divided area with a large amount of parallax toward a divided area with a small
amount of parallax.
[0103]
In this case, even if the user does not allow the user to recognize the most visible part (the
divided area having the largest amount of parallax), the user recognizes to the user a plurality of
parts (some divided areas having the amount of parallax equal to or more than the threshold) I
hope that you can do it.
[0104]
The movement of the sound source position in the reproduction sound field may be performed
by switching the sound reproducing device 25 for reproducing the sound to move the actual
sound source position, or moving the virtual sound source position.
[0105]
Further, the present invention does not exclude the sound image localization of the reproduction
sound field based on the sound source position of the original sound field, and acquires the
sound source position information of the original sound field from the data storage unit 22 and
displays three-dimensionally. After sound localization is performed based on the parallax amount
for only a second), sound reproduction may be performed by switching to sound localization
based on sound source position information of the original sound field.
[0106]
Further, the present invention is not particularly limited in the case where stereoscopic display is
performed by one display device 24 as shown in FIG. 13 (A).
As shown in FIG. 13 (B), stereoscopic display may be performed by a plurality of displays
equivalent to the single display shown in FIG. 13 (A) using a plurality of display devices 24a to
24d.
04-05-2019
28
For example, when it is desired to display a large image in an exhibition hall or the like, a multidisplay is used.
[0107]
Further, the display control device is not particularly limited in the case of controlling
reproduction of an image and sound.
The present invention can be applied to various electronic devices such as a television and a
personal computer as well as various types of portable terminals such as digital cameras and
mobile phones.
[0108]
The present invention is not limited to the examples described in the specification and the
examples illustrated in the drawings, and it goes without saying that various design changes and
improvements may be made without departing from the scope of the present invention.
[0109]
22: data storage unit 24: display device 25: sound reproduction device 30: stereoscopic display
control device 32: data acquisition unit 40: CPU 42: parallax amount detection unit 43: sound
source position determination unit 44: Image reproduction control unit 45 Sound reproduction
control unit 100 Three-dimensional display system
04-05-2019
29
Документ
Категория
Без категории
Просмотров
0
Размер файла
42 Кб
Теги
jp2010199739
1/--страниц
Пожаловаться на содержимое документа