close

Вход

Забыли?

вход по аккаунту

?

JP2017022600

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2017022600
Abstract: To obtain a composite image without a sense of incongruity at the time of
communication between users. An image communication apparatus includes: a plurality of
imaging cameras arranged along a screen; a position detection unit for detecting position
information on a position of a caller relative to the screen; The image of the caller is selected
from the images of the plurality of imaging cameras 9 based on the direction detection unit 24
that detects the direction information on the direction of the speaker, and the position
information and the direction information of the caller, An image combining unit 27 that
combines an image and a plurality of images to generate a composite image, an image
transmission unit 29 that transmits the composite image to the communicator, and a composite
image received from the communicator and displayed on the screen 3 And an image receiving
unit 31. [Selected figure] Figure 4
Image communication device
[0001]
The present invention relates to an image communication apparatus that communicates in both
directions using images.
[0002]
BACKGROUND In recent years, it has become common to use a videophone system that
communicates interactively using an image when making a conference or a personal call.
03-05-2019
1
When a video telephone system is used by a plurality of users, it is required to obtain an image
with a wide field of view, and therefore, a configuration in which a plurality of cameras are
arranged around a display screen is used. At this time, images acquired by a plurality of cameras
are transmitted to the other party's videophone system.
[0003]
On the other hand, as a technique for combining a plurality of images to obtain one combined
image, those described in Patent Documents 1 and 2 below are known. According to such a
technique, a plurality of images acquired by a plurality of cameras can be connected with high
accuracy and synthesized.
[0004]
JP, 2011-124837, A JP, 2014-86097, A
[0005]
However, it is difficult to obtain a composite image without discomfort among the callers when a
plurality of users move freely around by simply applying the above-described conventional image
combining technology to a videophone system.
For example, the image of the caller in the composite image is captured from various directions,
and communication tends to be difficult.
[0006]
Therefore, the present invention has been made in view of the above problems, and an object of
the present invention is to provide an image communication apparatus capable of obtaining a
composite image without a sense of discomfort when communicating between users.
[0007]
In order to solve the above problems, an image communication apparatus according to an aspect
of the present invention includes: a plurality of imaging cameras arranged along a screen; and a
03-05-2019
2
position detection unit that detects position information regarding a position of a caller on the
screen; Selecting an image of the caller from among a plurality of images acquired by a plurality
of imaging cameras based on a direction detection unit that detects direction information on the
direction of the caller with respect to the screen, and position information and direction
information; An image combining unit that combines an image of a selected caller and a plurality
of images to generate a composite image, and an image that transmits one of the composite
image and a plurality of images acquired by a plurality of imaging cameras A transmission unit
and an image display unit for displaying on a screen a composite image generated on the basis of
an image of a communicator who is a communication partner of the caller.
[0008]
According to the image communication apparatus of the above aspect, information related to the
position of the caller with respect to the screen is detected, and information related to the
direction of the caller with respect to the screen is acquired, and a plurality of pieces are
arranged along the screen based on the information. The image of the caller is selected from the
images acquired by the imaging camera, and a composite image is generated using the image and
the plurality of images acquired by the plurality of imaging cameras.
Images or composite images acquired by a plurality of imaging cameras are transmitted to the
communicator, and a composite image generated based on the images of the communicator is
displayed on the screen.
With such a configuration, based on the position and the orientation of the caller, it is possible to
include the image of the caller with no sense of incongruity at the time of communication with
the communicator in the synthesized image and display it. As a result, smooth communication
between the interlocutor and the caller can be realized.
[0009]
Here, the image combining unit may select the image of the imaging camera closest to the line
extended in the direction identified by the direction information from the position of the party
identified by the position information as the image of the party. Good. In this way, it is possible to
select the image of the caller as the directly facing image, and it is possible to include the image
of the caller without a sense of incongruity at the time of communication with the synthesized
03-05-2019
3
image and display it. As a result, smooth communication between the interlocutor and the caller
can be realized accurately.
[0010]
Further, the information processing apparatus may further include an information acquisition
unit that acquires position information on the position of the interlocutor on the screen, and the
image combining unit may correct the composite image based on the interlocutor's position
information. In this case, based on the positional relationship between the interlocutor and the
communicator, it is possible to display the image of the communicator with no discomfort at the
time of communication, by including it in the composite image. As a result, smooth
communication between the interlocutor and the caller can be realized accurately.
[0011]
In addition, when the image synthesis unit reflects the position of the interlocutor and the
position of the caller in the virtual space across the screen, the image of the imaging camera
closest to the line connecting the interlocutor and the caller The composite image may be
corrected so as to select the image of the caller. In this case, it is possible to display an image of
the caller as viewed from the front as a composite image. As a result, it is possible to cause the
interlocutor to display an image of the near talker at the time of actual interaction in the virtual
space.
[0012]
Further, the position detection unit may include a camera for acquiring a distance image
arranged along the screen, and the direction detection unit may include a plurality of directional
microphones. In this case, information on the position or orientation of the caller relative to the
screen can be efficiently obtained.
[0013]
According to the present invention, it is possible to obtain a composite image without any
discomfort when communicating between users.
03-05-2019
4
[0014]
FIG. 1 is a perspective view showing the configuration of an image communication apparatus
according to a preferred embodiment of the present invention.
It is the top view which looked at the visual field of the camera 9 for imaging of FIG. 1 from the
upper direction along the screen 3. FIG. FIG. 2 is a side view of the field of view of the imaging
camera 9 and the distance measurement camera 11 of FIG. It is a block diagram which shows the
function structure of the image processing apparatus 7 of FIG. It is a conceptual diagram which
shows the method of the image selection by the image selection part 25 of FIG. It is a figure
which shows the image of a production | generation of the synthesized image by the image
synthetic | combination part 27 of FIG. It is a flowchart which shows the operation | movement
procedure of the image communication by the image communication apparatus 1 of FIG. It is a
side view which shows the structure of the camera 9 for imaging concerning the modification of
this invention, and the camera 11 for ranging. It is a side view showing the composition of
camera 35 concerning the modification of the present invention. It is a perspective view which
shows arrangement | positioning of the distance measurement camera 11 in the modification of
this invention, and the camera 9 for imaging. It is a perspective view which shows arrangement |
positioning of the distance measurement camera 11 in the modification of this invention, and the
camera 9 for imaging. It is a perspective view showing arrangement of distance measurement
camera 11 in a modification of the present invention.
[0015]
Hereinafter, preferred embodiments of an image communication apparatus according to the
present invention will be described in detail with reference to the drawings. In the description of
the drawings, the same or corresponding parts will be denoted by the same reference symbols,
without redundant description.
[0016]
An image communication apparatus 1 according to a preferred embodiment of the present
invention is an apparatus for performing communication by transmitting and receiving an image
between remote places, and by providing the image communication apparatus 1 as a set, A video
telephone system is configured to communicate while referring to images among a plurality of
users.
03-05-2019
5
[0017]
FIG. 1 shows a schematic configuration of an image communication apparatus 1 according to the
present embodiment.
As shown in the figure, the image communication apparatus 1 displays an image by projecting an
image light on a screen (screen) 3 arranged to face a plurality of users U and the front of the
screen 3 The apparatus (projector, image display unit) 5 and an image processing apparatus 7
for performing image processing are provided. The image communication apparatus 1 is referred
to as an apparatus having the same configuration (hereinafter, referred to as "opposite
apparatus"). , And transmits the composite image generated by the image communication
apparatus 1 to the opposing apparatus, receives the composite image generated by the opposing
apparatus, and displays the composite image on the screen 3). Do.
[0018]
The screen 3 includes a plurality of imaging cameras 9 for capturing an image on the front
surface (image display surface) side of the screen 3, and a plurality of distance measurement
cameras 11 for acquiring distance images on the front surface of the screen 3. A pulse light
source 13 for emitting near-infrared pulse light and a plurality of directional microphones 15 for
detecting the direction of a talker who is a user who is speaking from among a plurality of users
U are attached. The distance measurement camera 11 and the pulse light source 13 function as a
position detection unit that detects position information on the position of the caller with respect
to the screen 3, and the directional microphone 15 detects directional information on the
direction of the caller with respect to the screen 3. It functions as a direction detection unit to
detect.
[0019]
The imaging camera 9 is an image sensor arranged two-dimensionally at equal intervals along
the front surface of the screen 3 so that an image on the front side of the screen 3 can be imaged
through the minute holes formed in the screen 3 It is fixed to the rear side of the screen 3. For
example, the imaging camera 9 has a configuration in which five pieces are arranged at equal
03-05-2019
6
intervals along the long side direction of the screen 3 and the five arrays are arranged in two
rows in the short side direction.
[0020]
Two distance measurement cameras 11 are fixed side by side at the upper end of the center of
the front of the screen 3, and the pulse light source 13 is disposed between the distance
measurement cameras 11. The distance measurement camera 11 obtains a distance image
including distance information for each pixel by measuring in real time for each pixel the time at
which the light pulse emitted by the pulse light source 13 strikes the target and returns. Image
sensor of the. By controlling the operations of the distance measurement camera 11 and the
pulse light source 13 by the image processing device 7, a distance image is acquired in real time.
[0021]
A plurality of directional microphones 15 are fixed side by side at the lower end of the front of
the screen 3 and detect audio signals emitted from a sound source including the user U on the
front side of the screen 3. The audio signal detected by these directional microphones 15 is
processed by the image processing device 7 so that the arrival direction of the sound from the
sound source including the user U to the surface of the screen 3 is detected in real time. As this
detection method, a known method of calculating the direction of arrival of sound using a
microphone array (for example, a correlation function method or a delay-and-sum array method)
is used.
[0022]
2 is a plan view of the field of view of the imaging camera 9 viewed from above along the screen
3, and FIG. 3 is a side view of the fields of view of the imaging camera 9 and the distance
measurement camera 11 along the screen 3. FIG. As shown in these figures, the focal length and
the angle of view of the imaging camera 9 are set so that an image of the space S 0 in a
predetermined range from the screen 3 can be imaged. Furthermore, in the imaging camera 9, in
the space S 0, the optical axes and the angle of view thereof are set such that the fields of view V
1 of the imaging cameras 9 adjacent vertically and horizontally and horizontally overlap.
Thereby, when combining the image acquired by the camera 9 for imaging and producing |
generating a synthetic | combination image, the discontinuity of the object caught by the image
03-05-2019
7
can be reduced. In addition, in order to reduce the occurrence of a dead angle in the space S 0,
the distance measurement camera 11 has its field angle set so as to have a wider field of view V
2 as compared with the imaging camera 9. The optical axis is directed downward.
[0023]
Next, the configuration of the image processing apparatus 7 will be described with reference to
FIG. FIG. 4 is a block diagram showing the functional configuration of the image processing
apparatus 7. The image processing device 7 is physically configured by a data processing device
such as a personal computer and an image processing device, and includes an arithmetic
processing circuit such as a CPU, a memory such as a RAM and a ROM, a data storage device
such as a hard disk device, and communication It has a built-in device. Then, as shown in FIG. 4,
the image processing device 7 includes, as functional components, a position information
acquisition unit 21, a position detection unit 23, a direction detection unit 24, an image selection
unit 25, an image synthesis unit 27, and an image transmission unit 29 and an image reception
unit (image display unit) 31.
[0024]
The position detection unit 23 of the image processing device 7 uses the distance images
detected by the plurality of distance measurement cameras 11 and the plurality of images
acquired by the plurality of imaging cameras 9 to allow the caller to the screen 3 The position
information regarding the position of a certain plurality of users U is detected. Specifically, the
position detection unit 23 detects the face positions of the plurality of users U on the respective
images by performing known face recognition processing on the respective images acquired by
the plurality of imaging cameras 9. . When recognizing the face of the user U from the image, a
known face recognition method using machine learning is used. Thereafter, based on the face
position of the user U recognized on the image and the center position of the lens of the imaging
camera 9 corresponding to the image, the position detection unit 23 starts the center position in
the three-dimensional space The position of a straight line passing through the face position of
the user U is specified. Furthermore, the position detection unit 23 combines a plurality of
distance images to generate a combined distance image (depth map) indicating the threedimensional shape of the object located in the space S 0, and determines this combined distance
image and each The three-dimensional coordinates based on the screens 3 of the faces of the
plurality of users U are calculated as position information by using the information specifying the
straight line corresponding to the face position of the user U. That is, the position detection unit
23 extracts an object which is determined to intersect the straight line in an object which is
03-05-2019
8
recognized as having a shape similar to a person in the composite distance image, and the threedimensional coordinates of the object are Calculated as 3D coordinates of face. Further, the
position detection unit 23 outputs the detected three-dimensional coordinates of the faces of the
plurality of users U to the direction detection unit 24 and the image selection unit 25. Here, the
position detection unit 23 may detect the three-dimensional coordinates of the face of the user U
by separately processing a plurality of distance images without combining the distance images.
[0025]
The direction detection unit 24 continuously detects the arrival directions of the sound from the
sound source with respect to the surface of the screen 3 for the plurality of sound sources based
on the plurality of sound signals obtained by the directional microphone 15. Then, based on the
detected direction of arrival of the sound, the direction detection unit 24 makes the directions of
the faces of the plurality of users U detected by the position detection unit 23 equal to the
directions of arrival of sounds from the plurality of detected sound sources. , And outputs, to the
image selection unit 25, three-dimensional coordinates of faces of the plurality of users U and
direction information on the orientation of the faces of the respective users U. Furthermore, the
direction detection unit 24 transmits the acquired position information (three-dimensional
coordinates) of the user U to the opposing apparatus.
[0026]
The position information acquisition unit 21 receives (acquires) position information on the
position of the communicator, who is the communication partner of the caller, with respect to the
screen of the caller from the opposite device. This position information is obtained as position
information of the face of the caller in the opposite device in the same manner as the image
communication device 1. Specifically, the position information acquisition unit 21 acquires threedimensional coordinates based on the screen on the facing device side of the face of the
communicator.
[0027]
The image selection unit 25 selects, for each of the plurality of users U detected by the position
detection unit 23, an image of a combination source from among the plurality of images acquired
by the imaging camera 9. For example, for the user U whose face direction is not detected, the
03-05-2019
9
image of the imaging camera 9 closest to the three-dimensional coordinates is selected based on
the three-dimensional coordinates calculated for the user U. On the other hand, for the user U
whose face orientation has been detected, an image is selected as follows. That is, based on the
three-dimensional coordinates of the face of the user U output from the direction detection unit
24 and the direction information of the user U, the image of the combination source is selected
from the plurality of images. FIG. 5 is a conceptual view showing a method of image selection by
the image selection unit 25. As shown in FIG. As shown in the figure, the image selection unit 25
sets the coordinates of the screen 3 and the coordinates of the imaging camera 9 in the virtual
space, and further, coordinates of the screen 3 in the virtual space across the screen 3.
Coordinates P 2 corresponding to the three-dimensional coordinates of the two users U 1 and U
2 detected by the position detection unit 23 are reflected on the basis of the reference. Here, the
virtual space is a virtual space in which the interlocutors are virtually arranged based on the
position information with the screen 3 interposed in the real space including the users U 1 and U
2 who are the talkers, the screen 3 and the imaging camera 9. It is a space that Then, the image
selection unit 25 extends the position of the line L extended from the coordinates P 2 of the user
U 1 in the virtual space to the direction specified by the direction information (for example, angle
information α with respect to the screen 3) of the user U 1. It recognizes and selects the imaging
camera 9 corresponding to the position closest to the line L. According to the example of FIG. 5,
the third imaging camera 9 from the left is selected. Here, the imaging camera 9 selected for the
user U 1 is limited to the imaging camera in which the three-dimensional coordinates of the face
of the user U 1 are within the angle of view. That is, the number of imaging cameras 9 to be
selected for the user U 1 depends on the distance from the screen 3 of the user U 1.
[0028]
Further, the image selection unit 25 may correct the selection result of the imaging camera 9
obtained as described above based on the position information on the position of the
communicator acquired by the position information acquisition unit 21. Then, the image
selection unit 25 may select an image of a combination source from a plurality of images based
on the corrected selection result. In this case, the image selection unit 25 further reflects
coordinates P 1 corresponding to the three-dimensional coordinates of the interlocutor acquired
in the virtual space. Then, the image selection unit 25 recognizes the position of a line L 1
connecting the coordinates P 1 of the interlocutor and the coordinates P 2 of the user U 1 in the
virtual space, and corresponds to the position closest to the line L 1. The selection result of the
imaging camera 9 is corrected so that the imaging camera 9 is selected for the user U 1.
According to the example of FIG. 5, the fourth imaging camera 9 from the left is selected.
Thereafter, the image selection unit 25 outputs the information of the imaging camera 9 selected
for each of the plurality of users U 1 and U 2 and the images acquired by all the imaging cameras
9 to the image combining unit 27 in real time. .
03-05-2019
10
[0029]
Returning to FIG. 4, the image combining unit 27 combines the images acquired by the imaging
camera 9 using the information of the imaging camera 9 selected for each of the plurality of
users U to generate a composite image. At this time, the image combining unit 27 combines the
images of the imaging camera 9 arranged in a two-dimensional manner on the screen 3 so as to
correspond to the arrangement for the parts corresponding to the background. Generate At that
time, the image combining unit 27 refers to the combined distance image so that corresponding
points coincide with each other at the boundary between two images of the imaging cameras 9
adjacent to each other, that is, to correct the parallax. Adjust the position of the combination. In
addition, the image combining unit 27 superimposes the images of the plurality of users U on the
background combined image created in this manner. Specifically, for each user U, an image of a
part including the face of the user U is cut out from the image of the selected imaging camera 9
and cut out at a position corresponding to the three-dimensional coordinates of the face of the
user U on the composite image Overlay the images. When cutting out the image of the user U
from the image, the search result of the three-dimensional coordinates of the face of the user U
for the combined distance image by the position detection unit 23 is referred to. The image
position of the face of the user U is specified by converting into the coordinates of.
[0030]
The unit of image selection can be set appropriately, and the image may be selected in units of
the entire user U, or the image may be selected in units of the part of the user U (face, neck,
body, etc.) It may be cut out. However, when cutting out and superimposing images in units of
regions of the user U, in order to prevent image discontinuities in the composite image, the
composite distance image is referred to and the boundaries between a plurality of images to be
superimposed correspond to each other It is preferable to adjust the position of the superposition
of the images so that the points coincide, that is, the parallax is corrected.
[0031]
FIG. 6 shows a generated image of a synthesized image by the image synthesis unit 27. 6 (a) to 6
(c) show images acquired by the second, middle, and right second imaging cameras 9 from the
left corresponding to FIG. 5, respectively. In, the synthesized image generated by the image
03-05-2019
11
synthesizing unit 27 using those images is shown. Thus, for the user U 1, as shown in FIG. 5, the
image of the middle imaging camera 9 closest to the line extending from the user in the direction
of the face direction as shown in FIG. 5, ie, FIG. Is selected, and an image G 1 cut out from the
image is synthesized into a composite image. On the other hand, for the user U 2 whose face
direction is not detected, the image of the second imaging camera 9 closest to the position of the
user U 2, that is, the image of FIG. An image G 2 cut out from the image is synthesized into a
composite image.
[0032]
Referring back to FIG. 4 again, when the image transmission unit 29 is obtained by repeating the
processing of the position information acquisition unit 21, the position detection unit 23, the
direction detection unit 24, the image selection unit 25, and the image combination unit 27. The
plurality of composite images of the series are transmitted as a continuous moving image to the
opposing apparatus on the communicator side. In addition, the image reception unit 31
continuously receives, from the opposite device, a time-series composite image generated on the
basis of the image of the communicator, and transmits the time-series composite image to the
image display device 5. To display as a moving image.
[0033]
An operation procedure of image communication by the above-described image communication
apparatus 1 will be described with reference to FIG. FIG. 7 is a flowchart showing an operation
procedure of image communication by the image communication apparatus 1.
[0034]
First, in the image processing apparatus 7, when an instruction to start communication from the
user U is accepted, the image communication process is started (step S01). Then, the position
detection unit 23 of the image processing device 7 detects the position of the face of the user U
who is a plurality of talkers on the front side of the screen 3 (step S02). Next, the direction
detection unit 24 of the image processing device 7 estimates direction information on the
direction of the face of the user U (step S03). At the same time, the positional information
acquisition unit 21 of the image processing device 7 acquires information of three-dimensional
coordinates on the screen of the communicator from the opposing apparatus (step S04).
03-05-2019
12
[0035]
Thereafter, the image selecting unit 25 of the image processing device 7 uses the position
information on the positions of the faces of the plurality of users U and the direction information
on the directions of the faces of the plurality of users U to perform a plurality of imagings for
each user U An image of a combination source is selected from the images acquired by the
camera 9 (step S05). Then, the image combining unit 27 of the image processing device 7
combines the images acquired by the plurality of imaging cameras 9 based on the selection result
by the image selecting unit 25 to generate a combined image (step S06). .
[0036]
Furthermore, the generated composite image is transmitted to the opposing apparatus by the
image transmission unit 29 of the image processing apparatus 7 each time (step S07). At the
same time, the image reception unit 31 of the image processing device 7 receives the composite
image from the opposite device, and the received composite image is displayed on the screen 3
(step S08). The processes in steps S02 to S08 described above are repeated until an instruction
to end communication from user U is received (step S09), whereby real-time image
communication between remote places is performed.
[0037]
According to the image communication apparatus 1 described above, positional information on
the position of the caller with respect to the screen 3 is detected, and directional information on
the direction of the caller with respect to the screen 3 is obtained. From among the images
acquired by the plurality of imaging cameras 9 arranged along the line, the image of the caller is
selected, and the image is synthesized using the images and the plurality of images acquired by
the plurality of imaging cameras 9. An image is generated. Further, the composite image is
transmitted to the communicator, and the composite image received from the communicator is
displayed on the screen 3. With such a configuration, based on the position and the orientation
of the caller, it is possible to include the image of the caller with no sense of incongruity at the
time of communication with the communicator in the synthesized image and display it. In
addition, it is possible to display an image of a communicator who is not uncomfortable at the
time of communication even for the caller. As a result, smooth communication between the
03-05-2019
13
interlocutor and the caller can be realized.
[0038]
Specifically, based on the position of the caller's face and the direction of the caller's face with
respect to the screen, it is possible to display an image of the caller who is not uncomfortable at
the time of communication in the synthesized image. . As a result, smooth communication
between the interlocutor and the caller can be realized more accurately. In particular, the
position of the caller is reflected in the virtual space, and the image of the imaging camera 9
closest to the line extending from the position of the caller to the direction of the caller in the
virtual space is selected as the image of the caller Therefore, it is easy to reflect an image of the
talker facing the talker in front of the talker when the talker talks with the talker in the
composite image. As a result, it is possible to cause the interlocutor to display an image of the
near talker at the time of actual interaction in the virtual space.
[0039]
In addition, the position of the interlocuator is further reflected in the virtual space across the
screen, and the image of the imaging camera 9 closest to the line connecting the interlocutor's
position and the caller's position in the virtual space is called Correct to select as a person's
image. As a result, it is possible to display an image of the caller as viewed from the front as a
composite image. As a result, it is possible to cause the interlocutor to display an image of the
near talker at the time of actual interaction in the virtual space.
[0040]
In addition, since the distance detection camera 11 is included as the position detection unit and
the directional microphone 15 is included as the direction detection unit, information regarding
the position or direction of the caller relative to the screen 3 can be efficiently obtained. Can.
[0041]
The present invention is not limited to the embodiments described above.
[0042]
03-05-2019
14
In the above embodiment, the distance measurement camera 11 as the position detection unit
and the imaging camera 9 for acquiring the image of the synthesis source of the synthesized
image are separately disposed. It may be arranged at a place.
FIG. 8 is a side view showing the configuration of an imaging camera 9 and a distance
measurement camera 11 according to a modification of the present invention.
In the modification shown in the figure, on the rear surface side of the hole 3a of the screen 3, a
lens 32 and a dichroic mirror 33 which receive the light passing through the hole 3a are
disposed. The dichroic mirror 33 transmits near-infrared light for distance image generation and
reflects visible light detected by the imaging camera 9. Furthermore, an imaging camera 9
including an imaging lens 9b and an imaging element 9a is disposed at a position adjacent to the
dichroic mirror 33 in the reflection direction of visible light that has passed through the hole 3a.
A distance measuring camera 11 including an imaging lens 11 b and an imaging element 11 a is
disposed at a position adjacent to the dichroic mirror 33 in the transmission direction of near
infrared light passing through the hole 3 a. According to such a modification, since the field of
view can be reliably made to coincide between the imaging camera 9 and the distance
measurement camera 11, the parallax correction processing at the time of generation of the
composite image can be simplified.
[0043]
Further, as shown in FIG. 9, the imaging camera 9 and the distance measurement camera 11 may
be substituted by an integrated camera. In the modified example shown in the figure, on the rear
surface side of the hole 3 a of the screen 3, a camera 35 including a lens 35 a that receives the
light passing through the hole 3 a and an imaging device 35 b is disposed. This camera is an
integrated image sensor having a function of generating a distance image by the TOF method and
a function of detecting visible light and generating a normal image.
[0044]
In the above embodiment, the number and arrangement of the distance measurement camera 11
and the imaging camera 9 may be variously changed. FIGS. 10 to 11 show the arrangement of
the distance measurement camera 11 and the imaging camera 9 in a modification of the present
03-05-2019
15
invention. In these figures, illustration of the directional microphone 15 is omitted.
[0045]
In the modification shown in FIG. 10, the number of distance measuring cameras 11 is four, and
these are divided into the upper end and the lower end of the screen 3 and arranged, and the
imaging camera 9 is arranged in one row in the short side direction of the screen 3 A
configuration in which minutes (5 in total) are added is adopted. Further, in the modification
shown in FIG. 11, a configuration in which the imaging camera 9 and the distance measurement
camera 11 are alternately arranged along the short side direction and the long side direction on
the screen 3 is adopted.
[0046]
In the above embodiment, the distance measurement camera 11 may not be included. FIG. 12
shows the arrangement of the imaging camera 9 in a modification of the present invention. In
FIG. 12, the directional microphone 15 is not shown.
[0047]
In the modification shown in FIG. 12, only the imaging camera 9 is provided on the screen 3.
Then, a composite image is generated by combining the images acquired by these imaging
cameras 9. In addition, these imaging cameras 9 also function as position detection units that
detect the positions of the faces of a plurality of users U. That is, in this modification, the position
detection unit 23 of the image processing device 7 generates a distance image by matching the
images of the two adjacent imaging cameras 9 as images of the stereo camera, and generates a
plurality of imaging A composite distance image is generated by combining a plurality of distance
images obtained by the pair of cameras 9.
[0048]
Furthermore, in the above embodiment, the image selection unit 25 of the image processing
device 7 selects the image of the imaging camera 9 closest to the position of the user U when
03-05-2019
16
selecting the image of the user U. It is not limited to such processing. That is, for the user U, the
image selection unit 25 of the above embodiment selects the image of the imaging camera 9
closest to the three-dimensional coordinates based on the three-dimensional coordinates
calculated for the user U. On the other hand, as a modification, various image selection methods
may be adopted. For example, the direction of the face of the user U is detected using images
obtained by a plurality of imaging cameras 9 or distance images obtained by the distance
measurement camera 11, and imaging at a position close to the direction of the face The image
of the camera 9 may be selected. For example, the direction of the face is detected by specifying
the position of the face part such as the user U's mouth and eyes and calculating the relationship
of those positions. Further, as a further modification, the gaze direction of the user U is detected
using the image obtained by the imaging camera 9 or the distance image obtained by the
distance measurement camera 11, and from the position of the user U's face It is also possible to
select the image of the imaging camera 9 at a position close to the detected gaze direction vector
and extending on the gaze direction vector. As a mechanism of detection of the gaze direction of
the user U, an existing technology (for example, a configuration described in Japanese Patent No.
4604190, a configuration described in Japanese Patent No. 4517049) can be used. According to
such a modification, it is possible to display an image in which communication with all the users
U is easy for the communicator.
[0049]
Moreover, although the direction detection part 24 of the said embodiment estimated direction
of the face of the user U based on the audio | voice signal detected by the directional microphone
15, the direction of the face of the user U was detected by the following method You may For
example, the direction detection unit 24 detects the direction of the face of the user U by image
processing using an image obtained by the plurality of imaging cameras 9 or a distance image
obtained by the distance measurement camera 11. Good. At this time, the orientation of the face
is detected by specifying the position on the image of the part of the face such as the mouth and
eyes of the caller and calculating the relationship between the positions. Further, the direction
detection unit 24 detects the gaze direction of the user U using the image obtained by the
imaging camera 9 or the distance image obtained by the distance measurement camera 11, and
the face direction from the gaze direction You may estimate
[0050]
In the above embodiment, the image communication apparatus 1 on the caller side generates a
composite image including the image of the caller and transmits the composite image to the
03-05-2019
17
opposing apparatus. The function of generating an image may be provided on the opposite
device side. In this case, the image transmission unit 29 of the image communication apparatus 1
transmits the images of the plurality of imaging cameras 9 and the information detected by the
position detection unit 23 and the direction detection unit 24 to the opposing device.
[0051]
Further, the size of the screen or the resolution of the image displayed on the screen may be
different between the image communication apparatus 1 on the caller side and the opposite
apparatus on the communicator side as the communication partner. Further, the arrangement
and the number of the plurality of imaging cameras 9 may be different between the image
communication apparatus 1 on the caller side and the opposite apparatus which is the
communication counterpart, and the arrangement and the number of the distance measurement
cameras 11 are different. It may be different. In such a case, when the image combining unit 27
of the image communication device 1 generates a composite image, the user U's user in the
virtual space divided by the screen 3 of the image communication device 1 and the screen of the
opposite device. It is desirable to execute coordinate conversion processing, pixel interpolation
processing, image enlargement processing, or image reduction processing so that the image of
the user U is displayed at the position on the display of the facing device corresponding to the
position.
[0052]
DESCRIPTION OF SYMBOLS 1 ... Image communication apparatus, 3 ... Screen (screen), 5 ... Image
display apparatus (image display part), 7 ... Image processing apparatus, 9 ... Camera for imaging
(position detection part), 11 ... Camera for ranging (position detection 13) Pulse light source 15
Directional microphone (direction detection unit) 21 Position information acquisition unit 23
Position detection unit 24 Direction detection unit 25 Image selection unit (image combining
unit) 27 ... image combining unit, 29 ... image transmitting unit, 31 ... image receiving unit, 35 ...
camera, U, U 1, U 2 ... user (caller).
03-05-2019
18
Документ
Категория
Без категории
Просмотров
0
Размер файла
32 Кб
Теги
jp2017022600
1/--страниц
Пожаловаться на содержимое документа