The ability to enrich remote live video by virtual information does not exist in standard teleconferencing systems yet, partly because transmitting real-time captured 3D information over network has some disadvantages. For example, bandwidth requirements of 3D information are much higher than normal video. Also, in order to generate an accurate 3D representation, the environment must be captured from several directions. Normally, users are accustomed to pointing a video camera so that it captures only those parts of the environment that the user wants to show to remote participants of the video conference.
In the new InterDigital solution [US 62/316,884], the user may restrict the 3D captured information transmitted to the remote users using the same method as in traditional video conferencing: by pointing a video camera in the desired direction. Even though the local 3D capture set-up generates 3D information of the whole environment, the system computes which 3D objects are visible in the camera view and transmits only those to the remote participants.