3-D image detector

Wee, Siang Tze Reginald; Liang, Jie;

A three dimensional image detector is disclosed. The three dimensional image detector has a lensing system to focus incident light reflected from an object field upon a first and second image detector. The first and second image detectors are matrices of charge coupled devices. The first and second image detectors convert the incident light to electrical signals. The electrical signals from the first and second image detectors are converted in an analog to digital converter to a plurality of digital words that represent the magnitude of the light that impinges upon the first and second image detectors. The plurality of digital words are retained in a random access memory for processing within a digital signal processor. The digital signal processor will locate an object within the object field and determine the depth and contour information of the object and present this information with the plurality of digital words to external circuitry for further processing or display. The first and second image detectors, the analog-to-digital converter, the random access memory, and the digital signal processor are to be disposed upon an integrated circuit substrate.

BACKGROUND OF THE INVENTION

1. Field Of The Invention

This invention relates to the field of electronic image detection and more particularly to circuits and apparatus used to process digital image data to derive three dimensional information within images for further processing or display.

2. Description of Related Art

Various methods and apparatus for the capture, transmission and presentation of three dimensional (3D) imaging information using stereoscopic visualization techniques are well known in the art. However, all of the methods and apparatus examined involved complex techniques for capturing the visual information and presentation in a fashion that the human eye and brain perceived a qualitative view of the 3D view. Any quantitative depth and contour information was not preserved and measurement of said depth and contour information must be accomplished using reference standards present within the view or by the use of measurement devices. Said measurement devices often included an external energy source such as sound, fight, or radio frequency waves that are transmitted to an object of interest as impulses and the reflected energy examined to determine distance and thus depth and contour information.

There is a large body of literature exploring the projective geometry necessary to calculate the location of objects within an object field of interest. Furthermore, the algorithms necessary to calculate the depth and contour information are well understood and is presented in such literature as Three Dimensional Computer Vision: A Geometric Viewpoint, by Olivier Faugeras, The MIT Press Cambridge Mass., 1993.

The complexity of the algorithms for the identification of objects and the calculation of depth and contour information has required either specialized computational circuitry or extremely high performance computer systems that have obviated the integration of the light collection, light conversion, and image processing functions within image detection systems. VLSI Image Processing, by Raymond J. Offen, McGraw-Hill Book Company, New York, N.Y. 1985, discusses the architectures, algorithms, and computational structures required for object location and the calculation of depth and contour information within an object field.

U.S. Pat. 5,101,269 (Shelley, et al.) is an example for the viewing of stereoscopic images. The stereo images are projected upon a pair of CCD sensors. The signals from the CCD sensors are transmitted, to synchronized electronics for processing, and thence to an electronic viewer to manipulate the image to produced to provide a 3D image. The display of the image is for non-real time viewing of still images.

U.S. Pat. No. 5,007,715 (Verhulst) illustrates a device for capturing of stereoscopic images upon a CCD array and transmitting the captured image to a display device providing a 3D image. Each display of each image provides only a visual display without providing any depth or contour information.

U.S. Pat. No. 5,381,784 (Adau) describes a stereoscopic endoscope wherein light is focused on two distally disposed CCD's to provide a signal that when suitably processed and displayed upon a television monitor will provide a 3D image. The application of this patent is for viewing channels and cavities during medical procedures and does not provide any capabilities for mensuration data of depth and contour.

U.S. Pat. No. 5,222,477 (Lia) provides a system of stereoscopic viewing using a single lens and a CCD array with two apertures, each placed to one side of the optical axis of the lens. The apertures are alternately opened and closed to cause light from each aperture to successively impinge upon the CCD array. The system then processes the electrical signals fro display such that the human eye will perceive 3D images.

U.S. Pat. No. 5,416,509 (Sombrowsky) describes a method and apparatus for capturing a 3D image and displaying this image using a glass plate having elongated apertures, scanning said apertures in a manner such to preserve the perspective information, and presenting said information on a modified display.

U.S. Pat. No. 5,049,963 (Yamada) discloses a holographic television apparatus, wherein a 3D image is captured and displayed using holographic interference patterns.

U.S. Pat. No. 5,446,529 (Stettner, et al.) develop a 3D underwater laser radar imaging system, wherein an object field is scanned by an impulse of laser light. The two dimensional information is captured using a two dimensional array of light collectors and the third dimension is calculated from the time of the return of the impulse of laser light.

SUMMARY OF THE INVENTION

An object of this invention is to receive incident light reflected from an object field and preserve three dimensional depth and contour information. Another object of this invention is the location of objects within an object field. Further another object for this invention is the extraction of the three dimensional depth and contour information of objects within the object field. Still another object for this invention is the integration upon a single integrated circuit substrate of the receiving of the incident light reflected from the object field, preserving of the three dimensional depth and contour information, the locating of objects within the object field, and the extraction of the three dimensional depth and contour information.

To fulfill these and other objects, a system for the detection and presentation of depth and contour information of a three dimensional object has a first and a second array of charged coupled devices to convert the incident light to electrical signals. The first and second array of charged coupled devices are distally disposed upon an integrated circuit substrate. Incident light, reflected from the object field, impinges upon the first and second array of charged coupled devices through a light focusing means. The light focusing means has two distally disposed lenses, each with a focal plane and an optical axis. Each lens is adjusted such that its focal plane aligns with each of the first and second array of charged coupled devices and the optical axis of each lens aligns with a center point of the first and second detection means.

An analog-to-digital converter, disposed upon the integrated circuit substrate and coupled to the first and second array of charged coupled devices, converts the electrical signals from the first and second array of charged coupled devices to a first and second plurality of digital words, wherein each word represents the magnitude of incident light impinged upon each charge coupled device within the first and second detection means.

The first and second plurality of digital words are retained in a frame memory that is disposed upon the integrated circuit substrate and coupled to the analog-to-digital converter.

Finally disposed upon the integrated circuit substrate and coupled to the frame memory is a digital signal processor which will detect the presence of objects within the object field and calculate the three dimensional depth and contour information. This depth and contour information will be presented to external circuitry for display or further processing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a shows diagramatically a system for the detection and presentation of 3D images of this invention.

FIGS. 1b and 1c are schematic block diagrams of the frame memory and digital signal processor of this invention.

FIG. 2 illustrates the geometry employed for the calculation of the depth and contour information.

• Motor-driven shutter mechanism Biologically active vasopressin analogues
• Combined rail clip and anchorage Pectate lyase
• Battery charging apparatus Magnetic thin film memory device
• Oxygen pulping with recycled liquor Segmented cold cathode display panel
• Process of drawing fibers Spectrophotometer
• Security bulb assembly Intensifying x-ray film cassette
• Radome for aircraft Web butt-splicing apparatus
• Collapsible non-strapped paste bin Barrel bypass system--full length groove
• Pyrrole stabilizers for chlorine-containing thermoplastics Rotary weed extractor
• Genetic marker test for lupus Ballasts with built-in ground-fault protection
• Pulse laser Assemblage for rearing nematodes
• Hand held electric caulking gun Finder optical system
• Hand-held telecommunication tester Candle apparatus
• Bullet sizing and lubricating system Zero setup time flip flop
• Particle orientation imaging system Non-invasive hyperthermia method and apparatus
• Integrated nonlinear waveguide spectrometer Flat bed sorter
• Electrode tool for electrochemical machining RGS-containing molecules and uses thereof
• Magnifying lollipop Small omni-directional, slot antenna
• Toothbrush and dental floss holder Vehicle display device
• Utilization of wave motion Bar storage apparatus
• Switch operating mechanism Electric toothbrush assembly
• Wart treatment Advanced graphics controls
• Semiconductor apparatus High-energy brake and brake components
• Production of thebaine and oripavine System for transfer of data
• Brushless motor having double insulation Perimeter weld flanges
• Transmission housing CCD demodulator/correlator
• Fuel dispenser catchment box Motor vehicle air-conditioning system
• Wheelchair automatic anti-rollback assembly Method of manufacturing bipolar transistor
• Multipipet Learning machine
• Hydraulic antivibratory devices

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 diagramatically shows the 3D image detection system (3D system) of this invention. The lines of reflection 6a and 6b define the object field 3 within the total world domain 2. The object field 3 is the total limit of viewable area capable of detection by the 3D system 1. At least one object 4 will occupy the object field 3.

The lines of reflectance 6a and 6b are the totality of the incident light reflected from the object field 2 and captured by the 3D system 1. These lines are transmitted through the lenses 8a and 8b into the optical subsystem 8. The optical subsystem 8 contains necessary lenses, focusing mechanisms, and prisms, to project the incident light from the object field 2 onto the matrices of CCD's 14a and 14b. The lines of reflectance 6a and 6b are transformed within the optical subsystem 8 into the lines of projection 10a and 10b as projected from the lenses 8c and 8d. The transformation and projection allows the area of each matrix of CCD's 14a and 14b to circumscribe a projection of the all of the object field 3.

The lenses 8a and 8b are distally disposed within the optical subsystem 8 such that the optical axis of each of the lenses 8a and 8b is offset from the center of the object field 3 in a manner similar to stereoscopic viewers of prior art.

Each CCD element within the matrices of CCD's 14a and 14b is composed of an optical sensor such as a photodiode and a CCD cell to act as an analog storage element for each photodiode. The charge stored in each CCD cell is proportional to the light impinging upon each cell. The CCD elements of the matrices of CCD's 14a and 14b are organized into rows and columns. Each CCD element within each column is connected to each adjacent element forming a serial shift register. The last CCD element in each of the columns of CCD's is connected to a final serial shift register.

The columns of CCD elements are synchronously shifted one element per column into a final serial shift register. The final serial shift register is then synchronously shifted to an output port to form a serial analog signal that represents the magnitude of the charge stored in each CCD element.

There is often an automatic gain control function attached to the CCD's to prevent saturation of the charge stored in each element when the magnitude of the light is sufficiently great.

The output of the final serial shift register of each of the matrices of CCD's 14a and 14b is connected to an analog-to-digital converter 16. The analog-to-digital converter 16 changes each serial analog signal from each of the matrices of CCD's 14a and 14b to a digital word representing the magnitude of the light impinging upon each picture element of the each matrix of the CCD's 14a and 14b.

The digital code words representing the magnitude of the light impinging upon each of the picture elements are stored in a frame memory 18. The frame memory 18 is a random access memory configures as shown in FIG. 1b. Frame memory 1 18a provides for the storing of each of the digital code words that are the picture element information (pixel) as they are transmitted from the matrices of CCD's 14a and 14b of FIG. 1a through the analog-to-digital converter 16 of FIG. 1a. When frame memory 1 18a is completely filled the timing generator 19 transfers the image information from frame memory 1 18a to frame memory 2 18b for processing by the digital signal processor 20. This double buffering pipeline allows the current image present on the matrices of CCD's 14a and 14b to be transferred to the frame memory 1 18a while the digital signal processor 20 is processing the previous image from the matrices of CCD's 14a and 14b to detect an object image within the digital code words and to calculate depth and contour information.

The timing generator circuit 19 provides the necessary timing signals to store the digital code words representing the pixel information to the frame memory 1 18a, to transfer data from frame memory 1 18a to frame memory 2 18b and to provide synchronization between frame memory 3 18b and the digital signal processor 20.

Since the frame memory 1 18a and the frame memory 2 18b are updated and refreshed constantly, they can be implemented as a volatile memory in current technology this could be static random access memory (SRAM), dynamic random access memory (DRAM), or video random access memory (VRAM). Additionally, the simplicity of the structure of the frame memories 18a and 18b would permit each frame memory to be implemented as a two port memory supporting simultaneous reading and writing of data.

Returning to FIG. 1a, the digital signal processor 20 is a computation unit predesigned to locate an object 4 within the object field 3 and calculate the depth and contour information describing said object 4. The digital signal processor 20 accesses the random access memory 18 and retrieves the digital code word for each element of each matrix. The digital signal processor then uses a triangulation algorithm which compares light intensity between adjacent cells of the CCD matrices 14a and 14b and then computes the depth and contour information.

As improvements in technology permit more complex designs, the frame memory 18 would be able to retain multiple images from the CCD matrices 14a and 14b. This permits the digital signal processor to compare past and present digital codes words of the image data to create improved depth and contour information.

The depth and contour information in conjunction with the digital code words representing the two images of the object field 3 are transferred from the is digital signal processor 20 to external circuitry for further processing or display.

Referring now to FIG. 1c, the digital signal processor consists of a central processing unit 20a and three memory modules 20b, 20c, and 20d. There is a program read only memory (ROM) 20b for storing the instructions and instruction sequences to perform the triangulation algorithm to detect the location of an object within the image and to calculate the depth and contour information. There is a data ROM 20c for storing the signal processing coefficients necessary to perform the triangulation algorithm and to perform other signal processing algorithms such as low pass filtering and edge enhancement necessary for the detection of an object within the image field and calculation of the depth and contour information.

A data random access memory 20d will provide temporary retention of image data and intermediate processing image data while the central processing unit 20a is processing each image.

The geometry used for calculating the depth and contour information is shown in FIG. 2. The object 4 has two points A and B that are at different locations relative to the axes X, Y, and Z. The X and Y axes define the retinal plane that is aligned with he matrices of CCD's 14a and 14b of FIG. 1 The point C.sub.1 is the center for the lens 8a of FIG. 1 and the point C.sub.2 is the center for the lens 8b of FIG. 1. The optical subsystem 8 of FIG. 1 has a an optical length d that is the effective distance from the retinal plane to the center of the lenses C.sub.1 and C.sub.2. The lenses are at a distance f apart and a distance e on the Y axis from the X axis.

The points A and B respectively form the projection points A', A", and B', B" upon the retinal plane as projected by the lines of reflectance 6a and 6b.

While this invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention.