From heller Mon Sep 21 00:33:27 1998 Date: Mon, 21 Sep 1998 00:33:05 -0700 From: "Aaron J. Heller" To: Michele Motsko , George Lukes , Doug Climenson CC: Bob Bolles , Marty Fischler , Donna Linne , Ann Johnston Subject: APGD IFD Monthly technical report for Aug 1998 (SRI Proj. 1515) X-Status: X-Keywords: X-UID: 4 Monthly Technical Report August 1998 SRI Project Number: ECU-1515 Contract Number: NMA202-97-C-1004 DARPA Order Number: E645 Prepared by: Aaron J. Heller, Sr. Computer Scientist Prepared for: Michele Motsko, Physical Scientist National Imagery and Mapping Agency Reston, VA SUMMARY OF ACTIVITIES: 1. Creation of procedures to ingest CIB and DPPDB imagery and instantiation of appropriate sensor geometric models. (Heller,Quam) 2. Road network extraction experiments on NTM imagery. (Heller) 3. Further refinement of low-level linear delineation algorithms. (Fischler,Wolf) 4. Major rewrite of FEM and CBACS, addition of FICS. (Connolly) 5. Creation of Reference Model using new Ft. Benning imagery. (GDE:NVO) 6. New triangulations of Ft. Hood imagery. (GDE:Pederas,Pearson) 7. Further work on extraction of building footprints and height estimates from dense DEMs. (Vexcel:Lesher,Wilson) DETAILED DESCRIPTION: 1. Creation of procedures to ingest CIB and DPPDB imagery and instantiation of appropriate sensor geometric models. (Heller,Quam) In the case of DPPDB, the distribution media consists of a "Master Product Descriptor File" and overview graphic, followed by a number of sets of four images, each set comprising a stereo pair at full resolution and 1/8th scale reduced resolution versions. Each image is put on the tape as a separate file and is read on a Unix system with the "dd" command using a block size of 64kb. The images are encoded according to the NITF 2.0 (clevel 6) standard, with 1k x 1k pixel tiles and each tile is lossy-JPEG compressed with a quality factor of 70%. Under RCDE/BOS control, the images are sequentially read off the tape and uncompressed using the program nj_unpac. RCDE reads header of the resulting image to get the name and rational polynomial coefficients and scale factors. The name is used to create an RCDE "2d-world" and the RPC information is used to compute a an LVCS-to-image sensor geometric model, which is then saved in the 2d-world directory. RCDE reads the NITF image and computes the reduced resolution gaussian pyramid. Each image in the pyramid is then saved as a blocked JPEG-compressed tiled TIFF. This entire process takes about 45 minutes per image on a Sun SPARCStation 20. In implementing this process, we discovered a number of errors in the DPPDB RPC documentation, the bulk of which are typesetting errors, however the major difficulty was that the definition of the RPC polynomials refers to image X and Y coordinates, whereas the NITF spec uses ROW and SAMPLE as pixel addresses, hence we had to experiment to find the correct relationship between these. For CIB imagery, we use the rast_imp (raster import) program that is part of the NIMA Muse package. We have implemented a function for RCDE that takes the same information used for the image extraction from the CIB CDROM (image extent in geodetic coordinates and scale) and computes the appropriate sensor geometric model. The image is saved from rast_imp in TIFF, which is then read into RCDE, reduced resolution pyramid calculated and saved in tiled JPEG compressed TIFF. Since the rast_imp program uses a GUI, this process cannot be put under RCDE/BOS control and the user must manually keep track of the extraction specifications used with rast_imp, so that they can later be supplied to the system. If a command line version of rast_imp were available this process could be placed completely under RCDE/BOS control. A further issue with rast_imp is that it appears to have a limitation on the amount of virtual memory it can use, which places an upper bound on the size of the image that can be extracted from a CIB CDROM. In the case of Ft. Irwin/NTC, this limits the extraction to approximately 1/4 of the image data on the CDROM. These procedures are adequate for experimental use, however with additional programming RCDE could be made to work directly with DPPDB and CIB products. 2. Road network extraction experiments on NTM imagery. (Heller) These experiments were carried out on the Ft. Irwin/NTC data and involved using CIB imagery and reduced resolution DPPDB for the low-level 2D extraction, monoplotting these results against the DTED2 elevation model and then using the DPPDB stereo pairs for the high-resolution refinement. The vertical datum for DPPDB is the WGS-84 ellipsoid where as the vertical datum for DTED is the geoid, hence we adjust the DTED2 adding the geoid separation interpolated from the standard table to each grid post. The first aspect I evaluated was the quality of the registration between CIB, DPPDB, and DTOP. I observe a Y-parallax errors between the in the stereo DPPDB imagery as large as three pixels in some areas, although more typically it is close to zero. Between CIB and DPPDB I observe registration errors as large as 10 meters. CIB to DTOP errors are also in the 10 meter range. All of these are within the specifications for these products, however without further registration refinement the observed 10 meters errors between CIB and DPPDB make it difficult to use low-level results computed from CIB. As noted previously, the severe compression artifacts present in CIB data also compromise the performance of the low-level extraction. On the other hand, experiments using reduced-resolution DPPDB for the low-level extraction and full-resolution for the high-level extraction work well. These results will be demonstrated at the IPR. Further work is needed to more fully characterize the results and evaluate the results using the DTOP as reference. 3. Further refinement of low-level linear delineation algorithms. (Fischler,Wolf) The main problems being addressed include: (a) Code modification to permit efficient extraction from very large images. (b) Parameter-sets for different types of degradation of road visibility. The current approach is to assume long sections of road with very good visibility intermixed with areas of very extensive occlusion. We have found a way to use the current processing sequence with different parameter-sets to also be able to handle the case where the road has intermittent visibility (i.e., appears as a "dashed-line"). (c) To increase completeness in detecting the roads in a give area, it is often necessary to view the area at different resolutions just for the purposes of detection; the problem of merging the results of the multi-resolution analysis is currently being addressed. (d) Suburban/city streets often exhibit regular geometric patterns, such as a rectangular grid or a set of parallel curvilinear structures. We have devised methods to detect such patterns. We will be able to demonstrate significant progrees in the above areas at the comming IPR. 4. Major rewrite of FEM and CBACS, addition of FICS. (Connolly) At this point we have a working prototype of a system for context-based algorithm selection and site-model construction, annotation, and editing. We have re-implemented the BOS, rewriting parts of it and stripping out older (useless) code. The three components of the system, FEM, CBACS, and FICS, are working and talking to each other. We have implemented rules in the FEM for performing extraction and annotation tasks on features. The FICS has also been implemented, and consists of rules for requesting tasks from the FEM to construct a "complete" site model. One may add restriction regions and features to the "context" object, and these can be passed along the pipeline. In Prolog terms, the top-level predicate of the FEM is "valid-task", which, in an attempt to satisfy the predicate, constructs a list of "contexts" within which the specified task can be performed. Each "context" is sent to CBACS, which invokes appropriate algorithms and parameters. The results are passed to the FICS. The FICS is really the top-level controller for the site modeling process. It attempts to satisfy the predicate "site-complete", which means that the site model that the FICS has been operating on is complete. In the process of verifying this predicate, the FICS will ask the FEM to perform various tasks. We have gotten this new BOS system to do extraction and annotation tasks (where the annotation is a simple grade computation). In particular, the system can perform the following two tasks: 1) delineate curves in images 2) annotate each curve with grade (%) at each point The following steps were completed in preparation for the road demo: 1) Determining a parameter set for the low-level road detection that results in almost zero missed detections, however with a correspondingly higher false positive rate. 2) Hooking in the high-resolution 3d refinement step as an algorithm with appropriate image selection rules. 3) Implemented curvature computation (annotation) 4) Implemented rules for (re)classifying curves based on annotations 5) Implemented rules and machinery for interpolation tasks 6) Vexcel has implemented a program to add land-use classifications to road segments. In the demo, we will show an improved result at Ft. Benning that has been arrived at by using the the "more liberal" parameters for the low-level detection and eliminating the false positives with the new implementation of the FEM/CBACS/FICS described above. We still need refinement algorithms, and the FICS needs some more machinery for classification of features (roads, protoroads, junk, unknown, etc.). 5. Creation of Reference Model using new Ft. Benning imagery. (GDE:NVO) The extraction work of the Ft. Benning reference model using the new imagery is complete. The new imagery and the extraction results have not yet been delivered to SRI for evaluation. 6. New trianulations of Ft. Hood imagery. (GDE:Paderas,Pearson) GDE remeasured all of the CMU ground control and tie points and added a significant number of new tie points. Due to previously reported problems encountered with the new Universal Triangulation Package, the existing package (HAATS) was used to compute new camera parameter solutions. When accepted, these will be recomputed with to UTP to provide parameter covariances. These new solutions were translated to TEC format and delivered to SRI on 8/27. Initial evaluation showed that the verticals were acceptable, but errors in the 20 to 150 pixel range were noted on the obliques. GDE said that they were seeing much lower residuals. In a conference call between GDE (Pearson & Paderas) and SRI (Heller & Quam), we identified that the problem was caused by an errors arising from an incomplete understanding of the SocetSet frame camera math model in the program GDE wrote to create TEC header files from SocetSet support files. As of this writing GDE has not delivered corrected files. SRI has given GDE an action to supply a description of the implementation of the math models used in SocetSet. 7. Further work on extraction of building footprints and height estimates from dense DEMs. (Vexcel:Lesher,Wilson) Vexcel continued to work on software to refine and "clean up" building edges extracted from dense DEMs. For test data he has been using DEMs of downtown Denver and Ft. Hood. The software now supports computing straight edges, squaring off corners, and computing local preferred orientations from road/street networks and averaging building edges. We hope to show results of this work at the 24 September IPR. The plan is for SRI to evaluate the use of the footprints and height estimates to initialize wireframe models that will be refined using multi-image model-based optimization. If sucessful, it will be the first demonstration of segementation structures and wireframe geometry extraction of buildings from dense DEMs. ============================================================================= GDE Systems APGD Monthly Report September 10, 1998 Technical Section Summary All of the work during this period has been on data set generation. Detailed Work Description (by SOW item): 1. Architecture Refinement No activity 2. Algorithm Survey No activity. 3. Algorithm Development No activity. 4. Multi-Sensor Registration No activity 5. Persistent Store. No activity 6. Dataset Production & Distribution: A thorough re-examination of the Fort Hood triangulation, including re-measurement of the CMU control and tie points and performance of a significant number of additional image measurements, has resulted in a solution which we believe to be as solid as possible with the available ground control. The resulting project has been subjected to an independent verification on the FPE system by one of that program's experts, and his report confirms the quality of the results. Because of the poor distribution of control on the oblique images, variations of the parameters for those images using different triangulation systems are to be expected, but these produce no significant differences in areas of interest on the ground. Our recommendation is that the latest SOCET SET set results be accepted and that feature extraction be allowed to continue at the NVO using those parameters. We will then compute the covariances and supply a set of TEC headers reflecting the results to SRI for distribution to the community. NVO personnel have completed the feature extraction work on the new Fort Benning imagery and are ready to continue the Fort Hood work as soon as agreement has been reached on the triangulation results. 7. Evaluations: No activity 8. Interface to FRE Contractors: No Activity 9. Demonstration Scenarios: No activity 10. Technology Transfer No activity 11. Option Years No activity 12. Program Management We are continuing to work according to the priorities agreed on with the prime. We keep in close contact with team members via weekly conference calls and e-mail communications as appropriate. We supported the IPR at SRI on July 21. ============================================================================= Automatic Population of Geospatial Databases Monthly Report to SRI for August Bob Wilson Vexcel Corporation 16 September 1998 1. MAJOR TECHNICAL ACCOMPLISHMENTS 1.0 Chris Lesher continued to work on software to refine and "clean up" building edges extracted from dense DEMs. For test data he has been using DEMs of downtown Denver and Ft. Hood. The software now supports computing straight edges, squaring off corners, and computing local preferred orientations from road/street networks and averaging building edges. We hope to show results of this work at the 24 September IPR. 1.1 Vexcel has provided code to SRI to read extracted road results provided in APGD evaluation format, compute attributes for the segments based segment geometry and LULC results, and write the annotated segments back out in APGD evaluation format. 1.2 Vexcel contributed illustrations and evaluation results to the First Year APGD Annual Report. Some of the evaluations were performed again for new versions of algorithms. 1.3 Vexcel continued to support Seth Teller's APGD effort by starting to process 24 low-flown, high-resolution, color, aerial oblique photos of MIT. Bob Wilson performed an evaluation of the quality of the digital scans (by Image Scans Inc., Wheat Ridge, CO). Aeotriangulation was started using SOCET Set, but this work was not charged to APGD; it was funded by a separate contract for photogrammetric services with MIT. 2. ACCOMPLISHMENTS VIS-A-VIS STATEMENT OF WORK 2.1 Refine the BOS architecture 2.2 Survey automated model extraction techniques 2.3 Develop feature extraction and consistency enforcement algorithms 1.0 2.4 Refine the design of and implement the dynamic database 2.5 Produce, maintain, and distribute data sets and ground truth 1.2 2.6 Develop evaluation metrics and perform evaluations 1.1 2.7 Interface to FRE contractors 1.2 2.8 Develop and perform demonstrations 2.9 Transfer technology 2.10 APGD program management Weekly conference calls, and phone conversations and e-mail coordinating writing the annual report. -- Aaron Heller, Senior Computer Scientist AI Center, Room EK264 Internet: heller@ai.sri.com SRI International Phone: +1 (650) 859-2618 333 Ravenswood Ave. Fax: +1 (650) 859-3735 Menlo Park, CA 94025 WWW: http://www.ai.sri.com/~heller