From heller Fri Oct 16 13:26:45 -0700 1998 Date: Fri, 16 Oct 1998 13:26:45 -0700 From: Aaron Heller To: Michele Motsko , George Lukes , Doug Climenson CC: Bob Bolles , Marty Fischler , Donna Linne , Ann Johnston Subject: APGD IFD Monthly technical report for Sep 1998 (SRI Proj. 1515) Monthly Technical Report September 1998 SRI Project Number: ECU-1515 Contract Number: NMA202-97-C-1004 DARPA Order Number: E645 Prepared by: Aaron J. Heller, Sr. Computer Scientist Prepared for: Michele Motsko, Physical Scientist National Imagery and Mapping Agency Reston, VA SUMMARY OF ACTIVITIES: 1. Planning for November demos. 2. Experiments to compare performance of low-level algorithm on 5m CIB vs. reduced resolution NTM. 3. Refinement of low-level linear delineation algorithm (Fischler,Wolf): 4. Work on camera parameters and reference models for Ft. Benning and Ft. Hood. 5. Improved building footprints from dense DEMs. DETAILED DESCRIPTION: 1. Planning for November demos (Heller,Fischler) In preparation for the November demos, we have inventoried the availability data and prepared top-level scenarios. The centerpiece of the classified demo will be: - Canned road-network results for a 10'x10' area of NTC that is roughly centered on the cantonment area. - live results over one or more small windows of OCONUS site "S" We will also show building extraction results from Ft. Hood, but mainly as an advertisement for the demo at the IUW. I am in contact with Eamon Barret to coordinate our demo. At this point it appears that due to security issues, we will not be able to use one of the machines at the SAC, but will need to bring our own machine. At the IUW we will demonstrate the end-to-end road network extraction on Ft. Benning and building extraction from from footprints at Ft. Hood. 2. Experiments to compare performance of low-level algorithm on 5m CIB vs. 5m reduced-resolution NTM. (Heller,Fischler,Wolf) We have completed initial experiments over small sections of NTC and site "S" to assess the impact on the low-level linear delineation system of the vector quantization (VQ) lossy-compression used for CIB. There are actually two sources of degradation present in CIB -- the resampling and blending to produce the orthomosaic and compression with the VQ algorithm. Because the tests compared reduced-resolution NTM to CIB, we are really evaluating the effects of both degradations, however since the NTM used for these experiments is near-nadir, and we avoided obviously blended areas (not hard to spot despite Hughes' claims to the contrary), I feel that the VQ artifacts are dominant. The experimental procedure consisted of running the LD system on selected window of CIB imagery. Locating discontinuities in the extracted results and examining the corresponding area in the NTM imagery to understand the source of the failure. At site S, in roughly half of the cases, there was occlusion of the road from trees and other vegetation. However in other cases at site S and almost sll cases at NTC, the effect observed was that where the local contrast in the scene was low, the VQ reduced it further and past the threshold that the LD system detects. In some cases the local contrast was reduced to zero and the road effectively disappears in the image. 3. Refinement of low-level linear delineation algorithm (Fischler,Wolf): The main problems being addressed include: (a) Code modification and additional infrastructure to permit efficient road extraction from very large images. This includes - methods to reduce the storage and processing time required; we have cut the required processing time for an 8kX8k image (5m or 10m CIB) from over 20 hours to less than 8 hours and made significant reductions in computer storage requirements. - methods to exploit the greater coherence of road structurs that can be viewed over larger geometric extents. - methods to inspect the results of large image models. (b) Parameter-sets for different types of degradation of road visibility. The current approach is to assume long sections of road with very good visibility intermixed with areas of very extensive occlusion. We have found a way to use the current processing sequence with different parameter-sets to also be able to handle the case where the road has intermittent visibility (i.e., appears as a "dashed-line") because of occlusion, loss of contrast, etc. This problem is especially prominent in low resolution CIB imagery where we also have to be able to do long range linking. We are attempting to formalize a process that can deal with these problems as an adjunct to our standard modeling algorithm. (c) To increase completeness in detecting the roads in a give area, it is often necessary to view the area at different resolutions just for the purposes of detection; the problem of merging the results of the multi-resolution analysis is currently being addressed. In particular, we have found, in recent experiments, that we can make significant completeness improvements by merging results of running our standard extraction algorithms at 2 or 3 different resolutions. We are building the infrastructure to formalize this ability as part of our standard modeling algorithm. (d) Suburban/city streets often exhibit regular geometric patterns, such as a rectangular grid or a set of parallel curvilinear structures. We have devised methods to detect such patterns and have conducted experiments on both our ability to extract such patterns and to use them in street modeling. In particular, we have been able to improve our ability to model the Ft. Irwin cantonment area through the use of these tools in ad-hoc experiments, and are now attempting to generalize these results and incorporate them into the BOS road-modeling subsystem. (e) Presentation of road and street modeling progress and capabilities at the 24 Sept IPR and at a subsequent visit to SRI by ARPA and NIMA visitors on 29 Sept. 4. Extension of the FICS (Feature Integration and Classification System) (Connolly) The FICS is responsible for integrating features into a consistent site model by requesting tasks from the feature extraction manager, classifying features, and creating feature sets that represent different "layers" of a site model. The FICS has been implemented as a set of prolog rules that decide how to classify features, and what tasks to request of the FEM. At any time, the FICS has a set of "features of interest" that may be reclassified or sent to the FEM for refinement. The addition of the FICS required that the Feature Extraction Manager be modified in the following ways: 1) A more specific breakdown of tasks that the FEM is to perform. 2) Provisions for passing feature and restriction information to CBACS and subsequently to invoked algorithms. 3) Addition of new algorithms to the CBACS algorithm table. In particular, the FEM now handles 5 task types, as follows: 1) Extraction: creation of features directly from imagery. 2) Refinement: refinement of existing features using imagery. 3) Annotation: computation of intrinsic feature properties (e.g. geometric, object-centered properties such as curvature or grade). 4) Verification: computation of feature properties from the existing imagery in which the feature is visible (e.g., material properties from SAR imagery). 5) Interpolation and Extrapolation: use of existing feature and image information to deduce the presence of features that were not previously extracted (e.g., filling in gaps in roads). These tasks require that each component of the BOS be able to pass features through to other components. In addition, regions (area features) may be predefined to restrict algorithm operation. For example, SRI's rural road extraction algorithm does not work well on urban areas, and so must be restricted from operating in such regions. Several algorithms have been added to CBACS for use on linear features. Of note is the incorporation of Vexcel's land use classification algorithm, which is an example of the ability of the BOS architecture to incorporate externally-supplied algorithms using an appropriate data-exchange standard. Vexcel's algorithm is implemented as a stand-alone executable that uses the APGD evaluation file format to communicate feature and property information. A CBACS "wrapper" is used to pass a georeferenced linear feature to the algorithm. This feature is returned with annotations that describe the land use categories over the linear feature's area of coverage. Other algorithms include: - radius of curvature (meters) - grade (%) - tilt (lateral grade %) - terrain variation (meters^-1) Some simple rules have been implemented in the FICS to reclassify linear features based on combinations of the above properties. In its current state, the system is able to automatically produce a version of the Ft. Benning site model that is comparable to a human-edited version where only deletions are allowed. 4. Work on camera parameters and reference models for Ft. Benning and Ft. Hood. (GDE,Heller) The Ft. Benning extractions using the new coverage are complete, but have not been delivered to SRI. A copy of the new coverage of the new coverage was delivered to SRI, but we have not yet done anything with it other than read it off the tape and examine a few of the images. As described in their report, GDE has retriangulated the Ft. Hood imagery. The new parameters have been independently evaluated by a GDE engineer and by Max Roos, both of whom find them acceptable. However, the TEC header translations delivered to SRI still exhibit the same errors as earlier sets. (I have prepared a WWW page that illustrates the errors I am seeing, which is accessible at http://www.ai.sri.com/~apgd/comparison.html.) There are three possibilities that I am considering: (1) The errors I observe are actually in the parameters and I have higher standards than the other personnel who have evaluated the camera parameters. (2) There is an error in the program used to translate SocetSet support files to TEC header format. (3) There is an error in RCDE's TEC header reader. I beleive that (3) is not likely since the reader has been in use for over five years with files generated by CMU and TEC without problems. In addition, both Quam and I have reviewed the reader code and cannot find any errors. I have also reviewed the code used by GDE to perform the translations (2) and cannot identify any obvious flaws. GDE supplied SRI with a document that defines the entries in the SocetSet support files and outlines the ground-to-image transformation pipeline used by SocetSet for frame cameras. I am using this to do an independent implementation of the SocetSet transformation pipeline and will use it to evaluate the parameters without going through the TEC header translation stage. 5. Improved building footprints from dense DEMs. (Vexcel) Vexcel has improved the quality of the extraction of building footprints from DEMs and delivered the latest result to SRI. One hitch is that Vexcel did their own aerotriangulation of the Ft. Hood nadir coverage to produce the dense DEM because the found that the CMU solutions yielded tears. SRI has computed rigid transformations that bring the Vexcel-produced footprints into alignment with the CMU and current GDE parameters. We are currently conducting experiments with these results and will report on them when complete. --------------------------------------------------------------------------- GDE Systems APGD Monthly Report September 10, 1998 Technical Section Summary All of the work during this period has been on data set generation. Detailed Work Description (by SOW item): 1. Architecture Refinement No activity 2. Algorithm Survey No activity. 3. Algorithm Development No activity. 4. Multi-Sensor Registration No activity 5. Persistent Store. No activity 6. Dataset Production & Distribution: A thorough re-examination of the Fort Hood triangulation, including re-measurement of the CMU control and tie points and performance of a significant number of additional image measurements, has resulted in a solution which we believe to be as solid as possible with the available ground control. The resulting project has been subjected to an independent verification on the FPE system by one of that program's experts, and his report confirms the quality of the results. Because of the poor distribution of control on the oblique images, variations of the parameters for those images using different triangulation systems are to be expected, but these produce no significant differences in areas of interest on the ground. Our recommendation is that the latest SOCET SET results be accepted and that feature extraction be allowed to continue at the NVO using those parameters. We will then compute the covariances and supply a set of TEC headers reflecting the results to SRI for distribution to the community. NVO personnel have completed the feature extraction work on the new Fort Benning imagery and are ready to continue the Fort Hood work as soon as agreement has been reached on the triangulation results. 7. Evaluations: No activity 8. Interface to FRE Contractors: No Activity 9. Demonstration Scenarios: No activity 10. Technology Transfer No activity 11. Option Years No activity 12. Program Management We are continuing to work according to the priorities agreed on with the prime. We keep in close contact with team members via weekly conference calls and e-mail communications as appropriate. We supported the IPR at SRI on July 21. --------------------------------------------------------------------------- Automatic Population of Geospatial Databases Monthly Report to SRI for September Bob Wilson Vexcel Corporation 6 October 1998 1. MAJOR TECHNICAL ACCOMPLISHMENTS 1.1 Computing Land Cover Attributes for Road Segments Chris Connolly sent us some segments of roads at Ft. Benning in APGD evaluation format. Chris Lesher wrote a program to read these data, construct road ribbons, and associate with each segment the number of covered pixels belonging to each land cover class and the total number of pixels. The program outputs road segments with attributes in the APGD evaluation format. Chris Connolly demonstrated this capability, invoked via the FICS, at the September '98 IPR. 1.2 Extraction of Buildings from Dense DEMs Chris Lesher continued to work on software to refine and "clean up" building edges extracted from dense DEMs. For test data he has been using DEMs of downtown Denver and Ft. Hood. The software now supports computing straight edges, squaring off corners, and computing local preferred orientations from road/street networks and averaging building edges. I presented some of the results at the September '98 IPR. 1.3 Writing Papers for the '98 IUW We submitted two APGD papers to the Proceedings: R. Xiao, C. Lesher, and R. Wilson. Building detection and localization using a fusion of interferometric synthetic aperture radar and multispectral imagery. R. Ledner, M Maresch, and C. Lesher. 3-D reconstruction of urban environments from dense digital elevation models. 1.4 Aerotriangulation of MIT Photos Vexcel continued to support Seth Teller's APGD effort by processing 24 low-flown, high-resolution, color, aerial, oblique photos of MIT. We have obtained very good exterior orientation estimates for 17 of the photos, but are delayed in making further progress by expiration of our license for SOCET Set. (Most of this work was supported by a separate purchase order from MIT, rather than by our subcontract to SRI.) 1.5 Incipient Collaboration with UMass on Building Extraction from IFSAR. Bob has been discussing with Alan Hanson and Kieth Hoepfner getting them slant-range Sandia Spotlight data of the Ft. Benning MOUT site with ancillary data required for radargrammetry. Sandia has the data on tape, but has not yet located it for Vexcel. (Note that Sandia is constructing a new high-res IFSAR sensor for the RTV project and Vexcel is contributing radar processing algorithms. Thus, Vexcel is maintaining fairly close technical contacts with Sandia.) 1.6 IPR at SRI Bob met at SRI with the IFD team on 23 September to discuss the project and prepare final viewgraphs of his presentation. All paperwork (including fingerprints) were submitted to SRI's security office to support clearances for Bob, Richard Carande, and Marty Marra. At the IPR (24 September) Bob presented a talk on recent work at Vexcel, "Context and Dense DEM Analysis for Feature Extraction." The following are action items for Vexcel arising from the meeting: 1.6.1 Determine suitability of Vexcel's Denver data set for use as a distributed APGD data set. Bob is pursuing this at a low level with Rob Ledner. Rob is quite protective of these data, which were gathered by Vexcel for a pilot project targeting customers in the telecom industry. 1.6.2 Determine suitability and availability of Seth Teller's low-altitude, aerial, oblique color images of MIT for use as a distributed APGD data set. The aerotrig for these images was almost completed at Vexcel (17 out of 24 images, each one ~1 GB), but is now on hold because our license for SOCET Set has run out. We hope to obtain another license in order to complete this work. 1.6.3 Compute new footprints for buildings at Ft. Hood and distribute them to APGD groups doing building extraction. Chris continues to develop the software to clean up extraction from dense DEMs. At what stage shall we distribute them? We are prepared at this time to provide them to SRI and GDE in the APGD evaluation format for feedback. 1.6.4 Radargrammetry toolkit for access of Sandia IFSAR We need to obtain complete slant-range data (including corrected sensor trajectory) from Sandia in order to assess the effort to develop such a toolkit. Could the same toolkit handle the new Sandia IFSAR sensor? 2. ACCOMPLISHMENTS VIS-A-VIS STATEMENT OF WORK 2.1 Refine the BOS architecture 1.1 2.2 Survey automated model extraction techniques 2.3 Develop feature extraction and consistency enforcement algorithms 1.1 and 1.2 2.4 Refine the design of and implement the dynamic database 2.5 Produce, maintain, and distribute data sets and ground truth 1.4 and 1.5 2.6 Develop evaluation metrics and perform evaluations 2.7 Interface to FRE contractors 1.4 and 1.5 2.8 Develop and perform demonstrations 1.1 and 1.6 2.9 Transfer technology 2.10 APGD program management 1.6, weekly conference calls, monthly report, and phone conversations.