LiDAR Day of Delivery

Whump! That"s the sound of your LiDAR data collection project being delivered to your desk. And given the ever-increasing size of LiDAR projects, your desk might bend a bit under the weight of the delivery (at least if bytes of data had any significant weight!)

Let's face it. Datasets are growing larger and larger. This trend is driven partly by the capacity of the hardware, the LiDAR data collection systems. Pulse rates continue to increase. The number of returns per pulse rate increases with optional hardware. Attribution per point can be increased by storing timestamps, return waveforms, or color attribution from correlated orthophotography.

Consider how much data will be delivered to you per flying day of your project. We"ll assume a 100Khz laser with a fair amount (40%) of multiple returns. Also, assume data is written with timestamps to an LAS file (more space efficient compared to ASCII), with 28 bytes per point. You can expect about 4 MB per second of data collection, or over 100 GB per an 8 hour flying day! And the hardware continues to improve.

But are our software systems keeping pace? When that data is delivered to your desk, will you have the software capable of loading and viewing the data? And will you see your LiDAR data together with your other geospatial datasets, all in a familiar software environment?

Capacity Issues

One seemingly obvious conclusion is you cannot load your entire LiDAR project into memory on your workstation. In fact, any one long flight line file would probably exceed the memory capacity of your computer. The entire project would be on the order of 100 times your computer memory capacity.

The traditional answer to the problem is to tile, or otherwise divide your LiDAR project. Each individual tile, or rectangular area, of data can then be loaded entirely into memory along with a few neighbor tiles. Often the project is delivered in tiles of data. However, there are issues with this approach. First and foremost, you see and work with your project in a piece-meal fashion. You cannot see your entire project, even at an overview level. Certain tasks may require the entire project. There may be tasks that span more tiles than you can easily load on your current system. And with all that, you may *still* be saddled with performance issues.

Fortunately, there is a better approach. New software systems are more clever about how a LiDAR dataset is brought into memory. For large LiDAR projects, you will definitely want software that avoids the old load-the-whole-file-into-memory approach. LiDAR software should bring into memory only parts of the data that are currently needed. For example, if you are zoomed in to the northwest corner of your project, the sections of your LiDAR data that are in this northwest corner should be brought in to memory, but nothing more! This selectivity can be achieved by preprocessing the data to create a spatial index of the LiDAR dataset. While preprocessing, one can also subset the data so that when viewing the entire project, or some overview area, a thinned set of points can be efficiently presented.

A few software tools do employ the new techniques for loading LiDAR data intelligently and efficiently. A standalone software tool that employs the above techniques is the MARS™ software from Merrick & Company. Another choice of tool is LP360 from QCoherent Software. LP360 is a LiDAR plug-in for ArcGIS and allows the integration of LiDAR and GIS data. Not only has QCoherent Software employed the techniques for loading and handling large amounts of LiDAR data, they have overcome the bottlenecks of drawing large amounts of LiDAR data in the ESRI environment. The combination of its ability to handle large datasets and integration into a GIS makes LP360 a very powerful tool in the LiDAR industry. Speaking of data integration...

Data Integration

The second big issue you face on the day your LiDAR project is delivered is getting your LiDAR data together with the rest of your GIS/geospatial data. Does your data line up? After all, you want to be able to validate and correlate your LiDAR data to data layers acquired in other ways and for other purposes. Perhaps there is a systematic error in the newly delivered data, which you want to identify as quickly as possible. Or perhaps you want to use the newly obtained LiDAR elevations to adjust some other data layer. Very often you want to use the new data to create something more, or to visualize the surface in a way that was not previously possible.

Admittedly, some LiDAR-focused tools can read other data formats. A LiDAR-focused tool may have implemented readers to bring in shapefiles, grids, imagery, etc. But ultimately you want to work in an environment that is familiar to you, not learn another tool. A high-performance LiDAR plug-in for the very popular ESRI™ environment has been lacking for some time. The ArcGIS environment has been designed for extensibility, so a modular, COM-based solution to accessing LiDAR data in this environment makes good sense. With a solution integrated to the ESRI environment, a vast multitude of other data types can be viewed together with your LiDAR data.

An interesting side note on data integration is the blending of imagery with LiDAR data. You may have orthophotography in any variety of formats for a project area. The ability to dynamically blend the orthos with the LiDAR point data, with additional adjustments in LiDAR point size and coloring, can yield very surprising, interesting, and revealing visual products.

QA/QC

Finally, when that LiDAR project is delivered, you"ll want to have special tools for performing QA/QC on the delivery. Several tools come to mind.

First, assuming you have survey control points for the project area, you"ll want to generate a control report. The control point report will compare your survey control points to an elevation calculated from the LiDAR data. Since LiDAR points and survey points usually don"t happen to be in the exact same X-Y location, a surface will need to be calculated from the LiDAR points classified as ground or classified as your surface model keypoints. Along with calculating the difference in elevation from control to LiDAR surface, the control report should provide statistics for the set of control points, i.e. RMSEz, vertical accuracy at 90% or 95% confidence interval, skew of control point differences, etc.

Second, along with this very statistical report, you"ll want tools to allow you to visually inspect the data. Of course, the data integration described above provides visual verification; you can literally see if there is a horizontal shift between your LiDAR data and other datasets. However, additional tools may prove useful, such as a profile viewer that lets you see your data "from the side". If the profile viewer is set to display LiDAR points by classification, you can quickly identify points not properly classified as ground by checking that the ground colored points, and only those points, are along the clearly identifiable ground surface line. With good profile view controls, you can literally walk through your data, checking that all is in order.

Conclusion

Fortunately, the industry is turning a corner in the availability of LiDAR software when a LiDAR project is delivered. New software offerings provide breakthroughs in performance, data integration, and QA/QC tools.

Based on the three topics discussed, the LP360 tool from QCoherent Software (www.qcoherent.com) is highly recommended. This plug-in for the ESRI ArcGIS environment has remarkable performance when run against even the largest datasets. It is very well integrated to the ESRI environment, and provides the QA/QC tools you need. To get a feel for the product, download a video demo from QCoherent"s website.

Don"t despair when those gigabytes (or terabytes!) of LiDAR data arrive. New LiDAR software tools are meeting the challenge!