Ruigang Yang, Matthew Joel Laibowitz
Department of Computer Science
Columbia University, New York
Before you start using AutoScanPlus, please read this manul THOROUGHLY.
AutoScanPlus is based on the frame work of Zipper program developed by Turk and Levoy . Our enhanced version provides an integrated software platform to acquire range data and register different views of range scans using our modified ICP algorithm. Combined with the range integration programs provided in another package(the name of this package hasn't been decided, but its name should come from M. Reed who wrote most of the original code, now those programs are in /n/moray/proj/ect/ryang/Geom) , it can be used in Automatic 3D model generation, reverse engineering and VR environment generation. Currently it can run under SGI machines.
All the source codes, tcl/tk libraries and one binary compiled under SGI
are provided in zplus.tar.gz. To install, you need to create a directory
for the software, then go the that directory and un-compress the package
using the following command
gzip -d zplus.tar.gz; tar xvf zplus.tar
We also provide the range data integration programs (most were written by
M. Reed, but I made significant changes to ppsweep, merge and int) , they
are now currently located at
/n/moray/proj/ect/ryang/Geom. Some programs may be needed to integrate models from different views. Please refer to Chapter 6. Misc. tools and more.
IMPORTANT: for every program/file mentioned in the manual, it can be found in the zplus package unless otherwise indicated.
Before you can run
4 environment variables need to be set
(Suppose the software is installed under /n/moray/proj/ect/ryang/gui)
Normal startup provides the complete set of functionality AutoScanPlus can provide. It requires a scanner server running on mistral, a Sun Sparc machine located in the Robotic Lab.The following diagram shows the hardware setup for the scan system. It contains a IBM robot, a laser range finder, and a precision turntable, they are all connected to the Sparc station's serial port. Matthew Joel Laibowitz built the switch box and the control software for the switch.
Setup the Scanner server on mistral
This server software only works on mistral due to the hardware limitation
Start the client
If every thing works, 3 windows will pop up, one is the Control Panel, indicating the current Robot parameters and various functions of AutoScanPlus, the second is the Selection Window, indicating how many files has been loaded, the last one is the object window where loaded objects are displayed using OpenGL. If the robot position values seem abnormal, type getpos in theAutoScanPlus shell. This is because of the delay between robot and server.
In this mode, the Remote Scan Module will be disabled.
This may be useful when the scanner server program is not running, or to
use AutoScanPlus as a data
registration tool. To start the program in this mode,
(without any parameter) to start the program. The GUI will be different, mainly remote control panel disappears.
AutoScanPlus integrated the functionality of Zipper and AutoScan, and added some new functions, including remote scan options and an integrated registration module. In the following section, only these new functions will be introduced. For the functions inherited from Zipper , please refer to Zipper's manual.
In the top part of Control Panel, as shown below, there are 3 sub
panels. The top one is the IBM robot control panel. It displays the current
position of the robot. The user can input some value to move the robot to
a certain position. The valid values for the Z Coordinate are between (-150,
0). Don't try to move the robot beyond (-150, 0) in the Z coordinate,
or the robot will be automatically shut down.
The bottom panel is the turntable control panel. It could manually control the turn table to turn to a certain degree. The current degrees of the turn table is also displayed there. You can either enter a positive value to turn it anti-clockwise or a negative one to move it clockwise.
The middle one is the scan control panel. It requires 5 parameters. # of scanlines, Average, start Z pos (in the robot frame), # of scans and the degrees of every rotation. the z-step between 2 consecutive scanlines is 1 mm. (Note: AutoScan for Sun has been improved so that user can change the Z-step value for every scan, at this time, this function hasn't been incorporated into this SGI version)
a remote scan can be started, first make sure the robot, range finder and
switch box are turned on, and the connection between
AutoScanPlus and scan server
is open. A good way to test this is to try to move the robot arm around.
If every thing is working fine, fill in the parameters in the Scan control
panel, then click the scan button. In the console, you can see dots been
printing out during the scanning process, and at the mean time, user can
manipulate the objects already loaded because
AutoScanPlus is a multi-process
application. A typical scan, which contains 100 scanlines average over 3
repeated scans, will usually take 5- 7 minutes.
After the scanning is finished, the newly scanned object will be displayed in the object window. You can view the object in the object window, the 3 mouse buttons' functions are as the follows: Left: rotation, Right: zoom, Mid: translation. The picture at above right shows the Selection window, where a list of loaded objects is displayed. In the picture, it indicates that one ply file(f.ply) was loaded. User can click on an object's corresponding Gray radio button(rightmost one) to display the gray scale range image. (Note: this operation requires the raw range data, if one object is loaded from a ply file, this operation is not allowed). In the gray image window, the user can view the histogram by clicking the Histogram button. The X-axis stands for the z-value of the rangedata points. Y-axis is the relative value of the data distribution. The user can use the 2 rulers to specify 2 thresholds, then clike Apply to get a new gray scale image. Only the datapoints that fall within the 2 thresholds will be displayed. Thus, the user can have an enhanced image of the object.
LIMITATION: in the current version, it requires the server and AutoScanPlus be within one NFS system, because range data is actually first written to a file, and AutoScanPlus reads from a file. This can be modified in future version.
Click on the Align button will bring up a Align Window as shown in right. You can either do a manual registration or automatic registration.
To start a manual registration,
you need first select the anchor (stationery) view and align view, which should be moved to be aligned with the anchor view. You can do this selection by clicking the corresponding radio button in the Selection Window. (Some time, it is difficult to remember which object is which, you can deselect the Vis(visiblity) checkbox in the Selection Window to hide corresponding object in the selection window, then you can tell which name is which object). Then you need to use the mouse to pick out at least 3 registration points in each view. The order of selection is important, the first selected point in the anchor view will be aligned with the first selected point in the source view. During the process of selection, you can click the Clear button to reset all points selected and start over. The Null button can be used to switch between selection model and view model (i.e. changing the view point of the mesh). After at least 3 points pairs have been selected, click the Exec button to start the alignment. The result will be displayed in the view window.
Automatic registration is easy to use, simply select the anchor and align view, then click Auto-Align to start the process. It first looks for a pts file which can be generated by another separate program(edect) which detects the faducial marks planted on the object and prints the coordinates of these marks to a pts file. If that file is found, it uses these points pairs to do the registration which is a very fast procedure. If that file is not found, it start a 2 stages matching process, first uses the feature-based registration method to get an initial transformation matrix(this stage may run about 10 minutes), then uses the modified ICP algorithm to further improve the accuracy. However, like most feature-based method, due to availability of dominant features, it may result in a erroneous match. You can also use Zipper's alignment routine by clicking the ZipperAlign button.
User can inspect the result of the match on screen. If the result is satisfactory, use the mprint command to save the transformation matrix to a text file (usually with a .mat suffix), which can be used in the integration process we will discuss next.
After all the data has been properly registered, we need to run the data
integration program to piece together a complete solid 3D model of the object
from scans in different viewpoint. Since the integration process is a very
lengthy process which may takes 30 mins to 3 hours according to the size
of the object, they are separate programs. First you need to
subsample the raw range data to generate a file with a suffix of i_sub_z.dat,
it is usually done by AutoScanPlus if you scan a
new image. But if you started from an existing ply file, you need to run
the BuildAll program to generate this file. The ppsweep (now in
/n/moray/proj/ect/ryang/Geom) program can convert every scan's raw data to
a solid extruded model. Here is the usage of ppsweep
usage: ppsweep [-l distance] [-s] [-d] [-b line] shortdatfilename
defaults: d = 0.0 (all surfaces included), s = FALSE (no swaps), d=FALSE (no dilation), b = 1 (base cutoff 1 scan line)
-ldistance distance threshold, if the distance of 2 points in one triangle is greater than distance, this triangle will
be deleted, note that 0.0 (the default value) means all surfaces included.The unit of distance is1mm.
-d[x|y] dilation option, dx means dilation in horizontal direction, dy means dilation in vertical direction, dxy
means in both direction.
-s swap the direction of edges in certain triangles. For every 4 points which forms a rectangle, the default
diagonal connection is NE to SW. If this option is given, the diagonal connection can be NE to SW or
NW to SE. This can smooth some edges.
-bline base cut off. The last few scanlines of an image ususally contain the turntable where the object was
placed on, the default is to cut only the last line (line = 1). If this option is given, last line will be cut.
shortfilename the filtered, subsampled dat file without .dat suffix
ppsweep will accept a file with the _i_sub_z.dat suffix to generate
2 files called swept-0.sat and swept-1.sat for every input file. Example:
ppsweep -dx -s -l 1000 scan0_i_sub_z will read the scan0_i_sub_z.dat
file to build a solid partial model with dilation in the horizontal direction,
swap the diagonal connection if necessary and discard and trangles bigger
than 2000 mms.
ppsweep will accept a file with the _i_sub_z.dat suffix to generate
2 files called swept-0.sat and swept-1.sat for every input file. Then you
can run the
merge (now in /n/moray/proj/ect/ryang/Geom) program
to merge those separate files to one sat file. Finally you can run the
int (now in /n/moray/proj/ect/ryang/Geom) program to piece
together scans from different viewpoint to a complete solid 3D model. The
int is a list of sat files needs to be united.
For every filename in the list,
int looks for the transformation
matrix file with a suffix of "trn", if this file is found, it will transform
the object according to the matrix provided by the trn file, otherwise, it
just performs a boolean intersection operation with other sat files. The result
will be saved in a file named "int.sat".
(TIP: mprint in AutoScanPlus will save the matrix information of every file loaded to one file, so you need to manually split the the saved file so int can find the transformation automatically. Please make sure that the trn file has the same name as the sat file, like for scan0.sat, you must save the trn file as scan0.sat.trn. In the trn file, only one forward transformation is needed and it is stored in text format as a 4x4 matrix). For sat file bigger than 10 MB, int might crash due to some unknown bugs. If you encoutner such case, you may use the ACIS test harness program to do the intersection.
In this section, a sample will be given to show how to use AutoScanPlus to generate a solid model from multiple range images.
AutoScanPlus's improvement over the original Zipper Program is mainly in 3 areas: Graphic routines, Registration Module and Remote Scan Module. In this chapter, we will first discuss the data flow in AutoScanPlus, including the data integration programs currently in /n/moray/proj/ect/ryang/Geom, to give a big picture how the program works. Then we will discuss the implementation about those major improvement.
The following chart depicts the complete process of 3D model generation from laser range image. It can be roughly divided into 4 stages: Laser Scan, Preprocess, Display/Registration, and Integration. The first three stages have been integrated to the AutoScanPlus. We also provide integration program using M. Reed's sweep-volume algorithm to generate complete model.
Display/ Registration: Display rountine has been substantially changed
to use OpenGL. The display of gray images are done by the extern program
bin2txt, which convert the raw range data to a pseudo rendered gray scale
image. Registration is pretty much integrated to
AutoScanPlus except the method
using fiducial marks. The detect of fudicial markes is done by the
edect program, it will look at the raw range data to generate
a pts text file, which looks like that
Turn:(-0.383066 0.080810 1.460000) Turn:(-0.052066 -0.189190 1.500000) Turn:(0.343934 -0.015190 1.360000)
AutoScanPlus's registraion program will first look for this file because this method is very fast. AutoScanPlus can generate the transformation matrix by using the
We don't have any program to view sat file directly and efficiently at this time. Yet we do provide a lot of useful filters that can convert SAT file to a lot of different popular formats like VRML. In Chapter 6, we listed all of them for your reference.
The Zipper program used the IRIS GL to implement the display and manipulation of 3D models. IRIS GL is only available for the SGI platform, and it is an obsolete library. OpenGL is the successor of GL, it is a platform independent graphic library which is available on almost every major platform, SGI, SUN or MS Windows. So, we decided to port our AutoScanPlus to OpenGL, so it can run on different platforms. But due in part to OpenGL's focus on portability, OpenGL and IRIS GL differ in several major areas. The most distinct area is the IO routines. OpenGL is a window-system independent "pure" graphics library, it contains no windowing, pop-up menus, event handling, color-map loading, buffer allocation/management, font file formats, or cursor handling routines, which are part of the IRIS GL and Zipper used heavily. Though Silicon Graphics provides a small auxiliary library that can be used to replace simple windowing, event handling, and color-map loading calls, Zipper is more complicated than it can handle. Fortunately, there is a publicly available package called OpenGL Utility Toolkit (GLUT). It provides a programming interface with ANSI C bindings for writing window system independent OpenGL programs. So, AutoScanPlus's graphic routine is written with a combination of GLUT and OpenGL. For backward compatibility, we provides 2 sets of codes for the graphic routines, one uses the IRIS GL, the other uses OpenGL. The one which uses OpenGL is currently located at /n/moray/proj/ect/ryang/ect/OpenGL, GLUT library is located at /n/moray/proj/ect/ryang/glut.
Zipper uses an ICP algorithm to precisely register pair of range images. It can generate a satisfactory result if the pair of range data that need to be registered are roughly aligned initially and there are some distinct features over the surface of the object. We provide more means of registration and integrate them into AutoScanPlus. For object without salient features, we can use the manual registration method or introduce some registration point by planting marks on the object. For object with salient features but not even roughly aligned, we can use the feature-based registration method to derive a initial estimation of the transformation matrix which may be used as the input the of the Zipper's ICP algorithm. In the paper Registering, Integrating and Building CAD Models from Range Data , we discussed these methods in detail and presented some models we build. So, here , I just briefly discuss the data structure we used to implement these methods. Range data is tessellated to a mesh in the preprocess stage. Every scan is associated with a a HashTable, where every vertex in that scan is stored. Vertex in the HashTable is indexed by its 2 curvature values, so we can find points of similar curvatures within O(1) time. These curvature values are pre-computed by
is the file where most registration functions are implemented.
Remote Scan Module:
Autoscan can only be started on the console of one particular machine in our lab, mistral, because it requires IO operations to some low-level devices, like serial port , audio port, which operation is only permitted for the console user. In AutoScanPlus, we use a client-sever architecture to circumvent this problem. There is a server programming running on Mistral, the host machine which the Scan System is physically connected to. AutoScanPlus includes a remote scan client module which can talk with the server program. The client can send command to the server, the server will execute the command and send back the reply from the robot system. Client and server are connected by the IP protocol, they use standard socket API to exchange packets. So, user can start a scan process from a remote machine, the result will be displayed on the screen right after the scan is done. It provides a more convenient way to acquire range data, and in some sense, we can use AutoScanPlus to "fax" a 3D object over the Internet.
tcl_remote.c is the file where all
the remote scan functions are implemented.
Here is a list of tools and script I have written or changed for the project.
The following is the list of models I generated during the last half year. Notice in the /n/moray/proj/ect/ryang/data directory, there are a lot of different models, every model has a few of obj files. The following list gives the obj files which have the best quality:
In /n/moray/proj/ect/ryang/office/, there are the data from the PennOffice
Project. SetA.iv is a integrated points set from all pictures(huge, about
40 MB). Individual data for every picture is stored as binary MatLab file(file
with a suffix
mat). One can use the mat2txt function I wrote
to extract points in the matlab mat file to a plain text file and then use
the iv2obj program to build a surface mesh or the iv2sat program to build
an extruded solid model. There are 3 setA's solid model , each in a
different resolution. We used the
60 by 192 model to build the real model on the quick slice machine.
This project is done under the supervision of Prof. Peter Allen. Thanks also goes to M. Reed for his guidance through the AutoScan project. Zoran Lazarevic gave excellent suggestions about the implementation of some numerical methods. I present my gratitude for their kind help.