Sasha Mirpour 3D Camera Tracking
Sasha Mirpour 3D Camera Tracking
Sasha Mirpour 3D Camera Tracking
Sasha Mirpour
September 2008
Creative Programming
Supervisor/Examiner: Sharon A Lazenby
Co-examiner: Ann-Sofie stberg
Sasha Mirpour
Abstract
In the past decade computer generated images have become widely used in the
visual effects industry. One of the main reasons is being able to seamlessly blend
three dimensional (3D) animation with live-action footage. In this study, different
3D camera tracking software (also referred to as matchmoving) is compared
focusing on workflow, user-friendly system, and quality of production.
Keywords: matchmoving, solving, 2D tracking, 3D calibration, parallax,
photogrammetry, computer vision.
Table of Contents
1 Introduction ............................................................................................................... 3
1.1 Problem definition ............................................................................................................ 4
1.2 Aim ................................................................................................................................... 5
1.3 Question at issue ............................................................................................................... 5
2 Research.................................................................................................................... 6
2.1 Past Research .................................................................................................................... 6
2.2 Current Research .............................................................................................................. 9
3 Preparations ........................................................................................................... 11
3.1 Cameras .......................................................................................................................... 11
3.1.1 Film back/focal length .......................................................................................... 12
3.1.2 Image/pixel aspect ratio ....................................................................................... 12
3.1.3 Frame rate ............................................................................................................ 12
3.1.4 Lens distortion ...................................................................................................... 12
3.2 Planning .......................................................................................................................... 13
1 Introduction
Matchmoving is a crucial element of many visual effects shots anytime computer
generated elements needs to be placed into live-action footage. The three dimensional
(3D) objects should be inserted in a method where they appear to move as if they were
part of the real footage with the correct position, scale, and orientation in relation to
the photographed objects in the scene.
Imagine a computer generated robot walking across a street or standing on a building
while the camera is moving, there would be a need for accurate 3D camera tracking
software. Matching the movements of a real camera seamlessly and being stable
enough over extended sequences is difficult while preventing problems with jitter and
drift which would otherwise ruin the appearance of the objects that are placed
correctly in the footage. Basically in the best case scenario, matchmoving allows
directors room for more freedom in terms of location, saving time and money by not
having to set up extensive bluescreen sets or motion control rigs in a limited
environment.
In the November 2001 issue of Millimeter, author Steven D. Katz simply explains
how 3D tracking performs. Matchmoving, optics, photogrammetry, and perspective
drawing are all part of an area of mathematics called projective geometry (Figure 1).
Applied to various spatial problems, it can provide solutions for measuring objects at a
distance, locating objects in space, and extracting 3D models from photographs. He
explains that "camera matching software utilizes a subset of projective geometry
called epipolar geometry (geometry of stereo vision) (Figure 2).
This branch of mathematics is used to describe the geometric relationship between two
optical systems viewing the same subject and can be used to locate points in space.
Because a moving camera offers a new view every frame, epipolar geometry works
for a single moving camera as well, and each new view is understood as a separate
optical system." Ultimately projective geometry is a method to accurately show how
3D space can be projected as 2D images while photogrammetry, shows us how 2D
images are used to calculate 3D space. [1]
Figure 1: Albrecht Drer, A Man Drawing a Line, 1525. The idea is that we can get a correct image of
some object seen through a veil or a window by tracing the outline of the object.
Figure 2: Two cameras take a picture of the same scene from different points of view. The epipolar
geometry then describes the relation between the two resulting views.
The final scene with a well created matchmoving shot is never noticed by the
viewer. Quoting Tim Dobbert, the author of Matchmoving - The Invisible Art of
Camera Tracking: If you`ve done your job right, no one should ever know you`ve
done your job at all. So despite its importance, it is completely invisible in the final
shot if it is done right. [2]
Choosing the right application can be difficult but evidently the one tool that provides
the most techniques for solving a matchmove is the preferred one.
1.2 Aim
The aim for this thesis is to compare different camera tracking software (also referred
to as matchmoving). The chosen applications range from being professional, freeware
and built-in 3D software`s.
I will also insert 3D objects into different video sequences of exteriors and interiors
and with the given results, determine which application would provide the best results
in the various situations.
I plan to use this current research in order to help advance my knowledge and skills in
creating a complete and correct matched environment in 3D, which would help
acquiring more accurate results.
When comparing the end results of each solution what will be the best choice in using
matchmoving software by determining speed of workflow, ease of use, and quality?
Cameras (lenses, formats, distortion etc.)
Understanding how cameras work, can help a great deal in putting all the pieces
together in matchmoving. What do I need to find out about the cameras and how do I
find that information?
2D Tracking (interactive/manual tracking)
Markers need to be added to important features in the image where the software can
calculate the cameras motion in order to re-create the three dimensional layout of the
scene. What features in an image should I look for when placing the markers? And
how do I deal with image sequences that suffer from motion blur, occlusion, noise
etc.?
3D Calibration (solving the cameras attributes)
Calibrating or solving the camera is probably the hardest part of matchmoving and
finding out the position and motion of the camera in the 3D scene. How do I find out
what is happening in the scene and how do I evaluate the solution in order to provide
the best results?
Automatic Tracking (editing, masking)
In the beginning, 2D tracking was always created manually. Ever since automatic 2D
tracking appeared, it has made more shots solvable and easier than before. What are
the technical differences between manual and automatic tracking and how are the
benefits compared?
Reconstruction (fitting everything in a 3D environment)
This is the last step in the matchmoving process. The goal is to make the camera fit the
digital set or 3D scene. Therefore, how do I accurately fit a matchmoving camera into
a computer graphic environment?
2 Research
2.1 Past Research
The science of extracting 3D information from 2D photographs predates Computer
Vision; it has a century-old history in the field of photogrammetry, which means
measuring from photos. [3] It has not been in widespread use, except in the field of
surveying, until computers became fast enough to perform the complex calculations.
This made researching in the field of Image Processing and Computer Vision
applicable to the matchmoving problem. Initially scientists built vision systems for
robots to autonomously navigate by building a full 3D representation of its
environment using only standard 2D cameras. Successful examples of this technology
can be seen in the NASA rovers Spirit and Opportunity, navigating their way around
the rocky surface of Mars (Figure 3). [2] [4]
Figure 3: NASA`s Mars Rover Spirit. Using photogrammetry to help navigate it`s way around the rocky
surface of Mars with a minimal amount of human intervention.
Figure 5: 3D tracking software developed at Digital Domain which was used on nearly every shot of the
movie Titanic.
Another interesting use of augmented reality with Computer Vision can be seen in the
memory game levelHead, created by Julian Oliver. A plastic 5x5x5cm cube with
unique markers on each face is held in front of a camera (Figure 8). The camera sends
the image to a screen while capturing all the moves by displaying tiny rooms on each
face of the cube, which are all connected by doors. And in one of the rooms is the
10
character. By tilting the cube the player directs the character from room to room in an
effort to find the exit. The software recognizes the marker where it can overlay 3D
content on a per-face basis giving a convincing impression that each room is somehow
inside the cube. [14]
A question I had about where matchmoving is heading in the coming years and how
software would change was answered by Russ Andersson with two main points. The
first he made was simply being able to extract more information from images. His
second point was software being able to analyze more kinds of shots automatically,
with less human input. If software can handle the more difficult shots, then obviously
less time and less skilled operators would be required, making those more difficult
shots called simple and filmmakers giving more outrageous extremely-difficult
shots to work on instead. [15]
3 Preparations
Knowing where to insert the effects or 3D objects and by planning ahead and
gathering as much information as possible helps a great deal in tracking and solving
the scene.
3.1 Cameras
For this research project, I have used a regular Digital 8 camera, DCR-TRV520 NTSC
from Sony. Digital cameras use a small chip called a charge-coupled device (CCD) to
11
record light coming through the lens and it is the shape of the CCD that defines the
shape of the image. There are a couple of important things to take into account when
figuring out a camera, because to a matchmover these can have a wide range of effects
depending on how they are calibrated. Many cameras feature built-in stabilization,
using a variety of operating principles which can alter the image in ways that might
disrupt the tracking and solving process. It is recommended and probably safest to turn
off the camera stabilization feature when possible working with visual effects,
SteadyShot in my case.
12
reapplied to the computer graphic elements in the final composite or it will appear to
tear away at the edges of the frame.
3.2 Planning
In the matter of preparing and planning how to insert effects or 3D objects, Russ
Andersson Illustrates this on his website by showing how an optimal camera path
would be best suited for 3D tracking. [16]
The red arrow (to the left) in Figure 9 shows an example of how to not shoot when
moving through your scene. It does not focus on anything long enough to obtain a
decent 3D track. By focusing on a subject or area while slightly rotating around it, the
perspective changes and this change is called parallax shift. This is important as
matchmoving software analyzes parallax in the image sequences and uses that
information to generate a 3D camera and scene.
Figure 9: The green arrow (to the right) shows the proper path to take in order to create enough parallax
shift for a successful solve.
When recording, it is always recommended to try and keep the camera steady as much
as possible for it will help a great deal when it is time to do the 2D tracking.
Steadycams are the best solution; however they do not come cheap. Obtaining similar
results is shown in a tutorial by Johnny Chung Lee (Figure 10) on how to build your
own camera stabilizer. [17]
13
Measurements of the camera height and distance to the subject or other objects in the
scene are also important steps to take as they can be a significant assistance during a
computer graphic set fitting. It aids a great deal in making the scale of everything
accurate.
Adding markers in the set or scene of a recording can help aid in finding the correct
scale and help improve the solution of the tracking procedure. An example of how a
tracking marker could look like is shown in Figure 11. This would be recommended in
environments with very few trackable features. The markers should be placed in
regular intervals rather than randomly spread around the set, and preferably with
known distances apart. And the distance between them will depend on how tightly the
shot is framed. Only the areas that are tracked will be known in 3D space. Therefore if
no tracking markers are placed in a particular area, the software will have no system of
accurately reproducing that portion of the 3D scene. [18]
Figure 11: This is a tracking marker. This will save you money. Written inside the triangle.
14
15
Figure 12: Multiple tracks that end or start on the same frame cause breaks in the cameras motion path.
Only stationary objects should be tracked, but objects like trees which are rich in
detail that might have subtle movements caused by wind should not be tracked as
matchmoving software is extremely sensitive to even the smallest movement. Features
such as lens flares, reflections, and specular highlights should also be avoided as they
create moving artifacts in the footage.
When heavy motion blur causes the tracker to fail expanding the search area helps as
it allows the tracking engine to look further during its search for a pattern. Expanding
the pattern size might also benefit but it will slow down the tracking. As a last resort
keyframing the track by hand through the motion blur can be very difficult and
inaccurate but will at least provide a solution. Temporary occlusion of objects that are
being tracked can also be helped by keyframing the gap a few frames. Heavy noise
from film grain or compression can cause the resulting camera motion to be jittery, by
removing the grain before tracking either by compositing software or using
uncompressed images which makes for a smoother motion. Another method is to only
track one channel for instance red or green where film grain is often heaviest in the
blue channel.
4.2 3D Calibration
In most matchmoving software, the calibration is a one-button operation. The user
simply instructs the program to solve the shot. The program analyzes the 2D tracking
that the user has provided and generates a camera that matches the real-world camera
by adding markers that represent the 3D locations of the features tracked in the 2D
tracking.
If the calibration is correct, the 3D markers should line up with the features that they
represent in the image. The easiest way to verify it is by looking through the realworld camera and the 3D camera created by the calibration process. A first-rate
16
calibration is the perfect or near-perfect alignment between the 2D track and the 3D
marker when viewed through the 3D camera.
Working with animation is usually an iterative process. A solution must be zeroed in
on rather than solved the first try which also applies to matchmoving. When dealing
with an inaccurate solution caused by bad 2D tracking, one needs to go back and
adjust it. Matchmove programs use an iterative process for determining the camera
location so it is a good reason to approach a calibration in the same manner. If for
example problems occur in a certain area of the shot, then troubleshooting in the same
area is advised.
Tim Dobbert writes in his book, Matchmoving - The Invisible Art of Camera
Tracking, about three simple steps when evaluating your solution and these are
checking the 3D nulls, 3D space, and the rendered movie. The first step would be to
compare the 2D and 3D markers (nulls) simply by looking through the 3D camera.
The nulls should appear to match up with the tracked features and the size of the nulls
should also correspond to whether they are close or far away. Checking the 3D space
is accomplished by looking through a perspective view of the 3D scene to see if the
camera moves as one would expect it too without any major spikes or jitter along the
path of the camera and to see if the 3D nulls are in the right position. And the final
check is made by rendering out a test movie to see whether the markers are sticking
throughout the sequence without signs of noise or subtle drifts. [2]
17
One benefit of automatic tracking is having a camera motion that is less noisy or
jittery when tracking a shot with a lot of noise or film grain. This happens when a
deviation from each 2D track causes similar small scale deviations in the solution.
Automatic tracking does not suffer from this problem as often, due to the large
number of tracks involved in the solution. It could be helpful to run automatic tracking
on top of manual tracking as it would have an overall dampening effect on noisy
camera motions.
While the 2D tracking process is greatly simplified, it will not always track the
features that are desired. Just as any automatic process, the 2D tracking needs to be
monitored and in some cases manipulated to produce the right results.
5 Set Fitting
Fitting the camera into the scene has two purposes. One is a method of checking the
final quality of the matchmove in the 3D animation software. Even if this can be
actually tested inside of the matchmoving software, it does not represent the exact
appearance that results in the final stages of rendering. And secondly it also provides
some sort of reference where a simplified version of the full scene is placed, which is
called a proxy set for the scene layout. Basically, a proxy set is used to determine
whether the various areas of the computer generated scene match up with appropriate
features in the real set.
For example, if a computer graphic robot is supposed to jump between two buildings
and then down to the ground, the matchmover should provide two simple boxes to
represent the buildings and a plane to show where the ground is located which the
robot will fall down on for the animator. It does not have to be detailed. It just has to
show the spatial relationship of key objects in the scene.
A problem during the initial building of a proxy set is figuring out the scale of objects
and different measurement in the scene if they are not available. Studying the footage
looking for any known dimensions, such as standard window heights, tiles on the
ground or cars on the street is a good place to start. When the proxy geometry is in
place adding transparent checkerboard textures on them enables one to see whether the
entire scene is properly matched or not. Alternatively enabling simple wireframes on
the proxy geometry is just as good. Checkerboards and wireframes show how well the
3D objects sticks to the original footage where there is lack of geometric detail. [2]
When putting the camera in the right position within the set, it has to be moved around
and placed to its starting position. The easiest method for doing this is to create a
simple camera rig by parenting the camera under another object e.g. a null. In this
approach, the camera retains its original path without being broken when moved
around.
18
The scale of the camera move is also of importance as knowing not only where the
camera started, but also how far it goes during the shot. Defining the scale of the
camera move leads to finding the cameras starting and ending positions. This is
prepared within the matchmoving software by setting up a coordinate system which
includes a field for the scale of the scene. The scale is set by defining the distance
between one or more features in the scene. If the height of a window is two meters
then tracking the corners of that window and defining it in the coordinate system as
having a scale of two meters, the camera will then move relative to that scale.
6 Applications
I have tried to select a broad range of different matchmoving software in order to
obtain a sense of which system might work best in any given situation. They vary in
price range and are very different in terms of how the user interface is laid out.
name
SynthEyes
2d3 boujou
PFTrack
Voodoo Camera Tracker
Maya (Unlimited) Live
version
2007
4.1
4.0
0.9.4 beta
2008
type
pro
pro
pro
research
plug-in
price
$
399
$ 10,000
$ 5,250
$
0
$ 4,995
6.1 SynthEyes
SynthEyes is featured packed and comes with a very low price tag, which is used by
many big names in the industry. It can handle shots of any resolution DV, HD, film,
and IMAX. It also exports to a whole range of software and offers a scripting
language, SIZZLE which makes it easy to modify the exported files, or even add
export type. Russ Andersson is the owner and creator of this software and also
maintained by himself providing everything from support to tutorials. [19]
19
6.3 PFTrack
Pixel Farm`s Track was launched in 2003 and brought many new tools to the tracking
scene such as integrated optical flow, geometry tracking, and multiple motion solving
which is used by a growing number of renowned visual effects companies. As of
version 5.0, stereo camera functionality is added amongst numerous improvements
and already considered an industry standard. PFtrack evolved and was developed
using a licensed technology from the Icarus program which is still available free for
non-commercial use. PFTrack is very similar to Icarus in terms of the UI layout and
workflow, and despite that it has not been developed in recent years it actually stands
out as a solid matchmoving software. [21] [22]
Maya Live which is available with Autodesk Maya Unlimited has not been developed
a great deal in recent years and is lagging in features such as automatic tracking
compared to other stand-alone matchmoving software. However, it serves as an
addition to an already great 3D modeling software. Autodesk recently acquired
Realviz which has a set of graphics software ranging from image based modeling to
motion capture and also a well known matchmoving software called Matchmover
which is currently discontinued but could probably be rebranded and replace Maya
Live. [24]
20
camera for examining the results but the final output of renderings where done mainly
with 3ds Max except for when Maya Live was used.
For practical reasons, I converted the imported DV footage to the TARGA (.tga) file
format, mainly because the Windows version of Voodoo Camera Tracker only imports
footage in .tga format. Maya also requires image files when previewing a sequence
through the 3D camera as long as the filename and numbering is set apart with a
period. Using multiple image files, will make it easier to work between different
software packages by never having to doubt the exact frame count when inside e.g.
Maya or 3ds Max. It is important when the original footage and the tracked camera
animation have to match up for only a one frame slip will break the entire sequence.
When comparing the provided software, five steps were chosen to test the software
which is listed below.
I was not able to explore every single feature included in each one of the software
other than finding out the most basic methods of achieving a solution. I planned on
finding features that are consistent and trackable throughout the image sequence. It
could be anything from a rock on the ground to an indicator on a car. These should
have a high contrast in relation to their immediate surroundings and not change due to
camera perspective. For example, an intersection where one object crosses another, a
highlight which moves across glass or an object that moves like leaves on a tree. All
of these will produce markers that do not represent any static locked property in the
scene.
7.1 SynthEyes
If I would describe SynthEyes in one word, it would be speed. It is by far the fastest
tracker and solver of the list. It is evident from the moment the program starts running
for it opens up instantly. The first impression of the user interface is that there are very
few menu items present. It is similar in layout to 3D modeling software with four
windows each showing top, front, left and perspective/camera view as shown in
Figure 13. Importing the footage brings up a settings window that can determine
start/end frames, frame rate, interlacing ,and different presets for aspect ratios and
back plate. There is also a button for image preprocessing for cropping and stabilizing
the image or even blurring the footage if there is too much noise throughout the
sequence. At the same time, it is possible to scrub back and forth throughout the
footage. In my case, I did not use any of the preprocessing. When importing is
completed, SynthEyes starts loading the footage into memory (caching) by simply
showing the time line changing from red to white just below the menu. It is effective
21
and very fast when scrubbing through the timeline. Before starting the tracking
process, choosing a motion profile that fits the camera movement helps the tracker to
work even faster and most importantly options to turn on whether the camera was
zooming during the shot or mounted on a tripod. Also SynthEyes by default assumes
that the shot is smooth from a steady cam, dolly or crane and by selecting Hand-held:
Predict/Sticky under the Tracking menu it adjusts the tracking engine further.
Going through the tracking and solving process within SynthEyes is aided by looking
at the error rate at which it solves the scene. Numbers above 1.0 is deficient while
obtaining close to 0 is the best, although being slightly below 1.0 will provide a
workable solution. I used the automatic tracker which supplied a fair amount of tracks
without cluttering the scene too much. Trying out the solver provided an error rate just
above 1.0 hpix. I identify tracks caught on reflections and moving parts of the scene
and delete them which supplied a much better error rate that made the solution
efficient. While the solution almost achieved the desired result, it was not entirely
perfect. I wanted to make it better by using supervised trackers together with
automatic trackers which was accomplished by marking long lasting features spread
throughout the footage.
The coordinate system operates by choosing three tracks on the ground; one is for the
origin of the scene, the second is for the X axis and third is for where the actual
ground plane should be. Adding 3D test objects is easy to create by using simple
controls of the viewports that feel very natural at the first try and immensely easy to
navigate with only the mouse. Playing through the sequence shows how proficient the
3D objects are sticking to the ground. Exporting the scene is done by choosing one of
many 3D animation packages available in the extensive list of software alternatives.
Final results are shown in Figure 14.
22
23
Figure 15: The wizard below the main window with the help documentation on the right side.
7.3 PFTrack
Importing footage did not bring up any options for settings instead they were found
under camera parameters. Even the dialog was pretty limited as it seemed to
interoperate the footage correctly and the focal length would be the only obvious
parameter to change. There was no zooming in the footage; therefore it was left as a
default. The automatic tracking was fairly time consuming as it tracked ahead and then
once more backwards. Similar to boujou, the tracking points almost built up the entire
scene.
24
There were a couple of great tools beneath the main viewport window, one of them
was the tracking quality graph (Track-E) which showed the error rate in green, yellow
and red where red was the worst (Figure 17). By choosing Clean Tracks under the
Camera menu, the tracking threshold could be adjusted and this was shown with a
dashed line on the colored graph that excluded everything above it with the most
amount of error. Taking out all the bad tracks and leaving only the green colored ones
would not produce a more accurate camera. While some tracks had errors in them,
they also contain good tracks along the length of the timeline. The feature projection
(Proj-E) and camera error graph (Cam-E) were used to find and edit additional tracks
that may only have big errors during a couple of frames. It was a measure of how well
the 3D feature points match their 2D track positions when viewed by the camera.
There were even green tracks that could interfere when solving as they might follow a
moving object without problems but this was the same for all tracking software and
they should just be deleted. It was a matter of working toward a solution that
eventually works.
Figure 17: Shows the Track-E feature under the main window.
The automatic solutions did not provide the best results and manual tracks had to be
added in addition to the automatic. Manual tracking was actually straight forward but
not as simple as setting up the coordinate system which was easily created by laying
out two lines parallel to each other, preferably there was a clear line of sight with
walls and straight paths on each of the three axes X, Y and Z.
Managing viewports and adding 3D objects in PFTrack was made more complicated
than it had to be when compared to some of the matchmoving software which resulted
in just exporting the camera directly into Maya. It was a matter of snapping the objects
to points and then adjusting them, but I found the process rather cumbersome while
navigating around the viewports at the same time. The memory cache was off by
25
default and could be set to any amount desired, but there was also a preview choice
that lowers the resolution of the image which had to be played through once in order
for smoother playback. Final results are shown in Figure 18.
26
Second thing that needed to be prepared was loading the initial camera which brought
up a range of settings from focal length, film back to aspect ratio. There was also a list
of pre-sets from various camera types which helped out in case no information was
available for the camera used in the footage.
The tracking process was automatic and solved the camera when finished. Under
View Controls the main settings guiding how Voodoo estimates the camera tracking
were found under the Detection parameter. One of three different algorithms could be
chosen for the corner detection in the footage. It was not clear exactly what the
different algorithms named Harris, Foerstner, and Susan performed other than being
able to try them out to see if the results differ for the better. Otherwise it was advised
to keep them at the default configuration.
My first try with the interior shot did not provide any accurate results and produced
points behind the camera when looking through the 3D scene viewer which was a very
bad sign. The process took about one hour and finished with an error rate (RMSE) of
10.670 where it usually should be around or under 1.0 for acceptable results. Also
every time the tracking/solving was finished with a high error rate it suggested doing a
bundle adjustment to refine the estimation, this could take up to several hours with no
significant improvements especially with the amount of errors that I obtained from the
interior shot. I did try out the Forestner algorithm which provided better results but
still insufficient at an RMSE of 5.16. As a last resort I chose rotation camera instead of
free move with the default settings which assigned a 2D point cloud and with an
RMSE of 5.15. It actually worked by keeping the camera steady and not jumping
around until it started to turn in the sequence which took away the illusion entirely.
27
With the recommendations of avoiding motion blur, long sequences e.g. not more than
200-400 frames, and that Voodoo Tracker was more robust in estimating the camera
parameters for a camera mounted on a tripod, it provided a notion about how far one
could use the Voodoo tracker. When I began with the exterior shot, the tracker did a
much better job in solving this particular scene. And it produced one of the best 3D
point clouds out of all the trackers and with an RMSE of 2.23. I exported the scene to
3ds Max. The position of the scene was adjusted directly in 3ds Max simply by
moving around the null or dummy already created by Voodoo. The image plane or
background plate of the sequence had to be set manually inside both 3ds Max and
Maya. There was some sliding in the camera path but overall it was working
sufficiently for inserting 3D objects. Final results are shown in Figure 20.
28
The playback of the footage was not smooth but scrubbing through forward and
backwards enabled it to work much better. When adding track points and starting the
tracking procedure, Maya Live would bring up a window with a video showing the
resulting motion of the track. The tracks should be mostly green indicating a good
solid track. Repeating the procedure with around ten tracks was sufficient and should
provide an overall green progress bar. The tracking points should be spread
throughout the image and last as long as possible. At this point a ground plane should
be defined. To define where the ground level was located, I had to switch to the Solve
settings and select Survey in the menu with all the points on the ground selected,
change the constraint type to Plane and make sure registration only was checked,
and finally clicking on the create button.
The scale and origin of the scene was basically set exactly in the same procedure. The
difference being selecting only two points for Distance which determines the scale,
and one for the origin being the Point constraint and making sure registration only
was un-checked in both cases. Switching back to Solve in the menu and starting the
solving process, the scene quickly changes and was centered around the origin and
everything was then lined up properly. With an overall pixel slip of 0.661 the solution
was solid enough and worked. Adding 3D objects was simply created directly through
any viewport within Maya as shown in Figure 21.
Figure 21: Maya Live Solve menu below the four viewports.
Initially, I had some issues with the tracking. After successfully adding a couple of
tracking points and suddenly when I added another, it failed to track in any direction.
Sometimes, continued tracking was allowed by deleting the last one and adjusting or
changing position of where I added the same tracker. Otherwise, it produced an error
in the command feedback. The worst issue was Maya Live suddenly crashing.
29
However, this issue was resolved by entirely re-installing Maya. Final results are
shown in Figure 22.
30
Automatic tracking was simply the best way of speeding up the matchmoving process.
And with the ability to add your own tracking points together with automatic tracking,
it created an even more solid solution. Depending on how well the camera motion was
acquired from the tracking process, it determined how far one could recreate the entire
scene. By using checkerboard textures or wireframes on the geometry, it could reveal
how well the scene was reproduced.
Having acquired the basic knowledge in order to begin in my research, I have tested
and compared five different matchmoving solutions. They all performed sufficiently
in most cases, enabling me to insert animated 3D objects. One important aspect of the
entire matchmoving process was that I cannot expect matchmoving software to just
solve any scene that was thrown at it. Although if lucky, and satisfactory results were
achieved on the first try. It was still generally an art form to get everything correct and
working towards a solution, which comes with a lot of practicing and with a sheer
amount of trial and error.
PFTrack provided the best end results, but it also demanded more work in order to
achieve it. Having a great set of tools and unique features not found in any other
application, it appeared to be an impressive software.
In boujou, the use of the help wizard did not result in any better solving of the camera,
but it did provide workable results. Being the most expensive package of the list, 2d3
offered a stripped-down version, named bullet SD at a much lower price range and a
simplified interface and with the same tracking and solving features. I believe that I
should have chosen this lighter solution from 2d3 in terms of diversifying the current
list even more.
Voodoo Camera Tracker which performed very poorly in one of the test footage, did
surprisingly well in the other. With Voodoo being one of the few free camera trackers,
I was not expecting it to be a 100% solution. However, I did have high hopes that it
would. Having tested the ICARUS tracker even though I have not included it in my
report, I would highly recommend it as an alternative for anyone wanting to use a
capable tool within matchmoving.
Maya Live had an intuitive 2D tracking and easy solving procedure. Being the only
fully manual tracking software made it a rather time consuming, but still proved to be
very robust in solving the camera path with good results.
Ultimately, I found that SynthEyes performed the best overall in terms of speed, ease
of use, and results. Since it is the most affordable and intuitive matchmoving software,
the combination is truly hard to beat.
All the videos of the final rendered matchmoving results can be found online at
http://mirpour.uppsapps.se/matchmoving or on a CD-ROM disk from the Department
of Mathematics, Natural and Computer Science at the University of Gvle.
31
References
[1] D. Katz, Steven. The New Cinematography. Millimeter. November, 2001.
http://digitalcontentproducer.com/mag/video_new_cinematography/, July, 2008.
[2] Dobbert, Tim. Matchmoving - The Invisible Art of Camera Tracking. SYBEX Inc,
2005. pp 1-273
[3] E. Debevec, Paul. ACM SIGGRAPH 1999. Image-Based Modeling and Lighting
http://www.siggraph.org/publications/newsletter/v33n4/contributions/debevec.html,
July, 2008.
[4] Robert, Luc. IBE - International Broadcast Engineer,
http://www.allbusiness.com/information/internet-publishing-broadcasting/6989221.html, July, 2008.
[5] Roble, Doug. ACM SIGGRAPH 1999. Vision in Film and Special Effects
http://www.siggraph.org/publications/newsletter/v33n4/contributions/roble.html, July,
2008
[6] Roberts, Mark. Motion Control. The 7 Uses of Motion Control
http://www.mrmoco.com/index2.htm?sidebar.htm&/Newsletter/7_uses_of_motion_co
ntrol.html, July, 2008.
[7] Interview by Mary Helen Stoltz with Steve Sullivan.
http://web.mst.edu/~w3/alumnus/fall02/sullivan.html, July, 2008.
[8] Mseymor7, fxguide. Art of Tracking Part 1: History of Tracking
http://www.fxguide.com/modules.php?name=News&file=article&sid=209,
July, 2008.
[9] Johnmont, fxguide. Art of Tracking Part 2: Tips & Apps Overview
http://www.fxguide.com/article212.html, July, 2008.
[10] Mendoza, Jorge. Accident reconstruction and visibility studies are enhanced
with camera-matched 3D video. Plaintiff. March 2008.
http://www.plaintiffmagazine.com/Mar08%20articles/Mendoza_Accident%20reconstr
uction%20and%20visibility%20studies%20are%20enhanced%20with%20cameramatched%203D%20video_Plaintiff%20magazine.pdf, July, 2008.
[11] Mseymor7, fxguide. test driving boujou bullet
http://www.fxguide.com/article199.html, July, 2008.
[12] MIRALab. University of Geneva. Lifeplus
http://lifeplus.miralab.unige.ch/HTML/introduction_main.htm, July, 2008.
32
33