代写COMP/ENGN6528 Computer Vision

COMP/ENGN6528 Computer Vision - 2023 S1
Computer Lab 3 (CLab-3)

Objectives:
This is CLab-3 for COMP/ENGN6528. The goal of this lab is to help you familiarize
yourself with, and practice:
Basic multi-view camera geometry, camera calibration. They are the basic
building blocks for 3D visual reconstruction systems.
The DLT method for two-view homography estimation.
You are free to choose Python or Matlab to complete this assignment.
Special Notes:
1. Each computer lab has three weeks. Tutors/Lab instructors will provide basic
supervision in the following three weeks when students can discuss the
potential questions with tutors.
2. Your Lab assignment will be marked based on the overall quality of your Lab
Report in PDF format. The report is to be uploaded to Wattle site before the
due time, which is usually on the Sunday evening of Week-3 session of your
lab (21st May, 2023). Please note that you must report on what you have done.
Only submitting code will get a low mark.
3. It is normal if you cannot finish all the tasks within the two 2-hour sessions —
these tasks are designed so that you will have to spend more time in order to
finish all the tasks including finishing your Lab report. This suggests that,
before attending the third lab session (in Week-3 session of each CLab), you
must make sure that you have almost complete 80% of the assignment.
Academic Integrity
You are expected to comply with the University Policy on Academic Integrity and
Plagiarism. You are allowed to talk with / work with other students on lab and project
assignments. You can share ideas but not code, you should submit your own work.
Your course instructors reserve the right to determine an appropriate penalty based on
the violation of academic dishonesty that occurs. Violations of the university policy
can result in severe penalties.


Task-1: 3D-2D Camera Calibration (18 marks)

Camera calibration involves finding the geometric relationship between 3D world
coordinates and their 2D projected positions in the image.
Four images, stereo2012a.jpg, stereo2012b.jpg, stereo2012c.jpg, and
stereo2012d.jpg, are given for this CLab-3. These images are different views of a
calibration target and some objects. For example, the diagram below is
stereo2012a.jpg with some text superimposed onto it:

(Do not directly use the above image for your camera calibration work
as it has been scaled for illustration. Use the original (unlabelled) image files provided.)

On the calibration target there are 3 mutually orthogonal faces. The points marked on
each face form a regular grid. They are all 7cm apart.
Write a Matlab function with the following specification
Function to perform camera calibration
Function C = calibrate (im, XYZ, uv)
Input: im: is the image of the calibration target.
XYZ: is a Nx3 array of XYZ coordinates of the calibration target points.
uv: is a N x 2 array of the image coordinates of the calibration target
points.
Outputs: C: is the 3 x 4 camera calibration matrix.

Requirements:
1) The variable N should be an integer greater than or equal to 6.
2) This function should also plot the uv coordinates onto the image of the calibration
target. It also projects the XYZ coordinates back into image coordinates using the
calibration matrix and plots these points too as a visual check on the accuracy of the
calibration process.
3) Lines from the origin to the vanishing points (namely, world coordinate system) in the
X, Y and Z directions are overlaid on the image.
4) The mean squared error between the positions of the uv coordinates and the
projected XYZ coordinates should also be reported.


Generally, we ask you to implement a function:
MATLAB user:
function C = calibrate(im, XYZ, uv)
Python user:
def calibrate(im, XYZ, uv)
return C
From the 4 supplied images (stereo2012a.jpg, stereo2012b.jpg,
stereo2012c.jpg, and stereo2012d.jpg), choose any image to work on and make
it clear in the report. Use the suggested right-hand coordinate system shown in the
diagram above and choose a sufficient number of calibration points on the calibration
target.
Store the XYZ coordinates in a file so that you can load the data into Matlab and
Python (You can choose your preferred datatype, for instances, mat in MATLAB and
numpy array in Python) and use them again and again. Note that each image can be
calibrated independently, so you can choose different calibration points to calibrate
each image. Neither do the numbers of calibration points need to be the same for
your chosen images.
The uv coordinates can be obtained using the MATLAB function ginput or calling
the function matplotlib.pyplot.ginput for Python users.
If one invokes ginput as follows:
>> uv = ginput(12) % e.g., to digitise 12 points in the image
and digitises a series of points by clicking with the left mouse button, then uv will be
a matrix containing the column and row coordinates of the points that you digitised.

As for Python users, you can get matplotlib.pyplot.ginput to get uv coordinates.
The operation is similar with that in MATLAB.
https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.ginput.html
After the above operation, the variable uv should be a 12 × 2 matrix, each row of
which should contain the coordinates of one image point.
Note: You need to ensure that, for each image, the numbers of 3D and 2D calibration
points are the same. For example, if your uv variable is a 12 × 2 matrix, then your
XYZ variable should be a 12 × 3 matrix. Also, the points should appear in the same
order in these two matrices.
Use the DLT algorithm to solve the unknown camera calibration matrix of size 3x4.
(Refer to lecture Notes and textbook:
Multiple View Geometry in Computer Vision Section 7 (page 178),

Hints
(1) In writing your code you may need to 'reshape' a 2D vector into a 1D vector. You
can use the function reshape or np.reshape (in MATLAB and Python respectively)
to reshape a matrix to arbitrary dimensions.
(2) You can save your calibration matrices using np.save and np.load for python
users.
np.save (file, arr, allow_pickle=True, fix_imports=True)
np.load(file, mmap_mode=None, allow_pickle=False, fix_imports=True, encoding='ASCII',
*, max_header_size=10000)

Python users would refer to the following link for the detailed explanations about the
function https://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html
https://docs.scipy.org/doc/numpy/reference/generated/numpy.load.html
For matlab users, please use the save and load functions for saving and loading the
calibration matrices. Here comes details about the Matlab's save command. For
example, the command
>> save mydata im1 im2 calibPts uv1 uv2 C1 C2% (please do not
use commas between variable names)
will save your variables im1 im2, calibPts, uv1, uv2, C1, and C2 in a file
called mydata.mat. At a later date you can load this data into the memory using the
command:
>> load mydata
The variables im1 im2, calibPts, uv1, uv2, C1, and C2 will then be restored in
your workspace.

For Task-1, you should include the following in your Lab-Report PDF file:
1. List calibrate function in your PDF file. [3 marks]
2. List the image you have chosen for your experiment, and display the image in
your PDF file. [0.5 mark]
3. List the 3x4 camera calibration matrix P that you have calculated for the
selected image. Please visualise the projection of the XYZ coordinates back
onto image using the calibration matrix P and report the reprojection error
(The mean squared error between the positions of the uv coordinates and the
projected XYZ coordinates using the estimated projection matrix) [2 marks]
4. Decompose the P matrix into K, R, t, such that P = K[R|t], by using the
following provided code (vgg_KR_from_P.m or vgg_KR_from_P.py). List the
results, namely the K, R, t matrices, in your PDF file. [1.5 marks]
5. Please answer the following questions:
- what is the focal length (in the unit of pixel) of the camera? [1 mark]
- what is the pitch angle of the camera with respect to the X-Z plane in the
world coordinate system? (Assuming the X-Z plane is the ground plane, then the
pitch angle is the angle between the camera's optical axis and the ground-plane.)
Please provide the calculation process. [2 marks]
- What is the camera centre coordinate in the XYZ coordinate system
(world coordinate system)? Please provide the calculation process. [1 mark]
6. Please resize your selected image using builtin function from matlab or python
to (H/3, W/3) where H, and W denote the original size of your selected image. Using
the interface function, (matplotlib.pyplot.ginput in Python and ginput in
Matlab, and) to find the uv coordinates in the resized image. [1 mark]
a. Please display your resized image in the report, list your calculated 3x4
camera calibration matrix P’ and the decomposed K’, R’, t’ in your PDF file.
[2 marks]
b. Please analyse the differences between 1) K and K’, 2) R and R’, 3) t and t’.
Please provide the reasoning when changes happened or there are no changes.
[2 marks]
c. Let us check the focal length (f and f’) (in pixel unit) and the principal
points extracted from K and K’, respectively. Please discuss their relationship
between (f and f’) and its connection to the image size of the original image
and the one after resizing. [2 marks]

Task-2: Two-View DLT based homography estimation. (10 marks)
A transformation from the projective space P3 to itself is called homography. A
homography is represented by a 3x3 matrix with 8 degrees of freedom (scale, as usual,
does not matter)

The goal of this task is to use the DLT algorithm to estimate a 3x3 homography
matrix.

Pick any 6 corresponding coplanar points in the images left.jpg and
right.jpg and get their image coordinates.

In doing this step you may find it useful to check the Matlab function ginput or
Python function matplotlib.pyplot.ginput

Calculate the 3x3 homography matrix between the two images, from the above 6 pairs
of corresponding points, using DLT algorithm. You are required to implement your
function in the following syntax.

Function to calculate homography matrix
H = homography (u2Trans, v2Trans, uBase, vBase)

Usage: Computes the homography H applying the Direct Linear Transformation
Inputs: u2Trans,
v2Trans:
are vectors with coordinates u and v of the transformed image
point (p')
uBase,
vBase:
are vectors with coordinates u and v of the original base
image point p
Output: H: is a 3x3 Homography matrix

In doing this lab task, you should include the following in your lab report:
1. List your source code for homography estimation function and display the two
images and the location of six pairs of selected points (namely, plotted those
points on images). Explain the steps about what you have done to estimate the
homogrqaphy and what is shown in the images. [5 marks]
2. List the 3x3 camera homography matrix H that you have calculated. [2 mark]
3. Warp the left image according to the calculated homography. Study the factors
that affect the rectified results, e.g., the distance between the corresponding
points, e.g the selected points and the warped ones. [3 mark] (Note: you can
use builtin image-warping functions in matlab and python.)
====================== End of CLab-3 ====================