代做Neural Networks Assignment Spring 2023代做留学生Python程序

Neural Networks Assignment

Spring 2023

Objective

Submit a 3 page report (please use the provided template) and your code. Please submit your assignment as 2 files: one is a pdf of your written report (NB: you can generate PDFs from most document writing software), the other is your code. Please name these files as CANDIDATE_NUMBER.pdf (your report) and CANDIDATE_NUMBER.foo (your code), e.g. 123456.pdf and 123456.ipynb.

Due date

16.00 (4pm), 11th May, 2023.

How to submit

Submit through Canvas Online (not Turnitin).

Grading

This assignment determines 100% of your grade for Neural Networks. It is separated into 2 components: the report (70%) and your code (30%).

Task Summary

The main task is to develop a deep neural network to perform. multi-class classification. The dataset for this assignment is the Fashion MNIST dataset (link), which is also available from the torchvision library in PyTorch (link). The dataset is made up of 28x28 grayscale images, which are split into a training set of 60,000 images and a test set of 10,000 images. The images have labels (one label per image) from 10 classes: 1) t-shirt/top; 2) trouser; 3) pullover; 4) dress; 5) coat; 6) sandal; 7) shirt; 8) sneaker; 9) bag; 10) ankle boot. The network should be trained to predict the labels using the training data and its generalisation performance is evaluated using the test data.

Task details

In this assignment, you need to implement a deep neural network with an input layer, three hidden layers (convolutional, or fully-connected, or a combination of both) with ReLU non- linear activation, and an output (classification) layer. Feel free to use PyTorch (recommended), or any other deep learning frameworks (JAX, TensorFlow) with automatic differentiation, or python. The neural network should be trained to classify the images from the Fashion MNIST dataset. You can use the built-in modules to load the dataset (e.g. in PyTorch, you can use torchvision.datasets.FashionMNIST) and to build the layers in your model (e.g. in PyTorch you can use Linear, Dropout, ReLU, Conv2d, BatchNorm2d among others). The training process should explore the following hyperparameter settings:

● Batch size: Number of examples per training iteration.

● Depth: compare deeper and shallower models. For example, compare performance when using two/three/four hidden layers.

● Width: Try using different numbers of hidden nodes and compare the performances. In a fully connected layer, this corresponds to the hidden layer size. In a convolutional layer, this corresponds to the number of filters used for convolution.

● Convolutional filter size: try to vary filter size (also called the kernel size of the filters) in convolutional layers and compare the performance. Try to analyse how the filter size affects the receptive field of the convolutional layers.

● Dropout: Dropout is an effective strategy to defend against overfitting in the fully connected layers. Try comparing the performance using different dropout rates.

● Batchnorm: batch normalisation is typically used in convolutional neural networks to prevent overfitting and speed up convergence. Compare performance with and without batch normalisation. Explore jointly with the batch size hyper-parameter.

● Max pool: max pooling is typically used to reduce the spatial dimensions of the layers (downsampling). Compare performance when using other types of pooling (average pooling), or no pooling.

● Non-linearity: Compare the performance when training with ReLU (this is the main task), leaky ReLU, and tanh non-linear activation functions.

● Optimiser: Try using different optimisers such as SGD, Adam, RMSProp.

● Weights initialisation: Try different weight initialisation strategies, such as He, Xavier, random noise.

● Regularisation (weight decay): L2 regularisation can be specified by setting the weight decay parameter in optimiser. Try using different regularisation factors and check what effect this has on the performance.

● Data augmentation: try different random transformations of an image, e.g. crops, horizontal/vertical flips, translations etc.

● Learning rate and learning rate scheduler: Learning rate is the key hyperparameter in model training. You can gradually decrease the learning rate to further improve your model. Try using different learning rates and learning rate schedules and compare the performance.

You should explore the learning rate and at least two other types of hyperparameters from those listed above; choose at least 3 different values for each hyperparameter (where applicable). For simplicity, you could analyse one hyperparameter at a time (i.e. fixing all others to some reasonable value), rather than performing a grid search.

You should describe your model selection procedure: 1) did you do a single train-val split or did you do cross validation, and how did you split the data? and 2) demonstrate your analysis of model selection based on learning curves: loss curves w.r.t. training epochs/iterations, accuracy curves on training and validation data. If you use TensorBoard to monitor your training, you can directly attach the screenshots of the training curves in your report.

To evaluate the performance of the model after hyper-parameter selection, you also need to have an evaluation part (for example a function), which uses (or loads) the trained model and evaluates its performance on the test set. In your report, please clearly state what hyperparameters you explored, how you did model selection, and what accuracy the model achieved on the train, validation and test sets.

Finally, use a pre-trained version of ResNet-18 (a smaller, 18-layer version of ResNet-152, which surpassed human performance in the ImageNet Challenge) on the Fashion MNIST dataset. NB: this will require some preprocessing of the input images to make them compatible with the ResNet-18 architecture. Compute the neuron activities in the final hidden layer (a 1000-dimensional vector) of ResNet-18 for a subset of 100 images from each class of the test set. Compute a 10x10 correlation matrix, where the i-j th entry is the mean correlation between representations of images from class i and class j. Repeat this analysis for the final hidden layer in your own neural network (the version that gives the best performance on the test data). Use these correlation matrices to compare representations from your model (which was trained on the Fashion MNIST dataset) and from ResNet-18 (which was trained on a very different dataset). What conclusions can be drawn from this comparison?

Let your interest in designing, and deploying neural networks be a driving force when exploring hyper-parameter search in this assignment. We value your creativity and critical thinking when analysing and presenting your findings.

Report details

You are expected to write a 3-page report detailing your solution and analyses for the problem. Please use the provided latex or word template. You may include supplementary plots in the appendix. References and appendix may go on an additional fourth page. Your report should include the following components:

1. Abstract (5% of the mark) - Present a short summary of your project (no more than 150 words), including brief statements about your goals, your results, and your conclusions.

2. Methodology (35% of the mark) - Describe the architecture of the neural network you will use, and other details/hyperparameters, e.g. layer sizes, pre-processing of input data (if any), nonlinear activation function details, connectivity (e.g. fully connected layers, convolutional layers and filter sizes, pooling layers etc. A schematic of the network will help. You can also describe the training method here, e.g. the optimiser and its hyperparameters, any splits in the data for training and validation. Describe solutions to overfitting and improving generalisation, if any. Describe the values used for the hyperparameters tested (e.g. in a table). The methodology should include enough details for someone else to replicate your work

3. Results (20% of the mark) - Present the results of your hyperparameter tests. Use tables and/or figures to show your results in a clear format. Remember to label axes of figures, and to provide a figure caption that explains everything the reader needs to know to understand the figure (note, this does not mean interpreting the results in the figure caption: you only need to provide a description of what the figure is showing). Plot graphs showing the changing performance of your model throughout training, e.g. the loss and the accuracy for the training (and validation) data. Report the model’s performance on the test data. Provide analyses of regularisation strategies used (if any) to mitigate overfitting and to improve generalisation.

4. Discussion (10% of the mark) - Note: this section can be incorporated with the Results section if you prefer. Draw conclusions from your results, and discuss what reasons (if any) might explain your results. Discuss additional/alternative methods you may have used that could improve on your results, with reasons (note: your reasons should be scientific, and not “if I had more time, I would have…..”). Don’t be afraid to point out flaws (if any) in the choices you have made for your model, and discuss alternatives that may produce better/more meaningful results, in hindsight.

5. Appendix - note: this does not contribute to your mark, but can be useful to help explain and to clarify those parts of your report that are assessed. This can be a useful space for additional figures, for example.

6. Code (30% of the mark) - You must also submit your implementation code. All code must be provided in a single file. Please make sure we will be able to run your code as is, i.e. without any further debugging from us, and provide any necessary instructions we may need to run it, e.g. command line code and any required input arguments (please provide these instructions in comments at the top of your code file). High quality code with a good structure and plenty of comprehensible comments will be marked favourably. A Colab/Jupyter notebook can be a very useful tool to achieve these criteria, e.g. using markdown sections for extended comments/explanations/referencing your report. Where you have generated code to produce figures, please state in the comments which figure(s) are being produced, e.g.:

# Generate Figure 2, panel C.

from matplotlib import pyplot as pl

pl.plot(x,y)

Important points:

● You are not expected to produce the perfect model, and there is rarely a unique solution when it comes to deep neural networks. However, you are expected to approach your investigation scientifically, and apply reason and constructive criticism to your own work.

● Your work will be assessed by humans who get tired/hungry/frustrated/intrigued/curious. While we make every effort to assess your work fairly, those humans are not perfect. Help them to assess your work, and to realise how talented you really are, by making your presentation clear and concise. Check references to figures, spelling and grammar. Every little helps.

● Use of chatGPT: we have given this project description (and paraphrased versions) to chatGPT and generated numerous responses from it. We therefore have a good idea of the kind of content it will produce for this assignment. I encourage you to refrain from using it as much as possible: time and effort are the seeds of learning. If, after having made several attempts on your own, you need to ask someone for help, you can ask me or the TAs (chatGPT can be a useful tool in those situations, as well).

Marking criteria

● 70%-100% - excellent:

○ Shows very good understanding, supported by evidence that the student has extrapolated from what was taught, through extra study or creative thought (e.g. incorporating additional regularisation or data augmentation strategies, in-depth analyses of more hyper-parameters than were requested). Work at the top end of this range is of exceptional quality. Report will be excellently structured, with proper references and proper discussion of existing relevant work. The report will be neatly presented, interesting and clear, with a disinterested critique of what is good and bad about the approach taken, and thoughts about where to go next with such work.

○ Important: the report should indicate extrapolated content, e.g. by having a specific section (ideally).

● 60-69% - good:

○ The work will be very competent in all respects. Work will evidence substantially correct and complete knowledge, though will not go beyond what was taught. Report should be well-structured and presented with proper referencing and some discussion/critical evaluation. Presentation will generally be of a high standard, with clear written style. and some discussion of related work.

● 50-59% - satisfactory:

○ The work will be competent in most respects. There may be minor gaps in knowledge, but the work will show a reasonable understanding of fundamental concepts. The report will be generally well-structured and presented with references, though may lack depth, appropriate critical discussion, or discussion of further developments, etc.

● 40-49% - borderline:

○ The work will have some significant gaps in knowledge but will show some understanding of fundamental concepts. The report should cover the fundamentals but may not cover some aspects of the work in sufficient detail. The work may not be organised in the most logical way and its presentation may not always be appropriate. There will be little or no critical evaluation or discussion. References may be missing, etc.

● 30-39% - fail:

○ The work will show inadequate knowledge of the subject. The work is seriously flawed, displaying a major lack of understanding, irrelevance, or incoherence. The report is badly organised and incomplete, possibly containing irrelevant material. The report may have missing sections, no discussion, etc.

● Below 30% - unacceptable (or not submitted):

○ The work is either not submitted or, if submitted, so seriously flawed that it does not constitute a bona-fide report/script.