Final project

The final project is on a topic selected by the students. See the topics page for potential topics and resources.

Students are strongly encouraged to work in pairs. On-campus and distance students can partner if desired. If you are looking for a partner, you may post in the project discussion forum on Blackboard.

Project reports are due December 3 at 11:59pm. The project report is worth 80% of the overall project grade, while the project presentation is worth 20% of the overall project grade.

Approaches to Final Project

There are three broad approaches available, which may appeal to students differently based on the existing skill set.

  1. Empirical projects: Identify, process and analyze an existing dataset on a security-related topic. It may be necessary to link two or more disparate data sources in order to conduct interesting analysis (e.g., linking data on US hospitals with records of data breaches so that you can report on the incidence of data breaches at hospitals and see if characteristics of the hospital affect the likelihood of breaches). Write up a report describing the dataset and providing basic analysis.
  2. Model projects: Create a model of security investment or a security game that extends prior work in the published literature. The extension can be a refinement of the model or an application of empirical data to the model. It is also possible to create a new model applied to a topic of your choice, but you must still review the existing literature first and see if there is any relevant published literature on the topic. If you find existing work, if you choose not to extend the model, then you must explain how your approach is different.
  3. Literature review projects: Write a 2500 word essay synthesizing at least 4-6 related security economics papers on a common topic.

Empirical Project Requirements

The purpose of the empirical project is to identify, process and analyze an existing dataset on a security-related topic. See the topics page for potential data sources and example project topics that you may choose from if you are having trouble identifying a topic on your own. Empirical projects also include attempts to quantify the probability of a security risk and its associated costs based on public sources.

Begin by stating hypotheses that you would like to examine with the data. Then write code in R (or whatever language you desire) to explore and then analyze the data. Apply any relevant statistical tests to confirm or refute your stated hypotheses.

The write-up should include the following sections: (results can be broken up into multiple sections if you prefer):

  1. Abstract (100-200 words)
  2. Introduction (200-400 words)
  3. Methodology (200-400 words)
  4. Results (600-1000 words)
  5. Conclusions, Limitations, and Future Work (200-400 words)

If there is any related work on the topic, please include references to these as well. References should follow the ACM format (see bottom of this page for examples.) In particular, for empirical projects that attempt to estimate costs, be sure to include references to how the costs are calculated.

I have included very rough indications of recommended word counts. Please note that what I really care about is that you write just enough to clearly communicate your project's motivation and results. These word counts are only an indication, so please do not add fluff to meet the minimum or cut out essential parts if you go over.

Abstract

Usually written after the rest of the report has been completed, the abstract should provide a pithy summary of the report's main findings. It must introduce the topic and describe the key results. As a general rule, abstracts should be no longer than 200 words (and closer to 100 is often acceptable).

Introduction

Here you should introduce and motivate your project topic. You should explain what your key questions to investigate are, why you think this is interesting, and how you expect the data you are analyzing to help answer the questions. Explicitly relate the question under investigation to any economics concepts discussed in the course. You should also briefly summarize the paper's key contributions and provide an outline of the subsequent sections of the paper.

Methodology

Here you should describe the data you used. In particular, explain for how long the data covers, how it was collected, whether it can be considered representative, and any limitations to its representativeness. Note that you do not need to go into too much detail here about the logistics of data collection. For example, you do not need to list the columns of all the tables in your database.

Results

Here you should describe the key results from your analysis. You can first include summary statistics to introduce the data set, followed by an explanation of the questions you are trying to answer and a description of the graphs and relevant statistics to answer the questions appropriately.

Conclusions, Limitations and Future Work

You should begin by restating your key results. Next you should describe the limitations of the current study -- ways in which the data set fell short, difficulties in generalizing results, etc. The discussion of limitations can transition into a discussion of opportunities for future work. What would you do if you had more time? Is there additional data you would collect, given what you now know about your data source? You can think of future work as a curated ``brain dump'' of what you would like to do if you were able to continue working on this topic.

Appendix

Include any code as an appendix to the report.

Model projects

The write-up should include the following sections: (results can be broken up into multiple sections if you prefer):

  1. Abstract (100-200 words)
  2. Introduction (200-400 words)
  3. Prior Work (200-400 words)
  4. Model Description and Results (800-1200 words)
  5. Conclusions, Limitations, and Future Work (200-400 words)

If there is any related work on the topic, please include references to these as well. References should follow the ACM format (see bottom of this page for examples.)

I have included very rough indications of recommended word counts. Please note that what I really care about is that you write just enough to clearly communicate your project's motivation and results. These word counts are only an indication, so please do not add fluff to meet the minimum or cut out essential parts if you go over.

Abstract

Usually written after the rest of the report has been completed, the abstract should provide a pithy summary of the report's main findings. It must introduce the topic and describe the key results. As a general rule, abstracts should be no longer than 200 words (and closer to 100 is often acceptable).

Introduction

Here you should introduce and motivate your project topic. You should explain what your key questions to investigate are, why you think this is interesting, and how your model can help to answer these questions. Explicitly relate the question under investigation to any economics concepts discussed in the course. You should also briefly summarize the paper's key contributions and provide an outline of the subsequent sections of the paper.

Prior Work

Briefly review relevant existing literature. In particular, explain in your own words how the model you are extending works

Model Description and Results

Here you should describe how the model works. Include any relevant equations. Be sure to make graphs that demonstrate how varying model parameters changes outcomes.

Conclusions, Limitations and Future Work

You should begin by restating your key results. Next you should describe the limitations of the current study -- ways in which the model fell short, difficulties in generalizing results, etc. The discussion of limitations can transition into a discussion of opportunities for future work. What would you do if you had more time? Is there additional data you would collect, given what you now know about your data source? You can think of future work as a curated "brain dump'' of what you would like to do if you were able to continue working on this topic.

Appendix

Include any code used as an appendix to the report.

Literature review projects

Conducting a literature review is an essential skill for any aspiring researcher. Literature reviews summarize existing research results on a topic in a way that synthesizes different perspectives and characterizes what is known about a topic as well as what is not known. The latter is often the most important result of a literature review, as it can reveal opportunities for future research.

See the slides from November 6 and the topics page for resources on how to conduct a literature review.

First, you should begin by collecting a large sample of potentially relevant papers by examining some of these curated paper lists and literature reviews:

You can complement your exploration by searching on Google Scholar and DBLP.

Include a few relevant CS papers if they describe the problem under study, particularly if you're focused on a particular attack. For economics papers, you should include descriptive, model, and empirical papers as appropriate.

Once you have collated a list of relevant papers, try to organize your thoughts based on several "big ideas" covered in the papers using a synthesis matrix. Include any paper that you read in the synthesis matrix, even if it is not included in the review itself.

The write-up should include the following sections: (results can be broken up into multiple sections if you prefer):

  1. Abstract (100-200 words)
  2. Introduction (200-400 words)
  3. Big Idea 1
  4. Big Idea n (Total of all big idea discussion should be 1500-2000 words)
  5. Opportunities for Future Research (300-600 words)
  6. Conclusions (200-400 words)

If there is any related work on the topic, please include references to these as well. References should follow the ACM format (see bottom of this page for examples.) In particular, for empirical projects that attempt to estimate costs, be sure to include references to how the costs are calculated.

I have included very rough indications of recommended word counts. Please note that what I really care about is that you write just enough to clearly communicate your project's motivation and results. These word counts are only an indication, so please do not add fluff to meet the minimum or cut out essential parts if you go over.

Abstract

Usually written after the rest of the report has been completed, the abstract should provide a pithy summary of the report's main findings. It must introduce the topic and describe the key findings of the review. As a general rule, abstracts should be no longer than 200 words (and closer to 100 is often acceptable).

Introduction

Here you should introduce and motivate your project topic. You should explain what your key questions to investigate are and why you think this is interesting. Explicitly relate the question under investigation to any economics concepts discussed in the course. You should also briefly summarize the paper's key contributions and provide an outline of the subsequent sections of the paper.

Big Idea 1--n

Present summaries of existing research papers into this area. Rather than organize sections simply by paper summaries, group them by themes that may be addressed by multiple papers. Be sure to include a discussion of limitations to existing approaches and any gaps in the literature that could be opportunities for future research. Your discussion must include explicit comparison between the approaches taken in different papers, explaining how they differ. When paper's reach different conclusions, point this out and take a side if one appears more convincing.

Opportunities for Future Research

Having described the research, you should briefly explain where there are gaps in current knowledge about the topic that you think could be suitable to future research.

Conclusions

You should begin by restating your key results, including the big ideas. Explain what we know and assess the prospects for improving on what we still don't.

Appendix

Include a synthesis matrix for the papers reviewed, plus a list of references to the papers included in the synthesis matrix. If you write individual paper summaries according to the template, you can include these in the appendix. Note that it is OK to copy and paste from parts of the review template provided that what you write there is original.

Project Proposal

By November 9, you must turn into me a brief project proposal, either by hard copy or via email to me with subject line "CSE 5/7390 Project Proposal". Include the topic of choice, the category of project (empirical, model, or literature review), project partner (if any), and a one-paragraph explanation of the proposed topic. For empirical projects, you should also include the data set(s) you will be using, and a brief statement of what you hope to learn from the data (incidence of attack, cost of attack, variation by target, etc.). For model projects, include a description of any existing model you will be extending, and a list of paper titles and links to PDFs for any relevant papers. For literature review projects, include a 1-2 sentence statement of the topic area, and include a list of at least three papers you are considering using.

You may optionally sign up for a time to meet with me to discuss project topics at this link: https://docs.google.com/spreadsheet/ccc?key=0AvJIU8FOWAZzdHF0UDkydjV0NGcwek1PTFF0OHdXbnc

Enter your name(s) and a phone number where I can reach you if you are a distance student. On-campus students should come to my office for the meeting. If you are planning to work in a team then both members should come.

The project proposal is designed so that I can provide feedback to you early in the process, and to ensure that you have selected a suitable topic. It is a completion grade. If you have a proposal ready before November 9, you are encouraged to send it sooner. I will review proposals as they are received.

Project Presentations

Each group will give 10-minute presentations during the last week of class summarizing your key findings. The total time allotted to each talk is 10 minutes. Plan on speaking for around 8 minutes, leaving the balance of time for questions. The presentation schedule can be found on the course calendar (Nov 27 and Nov 29). The project presentation is worth 20% of the overall project grade.

The goal of the presentation is to explain to others the topic you investigated, including the questions you hoped to answer, your results, and any lessons you learned from carrying out the study. An evaluation rubric is available here.

Distance students are not required to present, but they are welcome to do so if arrangements to attend in person can be made. If a distance student cannot present, then you must turn in presentation slides that include speaking notes for what you would plan to say if you were to give a talk. These slides should be turned in via Blackboard by December 3.

For all students, the project presentation slides will be posted on the course website. Please submit via Blackboard.