Syllabus
Policy Research: The Art of Creating Convincing Evidence
RPAD 399
: Stephen Holt, Ph.D.
: Mondays and Wednesdays
: 3:00 pm to 4:20 pm
: Massry Center for Business, B008
: sbholt@albany.edu
: Tuesdays and Thursdays from 3:00 pm to 5:00 pm or by appointment; in-person in Milne 324 or online in GatherTown.
: 518-442-3309
Course Description
How does child poverty influence health or academic growth? What can we do to reduce child poverty? How can we reduce carbon emissions without disrupting our economy? Do body-worn cameras reduce police involved shootings? Does tuition-free college reduce inequality? Or, perhaps most important of all, how do we go about answering these questions? This course will introduce students to the process of asking good research questions, figuring out what we already know or don’t know about the answer, and identifying the best approach for generating new answers and supporting evidence. In this course, students will develop an original research project using real data and centered on a research question they develop in the beginning of the course. Students will learn to identify and apply a method of analysis appropriate for effectively and persuasively answering their question. By the end of the semester, students will have produced a complete research project and data analysis in a policy area of their choice and will demonstrate basic familiarity with reading, writing, producing, and interpreting quantitative research to inform public policy.
Required Text
The Effect: Introduction to Research Design and Causality by Nick Huntington-Klein
- The book is available for free online here and can be found in hard copy here. Additional readings will be made available on the Class Zotero (see below) library.
Required Software
Stata17/BE - This course will be taking you through a research project from original conceptualization to the finished product, with an emphasis on causal analysis. Stata is required for the course to aid in conducting this analysis. The Basic Edition (BE) of Stata is available for a 6-month license and is sufficient for the course. The 6-month license costs $48 and can be found here.
Stata is a statistical software that will provide an opportunity for you to learn the basic logic and intuition using code to conduct data analysis. The coding language for Stata is very simple, clear, and straight forward, so it serves as a good entry point to getting comfortable using programming languages for analyzing data and learning more about the world around us. Once you adjust to the basics of analysis with an easy and intuitive language like Stata, transitioning to other software languages (e.g., R, Python) will be much less onerous.
Zotero - There are a variety of citation organizer software options out there for researchers to use and, having experimented with most of them, Zotero is by far the best. It is free; Open Source; designed and developed by researchers; and governed as a non-profit to protect the integrity and accessibility of the software. You can download the app here. I suggest also downloading the browser connector for whatever browser you use - it helps save a dramatic amount of time when conducting a literature review. I provide an introduction to Zotero and the features most applicable to this class in the embedded video below.
Additional Resources
The Workflow of Data Analysis Using Stata, by J. Scott Long Data Analysis Using Stata, by Ulrich Kohler and Frauke Kreuter
Assignments
Overview
The following assignments will form the basis of your grade in this course:
- Final Paper (60%)
- Literature Review (20%)
- Proposed Research Design and Analysis (20%)
- Final Write-Up (20%)
- Practice Problems (30%)
- Participation and Attendance (10%)
All assignments will be turned in via Blackboard
Final Papers
This course is an applied research based course. Students will be constructing a research paper in stages throughout the semester with intermediate deadlines for each stage of the project.
Late assignments. An assignment is considered late if it is submitted to Blackboard after class begins on the day the assignment is due. Late assignments will be deducted 10% of the available points if turned in late on the due date, 20% of the available points if turned in the day after the due date, and not accepted if later.
Practice Problems
Over the course of our semester together, students will be learning some advanced analytic techniques for causal inference. We will be going through the process of applying these methods together in class and students will have practice applications for each approach covered in class.
Homework assignments are to be submitted via Blackboard and are due before class the week they are due.
Participation
A big part of learning comes from class discussion and participation in class activities. You can do neither if you do not attend class. Attendance will be tracked and points will be subtracted for absences. In addition to attendance, sustained and regular contributions to class discussion will be factored in to the class participation grade for the semester. Note that while participation only affects 10% of your final grade, it can mean a difference of a full letter in the grade you receive. In education, as with most things in life, what you get out of an act is a function of what you put into that act. Consider this my small way of nudging you towards getting all you can out of this class.
Class Schedule
Overview of Weeks
Key:
| Symbol | Meaning |
|---|---|
|
Lecture in class |
 |
Readings due before class (listed in readings column) |
 |
Assignment due (clickable links in schedule) |
 |
Class activity |
 |
Reading is from required text |
 |
Reading is from an article |
| Date | Topic | Contents | Readings |
|---|---|---|---|
| 8/22 | Introduction | |
|
| 8/24 | Setting Up Computers | |
|
| 8/29 | Designing Research and Research Questions |
|
Chapters 1 & 2 |
| 8/31 | Designing Research and Research Questions |
|
|
| 9/12 | Reviewing the Literature | |
|
| 9/14 | Reviewing the Literature |
|
|
| 9/19 | Identification |
|
Chapter 5 |
| 9/21 | Identification |
|
|
| 9/26 | Creating Hypotheses and Identifying Data |
|
|
| 9/28 | Creating Hypotheses and Identifying Data |
|
|
| 10/3 | Cleaning Data (Variables) |
|
|
| 10/5 | Cleaning Data (Merges and Structure) |
|
|
| 10/12 | Describing Variables |
|
Chapter 3 |
| 10/17 | Describing Variables | |
|
| 10/19 | Describing Relationships |
|
Chapter 4 |
| 10/24 | Regression |
|
Chapter 13 |
| 10/26 | Regression | |
|
| 10/31 | Treatment Effects |
|
Chapter 10 |
| 11/2 | Treatment Effects | |
|
| 11/7 | Event Studies |
|
Chapter 17 |
| 11/9 | Event Studies | |
|
| 11/14 | Difference-in-Difference |
|
Chapter 18 |
| 11/16 | Difference-in-Difference | |
|
| 11/21 | Organizing Results |
|
|
| 11/28 | Writing Results from Analysis | |
|
| 11/30 | Organizing Presentation of Research | |
|
| 12/5 | Presentations of Final Papers | |
Detailed Assignments
5 page literature review
After working with the professor to select a broad topic with a set of potential research questions, students will be taught to conduct a literature review. The first draft of their literature review - a minimum of 5 single-spaced pages and a maximum of 10 single-spaced pages - should trace the theoretical debates about the topic from its origin and into recent years, summarize the empirical evidence (if any) from prior research, and outline the remaining open questions to be answered in this body of work. Students should have a minimum of 20 sources cited in their literature review. Acceptable sources are academic journals, policy reports from research firms (e.g., think tanks, evaluators), or official government reports (e.g., analysis from GAO, inspector generals, or other state/federal agencies).
Summary Statistics
The students will prepare a 1-2 page paper that describes, in detail, their data source, the variables they will be using, and why this data is relevant for testing the hypotheses generated by their research questions. The students will also prepare at least 1 tables summarizing their data on relevant variables.
Students will also be turning in a set of practice problems for practicing running and interpretting regressions from the previous week.
Research Model
Students will prepare a 2 page summary of the model they will use to test their hypotheses and answer their research questions and present their regression results. The assignment will also include 1 or 2 tables of regression output, clearly organized and interpreted correctly.
Presentations
Students will prepare a 15 minute presentation outlining their research paper, data, results, and conclusions for the class and deliver it in the final class. They will also turn in their research paper, which will include 5 pages of presenting and interpreting results and 3 pages discussing and concluding the paper (in addition to the previous sections developed in previous assignments).
Class Policies
- Public policy is a professional field; therefore, I emphasize professional skills in the classroom and assignments. Professional skills are punctuality, adhering to deadlines, and preparedness.
- After the first week, readings for each week should be completed by the Tuesday of that week (that is, BEFORE CLASS!). While much of the class reading comes from the text, I will cover other material in my lectures. You will be responsible for knowing this material too!
- A large body of well-designed research has demonstrated the detrimental effects of laptops on learning in a lecture/discussion based environment. Please be courteous and do not use your computers for anything other than class related work (taking notes and so on). Cellular phones are not to be used during class time!
- Letters of recommendation. If you are a hard working student and serious about a career in public service, I will be a dedicated advocate for you on the job market and will happily write letters of recommendation on your behalf. There is, however, one condition and one recommendation. The condition: I will not write a letter of recommendation for your while you are in my class. This is because to write a good faith, sincere, and thoughtful recommendation, I will need to be able to consider your work as a whole, and while the class is on-going, my assessment of you will be incomplete. After the semester is over, I am happy to help in any way I can, including writing letters. The recommendation: Make an appointment to visit my office hours at least once over the course of the semester to talk informally about your goals, career interests, and other professional ambitions so I can get a better sense of who you are as individuals. The better I know you, the more effective I can be at writing letters on your behalf and thinking of you when opportunities arise.
- Attend class! Again, class participation is 10% of your grade, and you can’t participate if you’re not in class. If you DO need to miss class (emergencies, sickness, etc.), please contact me as soon as possible and let me know. It will be your responsibility to notify me and to get any notes/materials from other students.
- Cell phones: we all have them, and they can be quite distracting. I ask that you please be courteous and silence your cell phone and leave it out of sight (in a pocket/purse/bag) during class.
- Feel free to eat and drink in class. I only ask that you do so quietly and in a manner that does not disrupt class.
- All assignments and non-textbook readings will be posted to the class Zotero Library. I will email any announcements or updates to the class and also post them in the Blackboard. Report any trouble accessing anything on the Blackboard as soon as you encounter the problem.
- I have a strict open door policy. If there is anything about the course, the assignments, the grading, the material, class, or anything related to public administration/policy or statistics broadly that you would like to discuss, do not hesitate to visit me during office hours or email me. I can respond via email, schedule a phone call, or schedule a separate meeting. I am here to help, so please do not hesitate to reach out to me. (But please be respectful of my time!)
- HAVE FUN! Public administration/policy is a broad topic that explores big, important questions that affect everyone. Discussing these topics should be as fun and interesting as it is challenging.
- The table below lays out the grading scale that will be used in assigning final course grades.
- Students with special physical and/or learning needs will be accommodated. Please notify the Disabilities Office and me as soon as possible so that reasonable accommodations can be made.
NOTE: Throughout the semester I may add or subtract readings as needed to adjust the course according to your progress, engagement, and interests.
Table. Grade Scale Used for Calculating Class Grades
| Percent | Grade | Points |
|---|---|---|
| 93-100 | A | 4.0 |
| 90-92 | A- | 3.7 |
| 87-89 | B+ | 3.3 |
| 83-86 | B | 3.0 |
| 80-82 | B- | 2.7 |
| 77-79 | C+ | 2.3 |
| 73-76 | C | 2.0 |
| 70-72 | C- | 1.7 |
| 67-69 | D+ | 1.3 |
| 63-66 | D | 1.0 |
| 60-62 | D- | 0.7 |
| < 60 | F | 0.0 |
Note: the percent refers to the percent of available weighted points earned. Each assignment is weighted by the proportion of the final grade made up by the assignment itself, as described above.
Academic integrity
Academic honesty is something your professor takes very seriously. Cheating in any form will not be tolerated. Students are required to be familiar with the university’s academic honesty policies; ignorance is not an excuse for dishonest behavior. In all cases of cheating, a Violation of Academic Integrity Report will be submitted to the Dean of Graduate Studies to be placed in your university file, with copies provided to you, the department head, and the Dean of Rockefeller College. Additional penalties may include some combination of the following: revision and re-submission of the assignment, reduction of the grade or failure of the assignment, reduction of the course grade or failure of the course, filing of a case with the Office of Conflict Resolution and Civic Responsibility, suspension, or expulsion. For a more detailed description of the university’s academic honesty policies, visit the site.
Students with Disabilities
We are committed to providing an accessible learning environment for all students. This includes students with physical, sensory, medical, cognitive, learning, mental health, and other disabilities. If you have, or think you may have a disability, please contact Disability Access and Inclusion Student Services (DAISS) by emailing daiss@albany.edu or calling 518 -442-5501. DAISS staff will explain the documentation and registration process, and set you up with an appointment. Once you have completed registration, you will be provided with a letter to inform your instructors that you are a student with a disability registered with DAISS, and which lists the recommended reasonable accommodations for your courses.
Counseling Center
The Counseling Center (518-442-5800; 400 Patroon Creek Blvd, Suite 104) offers counseling and consultations regarding personal concerns, self-help information, and connections to off-campus resources. More information can be found at their site.
Library Assistance
SUNY-Albany offers a great collection available in several different media. Access to research help and library tutorials can be found online at the library’s site.
For information about SUNY-Albany’s Dewey Graduate Library, which is located on the Downtown Campus, visit their site.
Writing Center
The university offers a number of services for students who need assistance with writing and research projects. Support is available in the Writing Center (518-442-4061; 140 HU) and at the University Library. Information about the Writing Center can be found at their site.
Title IX and Sexual Violence Prevention
Title IX of the Education Amendments of 1972 is a federal civil rights law that prohibits discrimination on the basis of sex in federally funded education programs and activities. The SUNY-wide Sexual Violence Prevention and Response Policies prohibit offenses defined as sexual harassment, sexual assault, intimate partner violence (dating or domestic violence), sexual exploitation, and stalking. The SUNY-wide Sexual Violence Prevention and Response Policies apply to the entire University at Albany community, including students, faculty, and staff of all gender identities. The University at Albany provides a variety of resources for support and advocacy to assist individuals who have experienced sexual offenses.
Confidential support and guidance can be found through the Counseling Center (518-442-5800, or online), the University Health Center (518-442-5454, or online), and the Interfaith Center (518-489-8573, or online). Individuals at these locations will not report crimes to law enforcement or university officials without permission, except for in extreme circumstances, such as a health and/or safety emergency. Additionally, the Advocates at the University at Albany’s Advocacy Center for Sexual Violence are available to assist students without sharing information that could identify them (518-442-CARE, or online).
Sexual offenses can be reported non-confidentially to the Title IX Coordinator within The Office for Equity and Compliance (518-442-3800, or online, Building 25, Room 117) and/or the University Police Department (518-442-3131, or online).
PLEASE NOTE: Faculty members are considered “responsible employees” at the University at Albany, meaning that they are required to report all known relevant details about a complaint of sexual violence to the University’s Title IX Coordinator, including names of anyone involved or present, date, time, and location.
In case of an emergency, please call 911.
Incomplete Grade Policy
A tentative grade given only when the student has nearly completed the course but due to circumstances beyond the student’s control the work is not completed on schedule. The date for the completion of the work is specified by the instructor. The date stipulated will not be later than one month before the end of the session following that in which the Incomplete is received. The grade I is automatically changed to E or U unless work is completed as agreed between the student and the instructor.
Absence due to religious observance
Students are excused, without penalty, to be absent because of religious beliefs, and will be provided equivalent opportunities for make-up examinations, study, or work requirements missed because of such absences. Students should notify the instructor of record in a timely manner, and the instructor will work directly with students to accommodate religious observances. Online courses will not schedule any assignment deadlines on religious holidays.