Harvard’s CS50: Analysis of a Learning Culture

Harvard University is home to the popular Introduction to Computer Science (CS50), a course available from the Computer Science Department. This course covers the introduction to intellectual enterprises of computer science and the art of programming.  In 2007 new instructor, Dr. David Malan reengineered the course with a new pedagogical spin.  Prior to Dr. Malan, CS50 had an enrollment of 132 students. In the span of four years, class enrollment has soared to over 600 students. CS50 has been historically delivered in a traditional face-to-face lecture hall on Harvard’s campus in Cambridge, MA. Today, it is offered through a variety of delivery methods.   Beyond the face-to-face format, CS50 is delivered online via https://www.cs50.tv and as CS50x through edX, a massively open online course platform created by Massachusetts Institute of Technology and Harvard University to offer university level courses in a wide range of disciplines (Lewin, 2012).

When Dr. Malan, a Harvard graduate himself, was asked to reengineer the CS50 course, it held a strong stigma as a class reserved for individuals concentrating in Computer Science (Oblinger, 2012). In addition to being held in the historic Sanders Theatre on Harvard’s campus, the course materials, lectures and sections can be followed by anyone able to access the course website.  As a result CS50 transposed the materials, lectures and resources from the open course repository, normally accessed by traditional students, to a structured management system hosted by edX. For Harvard undergraduates CS50 counts for both the Gen Ed requirement in Empirical and Mathematical Reasoning and for the computer science concentration.  Conversely, through edX students are offered a HarvardX certificate of completion upon finishing CS50x.

Prior to 2007, the evaluations and feedback had been increasingly uninspiring, regardless of the faculty teaching CS50. When Dr. Malan planned to reinvent the course, he did so with purposeful intentions and clear academic expectations, combined with extensive and robust online and human-support resources. Dr. Malan has even taken into account the physical environment by creating a fun and inviting atmosphere: each lecture begins with hand-selected music from artists like Skrillex and Kanye West (Oblinger, 2012).

The following report will review CS50 in both the traditional classroom and online modalities. Measures of quality will be used through Fink’s (2003) Model for Significant Learning, Quality Matters (QM) (2011) and Bloom’s Taxonomy (n.d.).  Through this exercise, a model will emerge for reenergizing an inoperative model for instruction in both traditional classroom and online instruction for which higher education institutions can apply to antiquated course designs.

Course Overview

CS50 is an introduction to the intellectual enterprises of computer science and the art of programming for majors and non-majors alike, with or without prior programming experience (Malan, 2013). An entry-level course, CS50 teaches students how to think algorithmically and solve problems efficiently. Topics include abstraction, algorithms, data structures, encapsulation, resource management, security, software engineering, and web development. Languages include C, PHP, and JavaScript plus SQL, CSS, and HTML. Problem sets inspired by real-world domains of biology, cryptography, finance, forensics, and gaming. As of Fall 2012, the on-campus version of CS50 is Harvard's largest course (Oblinger, 2012).

According to Quality Matters(2011) standards 1.1 through 1.8, the course syllabus for CS50 both meets and exceeds the communication of the course components, structure, policies and minimum prerequisite knowledge required to be successful in the course.  According to Fink’s (2003) model, CS50 introductory syllabi offers and explains the situational factors students will encounter in the course. The syllabus clearly identifies the subjects and learning goals beyond learning program languages.

One very unique aspect of the newly designed CS50 course is the acknowledgement of prerequisite skills for prospective students. The syllabus even offers varying degrees of textbooks designed for moderately comfortable programmers to individuals that are just beginning the art of programming.

Learning Objectives

The learning objectives are clearly stated for the types of programming languages that will be covered. However, this class focuses less on teaching a student how to be a good programmer, but rather how to think programmatically. Dr. Malan’s primary objective is to teach a student to think algorithmically and solve problems efficiently.

The course expectations charge students to attend or watch all lectures, class sections, submit problem sets, complete two quizzes and submit the final project. The CS50 syllabus clearly outlines the core course objectives. Approaching CS50 from an instructional design standpoint and measuring against Fink’s (2003) taxonomy of “Significant Learning”, the course objectives, assessments, structure, resources and technology satisfy the six major types of significant learning.  For example, CS50’s course design component fulfills the ‘application’ section of the pie, which includes skills, thinking and managing projects.

Assessment and Measurement

CS50 may be taken either for a letter grade or pass/fail. The syllabus clearly outlines the subjects that are covered in each class session. Sections, quizzes and final project – Fink (2003) categorizes this as forward-looking assessment for which the entire premise of CS50 is built upon. The final project states that the end product should be relevant well beyond the life of the course.  Measuring CS50 against QM standard 3.2, the grading policy is clearly stated in the course syllabus.

Instructional Materials

Prior to CS50, it was very rare for a face-to-face course to host such a robust online presence for resources (Oblinger, 2012). For most online courses, materials are password protected or hidden behind a learning management system tied to a student’s institutional credentials. For CS50 and CS50x, the materials are by and large, open to the public. In the end, this feature set the course apart and is why it is successful. There are a spectrum of digital tools, which includes a course website that hosts lecture, section, and seminar videos, virtual office hours, anonymous bulletin boards, lecture notes, an evening telephone hotline, FAQ’s and links to helpful materials related to projects.  As a part of the student course evaluation, comments like, “Most organized course ever” emerged and were recorded for the Fall 2012 semester (Oblinger, 2012).

Learner Interaction

Measuring the learner interactions through the Quality Matters rubric (2011), the learning activities within CS50 accurately promote the learning objectives. Each CS50 Teaching Fellow, or teaching assistant, is equipped with tablet devices for rapid grading. Beyond grading, the teaching fellows are highly utilized within class transactions.  Examining Quality Matters standard 5.2, the final group project accurately promotes learner interaction that supports active learning.

Course Technology

It is expected for a computer science course to leverage certain technologies. However, the pervasiveness surrounding the technological resources for student success is well beyond the academic status quo.  From tablet wielding Teaching Fellows, to recorded lectures in both MP3 and MP4 formats, and an after-hours telephone hotline, the use of course technology sets this course apart from traditional undergraduate lecture hall classes. From an online perspective, the course repository is simple, but effective. Located at cs50.tv, the lectures, sections, problem sets, quizzes and seminars are easily accessible complete with an archived version of the CS50 soundtrack available on iTunes.

Beyond the course technology, Dr. Malan has worked with Harvard’s information technology department to open up data sets such as the bus schedule and cafeteria data so students can apply real world data to class projects(Oblinger, 2012). This brings a sense of reality to the problem sets and final projects – contributing towards active engagement by solving real campus issues.  As a result of the increase in class enrollment, core changes had to take place with the university wireless infrastructure due to the increase in laptop & tablet use.

Although technology is omnipresent in CS50, it is not chief in the success of the course. This was accomplished by a clear goal of Dr. Malan to revitalize this course through strict pedagogical goals and reimagining the course as a whole combined with independent and open support structures. 

Learner Support

When measuring the learner support based on the Quality Matters (2011) rubric, CS50 rated very poorly, which paradoxically is the reason why this course is so highly revered amongst students and researchers. Objectively measuring based on the language of Quality Matters standards 7.2, 7.3 and 7.4, CS50 did not rate a score based on the verbiage on QM’s rubric. Based on the behavioral reactions and course assessments for CS50, this measurement seems to be an anomaly for this course. The language and verbiage did not lend itself to measuring courses that have an independent support system separate from university learning support outlets, like www.cs50.net and the team of Teaching Fellows that assist Dr. Malan in teaching and grading the course.  Indications that this course includes adequate support structure are evident on many levels. This level of support exists in dedicated Teaching Fellows, a robust website and a network of connected students. Additionally, the provisioning of open and online resources through unrestricted domains (cs50.net and cs50.tv) helped to create a clear pathway for Harvard students to access the material. Beyond that, open online resources expanded the course to the world many years before it was incorporated into edX’s offerings.

Accessibility

In terms of accessibility, the course materials were easy to find, query and read. The course repository website hosts a Google translation tool and offered a text-only feature satisfying QM standard 8. The text only feature offers alternate outputs converting the page into text only for screen readers. To be clear, CS50.tv is not a learning management system in the traditional sense; it is simply a content repository. In this way, the learning materials and resources are clearly marked, categorized and accessible.

Quality Measures and Taxonomy

How did CS50 score when evaluating using the Quality Matters 2012-13 rubric? After carefully measuring CS50 based on the course syllabus, website, objectives, materials and academic reviews, it scored an 87/95. Yielding an overall score of 91%. The course failed to meet some of the traditional measures included in the rubric, primarily accessibility. This particular standard failed to realistically measure CS50’s expansive level of accessibility.  According to Quality Matters, a course is never described as having "failed" a review; it is always "in the process of meeting standards" and is described as such until it meets the QM rubric standards at a rate of 85% or better (Quality Matters Program, 2011). So, according to the score, this course would have clearly met the standards. The words failed are not used when measuring online courses with the Quality Matters rubric. It either meets or does not meet the standards. The goal with QM is continuous improvement.

             The unique feature sets of this class could not be accurately captured in the point value system for quality matters. However, per Fink’s (2003) taxonomy for Successful Learning, CS50 fulfills every element.  When reviewing CS50 based on Bloom’s Taxonomy (n.d.)? CS50 met all of Fink’s taxonomy for Successful learning when considering the course objectives, assessments and resources. Likewise, when aligned with the three core domains within Bloom’s taxonomy, CS50’s seems to fair well. The cognitive learning structures promote new knowledge in computer science while applying principles to real world applications. Within these cognitive disciplines, analysis is aided by the teaching fellows and evaluation is conducted in the form of quizzes. 

Would the quality of instruction be high without technology? No, this class thrives on leveraging technology to deliver its course content. Yes, technology is necessary in this review, maybe not as much for face-to-face – but according to researchers and data from class assessments – this is the resource that is most liked.  The effective parameters of CS50 are accomplished through the interactive nature of the lecture and seminars and through social discussion on and offline. As most of the data sets and problem-based scenarios are pragmatic: valuing, organizing and characterization become key growth points for a student in CS50.  Problem sets are introduced early in the course, which challenge students to craft projects using programing software, languages which require critical reasoning and algorithmic analysis. Through eight tiered problem sets, students acquire psychomotor competencies for which they develop various technical and analytical skills as a result (Bloom’s Taxonomy, n.d.).

Conclusion and Recommendations

As traditional public universities seek to find new and innovative paths to connect students with content and teachers, as well as make large classrooms more relevant, intimate and engaging – we must look at courses like CSS50 and instructors like Dr. David Malan. Harvard University and Dr. Malan committed to purposefully design a course that transcends major concentration, genres and departments. The quality is clearly calculated by standards, but at root are the course design, pedagogical makeup through project-based learning and a strong relevance to real world problems and opportunities on and off campus.

The overall course value is explained through the Quality Matters (2011) score, but this is not the only beneficial portion. The enthusiasm associated with this course in both face-to-face and online environments have created what people call the “CS50 effect” (Oblinger, 2012).  It seems that experiential course design has been missing for some time. This is a factor that existed within CS50, which could not be empirically measured, by any rubric, model or design standard. This attribute, contributes towards a subject and class developing a personality for which students can connect, engage and obtain knowledge that will be shared, cultivated and applied.  The role of technology within the build for this course is absolute. Without leveraging open resources and in-time lectures, the relevance and applications would never be accomplished. However, technology aside, this is where a new model emerges for education technology enthusiasts.  There are many ways in which a course can be revived and produce successful results. However, for CS50 a bolstered infrastructure, successful technology integration and vast open online resources, are simply elements towards a bigger solution. Technology is simply an avenue where great instructional design; passionate support structures, experiential learning and shared enthusiasm lead to greater learning for students in all modalities.

References

Bloom’s Taxonomy. (n.d.). Retrieved April 1, 2013 from Wikipedia: http://en.wikipedia.org/wiki/Bloom%27s_Taxonomy

Fink, L. (2003). Self-directed guide to designing courses for significant learning. San Francisco, CA: Jossey-Bass.

Lewin, T. (2012, May 12). Harvard and m.i.t. team up to offer free online courses. New York Times. Retrieved from http://www.nytimes.com/2012/05/03/education/harvard-and-mit-team-up-to-offer-free-online-courses.html?_r=0

Malan, D. (2013).  CS50 website. Retrieved from https://www.cs50.net/

Oblinger, D. (n.d.). Case study: Cs50 at harvard. (2012).Game Changers: Education and Information Technologies: Educause, 361-367. Retrieved from http://www.educause.edu/research-publications/books/game-changers-education-and-information-technologies

Quality Matters Program. (2011). Quality Matters rubric workbook for higher education (2011-2013 ed.). Annapolis, MD: Maryl and Online, Inc.