CS 572
Information Retrieval and Web Search Engines

Fall Semester, 2014
Location: MRF 340
Time: TTh 5:00pm-6:20pm
Class number: 30387D

Instructor | Overview | Academic IntegrityTextbook and ReadingsAssignments and Exams Project Schedule





The plethora of information and content available on the public Internet wrought forth a boom in the early 2000s in the area of web search engines. Companies such as Altavista, Excite, and Yahoo were the players, and the game had an ambitious objective: for all the unknown web pages that existed in cyberspace: (1) locate them in some fashion (through links, through guessing, etc.); (2) obtain the content from those pages; and (3) make that content available to users who enter in a few search terms into an input box on a web page.

As the web grew (into the billions of pages range circa 2000) , search engines had to become smarter to deal with the data management problems induced with collecting such an archive -- storage, availability, processing, failover, redundancy were only some of the challenges that had to be addressed.

This course will afford the student a complete treatment of web search engines, its foundation, principles, and elements, including that which is described above. The class is centered around paper reading assignments, and an individual presentation that will test comprehension and understanding of the course reading material. A class project will require the student to leverage the search engine techniques learned during the course (e.g., ranking, crawling , content analysis and detection, query models ) to, coupled with programming/implementation effort leveraging modern open source search technologies from Apache, design and implement a component of a real-world search engine.

In addition to foundations, and practical experience with search engines, the class will also introduce the student to the state-of-the-art in search engine research, future trends and state-of-the-practice. Students are expected to attend class regularly, and participate (as directed) in all class discussions, and most importantly, have fun!

Academic Integrity

Students must work independently on all individual assignments; collaborating on individual assignments is considered cheating and will be penalized accordingly. All USC students are responsible for reading and following the USC Student Conduct Code, which prohibits plagiarism. Some examples of behavior that is not allowed are: copying all or part of someone else's work (by hand or by looking at others' files, either secretly or if shown), and submitting it as your own; giving another student in the class a copy of your assignment solution; consulting with another student during an exam; and copying text from published literature without proper attribution. If you have questions about what is allowed, please discuss it with the instructor.

Students who violate University standards of academic integrity are subject to disciplinary sanctions, including failure in the course and suspension from the University. Since dishonesty in any form harms the individual, other students, and the University, policies on academic integrity have been and will be strictly enforced.

Textbook and Readings


Supplemental Readings:

Assignments and Examinations





An exam testing your understanding of the lecture materials including crawling, ranking indexing, deduplication, querying, etc.



Assignments where you will build on the search engine topics in course (information retrieval, ranking, content detection and analysis, indexing, processing, etc.) and make a contribution to one of the existing Apache search technologies (Nutch, Lucene, Solr, Tika, OODT, etc.).


Project Submission Guidelines

Please refer to this document for guidelines on submitting your assignments.

Schedule (subject to change; check regularly)


Lecture Topic


Assignments and Deadlines


  • Tika in Action, Chapter 1


  • Tika in Action, Chapter 2
  • Tika in Action, Chapter 3




  • Tika in Action - Chapter 6
  • Tika in Action - Chapters 7, 8




  • M. Bernaschi, M. Cianfriglia, A. Di Marco, A. Sabellico, G. Me, G. Carbone, G. Totaro. Forensic Disk Image Indexing and Search in an HPC environment. 2014.


  • Spelling Correction
    (Taped Lecture)








  • Bostock, Michael, Vadim Ogievetsky, and Jeffrey Heer. "D³ data-driven documents." Visualization and Computer Graphics, IEEE Transactions on 17.12 (2011): 2301-2309.
  • Geospatial Search


  • Work on Assignments
  • Possible Guest Lecture
  • TBD


Thanksgiving Recess (November 26-29)
  • Exam Review - cont.
  • Exam