Guidelines for winning at Pacwar

A Call to Arms!

The Pacwar project is a slightly reduced version of a famous research problem that has remained unsolved for several years. Rice students in COMP 440 have been working on this version to find the identity of the ultimate killer species in this problem space. They have found some very strong species. In this 11th year of the Pacwar competition, you and your partner have the opportunity to find one that beats all known champions and be crowned uber champion.

A New Kind of Challenge

Same old, same old: Most of you have a lot of experience in structured problem-solving situations that are normal parts of an upper-level computer science or engineering class: problem sets and laboratory/programming assignments. Those tasks are typically precisely specified, and require an explicit or implied approach that can be evaluated against clearly defined success/termination criteria.

Radically cool: Most problems in the real-world and in research are much less structured. AI problem-solving wizards have to analyze the problem and determine what is required to solve it. You and your partner will have to be wizards, too, in the PacWar competition. Here’s how you, as potential PacWar winners, differ from plug –and-chug routine problem solvers. You will Apply your conceptual knowledge to design an approach to the solution Implement your approach with your experimental/programming skills Decide what criteria to use in evaluating how well you are doing Determine whether your approach is working or not Refine or modify your approach along the way, or start afresh with a brand new approach Notice similarities between the problem you are solving and previously solved problems in the literature, and Decide which ideas explored by others might work for you.

How to keep a research log

Keeping a journal of your work on this project is critical to developing the kind of intuitive feelfor solving ill-defined problems that you’ll need to win. Keeping a journal of your work on this project is critical. As David Chapman says in this punchy article

Writing down your ideas is the best way to debug them. Usually you will find that what seemed perfectly clear in your head is in fact an incoherent mess on paper.

Research logs give you a place to do the debugging, as follows. Write down hypotheses, interesting problems, possible solutions, notes on papers you've read, and interesting quotes. Describe your current species hunting strategy; explain how you are implementing it and how you plan to evaluate it. Imagine somebody at your elbow saying, Yes, but . . . ,So what? Why do you think so? Where did you get that? In response to that voice, grab a pencil or a keyboard and defend your approach and choices in your log. Rethink when you have a new insight. On a regular basis, record the best species you have found so far along with a summary of its performance and the reasons you think account for the record. Read back through your log periodically. Make a summary of it every four weeks. This summary will serve as your progress report.

How to read a research paper

The following advice is adapted from Marie desJardins excellent document on how to do research. Before bothering to read *any* paper, make sure it's worth it. Scan the title, then the abstract, then -- if you haven't completely lost interest already -- glance at the introduction and conclusions. (Of course, if your instructor tells you that this is an important paper, skip this preliminary step and jump right in!) Before you try to get all of the nitty-gritty details of the paper, skim the whole thing, and try to get a feel for the most important points. If it still seems worthwhile and relevant, go back and read the whole thing. Many people find it useful to take notes while they read. Even if you don't go back later and reread them, it helps to focus your attention and forces you to summarize as you read. And if you do need to refresh your memory later, rereading your notes is much easier and faster than reading the whole paper. A few other points to keep in mind as you read and evaluate papers:

Make sure the ideas described really worked (as opposed to just being theoretically valid, or tested on a few toy examples).
Try to get past buzzwords: they may sound good, but not mean much. Is there substance and an interesting idea underneath the jargon?
To really understand a paper, you have to understand the motivations for the problem posed, the choices made in finding a solution, the assumptions behind the solution, whether the assumptions are realistic and whether they can be removed without invalidating the approach, future directions for research, what was actually accomplished or implemented, the validity (or lack thereof) of the theoretical justifications or empirical demonstrations, and the potential for extending and scaling the algorithm up.

Your team's stash of great resources: Keep the papers you read filed away so you can find them again later, and set up an online bibliography (BibTeX is a popular format, but anything consistent will do). I find it useful to add extra fields for keywords, the location of the paper (if you borrowed the reference from the library or a friend), and a short summary of particularly interesting papers. This bibliography will be useful for later reference, for writing your final report, and for sharing with other students. It is very important that you cite and attribute sources (including code) correctly. See the UCLA Guide for citations to avoid common mistakes.

Writing standards

I expect professionally done documents, without spelling errors, with appropriate citations to papers/code that you reference or use in your work. If you have never written a technical document before, you should read ``The Elements of Style'', by Strunk and White, Macmillan Publishing Co.

Pitfalls to avoid and Advice for Winning

This is the 11th year of the Pacwar competition. I have summarized for you below the major pitfalls to avoid as well as some general advice for winning, based on my experience from previous years.

Major problems result from starting late.If you do not start by the first or second week of class, it is difficult to compete effectively at the end.
Slow and steady wins the race. Since the major deliverable for the project isn't due till December, there is the temptation to put off working on it till the deadline is imminent. Without exception, the past winners of Pacwar were groups who steadily put in a modest amount of work each week on the project.
Students routinely underestimate the amount of time it takes to ``write up'' the final report. You should realize that ``writing up'' includes not only the physical writing, but also and, more importantly, digesting the results you have found. Often, in writing, you realize that there is a better method that you should have tried. Rather than regretfully announce in your final report that you just discovered a much better way to solve the problem, make sure you get that insight early by writing early and writing often.

Devika Subramanian and Linda Driskill

Last modified: Tue Aug 29 08:30:37 Central Daylight Time 2006