At UTSC, we tend to use Crowdmark for exams in mathematics. To get the papers in to Crowdmark, we need to scan them all. This requires that we get the papers in bundles of a particular size and a very specific order. The image above shows a pile of exams ready to be scanned. In this note, I describe how I ask teaching assistants and assistant invigilators to collect and sort papers.
The first thing to note is that every paper has a unique ID number.
The example above shows booklet #192. This uniquely identifies the whole booklet. It occurs in the top right corner of every page beside the QR code. If Crowdmark encounters an error, it will say something like “Page 3 of #192 is missing.” We want the papers in numerical order so that we can look-up missing pages.
As invigilators, we want to make sure that we have the right number of papers. Sometimes, students will remove exam papers and sell them to other students. We check that we have the right number of papers while the students are in the room, so this check has to happen quickly.
To facilitate counting and sorting, we use sorting placards. The first step is to group the papers numerically in to bundles of about fifty. All papers with booklet IDs between 1 and 50 would go on 50-sorting placard shown below.
Once all the papers are in bundles of approximately 50, we count them. We fill out the little box $N / 50$ with the number of papers in the pile. For example, there might be 46 papers in a pile. We would write $46/50$ in the box. We then add up all numbers in boxes and get a number equal to the head count. At this point, we can dismiss the students.
Our ultimate goal is to get the papers in to bundles of size at most ten. We do this because the scanner can only handle 150~200 pages at a time. Once we’ve confirmed that the paper count matches the head count, each TA or assistant invigilator gets a pile of approximately fifty papers and the relevant 10-sorting placards for piles of size ten.
The invigilator then arranges their pile in groups of size ten, in numerically descending order. Small numbers on top, large numbers on the bottom. For example, the pile with papers 21-30 should be sorted as follows.
| Top |
| 21 |
| 23 |
| 24 |
| 25 |
| 26 |
| 29 |
| 30 |
| Bottom |
Notice that not every number 21-30 is present. This happens because there are blank papers that students don’t write. The invigilator then counts the number in the bundle, in this case seven, and writes $7/10$ in the box.
The invigilator has to wrap the 10-sorting placard around the pages as shown in the image at the top of this page. This makes the papers easy to handle while scanning.
The invigilator then tallies all the small numbers $n/10$ and checks that they get the big number $N/50$ for their pile. Once this is done, the papers are ready to be put in to a big pile. Again, we order things in numerically descending order.
| Top |
| 1-50 |
| 51-100 |
| 101-150 |
| 151-200 |
| 201-250 |
| Bottom |
We do the following:
This seems like a lot of work. However, when it is done smoothly, it runs very quickly. Roughly, this is an implementation of bucket sort. Everything, except the sorting in to groups of fifty, happens in parallel. The whole setup was designed intentionally to make Crowdmark scanning and total paper count verification go quickly.
Published: Apr 15, 2025 @ 14:53.
Last Modified: Apr 15, 2025 @ 16:27.
Home / Now / Blog / Notes / Reading / Office Camera / Tags / RSS Feeds / Top of Page
Thanks for reading! If you have any comments or questions about the content, please let me know. Anyone can contact me by email.