Why don’t students hack exams when they can?

This year, I created a series of tests for my course at IITM and to recruit for Gramener.

The tests had 2 interesting features.

One question required them to hack the page

Write the body of the request to an OpenAI chat completion call that:

• Uses model gpt-4o-mini
• Has a system message: Respond in JSON
• Has a user message: Generate 10 random addresses in the US
• Uses structured outputs to respond with an object addresses which is an array of objects with required fields: street (string) city (string) apartment (string) .
• Sets additionalProperties to false to prevent additional properties.

What is the JSON body we should send to https://api.openai.com/v1/chat/completions for this? (No need to run it or to use an API key. Just write the body of the request below.)

There’s no answer box above. Figure out how to enable it. That’s part of the test.

The only way to even attempt this question is to inspect the page, find the hidden input and make it visible. (This requires removing a class, an attribute, and a style – from different places.)

Here’s the number of people who managed to enable the text box and answer it.

College	# students	Enabled	Answered
NIT Bhopal	144	4 (2.8%)	0 (0.0%)
CBIT	277	16 (5.8%)	0 (0.0%)
IIT Madras	693	74 (10.7%)	4 (0.6%)

A few things surprised me.

First, I think students don’t inspect HTML. Less than 10% of students managed to modify the HTML page, even after being told they need to. But they know web programming. 49 students at CBIT scored full marks on the rest of the questions, which includes CSS selectors and complex JS code. Maybe editing in a browser instead of an editor is a big mental leap?

Second, almost no one could solve this problem. There are 3 ways to easily solve it.

1. Copy the question and relevant test cases from my exam page’s JavaScript into ChatGPT and ask for an answer. (I test it and it works.)
2. Copy the question and structured output documentation to ChatGPT and ask for an answer. (I tested it and it works.)
3. Create a random JSON and just keep fixing the errors manually until it passes. (The exam gives detailed error messages like “The system message must be ‘Respond in JSON'”, “addresses items must be an object”, etc.)

Maybe questions from a curriculum are easier to solve than questions not in a curriculum? Or is JSON schema too hard?

The exam was officially hackable

All validation was on the client side. The JS code was minified and answers are dynamically generated. But a student can set a breakpoint, see the answers, and modify their responses.

The students at NIT Bhopal and CBIT were not explicitly told that. The students at IITM were explicitly told that they could (and are welcome to) hack it.

Out of the 1,114 students who took these tests, only one student actually hacked it.

(How do I know that? No other student got full marks. This student got full marks with empty answers.)

It’s probably not that difficult. My course content covers scraping pages using JavaScript using DevTools. Inspecting JS is just a step away.

I did chat with the student who hacked it, asking:

Anand: How come you didn’t share the details of the hack with others?

Student: I did with a few but I am not sure whether or not they were able to figure it out still.
Most students in the program still require a lot of handholding even with basic things.
Experience from being a TA [Teaching Assistant] past term.

Why didn’t they hack?

Maybe…

1. They don’t believe me. What if hacking the exam page is considered cheating, even if explicitly allowed?
2. The time pressure is too much. They’d rather solve what they know than risk wasting time hacking.
3. It feels wrong. They’d rather answer based on their knowledge than take a shortcut.
4. They don’t know how. Using DevTools is more sophisticated than web programming.

Issue #1 – the trust issue – is solveable. We can issue multiple official notices.

Issue #4 – capability – is not worth solving. My aim is to get students to do stuff they weren’t taught.

Issue #2 & #3 – a risk-taking culture – is what I want to encourage. It might teach them to blur ethical lines and neglect fundamentals (which are bad), but it might also build adaptability, creativity, and prepare them for real-world scenarios.

Personally, I need more team members that get the job done even if they’ve never done it before.