ET-2 — Revision session (28 Aug 2026)¶
This session walked through the September 2025 FN paper. It also carried the new exam format announcement and the syllabus exclusion. Links go to the solved paper.
The new exam format (the biggest reveal)¶
Straight from the professor — memorize this
- Two question types: standard MCQs matching previous papers, plus new subjective scenario-based questions testing critical thinking rather than rote definitions or technical trivia.
- Subjective questions are graded by LLMs (AI-graded, not humans). Success depends on organizing, structuring, and justifying your answers clearly enough to "convince the LM."
- Example subjective scenario: you're talking to a client — "What are the three most important follow-up questions you would ask based on these requirements?" — testing whether you think about capabilities, constraints, and business context before building.
- For the LLM grader: use bullet points and clear headers. It grades on logical markers and "thinking" keywords: constraints, assumptions, scalability.
- On answer length: "logic over length" — a reasoned solution that demonstrates critical thinking, not a high word count.
Why this matters more than any single topic: the subjective section is new (first appeared in Jan 2026) and growing. Students who can't write for an LLM grader lose marks that no amount of MCQ knowledge recovers. Our full method with worked examples: Writing answers an LLM grades well.
Syllabus exclusion
OpenRefine is NOT on this term's syllabus. If you encounter OpenRefine questions in older papers (text clustering, name standardization), skip them immediately. You will not be asked.
Parquet — the "odd one out"¶
JSON, XML, and CSV are all text-based formats — human-readable, "flat," and carrying significant overhead when storing large numeric arrays (every number becomes a string of characters, and the parser must convert back).
Parquet uses a columnar approach — values of each column live together in a binary format, which makes it "the odd one out" and the most efficient choice for heavy numerical data.
Why columnar wins for analytics: when you query two columns of a twenty-column table, a row-based format (CSV) forces you to scan every row to extract those two values. A columnar format lets you read only the two columns you need — a fraction of the bytes, a fraction of the network transfer for remote files.
The exam pattern he named: when a question groups XML, JSON, and CSV together and asks about efficiency, the columnar outlier (Parquet) is almost certainly the correct answer.
Git lifecycle — the "safety goggles" before welding¶
He walked through the full inspection family, then gave it a metaphor worth keeping:
git statusandgit diffare the safety goggles you must wear before performing the welding of a commit or push. They prevent the accidental introduction of breaking changes into the shared codebase.
| Command | What it shows | When to use it |
|---|---|---|
git status |
Which files changed, staged, or untracked | Before every commit |
git diff |
What lines changed (working directory vs last commit) | Before every commit |
git log |
Commit history — authors, timestamps, messages | Finding when something broke |
git show |
One specific commit's changes in detail | Investigating a single change |
The trap he didn't mention but the exam tests: git diff after git add shows nothing — by default it compares the working directory to the staging area, not the last commit. git diff --staged shows what's actually about to be committed.
Asked in: T3-FN Q3 — git diff · T3-FN Q23–27 — the full git workflow scenario · Week 1 — git section
HTTP status codes — the full taxonomy¶
He taught the complete family with a distinction students consistently confuse:
| Code | Meaning | His example |
|---|---|---|
| 401 Unauthorized | Authentication required or failed — who are you? | Viewing private Instagram content without an account |
| 403 Forbidden | Authenticated but lacking permission — I know you, but you can't do this | Logged in, but viewing someone else's private content |
| 404 Not Found | The resource doesn't exist | — |
| 429 Too Many Requests | Rate limit exceeded | Often triggered/handled by the API library itself |
| 500 Internal Server Error | Generic server-side failure — the client request was valid | Check backend code and database |
The 401 vs 403 distinction is the one to nail: identity vs permission. One is a locked door with no key; the other is a key that doesn't fit this lock.
His note on 429: in many modern implementations, this code is handled by the API library itself — it retries with exponential backoff without you writing retry logic. But you should still know what the code means.
On 500: the generic catch-all for server-side failures where the client did nothing wrong. The debugging focus: backend code execution and the database layer — not the frontend.
Asked in: T3-AN Q4 — the 401 · T3-FN Q4 — the 429 · T1-FN Q28 — debugging the 500 · T3-AN Q27 — the 401 in a scenario
API validation — fail fast with schemas¶
His framing: external APIs (like Gemini or weather services) are unpredictable. Data may be missing, or the server may time out. Your architecture must survive this.
The strategy — fail fast:
- Audit data types and value ranges immediately upon receipt — at the boundary, before anything downstream
- Reject with clear error messages that a human can act on (which field, what's wrong)
- Explicit fallback strategies — so one API failure doesn't crash the entire downstream pipeline
Why "let downstream handle it" fails: bad data that flows past the boundary corrupts something far from its cause — the most expensive kind of bug to diagnose. The error appears in a dashboard three steps later, and nobody remembers which API call introduced the bad row.
Asked in: T3-FN Q7 — validation strategy · T3-AN Q17 — schema validation · T1-FN Q14 — Pydantic coercion
ETL — lookback windows, not full reprocessing¶
The problem: late-arriving or updated data that belongs to a previous processing cycle. Orders placed two days ago get updated today — but your pipeline already processed "two days ago" and won't look back.
The wrong fix: reprocess all historical data daily. He called this "cost-extensive" — wasted compute credits, excessive processing time, potential database lockups. In production, this is considered a failure of design.
The right pattern:
- Lookback window — process only the last N days (typically 3), capturing late arrivals without touching history
- Deduplication — key on (order_id, update_timestamp) so re-seeing a row is harmless: the newest version wins, duplicates collapse
The exam connection: this exact pattern appears as both an MCQ (T3-FN Q8) and as the TDS tip in a short-answer question: "use terms like 'Lookback Window' and 'Deduplication'" — the LLM grader rewards the professional vocabulary.
Asked in: T3-FN Q8 — the lookback pattern · T3-AN Q11 — idempotent ETL · GA2 (api-idempotency-pagination-server)
Dashboard caching — TTL arithmetic¶
The setup: a dashboard queries a large dataset that updates hourly but is viewed (queried) every few seconds by users.
His arithmetic: without caching, the database gets hit ~3600 times per hour (once per few seconds). With a 5-minute TTL, the database is hit once every 5 minutes — regardless of how many users view the dashboard. The cache absorbs the read load; the TTL bounds the freshness loss.
The trade-off TTL embodies: data freshness vs database conservation. A 1-minute TTL is fresher but hits the database more; a 1-hour TTL is very conservative but serves stale data. The right number matches your data's update frequency and your users' tolerance.
Timezones — the assumption that corrupts silently¶
His warning: assuming missing timestamps are UTC is "a common and dangerous pitfall that leads to silent data corruption."
What "silent" means here: no error, no warning — the data just shifts by hours. An order that was placed at 2 PM local time gets recorded as 2 PM UTC, which is actually 7:30 PM IST. Every join against another timezone-aware dataset is now off by 5.5 hours. Orders appear before they were placed.
The professional approach, in three steps:
- Infer the timezone from metadata or geographic location (where the data came from)
- Document the assumption — write down what you inferred and why
- Flag inferred values — mark the rows so auditors and downstream users know what's certain vs guessed
The contrast with the naive approach: assume-and-forget is the bug; assume-and-document is engineering.
Asked in: T3-FN Q10 — the timezone approach
Docker — the EXPOSE misconception and the build order¶
The distinction a student asked about:
EXPOSE serves as documentation — a "label" telling other containers which port the app listens on. It is not a "bridge" to the host. Only the
-pflag at runtime actually maps the port to the external world.
On seeing Dockerfiles in the exam: a student asked if they'd see a physical Dockerfile. His answer: the file itself may not always be displayed, but you're expected to understand the logic of the build process — the order of operations, what each instruction does, and why the sequence matters.
The sequence that matters most (the layer-caching question from T1-AN Q24):
FROM python:3.12
COPY requirements.txt . # ← dependency list FIRST
RUN pip install -r requirements.txt # ← install (stays cached)
COPY . . # ← code LAST
Reverse the COPY order and every code edit invalidates the install layer — dependencies re-download on every build.
Asked in: T3-FN Q31 — what EXPOSE does · T3-FN Q30 — the install line · T1-AN Q28 — EXPOSE vs -p · T1-AN Q24 — layer caching SA
The restaurant reviews case study (pandas)¶
He walked through the full scenario — the one that appears as five exam questions in T3-FN Q18–22:
The setup: a food delivery platform with review data in CSV (ratings, text comments, timestamps). They want trending cuisines and satisfaction patterns.
Step 1 — Missing review text:
His reasoning: for text analysis, rows with no text are pure noise. Keeping nulls "provides zero value for text analysis and skews your statistical averages." But note the scope: you drop these rows for the text analysis — their ratings still count elsewhere. Cleaning is per-question, not global.
Step 2 — Common words in negative reviews:
Filter first (rating < 3.0), then extract and count words from the filtered review_text. Two independent operations chained — don't count all words then filter.
Step 3 — Average rating by cuisine:
He noted the column names are case-sensitive — 'Cuisine' and 'cuisine' are different columns.
Step 4 — Sharing with a non-technical manager:
CSV — because it opens in Excel with zero friction. His reasoning: "portability and ease of use." A pickle file requires Python, JSON requires a programmer, a database requires a database.
Asked in: T3-FN Q18 — dropna · T3-FN Q19 — filter then count · T3-FN Q20 — CSV for stakeholders · T3-FN Q21 — pandas · T3-FN Q22 — groupby
Rapid-fire best practices¶
His list, verbatim — each one a potential MCQ or a bullet in a subjective answer:
- Versioned pipelines: always version transformation logic — it changes more frequently than the data schema
- Currency storage: store the original amount plus the specific exchange rate used, to enable future audits
- Recommendation auditing: use systematic segment analysis to fix poorly performing recommendations — not "retrain with more data and hope"
- Third-party assessment: evaluate security vulnerabilities in external libraries before performing updates — not after
- Docker reproducibility: use containers and pinned dependencies — "it works on my machine" must translate to the server
- PII minimization: retain only the fields necessary for the specific utility — balance analysis with privacy
Asked in: T3-FN Q11 — versioning transforms · T3-FN Q12 — currency storage · T3-FN Q13 — segment analysis · T3-FN Q14 — vulnerable dependencies · T3-FN Q15 — reproducibility · T3-FN Q17 — PII minimization
Exam-engineering tips (his own framing)¶
Two patterns he named explicitly
- The 'Odd One Out' technique: in serialization questions, if you see XML, JSON and CSV grouped together, recognize they're all text-based. Parquet, the columnar outlier, is almost certainly the correct answer for efficiency queries.
- The 'Longest Option' pattern: in engineering exams, the most detailed answer — the one defining constraints, documenting assumptions, addressing edge cases — is frequently correct because it mirrors professional best practices.
His MCQ prep advice
The MCQ section is designed to be accessible. Reviewing previous year questions is the single most effective use of your study time. The options themselves are designed to guide you — systematic elimination works. By reviewing the choices methodically, you can often eliminate two clearly wrong options and choose between the remaining two on principle.
Student doubts worth recording¶
| Student asked | The professor's answer |
|---|---|
| Do subjective answers need a specific length? (Disha Rani) | "Logic over length." A reasoned solution demonstrating critical thinking to the LLM grader, not word count. |
| What is "TDL"? | A slip of the tongue during the session. The correct term is TTL (Time-To-Live). TDL is not a standard term in this curriculum. |
| Will we see a Dockerfile in the exam? | The file itself may not always be displayed, but you're expected to understand the logic of the build process (e.g., the order of operations). |
| What does EXPOSE do? | Documentation (a "label") — only -p at runtime actually maps the port to the host. |