Laying out 11,206 worked solutions, and auditing the answer key

Akawntant, an online CPALE reviewer, has 11,000+ practice questions for the Philippine CPA Licensure Examination, and 97% of them come with an explanation. The platform around those questions is the easy part to talk about. This post is about the questions themselves, and two pieces of work on them that I think were worth the effort.

A solution is not a paragraph#

Accountancy questions are computation. A good explanation is a worked solution: take this figure, subtract that one, carry the result into the next line, arrive at the answer. Mine were stored as one block of prose per question. Everything was in there, but a student had to untangle the arithmetic from the sentences to follow it.

Akawntant is designed like an accountant's working paper, with ruled lines, caps column labels and a double rule under totals. A worked solution belongs on that paper as ruled steps, with the result of each step on the right and the final figure footed with a double rule.

So I wrote a formatter that reads an explanation and lays it out step by step. The CPALE sample questions page shows the result on real questions.

The test that mattered#

A formatter that parses free text is easy to get wrong in quiet ways. If it misreads a line, it can drop a figure, and a worked solution with a missing step is worse than a dense paragraph, because it looks authoritative.

The rule I set was that nothing may be lost. The formatter is tested against all 11,206 live explanations, and the test checks that every piece of content in the original appears in the formatted output. Fixtures are handy for edge cases, but the real explanations are where the oddities live, and running the whole bank is the only way I trust the claim.

An answer key I could not take on trust#

The second piece was an audit of the question bank. In an exam reviewer, one wrong answer key damages trust in everything else, because a student cannot tell which answers to doubt. I wrote about the same lesson on Arkitekto, the sibling project, where content review ended up being most of the work.

For Akawntant I tried to remove my own bias from the process. The audit started with the questions most likely to be wrong: the ones whose explanation disagreed with the stored key, the ones with no explanation at all, and a sample pulled for public posts. Each of those was solved from scratch, without looking at the existing answer key. A second pass then checked each result blind. No answer key was changed until the two passes had been compared. It has covered several hundred questions so far, not the whole bank, and the rest is still to do. The point of solving first is that reading an answer key before you solve a question pulls you towards agreeing with it, and I wanted the audit to be able to disagree.

Where the domain expert comes in#

A process like that can catch a wrong key, but it cannot be the final word on accountancy. I am a software engineer, not a CPA. I can check that the arithmetic in a solution follows from the question, and I can check a tax rate against the law, but judging whether a question tests a standard the way the board would, or whether two options are both defensible, takes someone who has lived in the subject.

That is where a friend of mine comes in. He placed second nationwide in the CPA board exam in 2024, and he is the domain expert behind Akawntant. The audit can flag a question, but on the accountancy itself I defer to him, because he knows it far better than I do. Where the checks cannot settle a question, I would rather switch it off than leave a guess live.

The engineering makes the content easier to check and easier to read. It does not replace knowing the material.

Two smaller things#

Pro is paid by GCash or InstaPay QR. Each receipt is checked before access is granted, so a payment is a verified event and not a screenshot someone has to eyeball.

I also ran a security review on a commit, and it found an open-redirect bug. It was a small mistake in a flow that every user passes through, which is a good reminder of why I review my own commits before they ship.

What I took from it#

The pattern across all of this is the same as on Arkitekto. The features that students notice, the mock exams, the mastery tracking, the spaced repetition and flashcards, only work if the content underneath is correct and readable. Presentation and correctness both count: a correct answer in a wall of text is still a poor explanation, and a beautiful layout around a wrong key is worse.

If you are preparing for the CPALE, the free plan gives you 10 practice questions a day and 1 full-length mock exam a month, so you can see whether the explanations work for you. You can start at akawntant.review.