Xiaodou Ledger: a screenshot is the entry, and the five times it got stuck on the way to the App Store

The hard part of an expense tracker isn't recording. It's keeping it up. Every purchase means opening the app, picking a category, typing an amount, and three days later you give up. Xiaodou Ledger does one thing: take a screenshot after you pay, and the entry is done.

The screenshot goes to a multimodal model (Doubao doubao-seed-2-0-mini), which returns structured JSON directly: amount, merchant, date, category. There's no separate OCR in the recognition step, and no "merge and correct" post-processing code at all.

That's on purpose. When recognition gets something wrong, I change only the prompt and don't patch it in code, because two sets of rules fighting each other are harder to debug than one that's inaccurate. Every constraint in the prompt has its reason written down in the docs: an order details page yields exactly 1 entry, the category must come from the given list, and if no date is visible, no guessing.

A few rules added later all came from real data:

• Refunds: the same model finds the original transaction among recent expenses. The id it returns must be in the candidate list, so the model can't invent one.
• Duplicates: same amount, same type, within 5 minutes counts as a duplicate. This came from looking at 14 days of recognition records: out of 135 successful recognitions, 7 were the same purchase screenshotted twice. That's roughly 1 in every 20.

The most-used entry point isn't the app. It's a Shortcut: tap the back of the phone to take a screenshot and send it straight to recognition. The first Shortcut called the backend directly and never went through the app at all. So the "agree to send screenshots to AI" consent stored in the app only ever stopped App Review. Real users' images were sent anyway. Once I found that, consent moved to the server. The server is what actually sends images to the model, so it's the only place that can control both paths, the app and the Shortcut. The Shortcut was also changed to go through an entry point in the app.

1. No iPad screenshots, so it couldn't be submitted. The project had iPad support, and submitting failed with SCREENSHOT_REQUIRED: screenshots for the 12.9-inch iPad were missing. It went through only after I switched to iPhone-only and rebuilt.

2. Rejected twice for privacy (5.1.1). The first time: the app didn't explain who screenshots were sent to or what was sent, and didn't ask for consent first. Apple was explicit that putting it only in the privacy policy doesn't count. This time the fix was in the product: a consent screen before upload, and without consent the camera and photo library don't even open. A page explaining it was added under the "Me" tab, and consent can be withdrawn at any time. The second time: the privacy policy was missing a commitment that "third parties provide the same level of protection". I added that, and while I was at it, made an English version. Before that it was Chinese only.

3. Approved, but not on the store. A version's release option defaults to "Manually release". After 1.0.2 was approved it sat in App Store Connect for several days because nobody pressed release. Swipe (now Flipfile) had hit the same problem a week earlier, but that lesson was written only in its own repository. Since then every version is set to "Automatically release after approval", and that rule also went into the knowledge base shared by all my apps.

4. China's ICP filing (备案). The entity name on the filing didn't match the name on the developer account. Before submitting, someone has to confirm once on the website with an admin account that "this is the same entity". The API can't do this step: Flipfile, under the same account, was submitted without that confirmation and the API returned a 409. An API for querying the filing information doesn't exist at all.

5. US reviewers couldn't reach the server (2.1(a)). At first the backend ran only on Alibaba Cloud. From a US probe, 5 out of 24 requests timed out. When the reviewer signed in, they didn't get the challenge code, and it failed silently. The final fix was two backends split by App Store region: mainland China goes to Alibaba Cloud, everywhere else goes to Cloudflare, with the same server code. I didn't split it into two apps, because the review guidelines discourage duplicate apps made just for regional differences.

The free tier gets 30 recognitions a month. Pro is ¥28 a year (with a 7-day trial) or ¥98 one-time. No monthly plan. By a September estimate, one recognition costs about ¥0.003 in model fees, so the free tier limits only the number of recognitions and cuts nothing else: the reason to pay and the source of cost are the same thing.

The backend and the first client lived in one repository: 189 of its 224 commits were made together with Claude, covering the backend, the recognition prompt and the release scripts. In October I rewrote the client in a new repository: 21 of 24 commits. The release process itself is also run by the agent: building, uploading, submitting for review, reading the review status. What I mainly do is set the direction, set the rules, and decide what structure to change after each thing goes wrong.

As of writing (October 22, 2026), the version on sale is 1.0.7, available in 175 countries and regions. The rewritten client, 1.1.0, was submitted for review on October 21. One thing it adds: before a screenshot is uploaded, the phone uses the system's text recognition to find addresses, phone numbers and names and grays them out before sending. The amount and merchant stay as they are.