We Put a GPU Job on Pay-Per-Second Cloud,
and It Sat There Crash-Looping for an Hour Without Us Seeing It

That night we had just finished wiring up a GPU pipeline to read receipts. The first job went off to a worker in the cloud, and then it went quiet. We ran modal container list, saw nothing stuck, and relaxed. Nothing was broken.

Except one of them had already broken and was busy crash-looping. That worker had failed at the moment it tried to load the model onto the GPU. When it failed, the platform retried it. The retry spun up a fresh machine, loaded again, and failed again. It went around like that, eating GPU time for roughly an hour, and container list never once showed it, because that command only lists what is currently active. A thing that dies and respawns does not count. We only caught it when Yim pulled up the web dashboard.

So this post is about putting heavy work, the kind that needs a GPU, onto serverless: where it pays off, and where the budget leaks in ways nobody warns you about.

Part 1Why put a GPU job on serverless at all

A GPU is expensive. Rent one and leave it running 24 hours and you pay for every hour, even the ones with no work coming in. But our receipt-reading work does not arrive around the clock. It comes in bursts, a handful of pages a day. Keeping a GPU warm all day to read a few pages is not worth it.

Serverless GPU fixes that. You do not babysit a machine. It wakes up when there is work, does the job, and shuts down, and you are billed by the second only for the time it actually ran. (We use Modal for this part.) It fits work that arrives in bursts far better than work that runs continuously.

A real measured number helps. The receipt model Typhoon OCR 2B, running on an L4 GPU (NVIDIA's small card), costs about 0.14 baht per page once the machine is warm, and takes about 18 seconds to read a page. At that price, paying for a machine left on all day to read a few pages would push the per-page cost up several times over.

Part 2Cold start is physics, not taste

The 18-seconds-per-page number is the warm price. When the machine wakes from zero, the story changes. It has to load the model onto the GPU before it can read anything, and that wake-up window (cold start) we measured at 344 seconds, close to 6 minutes.

This one number decides the shape of the whole system. If a person presses a button and has to stand around for 6 minutes waiting for the machine to wake, nobody waits that long. And if the caller, a cron job or a web page, waits the same way, most callers will time out and drop the job before the GPU is even awake.

That is why our system never makes anything wait on the GPU. The caller fires the job and goes off to do something else. When the job finishes, the result comes back by email, and we batch several pages into one run so we pay the wake-up cost once per run. This design did not come from a preference for async. It came from the physics of cold start forcing it.

Part 3Where the money burns quietly

Once the system is fire-and-forget, several spots can leak money with no alarm.

1. The caller must not wait on the result. If you tell it to call and block for the answer (.remote()), the 6-minute cold start is longer than the caller is willing to wait, and the job dies where it stands. You have to fire it and forget it (.spawn()), so it runs in the background on its own.
2. That fire-and-forget call has to happen inside an app that is actually deployed and running. If you fire the job from a temporary run on your own machine, the moment that command on your side ends, the cloud app dies with it, and the job hangs at 'processing' with nobody to pick it up. The right way is to let the deployed app's own cron pick the work up.
3. The one that really matters: if the worker fails at the moment it enters the GPU, it will retry forever and burn money every round. Each round lights up a fresh machine, which means paying again every round. And as noted up top, container list does not show a thing that crash-loops like this. You only see it on the web dashboard. The lesson: when something breaks, stop that app first (modal app stop) and then fix it, instead of letting it keep retrying. Teardown when things break is part of the run, not a chore for later.

Before you even get to the money, a couple of setup snags cost us time too. A newer vLLM, for instance, needs NVIDIA's compiler (nvcc) baked into the base image or the build fails. The specifics of that we keep in the code, but the principle is fully open.

Part 4How to budget a GPU so it does not bite you

If you boil it down to habits worth keeping, there are only a few.

1. Do not trust the CLI alone. The command line tells you part of the picture, and as we found out, it only shows what is active. The web dashboard and the billing page are where the truth lives about which machine is spending your money right now.
2. On a failure, tear down first, always. Do not leave a broken thing sitting there retrying.
3. Design the work as fire-and-forget with the result coming back later, from the start, so cold start never gets in the way and kills a job mid-flight.

Looking back, our GPU budget did not leak when work was heavy. It leaked when work broke and nobody was looking. The rent for the machine while it was actually reading receipts was startlingly cheap. What was expensive was the hour a machine spent crash-looping while we sat watching a green CLI and assumed everything was fine.

Part 5How it ended

Once we set our eyes on the right spot, the pipeline ran a full clean cycle for the first time. We switched to the Qwen3-VL 4B model paired with forcing it to answer in a fixed shape (structured outputs), and this time it read the totals off the receipts correctly on all 3 of 3 pages. The email landed in a real inbox with the right numbers. The machine we had been assembling over several nights finally turned a full first cycle.

If you are about to put a GPU job on serverless, the thing to take with you is not which model is more accurate. It is to set your eyes so you can see the thing that fails quietly, first. Because the most expensive GPU bill usually does not come from the work that runs. It comes from the work that breaks while nobody is watching.