Batch
Filed under
1 post
One Door to the Models, Part 4: Async Work Off the Request Path
Batch inference is a second door: a file goes in, a job runs, a file comes out, and no gateway policy is in the path. Part 4 keeps the accounting honest.
Batch inference is a second door: a file goes in, a job runs, a file comes out, and no gateway policy is in the path. Part 4 keeps the accounting honest.