Repository navigation
Fix/lambda cold start - #1
Merged
Merged
Conversation
Import ultralytics lazily on first evaluation so the IPC server opens its socket inside Lambda's 10 s init window, and install CPU-only torch/torchvision on Linux to drop the multi-GB CUDA libraries from the image. Also bump Poetry in the Dockerfile to 2.x to match poetry.lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The worker still took 12.7 s to open its IPC socket on Lambda, past the 10 s init limit. cv2 and numpy were the largest remaining module-level imports and are only needed for drawing, so import them there and compute the timing averages without numpy. Also run compileall after copying the function code so it is precompiled too; Lambda's read-only filesystem can't cache bytecode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the ipc transport, shimmy holds Lambda's 10 s init open until the worker has created its socket, which the Python imports still couldn't do on a cold container. Deferring the torch import only moved that cost into the first request, which then hit the 30 s FUNCTION_TIMEOUT. Use stdio like eduVisionAeroRC instead: init returns immediately and the request waits on stdin while Python imports. Restore the module-level ultralytics, cv2 and numpy imports and preload the default model so that work overlaps the cold start rather than the request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first request after a cold start still hit shimmy's 30 s send timeout while Python was importing torch/ultralytics. Raise it to 90 s, and temporarily enable PYTHONPROFILEIMPORTTIME so CloudWatch shows which imports dominate the cold start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See the commits for more detail, but changed the transport layer and lazy loaded large imports.