Repository navigation
Conversation
|
Thanks for the contribution. What is the failure mode that this addresses? It currently tries to download a file again if it failed to load it. So the file being preserved is no good. I'm not sure it is worth the extra complication. |
|
The concrete failure is an interrupted first download, rather than preserving an already-unloadable weight file. In the current call path, I added a regression test in 94529c2 for this exact path. Both HTTP-status failure and a stream interrupted after a partial chunk leave no cache file; a second The staging directory's purpose is to keep an incomplete transfer from becoming the cache entry. It does not validate whether a successfully downloaded model is trustworthy or loadable. |
|
Ok, I'll investigate this more. Good catch! |
|
I added more commits on a new PR in #200. I'll close this PR and merge there. |
The download helper writes to the public cache filename before a transfer has succeeded. An interrupted first download can therefore leave a partial file that
get_weights()treats as present on the next call:models.pychecksos.path.isfile()before downloading, while the DenseNet and ResNet load-error handlers print the filename and re-raise without deleting it or retrying the download.Stream into private sibling staging, check HTTP status and usable identity-encoded Content-Length, and replace the destination only after a successful transfer. Requests use connect/read inactivity timeouts. Responses and staging are closed on failures. This leaves an unsuccessful first download eligible for a later retry as well as preserving an existing destination during an explicit replacement attempt.
Successful replacement changes the destination inode and may change permission metadata; this does not authenticate model contents or provide crash durability.
Validation
Reused the retained eight-case focused handoff verification for the unchanged download implementation. Added
tests/test_download.pyto cover the actualget_weights()retry path: HTTP-status failure and an interrupted first stream each leave no cache file or staging directory, then a second call requests and publishes a complete response. Both subcases passed in an isolated source-function harness using the actualdownload()andget_weights()functions and the new repository test. Full package imports and the full test suite were not run locally; no model downloads or live endpoints were used.