Skip to content

fix: Add attribute check for sampler in close method#2173

Open
usernames122 wants to merge 1 commit intoabetlen:mainfrom
usernames122:main
Open

fix: Add attribute check for sampler in close method#2173
usernames122 wants to merge 1 commit intoabetlen:mainfrom
usernames122:main

Conversation

@usernames122
Copy link
Copy Markdown

This solves a bug I uncovered, that causes an AttributeError if constantly re-initializing a model in a loop and Python garbage collects it, such as testing the highest GPU layer count you can go before CUDA OOMs.
Specifically, it solves:

    if self.sampler is not None:
       ^^^^^^^^^^^^
AttributeError: 'LlamaModel' object has no attribute 'sampler'

This solves a bug I uncovered, that causes an AttributeError if constantly re-initializing a model in a loop and Python garbage collects it, such as testing the highest GPU layer count you can go before CUDA OOMs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant