Skip to content

VerifAI Dynamic Sampling Compatibility#417

Merged
dfremont merged 16 commits into
mainfrom
VerifAIDynamicSampling
May 11, 2026
Merged

VerifAI Dynamic Sampling Compatibility#417
dfremont merged 16 commits into
mainfrom
VerifAIDynamicSampling

Conversation

@Eric-Vin
Copy link
Copy Markdown
Collaborator

@Eric-Vin Eric-Vin commented Nov 13, 2025

Description

Adds compatibility with VerifAI dynamic sampling.

Issue Link

N/A

Checklist

  • I have tested the changes locally via pytest and/or other means
  • I have added or updated relevant documentation
  • I have autoformatted the code with black and isort
  • I have added test cases (if applicable)

Additional Notes

Merge with BerkeleyLearnVerify/VerifAI#58

@Eric-Vin Eric-Vin marked this pull request as draft November 13, 2025 01:44
@Eric-Vin Eric-Vin changed the title [WIP] VerifAI Dynamic Sampling [WIP] VerifAI Dynamic Sampling Compatibility Nov 13, 2025
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py
Copy link
Copy Markdown
Collaborator

@dfremont dfremont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine overall, although I'm not sure what prompted the introduction of extractOutput -- maybe we can talk offline.

@Eric-Vin Eric-Vin marked this pull request as ready for review April 7, 2026 16:15
@Eric-Vin Eric-Vin changed the title [WIP] VerifAI Dynamic Sampling Compatibility VerifAI Dynamic Sampling Compatibility Apr 7, 2026
@Eric-Vin Eric-Vin requested a review from dfremont April 7, 2026 16:15
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py Outdated
Comment thread src/scenic/core/external_params.py Outdated
param timeBound = 250

This value can also be set directly in VerifAI via the `maxSteps` parameter to the
`ScenicSampler`.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this reference will work if you add VerifAI to Intersphinx (see intersphinx_mapping in docs/conf.py).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just going to make this not be a link for the moment and we can improve it later.

Comment thread src/scenic/core/external_params.py
Copy link
Copy Markdown
Collaborator

@dfremont dfremont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good -- just a last few minor suggestions.

Copy link
Copy Markdown
Collaborator

@dfremont dfremont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this as-is and we can come back to the last two minor comments later.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 53.73134% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.56%. Comparing base (e17e6a4) to head (2b1573f).
⚠️ Report is 55 commits behind head on main.

Files with missing lines Patch % Lines
src/scenic/core/external_params.py 53.73% 31 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #417      +/-   ##
==========================================
- Coverage   89.90%   89.56%   -0.34%     
==========================================
  Files          54       48       -6     
  Lines       13536    13283     -253     
==========================================
- Hits        12169    11897     -272     
- Misses       1367     1386      +19     
Files with missing lines Coverage Δ
src/scenic/syntax/veneer.py 89.69% <ø> (-0.08%) ⬇️
src/scenic/core/external_params.py 77.31% <53.73%> (-12.34%) ⬇️

... and 52 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dfremont
Copy link
Copy Markdown
Collaborator

Thanks for the fixes, Eric! This is looking good -- we can revisit the coverage issue once the new version of VerifAI is out.

@dfremont dfremont merged commit cae39ca into main May 11, 2026
38 of 39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants