Skip to content

feat(agent): gzip wasm uploads over 25MB and persist encoding metadata#377

Closed
Nsttt wants to merge 2 commits intomainfrom
feat/wasm-gzip-over-25mb
Closed

feat(agent): gzip wasm uploads over 25MB and persist encoding metadata#377
Nsttt wants to merge 2 commits intomainfrom
feat/wasm-gzip-over-25mb

Conversation

@Nsttt
Copy link
Copy Markdown
Member

@Nsttt Nsttt commented Mar 2, 2026

Summary

  • gzip .wasm assets larger than 25MB before upload in zephyr-agent
  • propagate contentEncoding through build assets and snapshot contracts
  • preserve encoding metadata in snapshot creation and upload payloads
  • add transformer and snapshot tests plus README note

Verification

  • pnpm nx test zephyr-agent --runInBand
  • pnpm nx test zephyr-edge-contract --runInBand
  • manual local e2e against worker: uploaded compressed wasm + snapshot and verified response headers/body via curl (Content-Encoding: gzip, Content-Type: application/wasm)

@Nsttt Nsttt self-assigned this Mar 2, 2026
hash,
size: buffer.length,
buffer: buffer,
...(contentEncoding ? { contentEncoding } : {}),
Copy link
Copy Markdown
Member

@arthurfiorette arthurfiorette Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
...(contentEncoding ? { contentEncoding } : {}),
contentEncoding,

simpler to read

extname,
hash: asset.hash,
size,
...(contentEncoding ? { contentEncoding } : {}),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
...(contentEncoding ? { contentEncoding } : {}),
contentEncoding,

simpler to read

* path and extension for runtime lookup.
*/
export function compressLargeAssets(assetsMap: ZeBuildAssetsMap): ZeBuildAssetsMap {
return Object.values(assetsMap).reduce((memo, asset) => {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use https://nodejs.org/api/zlib.html brotli by default, should get us smaller file sizes.

Also, Im +1 on compressing everything by default.

*/
export function compressLargeAssets(assetsMap: ZeBuildAssetsMap): ZeBuildAssetsMap {
return Object.values(assetsMap).reduce((memo, asset) => {
if (!shouldCompressAsset(asset.size, asset.contentEncoding)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to prevent double encoding, maybe we should check if the filename doesn't end with .zip .gz or .br?


const buffer =
typeof asset.buffer === 'string' ? Buffer.from(asset.buffer) : asset.buffer;
const compressed = gzipSync(buffer, { level: 9 });
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use zlib.brotliCompress(buffer[, options], callback) (async api) to prevent it from blocking the whole build process while zip assets.

@Nsttt
Copy link
Copy Markdown
Member Author

Nsttt commented Apr 3, 2026

Will pick up in the future. Not needed right now.

@Nsttt Nsttt closed this Apr 3, 2026
@Nsttt Nsttt deleted the feat/wasm-gzip-over-25mb branch April 6, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants