Bulk Uploading App Crashing

[wilsonbhayblue]

When I was uploading full folders, the app would crash (twice over the whole vault)

I narrowed it down to the PRV-M10.avi file. For this one, it throw the warning of large file size (100mb). I am not able to click "Upload All"

For the BB120 folder the issue files were as shown below. The warning was the same as above

[Worxs3D]

Uploads > ~400 MB crash — verified not server-side

What we checked

We looked for size limits on the server side (Kong + Storage + DB) and could not find anything that would explain a crash at ~400+ MB.

Verified server-side config

Kong (Nginx)

• nginx-kong.conf contains client_max_body_size 0; (unlimited).
• docker-compose.override.yml sets:
  • KONG_NGINX_HTTP_CLIENT_MAX_BODY_SIZE: 0
  • KONG_NGINX_HTTPS_CLIENT_MAX_BODY_SIZE: 0

Storage service

• docker-compose.override.yml and docker-compose.s3.yml both set:
  • FILE_SIZE_LIMIT: 10737418240 (10 GiB)

Database

• No DB-level bucket/file-size limit found (no matches when searching for common values like 100MB/1GB and scanning limit-related numeric columns).

Logs during the crash

During a 400+ MB attempt, Kong logs show only:

• /rest/v1/... requests (normal app queries)
• /realtime/v1/websocket (HTTP 101)

No /storage/v1/object/... upload request (POST/PUT) appears in Kong logs at the time of the crash, and there is no clear Storage/MinIO “request too large” type error on the server side.

Result

Based on the above, we could not reproduce a server-side file-size limit around ~400 MB.

[Worxs3D]

Confirmed root cause and fix for the ~400 MB crash.

The server was not the cause. BluePLM read the complete file into Electron, converted it to a Base64 string, and transferred that string through IPC to the renderer.

That crosses V8’s maximum string length:

• 400 MiB binary data becomes about 533 MiB Base64.
• V8 rejects strings above 0x1fffffe8 characters.
• The effective raw-file limit is therefore about 384 MiB.
• The renderer/app terminates before a normal upload error can be reported.

Recommended implementation:

1. Do not use fs:read-file / Buffer.toString('base64') for uploads or initial sync.
2. Keep hashing streamed from disk (createReadStream / incremental hash).
3. Add a main-process IPC method that accepts only:
   • local file path
   • upload target URL
   • content type
4. In that IPC handler, pipe fs.createReadStream(filePath) directly into the HTTP request body and report progress as byte counts.
5. For Supabase Storage, create a signed upload URL in the renderer and pass that URL plus the local path to the main process. The actual file bytes never enter the renderer.
6. Keep the existing streamed download path unchanged.

Conceptually:

const input = fs.createReadStream(filePath)
const request = https.request(uploadUrl, {
  method: 'PUT',
  headers: {
    'content-length': fileSize,
    'content-type': 'application/octet-stream',
  },
})

await pipeline(input, request)

Validation:

• Typecheck and targeted regression tests pass.
• A local 401 MiB end-to-end streaming test completed successfully.
• A real-world upload and download test above the previous limit completed successfully.

This removes the client-side size crash while preserving content hashes and revision storage.