WIP: Video analytics

[daniel-graham-amplitude]

Summary

Checklist

• Does your PR title have the correct title format?
• Does your PR have a breaking change?:

---

Note

Medium Risk
Changes core event ingestion (Destination flush/send paths) and experimental delayed API behavior; video capture now depends on heartbeat correctness and shared-client queue semantics.

Overview
Adds delayed event upload to the SDK: events with a delay id are flushed to a separate /delayed endpoint (or delayedEventsServerUrl), using a DelayedPayload shape with instant_events vs timed events, no gzip, and queue logic to replace stale delayed events (including in-flight isFresh handling).

Video analytics moves from immediate track() calls to the shared heartbeat API: on play it sends Video Content Started immediately and queues a Video Content Stopped delayed event (default stop_reason: timeout, 1h timeout) that is updated on progress and flushed on real stop (paused, untracked, etc.) with play_id and richer properties (watch_duration, percent_completed). Buffering is modeled via a new waiting playback state in VideoObserver (stall detection when the playhead does not move), so sessions are not split across playing ↔ waiting.

Config surfaces delayedEventsServerUrl (experimental) through browser/core; test server and the HTML video demo wire the mock delayed endpoint.

^{Reviewed by Cursor Bugbot for commit 9a860a8. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

• ✅ Fixed: Percent completed mishandles zero duration
  • parseStopEventProperties now requires finite inputs and duration > 0, then clamps percent_completed to [0, 100], matching calculatePercentCompleted.
• ✅ Fixed: Batch mode builds wrong delayed URL
  • Delayed uploads now resolve the HTTP V2 base URL (or a custom non-Amplitude serverUrl) instead of appending /delayed to the Batch API path.
• ✅ Fixed: Shared delay object blocks stale removal
  • removeStaleDelayedEvents clones delay before setting isFresh so heartbeat-shared delay objects no longer keep the in-flight predecessor in the queue.

Or push these changes by commenting:

@cursor push 11e0afb1ad

Preview (11e0afb1ad)

diff --git a/packages/analytics-browser/src/video-capture/video-capture.ts b/packages/analytics-browser/src/video-capture/video-capture.ts
--- a/packages/analytics-browser/src/video-capture/video-capture.ts
+++ b/packages/analytics-browser/src/video-capture/video-capture.ts
@@ -216,11 +216,16 @@
   }
 
   parseStopEventProperties(nextState: VideoState): Record<string, string | number | boolean> {
-    const percentCompleted = ((nextState.position ?? 0) / (nextState.lastEvent?.duration ?? 0)) * 100;
+    const position = nextState.position ?? 0;
+    const duration = nextState.lastEvent?.duration ?? 0;
+    let percentCompleted = 0;
+    if (Number.isFinite(position) && Number.isFinite(duration) && duration > 0) {
+      percentCompleted = Math.min(100, Math.max(0, (position / duration) * 100));
+    }
     return {
       ...this.parseStartEventProperties(nextState),
       watch_duration: nextState.watchTime ?? 0,
-      percent_completed: percentCompleted || 0,
+      percent_completed: percentCompleted,
     };
   }
 }

diff --git a/packages/analytics-browser/test/video-capture/video-capture.test.ts b/packages/analytics-browser/test/video-capture/video-capture.test.ts
--- a/packages/analytics-browser/test/video-capture/video-capture.test.ts
+++ b/packages/analytics-browser/test/video-capture/video-capture.test.ts
@@ -628,5 +628,61 @@
         percent_completed: 0,
       });
     });
+
+    it('should return 0 percent_completed when duration is 0', () => {
+      const capture = new VideoCapture(mockAmplitude);
+      expect(
+        capture.parseStopEventProperties({
+          playbackState: 'paused',
+          lastEvent: { duration: 0, start_time: 0, last_position: 5 },
+          position: 5,
+          watchTime: 5,
+        }),
+      ).toEqual({
+        duration: 0,
+        start_time: 0,
+        position: 5,
+        watch_duration: 5,
+        percent_completed: 0,
+      });
+    });
+
+    it('should clamp percent_completed when position exceeds duration', () => {
+      const capture = new VideoCapture(mockAmplitude);
+      expect(
+        capture.parseStopEventProperties({
+          playbackState: 'ended',
+          lastEvent: { duration: 10, start_time: 0, last_position: 12 },
+          position: 12,
+          watchTime: 12,
+        }),
+      ).toEqual({
+        duration: 10,
+        start_time: 0,
+        position: 12,
+        watch_duration: 12,
+        percent_completed: 100,
+      });
+    });
+
+    it('should return 0 percent_completed for non-finite duration or position', () => {
+      const capture = new VideoCapture(mockAmplitude);
+      expect(
+        capture.parseStopEventProperties({
+          playbackState: 'paused',
+          lastEvent: { duration: Infinity, start_time: 0, last_position: 5 },
+          position: 5,
+          watchTime: 5,
+        }).percent_completed,
+      ).toBe(0);
+      expect(
+        capture.parseStopEventProperties({
+          playbackState: 'paused',
+          lastEvent: { duration: 10, start_time: 0, last_position: 0 },
+          position: NaN,
+          watchTime: 0,
+        }).percent_completed,
+      ).toBe(0);
+    });
   });
 });

diff --git a/packages/analytics-core/src/plugins/destination.ts b/packages/analytics-core/src/plugins/destination.ts
--- a/packages/analytics-core/src/plugins/destination.ts
+++ b/packages/analytics-core/src/plugins/destination.ts
@@ -161,7 +161,8 @@
           context.event.insert_id === incomingEvent.insert_id
         ) {
           if (this.inFlightDelayedEvents[incomingEvent.delay.id]) {
-            incomingEvent.delay.isFresh = true;
+            // Clone so isFresh is not set on a delay object shared with the in-flight event.
+            incomingEvent.delay = { ...incomingEvent.delay, isFresh: true };
             return true;
           }
           context.callback(buildResult(context.event, 0, 'Stale event overwritten'));
@@ -352,7 +353,15 @@
         this.config.enableRequestBodyCompression,
       );
       if (delay) {
-        serverUrl = this.config.delayedEventsServerUrl || `${serverUrl}/delayed`;
+        // Delayed ingest is HTTP V2 only; the Batch API has no /delayed route.
+        const { serverUrl: delayedBaseUrl } = createServerConfig(
+          this.config.serverUrl && !DEFAULT_AMPLITUDE_SERVER_URLS.has(this.config.serverUrl)
+            ? this.config.serverUrl
+            : '',
+          this.config.serverZone,
+          false,
+        );
+        serverUrl = this.config.delayedEventsServerUrl || `${delayedBaseUrl}/delayed`;
         this.translatePayloadToDelayedPayload(payload, list);
         shouldCompressUploadBody = false; // delayed events doesn't support compression
       }

diff --git a/packages/analytics-core/test/plugins/destination.test.ts b/packages/analytics-core/test/plugins/destination.test.ts
--- a/packages/analytics-core/test/plugins/destination.test.ts
+++ b/packages/analytics-core/test/plugins/destination.test.ts
@@ -14,7 +14,7 @@
 import { uuidPattern } from '../helpers/util';
 import { DiagnosticsClient, RequestMetadata } from '../../src';
 import { TrackEvent } from '../../src/types/event/event';
-import { AMPLITUDE_SERVER_URL } from '../../src/types/constants';
+import { AMPLITUDE_BATCH_SERVER_URL, AMPLITUDE_SERVER_URL } from '../../src/types/constants';
 
 const jsons = (obj: any) => JSON.stringify(obj, null, 2);
 
@@ -287,6 +287,61 @@
         });
         expect(send).toHaveBeenCalledTimes(3);
       });
+
+      test('should drop in-flight predecessor when replacement shares the same delay object', async () => {
+        const sharedDelay = { id: delayId };
+        const predecessor = {
+          event_type: 'before',
+          insert_id: '123',
+          delay: sharedDelay,
+        };
+        const replacement = {
+          event_type: 'after',
+          insert_id: '123',
+          delay: sharedDelay,
+        };
+        let resolveSend!: (value: Response) => void;
+        const sendPromise = new Promise<Response>((resolve) => {
+          resolveSend = resolve;
+        });
+        const successResponse = {
+          status: Status.Success,
+          statusCode: 200,
+          body: {
+            eventsIngested: 1,
+            payloadSizeBytes: 1,
+            serverUploadTime: 1,
+          },
+        } as Response;
+        const send = jest.fn().mockReturnValueOnce(sendPromise).mockResolvedValueOnce(successResponse);
+        destination.config = {
+          ...useDefaultConfig(),
+          transportProvider: { send },
+        };
+
+        const staleResult = destination.execute(predecessor);
+        const flushPromise = destination.flush(true);
+        const replacementResult = destination.execute(replacement);
+
+        expect(destination.queue).toHaveLength(2);
+
+        resolveSend(successResponse);
+
+        await staleResult;
+        await flushPromise;
+
+        expect(destination.queue).toHaveLength(1);
+        expect(destination.queue[0].event).toEqual(replacement);
+
+        await destination.flush(true);
+
+        await expect(replacementResult).resolves.toEqual({
+          event: replacement,
+          code: 200,
+          message: SUCCESS_MESSAGE,
+        });
+        expect(send).toHaveBeenCalledTimes(2);
+      });
     });
   });
 
@@ -1930,6 +1985,79 @@
       });
     });
 
+    test('should send delayed events to default HTTP V2 /delayed when serverUrl is empty', async () => {
+      const emptyUrlDestination = new Destination();
+      const emptyUrlTransport = {
+        send: jest.fn().mockResolvedValue(successResponse),
+      };
+      await emptyUrlDestination.setup({
+        ...useDefaultConfig(),
+        transportProvider: emptyUrlTransport,
+        apiKey: API_KEY,
+        serverUrl: '',
+        serverZone: 'US',
+      });
+      const delayId = 'delay-123';
+      const event = { event_type: 'delayed_event', delay: { id: delayId, timeout: 5000 } };
+      emptyUrlDestination.queue = [createContext(event)];
+      await emptyUrlDestination.flush(true);
+
+      expect(emptyUrlTransport.send).toHaveBeenCalledWith(delayedUrl, expect.objectContaining({ id: delayId }), false);
+    });
+
+    test('should append /delayed to a custom server URL', async () => {
+      const customDestination = new Destination();
+      const customTransport = {
+        send: jest.fn().mockResolvedValue(successResponse),
+      };
+      const customServerUrl = 'https://proxy.example.com/2/httpapi';
+      await customDestination.setup({
+        ...useDefaultConfig(),
+        transportProvider: customTransport,
+        apiKey: API_KEY,
+        serverUrl: customServerUrl,
+      });
+      const delayId = 'delay-123';
+      const event = { event_type: 'delayed_event', delay: { id: delayId, timeout: 5000 } };
+      customDestination.queue = [createContext(event)];
+      await customDestination.flush(true);
+
+      expect(customTransport.send).toHaveBeenCalledWith(
+        `${customServerUrl}/delayed`,
+        expect.objectContaining({ id: delayId }),
+        false,
+      );
+    });
+
+    test('should send delayed events to HTTP V2 /delayed when useBatch is true', async () => {
+      const batchDestination = new Destination();
+      const batchTransport = {
+        send: jest.fn().mockResolvedValue(successResponse),
+      };
+      await batchDestination.setup({
+        ...useDefaultConfig(),
+        transportProvider: batchTransport,
+        apiKey: API_KEY,
+        useBatch: true,
+        serverUrl: AMPLITUDE_BATCH_SERVER_URL,
+      });
+      const delayId = 'delay-123';
+      const delayTimeout = 5000;
+      const event = { event_type: 'delayed_event', delay: { id: delayId, timeout: delayTimeout } };
+      batchDestination.queue = [createContext(event)];
+      await batchDestination.flush(true);
+
+      expect(batchTransport.send).toHaveBeenCalledTimes(1);
+      expect(batchTransport.send).toHaveBeenCalledWith(
+        delayedUrl,
+        expect.objectContaining({
+          id: delayId,
+          timeout: delayTimeout,
+        }),
+        false,
+      );
+    });
+
     test('should send delayed events to /delayed endpoint', async () => {
       const delayId = 'delay-123';
       const delayTimeout = 5000;

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 9a860a8. Configure here.}

[cursor]

Percent completed mishandles zero duration

Medium Severity

parseStopEventProperties divides playhead position by duration with no finite/duration > 0 guard, so a 0 duration (unloaded media, some live streams) yields Infinity. percentCompleted || 0 does not catch that, and JSON.stringify later turns it into null. The same helper also skips clamping, so position past duration can exceed 100. calculatePercentCompleted in track-video.ts already handles these cases, but this path overwrites that value.

 

^{Reviewed by Cursor Bugbot for commit 9a860a8. Configure here.}

[cursor]

Batch mode builds wrong delayed URL

Medium Severity

Delayed uploads fall back to `${serverUrl}/delayed` after createServerConfig is called with useBatch. With batching enabled and no delayedEventsServerUrl, that becomes https://api2.amplitude.com/batch/delayed (or the EU batch equivalent) instead of the HTTP V2 delayed path (/2/httpapi/delayed). Video heartbeat events then post to an endpoint that does not exist.

 

^{Reviewed by Cursor Bugbot for commit 9a860a8. Configure here.}

[cursor]

Shared delay object blocks stale removal

Medium Severity

removeStaleDelayedEvents sets incomingEvent.delay.isFresh = true on the delay object itself. Heartbeat re-tracks queued events by passing that same delay reference into client.track(), so the in-flight queue entry is marked fresh too. removeEvents then keeps both copies, and the next flush can send the same video stop twice.

Additional Locations (1)

• packages/analytics-core/src/plugins/destination.ts#L552-L567

 

^{Reviewed by Cursor Bugbot for commit 9a860a8. Configure here.}

[github-actions]

size-limit report 📦

Path	Size
packages/analytics-browser/lib/scripts/amplitude-min.js.gz	66.36 KB (+3.23% 🔺)
packages/session-replay-browser/lib/scripts/session-replay-browser-min.js.gz	134.99 KB (+0.02% 🔺)
packages/unified/lib/scripts/amplitude-min.umd.js.gz	219.25 KB (+0.4% 🔺)
@amplitude/element-selector (gzipped esm)	3.4 KB (0%)