How does Segment process and retain data from the initial ingestion phase down to final workspace storage? – Twilio Support

Question

How does Segment process and retain data from the initial ingestion phase down to final workspace storage?

Twilio Segment

Segment Console

Data flow and retention vary depending on which stage of the source pipeline the events are currently passing through:

Pre-Ingestion (Red Step): If an event fails or is dropped before reaching our tracking API (e.g., due to local network blockers, ad-blockers, or client-side errors), Segment has no visibility into the event or why it failed.
Ingestion (Blue Step): A 200 OK response confirms that the event has successfully cleared our "front door" and been received by the Segment pipeline.
Processing (Yellow Step): Once inside the pipeline, events undergo JSON validation, Source Insert Functions execution, and Protocols filtering. Data is only retained for a 6-hour window during this active processing stage.
Storage (Green Step): Events that successfully pass all validation checks are classified as "Eligible Events." These are permanently stored in our internal Athena bucket according to the specific retention period configured for your workspace.

Learn more about Source Overview page.